-- Logs begin at Thu 2022-07-28 07:32:10 UTC, end at Thu 2022-07-28 09:29:52 UTC. -- Jul 28 08:00:17.187742 dv-cinder-851351-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 08:00:17.198087 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 08:00:17 2022] *** Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: nodename: dv-cinder-851351-1-iscsi Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: machine: x86_64 Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: clock source: unix Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: pcre jit disabled Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: detected number of CPU cores: 4 Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: current working directory: / Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: detected binary path: /usr/bin/uwsgi-core Jul 28 08:00:17.200025 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: your processes number limit is 47680 Jul 28 08:00:17.200569 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: your memory page size is 4096 bytes Jul 28 08:00:17.200569 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: detected max file descriptor number: 2048 Jul 28 08:00:17.200569 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: lock engine: pthread robust mutexes Jul 28 08:00:17.200569 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: thunder lock: enabled Jul 28 08:00:17.200569 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 08:00:17.200569 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 08:00:17.262417 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: Python main interpreter initialized at 0x55649f74c9c0 Jul 28 08:00:17.262417 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: python threads support enabled Jul 28 08:00:17.262417 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: your server socket listen backlog is limited to 100 connections Jul 28 08:00:17.262417 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: your mercy for graceful operations on workers is 90 seconds Jul 28 08:00:17.262950 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 08:00:17.263089 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: *** Operational MODE: preforking *** Jul 28 08:00:17.263604 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: *** uWSGI is running in multiple interpreter mode *** Jul 28 08:00:17.263604 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: spawned uWSGI master process (pid: 80356) Jul 28 08:00:17.263886 dv-cinder-851351-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 08:00:17.265048 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: spawned uWSGI worker 1 (pid: 80368, cores: 1) Jul 28 08:00:17.265561 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: spawned uWSGI worker 2 (pid: 80369, cores: 1) Jul 28 08:00:17.270197 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 08:00:22.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:00:22.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: from cryptography.utils import int_from_bytes Jul 28 08:00:22.862887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:00:22.862887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: from cryptography.utils import int_from_bytes Jul 28 08:00:22.913963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:00:22.914215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:00:22.914336 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] command line args: [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:00:22.914604 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:00:22.914757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ================================================================================ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:00:22.915013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] admin_token = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.915259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_dir = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.915423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.915615 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_source = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.915760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] debug = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.915964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.916320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.916551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.916751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.916994 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.917250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.917378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_config_append = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.917568 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.917701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_dir = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.917871 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.918166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.918468 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.918638 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.918840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.919012 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.919208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.919429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.919671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.919900 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.920128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.920348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.920585 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.920819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.921059 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.921331 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.921840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.922020 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] publish_errors = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.922276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.922506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.922677 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.922924 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.923151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.923379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] standard_threads = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.923605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.923837 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.924007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.924178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_journal = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.924349 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_json = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.924519 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_stderr = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.924690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_syslog = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.924919 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:22.925178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.925414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.925666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.925901 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.926167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.926406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.926649 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.926935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.927181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.927424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.927674 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.927904 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.928146 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.928379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.928622 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.928863 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.929097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.929347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.929597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.929839 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.930072 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.930350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.930611 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.930861 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.931105 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.931334 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.931562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.931792 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.932025 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.932271 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.932513 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.932740 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.932967 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.933211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.933454 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.933687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.933931 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.934190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.934424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.934663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.934889 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.935115 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.935348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.935578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.935811 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.936054 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.936282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.936518 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.936742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.936973 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.937203 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.937445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.937670 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.937906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.938168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.938394 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.938626 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.938858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.939082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.939328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.939561 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.939796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.940023 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.940283 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.940512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.940740 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.941205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.941471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.941703 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.941948 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.942216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.942517 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.942753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.942986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.943229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.943460 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.943693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.943929 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.944160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.944392 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.944641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.944844 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.945077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.945339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.945576 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.945812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.946045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.946309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.946540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.946773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.946998 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.947229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.947457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.947686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.947944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.948143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.948382 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.948617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.948861 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.949096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.949347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.949972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.950232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.950464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.external = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.950691 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.950988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.951217 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.951445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.password = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.951671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.token = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.951907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.952140 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.952383 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.952610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.952853 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.953095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.953326 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.953556 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.953798 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.954042 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.954322 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.954549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.954788 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.955018 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.955298 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.955545 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.955782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.956031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.956316 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.956568 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.956810 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.957052 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.957305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.957537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.957774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.958016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.958327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.958556 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.958801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.959037 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.959299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.959534 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.959784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.960019 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.960264 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.960497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.960738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.960981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.961215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.961448 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.961690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.961915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.962175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.962433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.962682 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.962911 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.963140 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.963369 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.963610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.963856 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.964098 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.964351 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.964593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.964859 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.965090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.965335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.965562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.965797 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.966024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.966279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.966519 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.966754 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.966998 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.967227 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.967472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.967716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.967947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.968190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.968429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.968657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.968889 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.969118 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.969352 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.969584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.969827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.970059 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.970316 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.970562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.970805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.971036 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.971266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.971510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.971752 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.971992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.972226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.972461 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.972688 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.972925 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.973181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.973410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.973652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.973966 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.974239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.974474 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.974701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.974933 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.975165 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.975466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.975610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.975858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.976098 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.976341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.976737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.976982 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.977204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.977452 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.977687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.977920 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.978215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.978420 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.978663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.978907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.979139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.979394 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.979614 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.979853 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.980085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.980315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.980548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.980790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.981027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.981257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.981489 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.981721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.981888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:00:22.981888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: from cryptography.utils import int_from_bytes Jul 28 08:00:22.982781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.982781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.983019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:00:22.983019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: from cryptography.utils import int_from_bytes Jul 28 08:00:22.983265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.983265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.983265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.983265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.983699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.983699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.983951 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.984085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.984259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.984491 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.984737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.984988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.985227 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.985471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.985715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.985958 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.986497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.986497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.986767 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.986904 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.987130 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.987379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.987619 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.988054 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.988289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.988534 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.988783 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.989023 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.989266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.989540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.989791 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.990034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.990297 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.990528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.990755 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.990987 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.991228 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.991487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.991729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.991960 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.992197 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.992424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.992658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.992888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.993130 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.993362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.993616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.993858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.994094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.994355 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.994596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.994838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.995069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.995308 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.995533 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.995783 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.996028 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.996253 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.996491 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.996784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.997041 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.997282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.997523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.997756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.997993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.998248 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.998483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.998721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.998944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.999185 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.999416 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.999655 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:22.999886 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.000129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.000359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.000601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.000833 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.001064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.001297 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.001462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:00:23.030066 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:00:23.030341 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:00:23.030465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] command line args: [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:00:23.030698 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:00:23.030875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ================================================================================ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:00:23.031175 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] admin_token = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.031355 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.031480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] config_dir = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.031603 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.031730 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.031844 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.031974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] config_source = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.031974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] debug = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.032190 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.032337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.032491 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.032775 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.033006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.033239 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.033430 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_config_append = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.033586 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.033783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_dir = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.033946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_file = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.034221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.034469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.034726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.034968 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.035148 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.035335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.035567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.035812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.036047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.036352 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.036589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.036805 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.037053 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.037301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.037582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.038096 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.038312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] publish_errors = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.038547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.038782 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.038968 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.039210 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.039448 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.039689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] standard_threads = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.039928 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.040175 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.040355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.040478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.040591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_journal = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.040704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_json = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.040860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_stderr = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.041037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_syslog = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.041272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:00:23.041534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.041778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.042042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.042315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.042569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.042811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.043063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.043302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.043553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.043804 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.044061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.044301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.044554 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.044795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.045047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.045300 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.045542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.045801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.046059 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.046335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.046579 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.046837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.047110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.047363 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.047616 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.047865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.048089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.048325 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.048566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.048835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.049095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.049330 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.049566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.049819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.050082 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.050349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.050604 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.050844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.051086 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.051325 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.051561 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.051839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.052335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.052335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.052557 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.052782 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.053022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.053022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.053691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.053691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.053941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.054293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.054293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.054793 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.054901 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.055140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.055372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.055742 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.055848 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.056078 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.056324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.056324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.056324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.056324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.056324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.056324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.056324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.058609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.058609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.058609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.058894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.059012 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.059269 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.059518 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.059791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.060068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.060366 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.061439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.061439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.061439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.061439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.061439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.062673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.062673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.062673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.062673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.062673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.063304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.063304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.063510 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.063615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.063963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.064124 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.064254 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.064504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.064743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.064949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.065196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.065437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.065693 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.065959 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.066237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.066482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.066729 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.external = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.067223 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.067562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.067816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.068043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.password = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.068279 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.token = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.068544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.068786 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.069011 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.069248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.069521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.069791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.070033 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.070256 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.070536 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.070786 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.070907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.071018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.071107 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.071281 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.071503 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.071744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.072000 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.072287 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.072526 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.072763 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.073051 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.073304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.073596 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.073845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.074083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.074348 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.074967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.075244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.075498 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.075736 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.075989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.076237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.076476 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.076697 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.076959 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.077202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.077455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.077721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.077949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.078249 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.078496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.078709 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.078963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.079226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.079465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.079718 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.079981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.080203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.080445 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.080665 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.080914 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.081163 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.081413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.081673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.081931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.082230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.082475 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.082716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.082953 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.083240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.083471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.083711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.083966 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.084211 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.084463 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.084695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.084946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.085198 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.085440 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.085691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.085943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.086206 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.086452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.086688 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.086930 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.087182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.087441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.087684 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.087925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.088183 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.088511 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.088766 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.089006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.089261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.089512 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.089754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.089999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.090265 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.090506 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.090754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.091023 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.091251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.091508 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.091761 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.092012 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.092252 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.092488 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.092722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.092963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.093203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.093444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.093695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.093937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.094217 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.094471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.094723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.094958 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.095214 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.095455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.095689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.095930 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.096170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.096423 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.096674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.096918 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.097177 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.097409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.097655 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.097895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.098161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.098569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.098894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.099156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.099392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.099637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.099893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.100150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.100400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.100643 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.100902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.101158 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.101402 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.101652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.101894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.102161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.102405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] role.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.102660 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] role.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.102904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.103159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.103409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.103661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.103921 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.104172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.104425 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.104671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.104924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.105163 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.105414 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.105664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.105916 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.106197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.106632 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.106876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.107137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.107392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.107646 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.107899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.108138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.108397 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.108645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.108889 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.109125 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.109362 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.109602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.109857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.110132 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.110388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.110631 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.110871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.111107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.111353 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.111594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.111841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.112091 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.112363 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.112624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.112859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.113102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.113348 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.113603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.113845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.114087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.114435 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.114685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.114931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.115166 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.115420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.115723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.115995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.116239 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.116498 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.116738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.116980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.117216 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.117462 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.117704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.117935 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.118213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.118464 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.118703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.118943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.119201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.119440 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.119687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.119914 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.120163 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.120405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:00:23.120585 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:00:23.124973 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.146095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.146373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.146548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.155466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.176126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.176329 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.176942 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.177178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.214678 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.244853 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.245067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.245201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.245378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.245616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.245808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.252780 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.252992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.269984 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.270254 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.270439 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.270719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.270934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.271148 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.271361 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.283015 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.296206 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.296419 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.296551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.296685 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.296844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.296961 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.297089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.297214 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.297336 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.297469 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.297585 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.297705 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.297825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.311865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.312105 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.312302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.312524 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.312739 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.312956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.313164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.314974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.315162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:00:23.315502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.315701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:00:23.315968 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.316226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.316479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.316726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.321550 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.325400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.325645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.326168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.326168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.326384 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.326501 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.326705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:00:23.327318 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.327443 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.327889 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.327889 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.335330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.337715 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.337934 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:00:23.338342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.338551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:00:23.338823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.339085 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.339345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.339608 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.340529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.340825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.341103 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.341374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.341648 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.341903 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.342200 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.342471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.342738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.343007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.343271 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.350023 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.350355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.350629 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.350924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.358340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.363705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.363995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.364315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.364630 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.364939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.365202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.365470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.365760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.366041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.366351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.366629 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.369124 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.370729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.371008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.375858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.376153 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.376433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.383501 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.383793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.388050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.388336 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.388629 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.388904 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.389184 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.389458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.389733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.390013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.390448 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.391936 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.392223 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.397232 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.397523 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.397811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.404791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.405085 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.406281 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.406578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.409408 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.409701 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.410037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.410353 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.410631 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.410721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.410916 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.411248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.411519 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.414045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.414376 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.418892 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.422159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.425413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.425714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.425996 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.426361 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.426588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.426932 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.428318 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.432623 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.435919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.436221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.440795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.443170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.443464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.443728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.443987 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.444157 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.447545 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.447852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.448145 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.448432 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.448701 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.448980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.451462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.453092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.453355 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.456314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.456581 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.456883 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.457158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.457445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.465928 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.466253 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.466526 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.466791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.469942 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.470346 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.470628 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.470906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.471180 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.471606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.471902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.474014 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.475663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.475946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.478907 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.479197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.479479 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.479766 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.480049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.488277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.491379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.491969 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.492277 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.492541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.492819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.493095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.493371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.493675 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.494344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.497111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.498757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.499034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.499317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.505829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.507464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.509551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.510383 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.510666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.510941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.511202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.512595 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.515591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.518331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.519289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.519570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.519840 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.519936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.520350 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.520602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.526804 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.527076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.527188 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.527354 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.527642 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.527905 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.528179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.528437 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.528696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.528827 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.529068 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.529348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.529616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.529879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.530176 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.530433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.531698 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.531993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.532281 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.532560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.540823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.541114 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.541396 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.548373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:00:23.548656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.548955 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.549220 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.549496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.549778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.550056 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.550352 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.550740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.551026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.551306 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.551923 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.552187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:00:23.563735 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55649f74c9c0 pid: 80368 (default app) Jul 28 08:00:23.567343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39000848-9bd6-489c-af50-cdb1079cebb4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:23.567710 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39000848-9bd6-489c-af50-cdb1079cebb4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:23.567979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39000848-9bd6-489c-af50-cdb1079cebb4 None None] PATH_INFO: `/v3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:23.572393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jul 28 08:00:23 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:23.586386 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55649f74c9c0 pid: 80369 (default app) Jul 28 08:00:31.379675 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee57f-e5b3-4c66-9ad5-a3a9167aaaaf None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:31.380277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee57f-e5b3-4c66-9ad5-a3a9167aaaaf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:31.380277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee57f-e5b3-4c66-9ad5-a3a9167aaaaf None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:31.381625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 2/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:00:31.389703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:31.390251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:31.390251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:31.460162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: INFO dbcounter [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] Registered counter for database keystone Jul 28 08:00:31.481880 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG oslo_db.sqlalchemy.engines [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80369) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:00:31.489790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writer thread running {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:00:31.673546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG passlib.handlers.bcrypt [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80369) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:00:31.673865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG passlib.handlers.bcrypt [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80369) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:00:31.731343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:31.733847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:31.866016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:00:32.026728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcee57f-e5b3-4c66-9ad5-a3a9167aaaaf None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:32.097435 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: INFO dbcounter [None req-8bcee57f-e5b3-4c66-9ad5-a3a9167aaaaf None None] Registered counter for database keystone Jul 28 08:00:32.117586 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG oslo_db.sqlalchemy.engines [None req-8bcee57f-e5b3-4c66-9ad5-a3a9167aaaaf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80368) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:00:32.124223 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writer thread running {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:00:32.245272 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcee57f-e5b3-4c66-9ad5-a3a9167aaaaf None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:32.249811 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72ba43-3a65-4e79-9550-f59beeb5f41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bed3d6665f7c40158ba248253deda4a1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:32.250494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ba43-3a65-4e79-9550-f59beeb5f41f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:32.250785 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ba43-3a65-4e79-9550-f59beeb5f41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:32.251058 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ba43-3a65-4e79-9550-f59beeb5f41f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:32.260065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72ba43-3a65-4e79-9550-f59beeb5f41f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:32.312232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cf72ba43-3a65-4e79-9550-f59beeb5f41f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:00:32.312443 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72ba43-3a65-4e79-9550-f59beeb5f41f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:32.323631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 3/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:00:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:00:32.330450 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:32.345987 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82c72e-ed75-4220-8254-f7aee570bcb8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:32.351262 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cd7ba-a235-4b00-9190-5d453665037b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bed3d6665f7c40158ba248253deda4a1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:32.352431 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd7ba-a235-4b00-9190-5d453665037b admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:32.352714 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd7ba-a235-4b00-9190-5d453665037b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:32.352989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd7ba-a235-4b00-9190-5d453665037b admin admin] PATH_INFO: `/v3/projects/bed3d6665f7c40158ba248253deda4a1` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:32.354667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd7ba-a235-4b00-9190-5d453665037b admin admin] RBAC: Authorizing `identity:get_project(project_id=bed3d6665f7c40158ba248253deda4a1)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:32.404982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3c0cd7ba-a235-4b00-9190-5d453665037b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:00:32.405654 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd7ba-a235-4b00-9190-5d453665037b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:32.410230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 2/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jul 28 08:00:32 2022] GET /identity/v3/projects/bed3d6665f7c40158ba248253deda4a1 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:34.410229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebf11f-142d-476e-8c63-f5d5a31641df None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:34.410802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebf11f-142d-476e-8c63-f5d5a31641df None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:34.410802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebf11f-142d-476e-8c63-f5d5a31641df None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:34.415360 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 4/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:34.421533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd32b6-4f2c-4c3e-9cc9-b894aaad7730 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:34.422029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd32b6-4f2c-4c3e-9cc9-b894aaad7730 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:34.422029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd32b6-4f2c-4c3e-9cc9-b894aaad7730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:34.445820 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ebdd32b6-4f2c-4c3e-9cc9-b894aaad7730 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:34.448037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ebdd32b6-4f2c-4c3e-9cc9-b894aaad7730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:34.456515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 3/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 28 08:00:34.479120 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebf11f-142d-476e-8c63-f5d5a31641df None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:34.493238 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebf11f-142d-476e-8c63-f5d5a31641df None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:34.498088 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30418224-c6c5-4762-a543-95d0498915b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bed3d6665f7c40158ba248253deda4a1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:34.498772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418224-c6c5-4762-a543-95d0498915b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:34.499060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418224-c6c5-4762-a543-95d0498915b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:34.499338 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418224-c6c5-4762-a543-95d0498915b1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:34.508238 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30418224-c6c5-4762-a543-95d0498915b1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:34.510070 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30418224-c6c5-4762-a543-95d0498915b1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:34.520024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 5/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:00:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:00:34.526338 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd32b6-4f2c-4c3e-9cc9-b894aaad7730 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:34.536084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd32b6-4f2c-4c3e-9cc9-b894aaad7730 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:34.541448 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dac0c-131d-4ff9-b63e-5c44c321f744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bed3d6665f7c40158ba248253deda4a1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:34.542129 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dac0c-131d-4ff9-b63e-5c44c321f744 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:34.542421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dac0c-131d-4ff9-b63e-5c44c321f744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:34.542691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dac0c-131d-4ff9-b63e-5c44c321f744 admin admin] PATH_INFO: `/v3/users/acf187c6216740528137e4e8f0aad7b7` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:34.544190 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dac0c-131d-4ff9-b63e-5c44c321f744 admin admin] RBAC: Authorizing `identity:get_user(user_id=acf187c6216740528137e4e8f0aad7b7)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:34.546045 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dac0c-131d-4ff9-b63e-5c44c321f744 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:34.547713 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 4/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jul 28 08:00:34 2022] GET /identity/v3/users/acf187c6216740528137e4e8f0aad7b7 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:36.623055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1fe6f-1d1b-4886-9c91-7b9163def508 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:36.623625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1fe6f-1d1b-4886-9c91-7b9163def508 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:36.623625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1fe6f-1d1b-4886-9c91-7b9163def508 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:36.624948 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 6/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:36.630207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75225bc8-44df-468b-9bc9-a9cdaad701b4 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:36.630616 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75225bc8-44df-468b-9bc9-a9cdaad701b4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:36.630710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75225bc8-44df-468b-9bc9-a9cdaad701b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:36.640937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052540-05ad-4c56-987d-8b8c54d03fb8 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:36.641223 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052540-05ad-4c56-987d-8b8c54d03fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:36.641500 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052540-05ad-4c56-987d-8b8c54d03fb8 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:36.642835 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 7/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:36.648378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:36.648666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:36.648947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:36.650383 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-75225bc8-44df-468b-9bc9-a9cdaad701b4 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:36.651567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-75225bc8-44df-468b-9bc9-a9cdaad701b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:36.696790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG passlib.handlers.bcrypt [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80368) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:00:36.697194 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG passlib.handlers.bcrypt [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80368) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:00:36.736578 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 5/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:36.743012 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:36.744654 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:36.758805 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75225bc8-44df-468b-9bc9-a9cdaad701b4 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:36.782788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:36.845245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 8/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:36.867701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:36.868244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75225bc8-44df-468b-9bc9-a9cdaad701b4 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:36.902867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ab094-fa51-4ec5-a56e-37c7d4b79694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:36.903416 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ab094-fa51-4ec5-a56e-37c7d4b79694 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:36.903695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ab094-fa51-4ec5-a56e-37c7d4b79694 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:36.903968 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ab094-fa51-4ec5-a56e-37c7d4b79694 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:36.915832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ab094-fa51-4ec5-a56e-37c7d4b79694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-412ab094-fa51-4ec5-a56e-37c7d4b79694 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:36.924871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:36.925816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:36.926707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 6/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:00:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:00:36.932119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ab094-fa51-4ec5-a56e-37c7d4b79694 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:36.970966 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737610e5-616b-49a5-b91e-050d5fb94846 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:36.973275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:37.009901 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f29a17-e83b-4b27-aaba-66a4cd7bd50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:37.010836 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f29a17-e83b-4b27-aaba-66a4cd7bd50c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:37.011128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f29a17-e83b-4b27-aaba-66a4cd7bd50c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:37.011399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f29a17-e83b-4b27-aaba-66a4cd7bd50c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:37.016305 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:37.035868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ab094-fa51-4ec5-a56e-37c7d4b79694 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:37.070905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:37.071538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:37.071817 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:37.072094 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:37.072833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:37.073140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:37.074939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:37.081466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f29a17-e83b-4b27-aaba-66a4cd7bd50c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:37.083031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f29a17-e83b-4b27-aaba-66a4cd7bd50c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:37.088415 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 7/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:00:37.094262 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:37.189535 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 9/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:00:37.195783 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f29a17-e83b-4b27-aaba-66a4cd7bd50c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:37.199121 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:37.300529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f29a17-e83b-4b27-aaba-66a4cd7bd50c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:37.307564 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf1782-96a3-4974-8447-6d8c90a037bd None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:37.337859 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:37.338772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:37.339055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:37.339329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:37.342132 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f613eb9-0e71-4b9a-8311-83d821c05f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:37.342788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f613eb9-0e71-4b9a-8311-83d821c05f13 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:37.343074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f613eb9-0e71-4b9a-8311-83d821c05f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:37.343338 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f613eb9-0e71-4b9a-8311-83d821c05f13 None admin] PATH_INFO: `/v3/users/acf187c6216740528137e4e8f0aad7b7` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:37.344538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f613eb9-0e71-4b9a-8311-83d821c05f13 None admin] RBAC: Authorizing `identity:get_user(user_id=acf187c6216740528137e4e8f0aad7b7)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:37.345990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f613eb9-0e71-4b9a-8311-83d821c05f13 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:37.347654 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 8/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:37 2022] GET /identity/v3/users/acf187c6216740528137e4e8f0aad7b7 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:00:37.354776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f613eb9-0e71-4b9a-8311-83d821c05f13 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:37.358546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:37.360061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:37.370153 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:00:37.378399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:37.379350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:37.380198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:00:37.380198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:00:37.380198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:00:37.380198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:00:37.380198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:00:37.385767 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:37.442727 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:37.459319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f613eb9-0e71-4b9a-8311-83d821c05f13 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:37.487044 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b95e1-eb7c-4639-8821-742f889df417 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:37.494203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbb60e-c937-4202-8f44-69a24f546f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:37.494888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbb60e-c937-4202-8f44-69a24f546f0f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:37.495180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbb60e-c937-4202-8f44-69a24f546f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:37.495449 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbb60e-c937-4202-8f44-69a24f546f0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:37.496623 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbb60e-c937-4202-8f44-69a24f546f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:37.498096 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbb60e-c937-4202-8f44-69a24f546f0f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:37.499731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:00:37.505719 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbb60e-c937-4202-8f44-69a24f546f0f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:37.522285 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a03b18-523a-4b48-a532-61f525d6693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:37.522926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a03b18-523a-4b48-a532-61f525d6693a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:37.523210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a03b18-523a-4b48-a532-61f525d6693a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:37.523488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a03b18-523a-4b48-a532-61f525d6693a None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:37.524197 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a03b18-523a-4b48-a532-61f525d6693a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:37.524522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a03b18-523a-4b48-a532-61f525d6693a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:37.525927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a03b18-523a-4b48-a532-61f525d6693a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:37.543507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:37.596338 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:37.619754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbb60e-c937-4202-8f44-69a24f546f0f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:37.659631 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf4baf-dcd2-4e50-b16e-6bed2211b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:37.660345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4baf-dcd2-4e50-b16e-6bed2211b6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:37.660540 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4baf-dcd2-4e50-b16e-6bed2211b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:37.660809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4baf-dcd2-4e50-b16e-6bed2211b6f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:37.661685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf4baf-dcd2-4e50-b16e-6bed2211b6f5 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.domain.id=default, user.id=acf187c6216740528137e4e8f0aad7b7) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:37.661995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf4baf-dcd2-4e50-b16e-6bed2211b6f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:37.663553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf4baf-dcd2-4e50-b16e-6bed2211b6f5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:37.671812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:00:37 2022] GET /identity/v3/role_assignments?user.id=acf187c6216740528137e4e8f0aad7b7&scope.domain.id=default&role.id=65cd37a73e4f498482fefabb8374006c => generated 233 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:39.962071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8a4e-5605-41f1-a747-24e7d10e670d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:39.962961 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8a4e-5605-41f1-a747-24e7d10e670d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:39.963374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8a4e-5605-41f1-a747-24e7d10e670d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:39.965201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:39.970222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2b70d-3106-4913-a9e3-33b22d387d65 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:39.971017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2b70d-3106-4913-a9e3-33b22d387d65 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:39.971434 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2b70d-3106-4913-a9e3-33b22d387d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:39.997752 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-87b2b70d-3106-4913-a9e3-33b22d387d65 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:39.998895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-87b2b70d-3106-4913-a9e3-33b22d387d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:40.076504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf26bba-9ae2-4d02-a588-a993aeb136b1 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.077169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf26bba-9ae2-4d02-a588-a993aeb136b1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.077666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf26bba-9ae2-4d02-a588-a993aeb136b1 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.079263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 13/23] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:40.084867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609efa-b134-46cc-98a3-b9537f282c95 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.085461 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609efa-b134-46cc-98a3-b9537f282c95 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.085938 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609efa-b134-46cc-98a3-b9537f282c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.101989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 11/24] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:40.135164 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-52609efa-b134-46cc-98a3-b9537f282c95 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:40.136712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-52609efa-b134-46cc-98a3-b9537f282c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:40.146824 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2b70d-3106-4913-a9e3-33b22d387d65 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:40.255718 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:40.276607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 14/25] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:00:40.277927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3088-b594-47f2-a8bd-bf44a0347e0c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.278237 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3088-b594-47f2-a8bd-bf44a0347e0c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.278510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3088-b594-47f2-a8bd-bf44a0347e0c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.279757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 15/26] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:40.285175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82c657-6351-434b-9190-870e053b1939 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.285452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2b70d-3106-4913-a9e3-33b22d387d65 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:40.285559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82c657-6351-434b-9190-870e053b1939 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.285730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82c657-6351-434b-9190-870e053b1939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.327243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8f82c657-6351-434b-9190-870e053b1939 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:40.328667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8f82c657-6351-434b-9190-870e053b1939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:40.330650 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f59ec-1ca2-4d9a-91af-f8d7fac58e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:40.331296 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f59ec-1ca2-4d9a-91af-f8d7fac58e48 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.331578 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f59ec-1ca2-4d9a-91af-f8d7fac58e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.331850 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f59ec-1ca2-4d9a-91af-f8d7fac58e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.333296 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f59ec-1ca2-4d9a-91af-f8d7fac58e48 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:40.334040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f59ec-1ca2-4d9a-91af-f8d7fac58e48 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:40.431407 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:40.469204 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 12/27] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:00:40 2022] POST /identity/v3/roles => generated 219 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:00:40.470229 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f59ec-1ca2-4d9a-91af-f8d7fac58e48 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:40.496541 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 16/28] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:40.530450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82c657-6351-434b-9190-870e053b1939 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:40.608891 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f59ec-1ca2-4d9a-91af-f8d7fac58e48 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:40.655588 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd93324-48c6-40ac-b45b-487129acd073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:40.656285 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93324-48c6-40ac-b45b-487129acd073 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.656576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93324-48c6-40ac-b45b-487129acd073 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.656849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93324-48c6-40ac-b45b-487129acd073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.659508 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:40.671207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd93324-48c6-40ac-b45b-487129acd073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:40.672855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd93324-48c6-40ac-b45b-487129acd073 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-7cd93324-48c6-40ac-b45b-487129acd073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:00:40.693753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:40.694647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:40.695398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:00:40.695398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:00:40.695398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:00:40.695398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:40.695398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 13/29] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:40.697909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d53a52-4f8d-464c-9ac1-677c60f70769 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.697909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d53a52-4f8d-464c-9ac1-677c60f70769 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.697909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d53a52-4f8d-464c-9ac1-677c60f70769 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.699216 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 14/30] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:40.706845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d53a52-4f8d-464c-9ac1-677c60f70769 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:40.799171 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82c657-6351-434b-9190-870e053b1939 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:40.848014 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e073f-870a-4398-9d94-7ccb10d4a774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:40.849467 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e073f-870a-4398-9d94-7ccb10d4a774 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.849893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e073f-870a-4398-9d94-7ccb10d4a774 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.851594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e073f-870a-4398-9d94-7ccb10d4a774 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.872922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e073f-870a-4398-9d94-7ccb10d4a774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f04e073f-870a-4398-9d94-7ccb10d4a774 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:40.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:40.882564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:00:40.883275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 17/31] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:00:40.884372 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab85c-47cb-46df-9f83-9c612f9a5f59 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.884565 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab85c-47cb-46df-9f83-9c612f9a5f59 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.884896 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab85c-47cb-46df-9f83-9c612f9a5f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.887067 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d53a52-4f8d-464c-9ac1-677c60f70769 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:40.931440 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:40.932467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:40.932996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:40.933478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:40.934544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:40.935106 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:40.936778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:40.959411 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 15/32] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:40.961077 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:41.022090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bc5ab85c-47cb-46df-9f83-9c612f9a5f59 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:41.023041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:41.023560 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bc5ab85c-47cb-46df-9f83-9c612f9a5f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:41.055350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:41.098197 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 18/33] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:41.099345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb99a7a-0a19-42b4-9a6f-06dc84cb55db None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:41.099529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ab85c-47cb-46df-9f83-9c612f9a5f59 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:41.146746 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:41.148022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.148464 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.149192 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.150385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:41.150847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:41.152410 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:41.164119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 16/34] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:41.165183 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:41.245447 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ab85c-47cb-46df-9f83-9c612f9a5f59 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:41.284548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c840ac-8cca-4c54-879a-b7873b80deef None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:41.297880 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:41.307855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567acf4b-1a94-4580-b37c-04784c0cf531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:41.309508 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acf4b-1a94-4580-b37c-04784c0cf531 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.310008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acf4b-1a94-4580-b37c-04784c0cf531 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.310488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acf4b-1a94-4580-b37c-04784c0cf531 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567acf4b-1a94-4580-b37c-04784c0cf531 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:41.313966 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567acf4b-1a94-4580-b37c-04784c0cf531 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:41.343978 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982386f5-e8c0-47b3-9320-d8c4020e31c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:41.345266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982386f5-e8c0-47b3-9320-d8c4020e31c6 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.346068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982386f5-e8c0-47b3-9320-d8c4020e31c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.346911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982386f5-e8c0-47b3-9320-d8c4020e31c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.349402 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982386f5-e8c0-47b3-9320-d8c4020e31c6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:41.352491 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982386f5-e8c0-47b3-9320-d8c4020e31c6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:41.373936 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 17/35] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:00:41 2022] POST /identity/v3/roles => generated 225 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:00:41.375118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982386f5-e8c0-47b3-9320-d8c4020e31c6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:41.459816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 19/36] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:00:41 2022] POST /identity/v3/projects => generated 301 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:00:41.461572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273ed4c-04e7-4910-ac04-357a0dff6800 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.461937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273ed4c-04e7-4910-ac04-357a0dff6800 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.462324 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273ed4c-04e7-4910-ac04-357a0dff6800 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.463866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 20/37] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:41.470229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe85f7b-125f-4bdf-b8b0-054669db333a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.470611 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe85f7b-125f-4bdf-b8b0-054669db333a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.470978 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe85f7b-125f-4bdf-b8b0-054669db333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.513168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982386f5-e8c0-47b3-9320-d8c4020e31c6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:41.516280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6fe85f7b-125f-4bdf-b8b0-054669db333a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:41.518092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6fe85f7b-125f-4bdf-b8b0-054669db333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:41.549534 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:41.598359 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23029f86-d93b-4550-87c3-79a237cf67f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:41.598984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23029f86-d93b-4550-87c3-79a237cf67f2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.599275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23029f86-d93b-4550-87c3-79a237cf67f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.599556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23029f86-d93b-4550-87c3-79a237cf67f2 None admin] PATH_INFO: `/v3/users/acf187c6216740528137e4e8f0aad7b7` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.600853 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23029f86-d93b-4550-87c3-79a237cf67f2 None admin] RBAC: Authorizing `identity:get_user(user_id=acf187c6216740528137e4e8f0aad7b7)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:41.602424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23029f86-d93b-4550-87c3-79a237cf67f2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:41.604379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 18/38] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:41 2022] GET /identity/v3/users/acf187c6216740528137e4e8f0aad7b7 => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:41.610534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23029f86-d93b-4550-87c3-79a237cf67f2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:41.629482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 21/39] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:00:41.631577 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc554-29af-4331-9681-98d9b27eae09 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.632395 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc554-29af-4331-9681-98d9b27eae09 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.632885 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc554-29af-4331-9681-98d9b27eae09 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.635843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 22/40] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:41.647898 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dccc52-8876-4edc-9d84-3526080962cb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.648459 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dccc52-8876-4edc-9d84-3526080962cb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.648958 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dccc52-8876-4edc-9d84-3526080962cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.656668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:41.674964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f9dccc52-8876-4edc-9d84-3526080962cb None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:41.676295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f9dccc52-8876-4edc-9d84-3526080962cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:41.748663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23029f86-d93b-4550-87c3-79a237cf67f2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:41.770179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 23/41] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:00:41.770179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dccc52-8876-4edc-9d84-3526080962cb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:41.794051 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bcca2-eb07-472c-b080-a0986a87bc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:41.794756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bcca2-eb07-472c-b080-a0986a87bc46 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.795037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bcca2-eb07-472c-b080-a0986a87bc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.795314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bcca2-eb07-472c-b080-a0986a87bc46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.796501 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bcca2-eb07-472c-b080-a0986a87bc46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:41.797996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bcca2-eb07-472c-b080-a0986a87bc46 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:41.799663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 19/42] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:41.800433 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bcca2-eb07-472c-b080-a0986a87bc46 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:41.853846 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:41.876447 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dccc52-8876-4edc-9d84-3526080962cb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:41.904421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bcca2-eb07-472c-b080-a0986a87bc46 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:41.906857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:41.910459 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36069262-a4cf-42bf-8edb-b6d95e8e75f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:41.911183 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069262-a4cf-42bf-8edb-b6d95e8e75f2 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.911504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069262-a4cf-42bf-8edb-b6d95e8e75f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.911744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069262-a4cf-42bf-8edb-b6d95e8e75f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.912492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36069262-a4cf-42bf-8edb-b6d95e8e75f2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:41.914110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36069262-a4cf-42bf-8edb-b6d95e8e75f2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:41.930661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 24/43] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:00:41 2022] POST /identity/v3/roles => generated 223 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:00:41.931473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36069262-a4cf-42bf-8edb-b6d95e8e75f2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:41.942372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a0eb5-28a6-40fc-a634-5475f3b3b061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:41.943057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a0eb5-28a6-40fc-a634-5475f3b3b061 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:41.943347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a0eb5-28a6-40fc-a634-5475f3b3b061 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:41.943633 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a0eb5-28a6-40fc-a634-5475f3b3b061 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:41.944913 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a0eb5-28a6-40fc-a634-5475f3b3b061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:41.946483 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a0eb5-28a6-40fc-a634-5475f3b3b061 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:41.948248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 20/44] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:41.954888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a0eb5-28a6-40fc-a634-5475f3b3b061 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:42.065595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36069262-a4cf-42bf-8edb-b6d95e8e75f2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:42.080135 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a0eb5-28a6-40fc-a634-5475f3b3b061 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:42.105224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7c5d4-497b-4401-8d97-6ff6bf6903ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:42.106699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7c5d4-497b-4401-8d97-6ff6bf6903ae None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:42.106699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7c5d4-497b-4401-8d97-6ff6bf6903ae None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:42.106699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7c5d4-497b-4401-8d97-6ff6bf6903ae None admin] PATH_INFO: `/v3/domains/default/users/acf187c6216740528137e4e8f0aad7b7/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:42.108888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7c5d4-497b-4401-8d97-6ff6bf6903ae None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=acf187c6216740528137e4e8f0aad7b7, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:42.110912 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7c5d4-497b-4401-8d97-6ff6bf6903ae None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:42.124351 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:42.124621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eb308-c7b5-41b0-9938-928b110f8f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:42.125077 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb308-c7b5-41b0-9938-928b110f8f10 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:42.125723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb308-c7b5-41b0-9938-928b110f8f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:42.125723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb308-c7b5-41b0-9938-928b110f8f10 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:42.126465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eb308-c7b5-41b0-9938-928b110f8f10 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:42.128075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eb308-c7b5-41b0-9938-928b110f8f10 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:42.133104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 25/45] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jul 28 08:00:41 2022] PUT /identity/v3/domains/default/users/acf187c6216740528137e4e8f0aad7b7/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:00:42.169844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 21/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:00:41 2022] POST /identity/v3/projects => generated 312 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:00:44.167082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02019a-3555-48c8-a12f-4d5553d4b307 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.167662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02019a-3555-48c8-a12f-4d5553d4b307 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.167662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02019a-3555-48c8-a12f-4d5553d4b307 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.169046 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 26/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:44.175072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76774b72-fb15-41f3-b938-f1ea0774c706 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.175584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76774b72-fb15-41f3-b938-f1ea0774c706 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.176249 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76774b72-fb15-41f3-b938-f1ea0774c706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.195817 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-76774b72-fb15-41f3-b938-f1ea0774c706 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:44.197369 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-76774b72-fb15-41f3-b938-f1ea0774c706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:44.212711 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9a60-c659-4ea7-9685-c999649af9e2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.212979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9a60-c659-4ea7-9685-c999649af9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.213276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9a60-c659-4ea7-9685-c999649af9e2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.214638 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 27/48] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:44.217865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:44.220320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b737-92e5-492b-ad1a-3654ca43f045 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.220630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b737-92e5-492b-ad1a-3654ca43f045 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.220875 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b737-92e5-492b-ad1a-3654ca43f045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.241909 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2618b737-92e5-492b-ad1a-3654ca43f045 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:44.243168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2618b737-92e5-492b-ad1a-3654ca43f045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:44.296658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 22/49] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:44.313177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 28/50] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:00:44.321754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76774b72-fb15-41f3-b938-f1ea0774c706 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:44.336959 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618b737-92e5-492b-ad1a-3654ca43f045 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:44.431995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76774b72-fb15-41f3-b938-f1ea0774c706 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:44.438082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618b737-92e5-492b-ad1a-3654ca43f045 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:44.468486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:44.468676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dd2a1-a88d-4dbe-bf69-bc0850e21782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:44.469194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dd2a1-a88d-4dbe-bf69-bc0850e21782 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.469479 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dd2a1-a88d-4dbe-bf69-bc0850e21782 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.469755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dd2a1-a88d-4dbe-bf69-bc0850e21782 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.477683 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f88539-eb33-4982-a978-74a659287e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:44.478388 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f88539-eb33-4982-a978-74a659287e53 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.478671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f88539-eb33-4982-a978-74a659287e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.478945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f88539-eb33-4982-a978-74a659287e53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.480093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f88539-eb33-4982-a978-74a659287e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:44.480887 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dd2a1-a88d-4dbe-bf69-bc0850e21782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:44.481572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f88539-eb33-4982-a978-74a659287e53 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:44.483248 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 29/51] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-099dd2a1-a88d-4dbe-bf69-bc0850e21782 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:44.485956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:44.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:44.487658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 23/52] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:00:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:44.489568 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f88539-eb33-4982-a978-74a659287e53 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:44.493256 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dd2a1-a88d-4dbe-bf69-bc0850e21782 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:44.510208 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:44.598447 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dd2a1-a88d-4dbe-bf69-bc0850e21782 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:44.600355 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f88539-eb33-4982-a978-74a659287e53 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:44.635492 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:44.636133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.636417 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.636693 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.637428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:44.637736 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:44.638803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37c812-ceed-4b32-9479-ac8374a04fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:44.639175 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:44.639502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37c812-ceed-4b32-9479-ac8374a04fc0 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.639790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37c812-ceed-4b32-9479-ac8374a04fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.640067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37c812-ceed-4b32-9479-ac8374a04fc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.640817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37c812-ceed-4b32-9479-ac8374a04fc0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:44.642328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37c812-ceed-4b32-9479-ac8374a04fc0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:44.648308 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 24/53] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:44.654322 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:44.671590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 30/54] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:00:44 2022] POST /identity/v3/projects => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:00:44.752036 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:44.781058 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c57ee-545b-4d2b-bda8-f3b0060cb4ba None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:44.825851 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5f7b0-32bc-4a2a-af58-7114be4534eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:44.826582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5f7b0-32bc-4a2a-af58-7114be4534eb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.826875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5f7b0-32bc-4a2a-af58-7114be4534eb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.827154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5f7b0-32bc-4a2a-af58-7114be4534eb None admin] PATH_INFO: `/v3/users/acf187c6216740528137e4e8f0aad7b7` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.828541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5f7b0-32bc-4a2a-af58-7114be4534eb None admin] RBAC: Authorizing `identity:get_user(user_id=acf187c6216740528137e4e8f0aad7b7)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:44.830180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5f7b0-32bc-4a2a-af58-7114be4534eb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:44.832202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 25/55] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:44 2022] GET /identity/v3/users/acf187c6216740528137e4e8f0aad7b7 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:00:44.838078 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37c812-ceed-4b32-9479-ac8374a04fc0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:44.895992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:44.950874 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37c812-ceed-4b32-9479-ac8374a04fc0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:44.986466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc2b27-293b-4d08-88be-275ed7cbd030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:44.987030 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc2b27-293b-4d08-88be-275ed7cbd030 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:44.987318 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc2b27-293b-4d08-88be-275ed7cbd030 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:44.987597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc2b27-293b-4d08-88be-275ed7cbd030 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:44.988796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc2b27-293b-4d08-88be-275ed7cbd030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:44.990356 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc2b27-293b-4d08-88be-275ed7cbd030 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:44.991977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 31/56] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:44.997916 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5f7b0-32bc-4a2a-af58-7114be4534eb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:45.103523 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5f7b0-32bc-4a2a-af58-7114be4534eb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:45.137974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12956018-a8cc-4aed-be8a-448ac00e8441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:45.138675 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956018-a8cc-4aed-be8a-448ac00e8441 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:45.138941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956018-a8cc-4aed-be8a-448ac00e8441 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:45.139224 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956018-a8cc-4aed-be8a-448ac00e8441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:45.140040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956018-a8cc-4aed-be8a-448ac00e8441 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.domain.id=default, user.id=acf187c6216740528137e4e8f0aad7b7) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:45.140349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956018-a8cc-4aed-be8a-448ac00e8441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:45.141822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956018-a8cc-4aed-be8a-448ac00e8441 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:45.148081 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 26/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:00:44 2022] GET /identity/v3/role_assignments?user.id=acf187c6216740528137e4e8f0aad7b7&scope.domain.id=default&role.id=65cd37a73e4f498482fefabb8374006c => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:46.734737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ae4a-c0f8-4338-aa63-3de813ca12cb None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:46.735187 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ae4a-c0f8-4338-aa63-3de813ca12cb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:46.735348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ae4a-c0f8-4338-aa63-3de813ca12cb None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:46.737553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 32/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:46.743996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd80710-d217-451e-81e7-db7b1f090a5a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:46.744388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd80710-d217-451e-81e7-db7b1f090a5a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:46.744567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd80710-d217-451e-81e7-db7b1f090a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:46.763674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4fd80710-d217-451e-81e7-db7b1f090a5a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:46.764903 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4fd80710-d217-451e-81e7-db7b1f090a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:46.766937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:46.841033 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 27/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:46.865085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90ae4a-c0f8-4338-aa63-3de813ca12cb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:46.975855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90ae4a-c0f8-4338-aa63-3de813ca12cb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:46.991845 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:47.011951 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c792ac-eb7c-464f-8e95-f85f266f3956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:47.012610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c792ac-eb7c-464f-8e95-f85f266f3956 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:47.012897 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c792ac-eb7c-464f-8e95-f85f266f3956 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:47.013174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c792ac-eb7c-464f-8e95-f85f266f3956 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:47.014425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c792ac-eb7c-464f-8e95-f85f266f3956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:47.015872 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c792ac-eb7c-464f-8e95-f85f266f3956 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:47.017482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 33/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:47.023836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd80710-d217-451e-81e7-db7b1f090a5a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:47.135013 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd80710-d217-451e-81e7-db7b1f090a5a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:47.174241 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aa021-3a2d-40d4-9a9a-8123e2db578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:47.174893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa021-3a2d-40d4-9a9a-8123e2db578f None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:47.175151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa021-3a2d-40d4-9a9a-8123e2db578f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:47.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa021-3a2d-40d4-9a9a-8123e2db578f None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:47.176162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aa021-3a2d-40d4-9a9a-8123e2db578f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:47.177768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aa021-3a2d-40d4-9a9a-8123e2db578f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:47.193314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:47.208743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 28/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:00:47 2022] POST /identity/v3/users => generated 275 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:00:50.294393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33d019-a999-4e81-aefc-6c273917cc04 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.294905 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33d019-a999-4e81-aefc-6c273917cc04 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.294977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33d019-a999-4e81-aefc-6c273917cc04 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.296302 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 34/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:50.307753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1f729-c13c-4747-b473-36fc60858837 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.308694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1f729-c13c-4747-b473-36fc60858837 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.309158 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1f729-c13c-4747-b473-36fc60858837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.339153 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8cf1f729-c13c-4747-b473-36fc60858837 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:50.340728 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8cf1f729-c13c-4747-b473-36fc60858837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:50.423083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 29/63] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:00:50.461134 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33d019-a999-4e81-aefc-6c273917cc04 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:50.499365 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed2c32-f01b-485a-8170-28eb11293112 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.500000 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed2c32-f01b-485a-8170-28eb11293112 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.500475 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed2c32-f01b-485a-8170-28eb11293112 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.502080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 30/64] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:50.507899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee086ef-7c2d-4f08-874f-66d962ff784c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.508462 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee086ef-7c2d-4f08-874f-66d962ff784c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.508936 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee086ef-7c2d-4f08-874f-66d962ff784c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.533772 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6ee086ef-7c2d-4f08-874f-66d962ff784c None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:50.535335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6ee086ef-7c2d-4f08-874f-66d962ff784c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:50.583202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33d019-a999-4e81-aefc-6c273917cc04 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:50.618738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acce574-8473-44aa-b407-3ff1f2b8f04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:50.619768 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acce574-8473-44aa-b407-3ff1f2b8f04f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.620255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acce574-8473-44aa-b407-3ff1f2b8f04f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.620985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acce574-8473-44aa-b407-3ff1f2b8f04f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.626514 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 31/65] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:50.627974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f7c3-2401-4961-b8c6-b2fd6cc7a07a None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.628270 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f7c3-2401-4961-b8c6-b2fd6cc7a07a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.628555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f7c3-2401-4961-b8c6-b2fd6cc7a07a None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.630222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 32/66] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:50.635925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570e16-6012-4610-9d93-5d9e079903c7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.636219 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570e16-6012-4610-9d93-5d9e079903c7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.636503 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570e16-6012-4610-9d93-5d9e079903c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.638250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acce574-8473-44aa-b407-3ff1f2b8f04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7acce574-8473-44aa-b407-3ff1f2b8f04f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:50.643922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:50.644750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:50.645527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:00:50.648161 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 35/67] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:50.654603 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acce574-8473-44aa-b407-3ff1f2b8f04f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:50.661698 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-eb570e16-6012-4610-9d93-5d9e079903c7 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:50.662941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-eb570e16-6012-4610-9d93-5d9e079903c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:50.704268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:50.736946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:50.762994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 33/68] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:50.764205 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570e16-6012-4610-9d93-5d9e079903c7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:50.797785 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acce574-8473-44aa-b407-3ff1f2b8f04f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:50.838747 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:50.839803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.840379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.840849 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.841771 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:50.842319 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:50.844054 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:50.861329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 36/69] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:00:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:50.862779 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:50.887400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570e16-6012-4610-9d93-5d9e079903c7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:50.924222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb250ad2-0be2-4f36-827a-b6345c589374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:50.924916 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb250ad2-0be2-4f36-827a-b6345c589374 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:50.925204 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb250ad2-0be2-4f36-827a-b6345c589374 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:50.925480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb250ad2-0be2-4f36-827a-b6345c589374 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:50.937857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb250ad2-0be2-4f36-827a-b6345c589374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-bb250ad2-0be2-4f36-827a-b6345c589374 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:50.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:50.943779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:50.944504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 34/70] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:00:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:50.945109 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb250ad2-0be2-4f36-827a-b6345c589374 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:50.978749 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699565ce-cb9e-49db-81fd-d223000276df None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:50.987215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:50.997387 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.015145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af78a4-57f2-4bc5-9898-19814c49aba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.015811 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af78a4-57f2-4bc5-9898-19814c49aba1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.016100 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af78a4-57f2-4bc5-9898-19814c49aba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.016379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af78a4-57f2-4bc5-9898-19814c49aba1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.027112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af78a4-57f2-4bc5-9898-19814c49aba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e4af78a4-57f2-4bc5-9898-19814c49aba1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:51.032993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:51.033809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:51.034574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:00:51.038250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 37/71] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:51.039630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a563ebb-bc55-460f-86cf-49a83ee05cb0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.039920 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a563ebb-bc55-460f-86cf-49a83ee05cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.040193 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a563ebb-bc55-460f-86cf-49a83ee05cb0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.041671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 38/72] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:51.046200 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a563ebb-bc55-460f-86cf-49a83ee05cb0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:51.061249 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb250ad2-0be2-4f36-827a-b6345c589374 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.097392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b2cb0-08e0-4bb2-b6b5-125055435791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.097963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2cb0-08e0-4bb2-b6b5-125055435791 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.098307 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2cb0-08e0-4bb2-b6b5-125055435791 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.098582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2cb0-08e0-4bb2-b6b5-125055435791 None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.112954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b2cb0-08e0-4bb2-b6b5-125055435791 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:51.114659 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b2cb0-08e0-4bb2-b6b5-125055435791 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:51.116364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 35/73] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:50 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:51.117142 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b2cb0-08e0-4bb2-b6b5-125055435791 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:51.153608 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a563ebb-bc55-460f-86cf-49a83ee05cb0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.189487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599047d-b26d-428e-bcf6-72852939f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.190154 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599047d-b26d-428e-bcf6-72852939f21d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.190441 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599047d-b26d-428e-bcf6-72852939f21d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.190719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599047d-b26d-428e-bcf6-72852939f21d None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.191445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599047d-b26d-428e-bcf6-72852939f21d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:51.191758 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599047d-b26d-428e-bcf6-72852939f21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:51.193172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599047d-b26d-428e-bcf6-72852939f21d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:51.202181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 39/74] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:00:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:51.203492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a3eb-9797-41eb-befb-a2ae2c7039cd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.203791 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a3eb-9797-41eb-befb-a2ae2c7039cd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.204112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a3eb-9797-41eb-befb-a2ae2c7039cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.220023 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b2cb0-08e0-4bb2-b6b5-125055435791 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.221674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.222221 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-06b0a3eb-9797-41eb-befb-a2ae2c7039cd None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:51.223468 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-06b0a3eb-9797-41eb-befb-a2ae2c7039cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:51.246388 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.255438 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2dc11-5d42-45e2-812b-44f22de966a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.256084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2dc11-5d42-45e2-812b-44f22de966a5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.256367 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2dc11-5d42-45e2-812b-44f22de966a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.256642 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2dc11-5d42-45e2-812b-44f22de966a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.257378 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2dc11-5d42-45e2-812b-44f22de966a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:51.257699 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2dc11-5d42-45e2-812b-44f22de966a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:51.259121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2dc11-5d42-45e2-812b-44f22de966a5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:51.267640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 36/75] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:51.268856 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37b2e-c0c5-4067-af42-ee0a7ef58748 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.269140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37b2e-c0c5-4067-af42-ee0a7ef58748 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.269446 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37b2e-c0c5-4067-af42-ee0a7ef58748 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.270950 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 37/76] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:51.271450 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a37b2e-c0c5-4067-af42-ee0a7ef58748 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:51.294852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 40/77] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:51.295715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a3eb-9797-41eb-befb-a2ae2c7039cd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:51.380894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a37b2e-c0c5-4067-af42-ee0a7ef58748 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.396086 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a3eb-9797-41eb-befb-a2ae2c7039cd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.415247 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9b9d7-aa80-4dd6-b081-57ac105780c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.415902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9b9d7-aa80-4dd6-b081-57ac105780c8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.416187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9b9d7-aa80-4dd6-b081-57ac105780c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.416443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9b9d7-aa80-4dd6-b081-57ac105780c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.429852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135373a-19f7-4118-9c11-3c05bdadfb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.430567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135373a-19f7-4118-9c11-3c05bdadfb63 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.430869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135373a-19f7-4118-9c11-3c05bdadfb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.431129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135373a-19f7-4118-9c11-3c05bdadfb63 None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.432345 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135373a-19f7-4118-9c11-3c05bdadfb63 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:51.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135373a-19f7-4118-9c11-3c05bdadfb63 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:51.435450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 41/78] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:51 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:51.436207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135373a-19f7-4118-9c11-3c05bdadfb63 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:51.453634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.463717 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.492682 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9b9d7-aa80-4dd6-b081-57ac105780c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:51.494220 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9b9d7-aa80-4dd6-b081-57ac105780c8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:51.537466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135373a-19f7-4118-9c11-3c05bdadfb63 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.572909 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682f079-bbfa-40b6-b70f-6f798b7ff3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.573573 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682f079-bbfa-40b6-b70f-6f798b7ff3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.573864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682f079-bbfa-40b6-b70f-6f798b7ff3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.574181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682f079-bbfa-40b6-b70f-6f798b7ff3f1 None admin] PATH_INFO: `/v3/users/acf187c6216740528137e4e8f0aad7b7` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.575750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682f079-bbfa-40b6-b70f-6f798b7ff3f1 None admin] RBAC: Authorizing `identity:get_user(user_id=acf187c6216740528137e4e8f0aad7b7)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:51.577233 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682f079-bbfa-40b6-b70f-6f798b7ff3f1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:51.578910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 42/79] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:51 2022] GET /identity/v3/users/acf187c6216740528137e4e8f0aad7b7 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:51.580204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b87e0-54d1-4303-a0f5-a2215501f697 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.580512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b87e0-54d1-4303-a0f5-a2215501f697 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.580773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b87e0-54d1-4303-a0f5-a2215501f697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.595698 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 38/80] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:00:51.596803 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9b9d7-aa80-4dd6-b081-57ac105780c8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:51.599282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f16b87e0-54d1-4303-a0f5-a2215501f697 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:51.600631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f16b87e0-54d1-4303-a0f5-a2215501f697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:51.662601 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.669652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 43/81] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:51.670760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b87e0-54d1-4303-a0f5-a2215501f697 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:51.697167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.705229 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9b9d7-aa80-4dd6-b081-57ac105780c8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.749122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421c653-0bea-4ba9-bc6f-cddd071b330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.749783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421c653-0bea-4ba9-bc6f-cddd071b330c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.750066 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421c653-0bea-4ba9-bc6f-cddd071b330c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.750374 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421c653-0bea-4ba9-bc6f-cddd071b330c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.763604 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421c653-0bea-4ba9-bc6f-cddd071b330c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0421c653-0bea-4ba9-bc6f-cddd071b330c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:51.769669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:51.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 39/82] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:51.775995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421c653-0bea-4ba9-bc6f-cddd071b330c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:51.791227 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b87e0-54d1-4303-a0f5-a2215501f697 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.832082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c31f4-f143-4604-8548-9b5820197377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.832903 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c31f4-f143-4604-8548-9b5820197377 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.833181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c31f4-f143-4604-8548-9b5820197377 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.833451 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c31f4-f143-4604-8548-9b5820197377 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.885227 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421c653-0bea-4ba9-bc6f-cddd071b330c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:51.902242 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c31f4-f143-4604-8548-9b5820197377 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:51.904075 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c31f4-f143-4604-8548-9b5820197377 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:51.911232 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.923435 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa3b0a-015a-4499-a537-4a7f079361cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:51.923992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa3b0a-015a-4499-a537-4a7f079361cb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:51.924259 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa3b0a-015a-4499-a537-4a7f079361cb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:51.924530 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa3b0a-015a-4499-a537-4a7f079361cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:51.929334 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:51.993525 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa3b0a-015a-4499-a537-4a7f079361cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:51.995043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa3b0a-015a-4499-a537-4a7f079361cb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.009684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 44/83] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:00:52.010594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c31f4-f143-4604-8548-9b5820197377 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.098416 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 40/84] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:00:52.099297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa3b0a-015a-4499-a537-4a7f079361cb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.114054 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c31f4-f143-4604-8548-9b5820197377 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.129499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:52.141259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:52.153400 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec45238-b645-48f6-acd0-33e11aa9ef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.154149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec45238-b645-48f6-acd0-33e11aa9ef40 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.154501 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec45238-b645-48f6-acd0-33e11aa9ef40 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.154895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec45238-b645-48f6-acd0-33e11aa9ef40 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.156231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec45238-b645-48f6-acd0-33e11aa9ef40 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.157785 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec45238-b645-48f6-acd0-33e11aa9ef40 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.159705 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 45/85] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:52.160390 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec45238-b645-48f6-acd0-33e11aa9ef40 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.214970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa3b0a-015a-4499-a537-4a7f079361cb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.250222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5130-0b19-4d34-8a4c-2e653aa29d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.250886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5130-0b19-4d34-8a4c-2e653aa29d46 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.251176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5130-0b19-4d34-8a4c-2e653aa29d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.251442 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5130-0b19-4d34-8a4c-2e653aa29d46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.252718 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e5130-0b19-4d34-8a4c-2e653aa29d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.254232 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e5130-0b19-4d34-8a4c-2e653aa29d46 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.255933 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 41/86] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:52.258077 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5130-0b19-4d34-8a4c-2e653aa29d46 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.270229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec45238-b645-48f6-acd0-33e11aa9ef40 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.305377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.306006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.306471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.306606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.307298 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:52.307612 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.309016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.320876 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 46/87] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:52.321768 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.372253 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:52.381618 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5130-0b19-4d34-8a4c-2e653aa29d46 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.391769 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:52.421370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7140b-a873-4fa0-925d-8983b5798927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.422037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7140b-a873-4fa0-925d-8983b5798927 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.422352 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7140b-a873-4fa0-925d-8983b5798927 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.422636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7140b-a873-4fa0-925d-8983b5798927 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.423942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7140b-a873-4fa0-925d-8983b5798927 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.425447 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7140b-a873-4fa0-925d-8983b5798927 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.427496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 42/88] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:52.428317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7140b-a873-4fa0-925d-8983b5798927 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.450605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f780ac2-eff1-4ac4-b92c-228a10710007 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.496278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfccb0a-19ec-468e-a79d-899c5b24541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.497292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfccb0a-19ec-468e-a79d-899c5b24541e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.497292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfccb0a-19ec-468e-a79d-899c5b24541e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.497471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfccb0a-19ec-468e-a79d-899c5b24541e None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.498660 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfccb0a-19ec-468e-a79d-899c5b24541e None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.500422 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfccb0a-19ec-468e-a79d-899c5b24541e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.502258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 47/89] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:52.503027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfccb0a-19ec-468e-a79d-899c5b24541e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.551925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7140b-a873-4fa0-925d-8983b5798927 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.590388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.591033 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.591315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.591580 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.592388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] RBAC: Adding query filter params (role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6, user.id=0ed09815b5214d75993e263bb3fd05c9) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:52.592696 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.594204 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.600137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 43/90] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/role_assignments?user.id=0ed09815b5214d75993e263bb3fd05c9&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=06907320d4bc47b3ba3375df728a605a => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:52.607491 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.633955 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfccb0a-19ec-468e-a79d-899c5b24541e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.638265 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:52.666157 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:52.680957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64e3de-8766-4475-8ced-0ef8390bb626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.681612 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64e3de-8766-4475-8ced-0ef8390bb626 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.681895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64e3de-8766-4475-8ced-0ef8390bb626 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.682220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64e3de-8766-4475-8ced-0ef8390bb626 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.682944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64e3de-8766-4475-8ced-0ef8390bb626 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.684475 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64e3de-8766-4475-8ced-0ef8390bb626 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.734330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 48/91] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:00:52 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:00:52.735451 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64e3de-8766-4475-8ced-0ef8390bb626 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.749582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c41c35-7d44-47b8-abd5-cc7a5cbf67b3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.802996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5db77-5a6e-4223-baee-aacda16d4676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.803741 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5db77-5a6e-4223-baee-aacda16d4676 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.804029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5db77-5a6e-4223-baee-aacda16d4676 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.804309 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5db77-5a6e-4223-baee-aacda16d4676 None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.805724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5db77-5a6e-4223-baee-aacda16d4676 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.808187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5db77-5a6e-4223-baee-aacda16d4676 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.810659 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 44/92] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:52.811811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5db77-5a6e-4223-baee-aacda16d4676 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.866610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64e3de-8766-4475-8ced-0ef8390bb626 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.907981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:52.908984 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:52.909458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:52.909947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:52.911113 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] RBAC: Adding query filter params (role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6, user.id=0ed09815b5214d75993e263bb3fd05c9) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:52.911451 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:52.913170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:52.923541 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 49/93] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/role_assignments?user.id=0ed09815b5214d75993e263bb3fd05c9&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:52.924816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:52.955749 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:52.968650 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5db77-5a6e-4223-baee-aacda16d4676 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:52.999116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:53.015437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:53.016474 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:53.017002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:53.017491 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:53.018563 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6, user.id=acf187c6216740528137e4e8f0aad7b7) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:53.019107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:53.021492 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:53.028575 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 45/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/role_assignments?user.id=acf187c6216740528137e4e8f0aad7b7&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=65cd37a73e4f498482fefabb8374006c => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:53.077501 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64215ee-4091-4d41-a1cc-c59a2a3ccc3c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:53.123381 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d2e84-6640-4c49-9332-ee6ab7a6125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:53.124045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2e84-6640-4c49-9332-ee6ab7a6125e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:53.124332 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2e84-6640-4c49-9332-ee6ab7a6125e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:53.124611 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2e84-6640-4c49-9332-ee6ab7a6125e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:53.230789 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d2e84-6640-4c49-9332-ee6ab7a6125e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:53.232769 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d2e84-6640-4c49-9332-ee6ab7a6125e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:53.296382 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:53.369233 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 50/95] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:00:53.375074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:53.500102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92ff72-096a-4051-8746-310e5f23f01c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:53.538841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae67a-4912-4371-a1da-e03d126986eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:53.539523 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae67a-4912-4371-a1da-e03d126986eb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:53.539811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae67a-4912-4371-a1da-e03d126986eb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:53.540093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae67a-4912-4371-a1da-e03d126986eb None admin] PATH_INFO: `/v3/projects/865a7ea820ca4699a96b9cc1a0e8879f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:53.541307 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ae67a-4912-4371-a1da-e03d126986eb None admin] RBAC: Authorizing `identity:get_project(project_id=865a7ea820ca4699a96b9cc1a0e8879f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:53.542796 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ae67a-4912-4371-a1da-e03d126986eb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:53.544519 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 46/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:53 2022] GET /identity/v3/projects/865a7ea820ca4699a96b9cc1a0e8879f => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:53.551473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d2e84-6640-4c49-9332-ee6ab7a6125e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:53.674008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d2e84-6640-4c49-9332-ee6ab7a6125e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:53.717633 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2de14a-a3cc-485e-a292-d09a4d6800ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:53.718313 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2de14a-a3cc-485e-a292-d09a4d6800ac None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:53.718629 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2de14a-a3cc-485e-a292-d09a4d6800ac None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:53.718883 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2de14a-a3cc-485e-a292-d09a4d6800ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:53.719767 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2de14a-a3cc-485e-a292-d09a4d6800ac None admin] RBAC: Adding query filter params (role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=865a7ea820ca4699a96b9cc1a0e8879f, user.id=0ed09815b5214d75993e263bb3fd05c9) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:53.720090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2de14a-a3cc-485e-a292-d09a4d6800ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:53.721567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2de14a-a3cc-485e-a292-d09a4d6800ac None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:53.727858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 51/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:00:53 2022] GET /identity/v3/role_assignments?user.id=0ed09815b5214d75993e263bb3fd05c9&scope.project.id=865a7ea820ca4699a96b9cc1a0e8879f&role.id=06907320d4bc47b3ba3375df728a605a => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:55.291160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9008a7-d035-4163-8e7d-045ca7377b32 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:55.292158 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9008a7-d035-4163-8e7d-045ca7377b32 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:55.292661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9008a7-d035-4163-8e7d-045ca7377b32 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:55.294262 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 47/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:55.299847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77003fe2-5fd3-4349-8c07-b46c8d26ab5a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:55.300705 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77003fe2-5fd3-4349-8c07-b46c8d26ab5a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:55.301218 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77003fe2-5fd3-4349-8c07-b46c8d26ab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:55.316431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:55.331258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-77003fe2-5fd3-4349-8c07-b46c8d26ab5a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:55.332880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-77003fe2-5fd3-4349-8c07-b46c8d26ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:55.406985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 52/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:55.428903 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9008a7-d035-4163-8e7d-045ca7377b32 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:55.457412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:55.535487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f408c-fc0a-401a-81fe-5dd6baea3e82 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:55.536126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f408c-fc0a-401a-81fe-5dd6baea3e82 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:55.536590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f408c-fc0a-401a-81fe-5dd6baea3e82 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:55.538263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 53/100] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:00:55.546882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9008a7-d035-4163-8e7d-045ca7377b32 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:55.549253 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf08b-b45d-4ec8-9160-30c5d3916337 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:55.549762 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf08b-b45d-4ec8-9160-30c5d3916337 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:55.550262 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf08b-b45d-4ec8-9160-30c5d3916337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:55.586772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-118bf08b-b45d-4ec8-9160-30c5d3916337 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:55.589729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-118bf08b-b45d-4ec8-9160-30c5d3916337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:55.631211 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf82dc-9501-448a-96a3-29f107bc2128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:55.635786 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf82dc-9501-448a-96a3-29f107bc2128 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:55.636354 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf82dc-9501-448a-96a3-29f107bc2128 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:55.636819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf82dc-9501-448a-96a3-29f107bc2128 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:55.659167 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf82dc-9501-448a-96a3-29f107bc2128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d4cf82dc-9501-448a-96a3-29f107bc2128 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:55.664346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:55.665168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:55.665890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 48/101] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:55.679131 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf82dc-9501-448a-96a3-29f107bc2128 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:55.698139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 54/102] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:00:55.735622 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf08b-b45d-4ec8-9160-30c5d3916337 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:55.879569 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:55.890480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:55.929931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf82dc-9501-448a-96a3-29f107bc2128 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:55.966185 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf08b-b45d-4ec8-9160-30c5d3916337 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.003983 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655e582-f2fd-441d-91c2-65438eeffce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.004721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655e582-f2fd-441d-91c2-65438eeffce9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.005008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655e582-f2fd-441d-91c2-65438eeffce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.005288 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655e582-f2fd-441d-91c2-65438eeffce9 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.006036 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655e582-f2fd-441d-91c2-65438eeffce9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:56.006490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655e582-f2fd-441d-91c2-65438eeffce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:56.007920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655e582-f2fd-441d-91c2-65438eeffce9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:56.027318 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 49/103] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:00:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:00:56.030738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ead987-b276-47ff-83ac-0d77abda0859 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.031035 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ead987-b276-47ff-83ac-0d77abda0859 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.031324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ead987-b276-47ff-83ac-0d77abda0859 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.032925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 50/104] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:56.035015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ead987-b276-47ff-83ac-0d77abda0859 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.053880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc62d3-47b2-4cfd-9d26-ad461ed00b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.054607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc62d3-47b2-4cfd-9d26-ad461ed00b0e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.054894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc62d3-47b2-4cfd-9d26-ad461ed00b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.055168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc62d3-47b2-4cfd-9d26-ad461ed00b0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.077411 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc62d3-47b2-4cfd-9d26-ad461ed00b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b1bc62d3-47b2-4cfd-9d26-ad461ed00b0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:56.082553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:56.083350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:00:56.084065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 55/105] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:00:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:00:56.095077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0772d65-6d49-4625-90cc-ef30aeb4f06a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.095388 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0772d65-6d49-4625-90cc-ef30aeb4f06a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.095638 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0772d65-6d49-4625-90cc-ef30aeb4f06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.124356 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b0772d65-6d49-4625-90cc-ef30aeb4f06a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:56.125716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b0772d65-6d49-4625-90cc-ef30aeb4f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:56.210918 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ead987-b276-47ff-83ac-0d77abda0859 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.220618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 56/106] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:56.221469 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0772d65-6d49-4625-90cc-ef30aeb4f06a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.271164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce71d6-3698-42d2-a6ee-0e7aee117c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.272309 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce71d6-3698-42d2-a6ee-0e7aee117c26 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.272716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce71d6-3698-42d2-a6ee-0e7aee117c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.272996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce71d6-3698-42d2-a6ee-0e7aee117c26 None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.275433 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce71d6-3698-42d2-a6ee-0e7aee117c26 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:56.277680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce71d6-3698-42d2-a6ee-0e7aee117c26 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:56.281982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 51/107] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:56 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:56.283791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3cb30-a1dc-4e23-8760-40898aa695b3 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.284129 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3cb30-a1dc-4e23-8760-40898aa695b3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.284414 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3cb30-a1dc-4e23-8760-40898aa695b3 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.287528 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 52/108] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:56.288757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3cb30-a1dc-4e23-8760-40898aa695b3 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.311070 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:56.312277 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:56.373655 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0772d65-6d49-4625-90cc-ef30aeb4f06a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.416910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.417681 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.418029 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.418405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.419286 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:56.419432 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3cb30-a1dc-4e23-8760-40898aa695b3 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.419740 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:56.421498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:56.433182 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 57/109] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:00:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:56.435021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.472359 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f7628-5f05-49ec-a30b-27c1edb452bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.473384 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f7628-5f05-49ec-a30b-27c1edb452bc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.473835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f7628-5f05-49ec-a30b-27c1edb452bc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.474316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f7628-5f05-49ec-a30b-27c1edb452bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.485751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f7628-5f05-49ec-a30b-27c1edb452bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-462f7628-5f05-49ec-a30b-27c1edb452bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:56.491283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:56.492101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:00:56.492823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:56.495302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 53/110] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:00:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:56.496666 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bf323-ca47-40bc-a2ce-7aa9e3b53540 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.497099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bf323-ca47-40bc-a2ce-7aa9e3b53540 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.497519 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bf323-ca47-40bc-a2ce-7aa9e3b53540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.517444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-286bf323-ca47-40bc-a2ce-7aa9e3b53540 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:56.518991 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-286bf323-ca47-40bc-a2ce-7aa9e3b53540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:56.565142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b8e99-bf2d-4f00-ad26-4c38a4568624 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.584765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:56.598928 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 54/111] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:56.598928 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bf323-ca47-40bc-a2ce-7aa9e3b53540 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.598928 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:56.615965 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc3046-36c3-46fd-a808-4eec7b1ce9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.616640 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc3046-36c3-46fd-a808-4eec7b1ce9fc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.616924 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc3046-36c3-46fd-a808-4eec7b1ce9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.617800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc3046-36c3-46fd-a808-4eec7b1ce9fc None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.619091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc3046-36c3-46fd-a808-4eec7b1ce9fc None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:56.620602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc3046-36c3-46fd-a808-4eec7b1ce9fc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:56.622491 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 58/112] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:56 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:56.623116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc3046-36c3-46fd-a808-4eec7b1ce9fc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.700259 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bf323-ca47-40bc-a2ce-7aa9e3b53540 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.738645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac7296-cd66-4cea-8c18-4c7aa49e5983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.739323 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7296-cd66-4cea-8c18-4c7aa49e5983 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.739609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7296-cd66-4cea-8c18-4c7aa49e5983 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.739890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7296-cd66-4cea-8c18-4c7aa49e5983 None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac7296-cd66-4cea-8c18-4c7aa49e5983 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:56.742621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac7296-cd66-4cea-8c18-4c7aa49e5983 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:56.744262 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 55/113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:56 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:56.744714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc3046-36c3-46fd-a808-4eec7b1ce9fc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.745494 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a345e-4634-49c0-aeb1-c0f4df57dc4b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.745790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a345e-4634-49c0-aeb1-c0f4df57dc4b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.746137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a345e-4634-49c0-aeb1-c0f4df57dc4b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.750168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 56/114] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:56.750168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a345e-4634-49c0-aeb1-c0f4df57dc4b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.789376 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.790012 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.790353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.790662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.791865 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:56.792181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:56.793980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:56.803632 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 59/115] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:56.804389 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.828408 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:56.856128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:56.860164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a345e-4634-49c0-aeb1-c0f4df57dc4b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.895722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f375056-188a-449f-87c9-5bc54d2d92bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.896377 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375056-188a-449f-87c9-5bc54d2d92bb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.896661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375056-188a-449f-87c9-5bc54d2d92bb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.896933 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375056-188a-449f-87c9-5bc54d2d92bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.907695 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f260aa-265b-48cc-8063-e4aa63604140 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:56.907976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f375056-188a-449f-87c9-5bc54d2d92bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-5f375056-188a-449f-87c9-5bc54d2d92bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:56.912924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:56.913816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 57/116] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:56.919941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f375056-188a-449f-87c9-5bc54d2d92bb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:56.940156 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f0532-11a9-4668-8809-3523cd1917ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:56.940843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f0532-11a9-4668-8809-3523cd1917ff None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.941147 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f0532-11a9-4668-8809-3523cd1917ff None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.941402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f0532-11a9-4668-8809-3523cd1917ff None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/users/0ed09815b5214d75993e263bb3fd05c9/roles/06907320d4bc47b3ba3375df728a605a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.943383 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f0532-11a9-4668-8809-3523cd1917ff None admin] RBAC: Authorizing `identity:create_grant(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6, user_id=0ed09815b5214d75993e263bb3fd05c9, role_id=06907320d4bc47b3ba3375df728a605a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:56.944856 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f0532-11a9-4668-8809-3523cd1917ff None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:56.959209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 60/117] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:00:56 2022] PUT /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/users/0ed09815b5214d75993e263bb3fd05c9/roles/06907320d4bc47b3ba3375df728a605a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:00:56.978330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfa6cd-a6d5-477d-80f6-50ce6ab7cc7a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:56.978330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfa6cd-a6d5-477d-80f6-50ce6ab7cc7a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:56.978330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfa6cd-a6d5-477d-80f6-50ce6ab7cc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:56.980528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e1dfa6cd-a6d5-477d-80f6-50ce6ab7cc7a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:56.981923 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e1dfa6cd-a6d5-477d-80f6-50ce6ab7cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:57.032062 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f375056-188a-449f-87c9-5bc54d2d92bb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.066138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 61/118] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:57.066138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfa6cd-a6d5-477d-80f6-50ce6ab7cc7a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.081001 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:57.088314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e27071-bb9b-4c37-b8fc-55a7866b26a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.089027 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27071-bb9b-4c37-b8fc-55a7866b26a3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.089317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27071-bb9b-4c37-b8fc-55a7866b26a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.089594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27071-bb9b-4c37-b8fc-55a7866b26a3 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.090834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e27071-bb9b-4c37-b8fc-55a7866b26a3 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:57.092337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e27071-bb9b-4c37-b8fc-55a7866b26a3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:57.094141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 58/119] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:56 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:57.095424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096fc93-2806-4056-8a4f-1e43cbb8c152 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.095752 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096fc93-2806-4056-8a4f-1e43cbb8c152 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.096002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096fc93-2806-4056-8a4f-1e43cbb8c152 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.097381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 59/120] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:57.097973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096fc93-2806-4056-8a4f-1e43cbb8c152 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.120441 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:57.193358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfa6cd-a6d5-477d-80f6-50ce6ab7cc7a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.223211 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096fc93-2806-4056-8a4f-1e43cbb8c152 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.232667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfcbcf-3d4a-4ee2-839f-1fd5b6fed4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.233261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfcbcf-3d4a-4ee2-839f-1fd5b6fed4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.233543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfcbcf-3d4a-4ee2-839f-1fd5b6fed4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.233821 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfcbcf-3d4a-4ee2-839f-1fd5b6fed4d4 None admin] PATH_INFO: `/v3/users/acf187c6216740528137e4e8f0aad7b7` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.235079 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfcbcf-3d4a-4ee2-839f-1fd5b6fed4d4 None admin] RBAC: Authorizing `identity:get_user(user_id=acf187c6216740528137e4e8f0aad7b7)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:57.236551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfcbcf-3d4a-4ee2-839f-1fd5b6fed4d4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:57.238714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 62/121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:57 2022] GET /identity/v3/users/acf187c6216740528137e4e8f0aad7b7 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:57.242307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfcbcf-3d4a-4ee2-839f-1fd5b6fed4d4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.267478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.268125 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.268413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.268688 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.269424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:57.269735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:57.271204 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:57.284472 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 60/122] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:00:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:57.284472 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.345368 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:57.348445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfcbcf-3d4a-4ee2-839f-1fd5b6fed4d4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.356828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:57.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54e63a-2f25-400f-b5af-7ceaf724e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.386681 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54e63a-2f25-400f-b5af-7ceaf724e902 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.386805 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299eb82-9984-4b57-ab98-ae6c431b1458 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.386984 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54e63a-2f25-400f-b5af-7ceaf724e902 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.387231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54e63a-2f25-400f-b5af-7ceaf724e902 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.388620 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54e63a-2f25-400f-b5af-7ceaf724e902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:57.391074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54e63a-2f25-400f-b5af-7ceaf724e902 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:57.392903 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 63/123] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:00:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:57.393808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44a88c-1249-439c-b6ce-550425211c92 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.398152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44a88c-1249-439c-b6ce-550425211c92 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.398152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44a88c-1249-439c-b6ce-550425211c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.418273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9c44a88c-1249-439c-b6ce-550425211c92 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:57.419389 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9c44a88c-1249-439c-b6ce-550425211c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:57.421164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c29911-bf63-4e4e-af4f-ec34c1afa5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.421953 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c29911-bf63-4e4e-af4f-ec34c1afa5a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.422278 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c29911-bf63-4e4e-af4f-ec34c1afa5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.422552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c29911-bf63-4e4e-af4f-ec34c1afa5a8 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/users/0ed09815b5214d75993e263bb3fd05c9/roles/94aa0ce403304ce8a21ef20db556d534` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.424928 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c29911-bf63-4e4e-af4f-ec34c1afa5a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6, user_id=0ed09815b5214d75993e263bb3fd05c9, role_id=94aa0ce403304ce8a21ef20db556d534)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:57.427910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c29911-bf63-4e4e-af4f-ec34c1afa5a8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:57.450135 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 61/124] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:00:57 2022] PUT /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/users/0ed09815b5214d75993e263bb3fd05c9/roles/94aa0ce403304ce8a21ef20db556d534 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:00:57.450135 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c29911-bf63-4e4e-af4f-ec34c1afa5a8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.519467 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 64/125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:57.520633 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44a88c-1249-439c-b6ce-550425211c92 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.583377 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c29911-bf63-4e4e-af4f-ec34c1afa5a8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.617202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:57.633998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6adaaf-e592-4e7b-a2e0-6b9f564045f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.634949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6adaaf-e592-4e7b-a2e0-6b9f564045f9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.635143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6adaaf-e592-4e7b-a2e0-6b9f564045f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.635422 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6adaaf-e592-4e7b-a2e0-6b9f564045f9 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.636623 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6adaaf-e592-4e7b-a2e0-6b9f564045f9 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:57.638196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6adaaf-e592-4e7b-a2e0-6b9f564045f9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:57.639905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 62/126] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:57 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:57.640661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6adaaf-e592-4e7b-a2e0-6b9f564045f9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.663888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44a88c-1249-439c-b6ce-550425211c92 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.667262 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:57.717450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334bf27-be8e-4965-a52b-e55f0540f0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.718137 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334bf27-be8e-4965-a52b-e55f0540f0da None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.718427 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334bf27-be8e-4965-a52b-e55f0540f0da None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.718699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334bf27-be8e-4965-a52b-e55f0540f0da None admin] PATH_INFO: `/v3/users/81d01c6d0b764715b1c523925b46b296` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.738947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334bf27-be8e-4965-a52b-e55f0540f0da None admin] RBAC: Authorizing `identity:get_user(user_id=81d01c6d0b764715b1c523925b46b296)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:57.740537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334bf27-be8e-4965-a52b-e55f0540f0da None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:57.742493 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 65/127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:57 2022] GET /identity/v3/users/81d01c6d0b764715b1c523925b46b296 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:57.743093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334bf27-be8e-4965-a52b-e55f0540f0da None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.763399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6adaaf-e592-4e7b-a2e0-6b9f564045f9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.798994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b20051-0bed-48f9-9c65-aca0be330a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.799959 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20051-0bed-48f9-9c65-aca0be330a6c None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.800466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20051-0bed-48f9-9c65-aca0be330a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.800925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20051-0bed-48f9-9c65-aca0be330a6c None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:57.801878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b20051-0bed-48f9-9c65-aca0be330a6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:57.803694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b20051-0bed-48f9-9c65-aca0be330a6c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:57.836743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 63/128] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:00:57 2022] POST /identity/v3/projects => generated 302 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:00:57.837891 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b20051-0bed-48f9-9c65-aca0be330a6c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:57.890697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334bf27-be8e-4965-a52b-e55f0540f0da None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:57.930348 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:57.979826 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2df4a-70cb-4819-bed2-fac1daa4ceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:57.980541 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2df4a-70cb-4819-bed2-fac1daa4ceb6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:57.982344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2df4a-70cb-4819-bed2-fac1daa4ceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:57.982344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2df4a-70cb-4819-bed2-fac1daa4ceb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:58.003999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b20051-0bed-48f9-9c65-aca0be330a6c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:58.031062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2df4a-70cb-4819-bed2-fac1daa4ceb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bff2df4a-70cb-4819-bed2-fac1daa4ceb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:58.036765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:58.037601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:58.038359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:00:58.042224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 66/129] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:58.044893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2df4a-70cb-4819-bed2-fac1daa4ceb6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:58.067462 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f0b22-bb01-4fce-ade0-9661a7cb0f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:58.068083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f0b22-bb01-4fce-ade0-9661a7cb0f9b None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:58.068365 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f0b22-bb01-4fce-ade0-9661a7cb0f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:58.068639 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f0b22-bb01-4fce-ade0-9661a7cb0f9b None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/users/acf187c6216740528137e4e8f0aad7b7/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:58.070926 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f0b22-bb01-4fce-ade0-9661a7cb0f9b None admin] RBAC: Authorizing `identity:create_grant(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6, user_id=acf187c6216740528137e4e8f0aad7b7, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:58.072384 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f0b22-bb01-4fce-ade0-9661a7cb0f9b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:58.080199 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:58.100317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 64/130] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:00:57 2022] PUT /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/users/acf187c6216740528137e4e8f0aad7b7/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:00:58.101458 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f0b22-bb01-4fce-ade0-9661a7cb0f9b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:58.171089 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2df4a-70cb-4819-bed2-fac1daa4ceb6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:58.209267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fbfe3-dc9d-49b0-a3ef-6ca4a9cc150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:58.211267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fbfe3-dc9d-49b0-a3ef-6ca4a9cc150e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:58.211267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fbfe3-dc9d-49b0-a3ef-6ca4a9cc150e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:58.211267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fbfe3-dc9d-49b0-a3ef-6ca4a9cc150e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:58.285175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fbfe3-dc9d-49b0-a3ef-6ca4a9cc150e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:58.286955 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fbfe3-dc9d-49b0-a3ef-6ca4a9cc150e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:58.312122 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:58.396695 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 67/131] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:00:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:00:58.401554 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f0b22-bb01-4fce-ade0-9661a7cb0f9b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:58.402369 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fbfe3-dc9d-49b0-a3ef-6ca4a9cc150e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:58.403954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:58.493484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:58.494875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:58.495158 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:58.495429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:58.496274 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:58.496764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:58.498537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:58.516859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 65/132] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:00:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:58.522430 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:58.561915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fbfe3-dc9d-49b0-a3ef-6ca4a9cc150e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:58.612444 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:58.629386 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d5bf3-d084-4345-a522-18e7be0cfbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:58.630059 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5bf3-d084-4345-a522-18e7be0cfbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:58.630368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5bf3-d084-4345-a522-18e7be0cfbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:58.630640 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5bf3-d084-4345-a522-18e7be0cfbb3 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:58.632095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d5bf3-d084-4345-a522-18e7be0cfbb3 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:58.633559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d5bf3-d084-4345-a522-18e7be0cfbb3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:58.636421 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 68/133] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:58 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:00:58.643191 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d5bf3-d084-4345-a522-18e7be0cfbb3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:58.716274 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7ba6c-c631-408b-a6ed-c22a9613f519 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:58.776264 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f49ba-fa6c-48f8-b24a-285b0548b081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:58.777408 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f49ba-fa6c-48f8-b24a-285b0548b081 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:58.777712 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f49ba-fa6c-48f8-b24a-285b0548b081 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:58.778015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f49ba-fa6c-48f8-b24a-285b0548b081 None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:58.780199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f49ba-fa6c-48f8-b24a-285b0548b081 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:58.783191 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f49ba-fa6c-48f8-b24a-285b0548b081 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:58.787022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 66/134] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:00:58 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:00:58.792328 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f49ba-fa6c-48f8-b24a-285b0548b081 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:58.871522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:58.880685 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d5bf3-d084-4345-a522-18e7be0cfbb3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:58.969741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:58.970331 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:58.970615 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:58.970891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:58.971708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] RBAC: Adding query filter params (role.id=becb35407660454b9982c279fd4ee506, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6, user.id=81d01c6d0b764715b1c523925b46b296) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:00:58.972017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:58.973600 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:58.992168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 69/135] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:00:58 2022] GET /identity/v3/role_assignments?user.id=81d01c6d0b764715b1c523925b46b296&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=becb35407660454b9982c279fd4ee506 => generated 259 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:59.042568 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f49ba-fa6c-48f8-b24a-285b0548b081 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:59.100743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb8867-ad4a-4f18-b698-37d8e0f866b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:59.101828 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb8867-ad4a-4f18-b698-37d8e0f866b0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:59.102381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb8867-ad4a-4f18-b698-37d8e0f866b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:59.102876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb8867-ad4a-4f18-b698-37d8e0f866b0 None admin] PATH_INFO: `/v3/projects/865a7ea820ca4699a96b9cc1a0e8879f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:59.104524 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb8867-ad4a-4f18-b698-37d8e0f866b0 None admin] RBAC: Authorizing `identity:get_project(project_id=865a7ea820ca4699a96b9cc1a0e8879f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:59.106325 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb8867-ad4a-4f18-b698-37d8e0f866b0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:59.108382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 67/136] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:00:58 2022] GET /identity/v3/projects/865a7ea820ca4699a96b9cc1a0e8879f => generated 312 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:00:59.114643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:59.212435 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:59.269673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3d027-b7f9-4699-a001-e4e31f299e35 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:59.313904 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33075a98-5e0d-4db5-98b2-4863031132ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:59.314633 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33075a98-5e0d-4db5-98b2-4863031132ec None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:59.314914 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33075a98-5e0d-4db5-98b2-4863031132ec None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:59.315185 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33075a98-5e0d-4db5-98b2-4863031132ec None admin] PATH_INFO: `/v3/projects/865a7ea820ca4699a96b9cc1a0e8879f/users/0ed09815b5214d75993e263bb3fd05c9/roles/06907320d4bc47b3ba3375df728a605a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:59.317250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33075a98-5e0d-4db5-98b2-4863031132ec None admin] RBAC: Authorizing `identity:create_grant(project_id=865a7ea820ca4699a96b9cc1a0e8879f, user_id=0ed09815b5214d75993e263bb3fd05c9, role_id=06907320d4bc47b3ba3375df728a605a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:00:59.318748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33075a98-5e0d-4db5-98b2-4863031132ec None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:00:59.340902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 70/137] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:00:59 2022] PUT /identity/v3/projects/865a7ea820ca4699a96b9cc1a0e8879f/users/0ed09815b5214d75993e263bb3fd05c9/roles/06907320d4bc47b3ba3375df728a605a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:00:59.581973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f61a5-b24a-4e28-9338-aff6f7f18325 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:59.583074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f61a5-b24a-4e28-9338-aff6f7f18325 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:59.583573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f61a5-b24a-4e28-9338-aff6f7f18325 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:59.585653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 68/138] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:00:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:00:59.591018 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138067-046e-4233-8175-c6e8e0c5a14f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:59.591488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138067-046e-4233-8175-c6e8e0c5a14f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:59.591558 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138067-046e-4233-8175-c6e8e0c5a14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:59.618618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-98138067-046e-4233-8175-c6e8e0c5a14f None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:00:59.619839 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-98138067-046e-4233-8175-c6e8e0c5a14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:00:59.716338 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 71/139] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:00:59.754727 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f61a5-b24a-4e28-9338-aff6f7f18325 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:00:59.887845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:00:59.892988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f61a5-b24a-4e28-9338-aff6f7f18325 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:00:59.934693 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bccde1-51d3-4d76-9887-348013c78a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:00:59.935364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bccde1-51d3-4d76-9887-348013c78a5f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:00:59.935661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bccde1-51d3-4d76-9887-348013c78a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:00:59.935943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bccde1-51d3-4d76-9887-348013c78a5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:00:59.947290 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bccde1-51d3-4d76-9887-348013c78a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-05bccde1-51d3-4d76-9887-348013c78a5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:00:59.953469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:00:59.954382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:00:59.955143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 69/140] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:00:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:00:59.961077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98138067-046e-4233-8175-c6e8e0c5a14f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:00.033044 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:00.072504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f91b39-59ac-4953-b087-d3c62b26f17b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.072965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f91b39-59ac-4953-b087-d3c62b26f17b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.073345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f91b39-59ac-4953-b087-d3c62b26f17b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:00.075480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 70/141] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:00.080971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bf0a0-5987-41ec-8574-eeacd894d9b3 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.081352 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bf0a0-5987-41ec-8574-eeacd894d9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.081733 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bf0a0-5987-41ec-8574-eeacd894d9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:00.097578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98138067-046e-4233-8175-c6e8e0c5a14f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:00.123220 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0c7bf0a0-5987-41ec-8574-eeacd894d9b3 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:00.124609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0c7bf0a0-5987-41ec-8574-eeacd894d9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:00.152854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:00.153944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.154399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.154806 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:00.156315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:00.156947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:00.158665 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:00.170689 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 72/142] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:00:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:00.179977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:00.296844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 71/143] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:00.313229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc87f8-b0a0-49b2-ab4a-0548f43a0079 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:00.321408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:00.331781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bf0a0-5987-41ec-8574-eeacd894d9b3 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:00.369428 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfcb4f-6ef0-4ecd-b105-25145b666bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:00.370520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfcb4f-6ef0-4ecd-b105-25145b666bcc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.370937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfcb4f-6ef0-4ecd-b105-25145b666bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.371590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfcb4f-6ef0-4ecd-b105-25145b666bcc None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:00.373293 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfcb4f-6ef0-4ecd-b105-25145b666bcc None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:00.374906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfcb4f-6ef0-4ecd-b105-25145b666bcc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:00.383169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 73/144] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:00 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:00.388854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfcb4f-6ef0-4ecd-b105-25145b666bcc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:00.559864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfcb4f-6ef0-4ecd-b105-25145b666bcc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:00.561212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:00.585933 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bf0a0-5987-41ec-8574-eeacd894d9b3 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:00.615605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde723c8-afd3-4ffa-bd08-42e6426c029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:00.616887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde723c8-afd3-4ffa-bd08-42e6426c029b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.617836 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde723c8-afd3-4ffa-bd08-42e6426c029b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.618360 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde723c8-afd3-4ffa-bd08-42e6426c029b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:00.666645 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:00.674823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923e229-7070-4039-910e-9f191c58f682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:00.675584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923e229-7070-4039-910e-9f191c58f682 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.675875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923e229-7070-4039-910e-9f191c58f682 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.676214 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923e229-7070-4039-910e-9f191c58f682 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:00.697601 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923e229-7070-4039-910e-9f191c58f682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-c923e229-7070-4039-910e-9f191c58f682 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:00.703478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:00.704280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:00.705009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:00.709686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde723c8-afd3-4ffa-bd08-42e6426c029b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:00.711507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde723c8-afd3-4ffa-bd08-42e6426c029b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:00.714356 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 72/145] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:00.721373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923e229-7070-4039-910e-9f191c58f682 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:00.860057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923e229-7070-4039-910e-9f191c58f682 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:00.898868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:00.899503 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.899782 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.900052 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:00.900779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:00.901082 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:00.902637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:00.911765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 73/146] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:00.918754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:00.937337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:00.964898 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 74/147] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:00.966239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905cf9-6018-435e-9a26-87af1469b4f9 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.966528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905cf9-6018-435e-9a26-87af1469b4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.966805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905cf9-6018-435e-9a26-87af1469b4f9 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:00.968048 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 75/148] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:00.975466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea5b21-ca27-486f-b259-e9af1d160a6b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:00.975948 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea5b21-ca27-486f-b259-e9af1d160a6b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:00.976386 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea5b21-ca27-486f-b259-e9af1d160a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.027149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-80ea5b21-ca27-486f-b259-e9af1d160a6b None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:01.028571 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-80ea5b21-ca27-486f-b259-e9af1d160a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:01.030259 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514460ee-4ba0-408f-9656-f4b4331044c0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:01.065883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00352ce-1115-408f-af9b-71fa4af03b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:01.066991 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00352ce-1115-408f-af9b-71fa4af03b7a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:01.067409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00352ce-1115-408f-af9b-71fa4af03b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:01.067816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00352ce-1115-408f-af9b-71fa4af03b7a None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.075237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00352ce-1115-408f-af9b-71fa4af03b7a None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:01.077933 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00352ce-1115-408f-af9b-71fa4af03b7a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:01.079929 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 74/149] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:00 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:01.082981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00352ce-1115-408f-af9b-71fa4af03b7a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:01.167965 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 76/150] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:01.168959 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea5b21-ca27-486f-b259-e9af1d160a6b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:01.177467 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:01.190839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:01.212723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00352ce-1115-408f-af9b-71fa4af03b7a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:01.248278 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ae652-0b62-4441-95fb-78be65db68bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:01.249331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae652-0b62-4441-95fb-78be65db68bf None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:01.249787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae652-0b62-4441-95fb-78be65db68bf None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:01.250251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae652-0b62-4441-95fb-78be65db68bf None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.251735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ae652-0b62-4441-95fb-78be65db68bf None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:01.253349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ae652-0b62-4441-95fb-78be65db68bf None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:01.255604 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 75/151] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:01 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:01.256722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ae652-0b62-4441-95fb-78be65db68bf None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:01.363999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea5b21-ca27-486f-b259-e9af1d160a6b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:01.378821 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ae652-0b62-4441-95fb-78be65db68bf None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:01.414337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718e587-ff60-41ae-9169-528e76ef9d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:01.415500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e587-ff60-41ae-9169-528e76ef9d7e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:01.415965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e587-ff60-41ae-9169-528e76ef9d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:01.416478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e587-ff60-41ae-9169-528e76ef9d7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.417868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718e587-ff60-41ae-9169-528e76ef9d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:01.419593 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718e587-ff60-41ae-9169-528e76ef9d7e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:01.421864 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 76/152] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:01.423057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718e587-ff60-41ae-9169-528e76ef9d7e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:01.442524 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:01.463404 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb3141-d121-47a7-af92-5f00bd86282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:01.464061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3141-d121-47a7-af92-5f00bd86282c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:01.464340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3141-d121-47a7-af92-5f00bd86282c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:01.464620 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3141-d121-47a7-af92-5f00bd86282c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.531882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718e587-ff60-41ae-9169-528e76ef9d7e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:01.570677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:01.571495 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:01.571871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:01.572250 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.573374 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] RBAC: Adding query filter params (role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6, user.id=0ed09815b5214d75993e263bb3fd05c9) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:01.573788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:01.575782 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:01.583086 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 77/153] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:01 2022] GET /identity/v3/role_assignments?user.id=0ed09815b5214d75993e263bb3fd05c9&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=06907320d4bc47b3ba3375df728a605a => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:01.584005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:01.655800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb3141-d121-47a7-af92-5f00bd86282c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:01.657335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb3141-d121-47a7-af92-5f00bd86282c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:01.660616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:01.690646 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:01.703911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4943004-f3ac-4a44-b285-6c5e4971efe9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:01.744434 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b707f61-52dd-4ef0-87a6-e01843fd720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:01.745277 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b707f61-52dd-4ef0-87a6-e01843fd720d None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:01.745653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b707f61-52dd-4ef0-87a6-e01843fd720d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:01.746021 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b707f61-52dd-4ef0-87a6-e01843fd720d None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.746954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b707f61-52dd-4ef0-87a6-e01843fd720d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:01.748605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b707f61-52dd-4ef0-87a6-e01843fd720d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:01.784753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 78/154] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:01 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:01.786248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce709-2f94-46aa-b8a6-f9499d3d7409 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:01.786535 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce709-2f94-46aa-b8a6-f9499d3d7409 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:01.786806 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce709-2f94-46aa-b8a6-f9499d3d7409 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.788195 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 79/155] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:01.794876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e12493-a333-4997-95e5-d7bde76e3152 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:01.795186 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e12493-a333-4997-95e5-d7bde76e3152 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:01.795413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e12493-a333-4997-95e5-d7bde76e3152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:01.835611 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f3e12493-a333-4997-95e5-d7bde76e3152 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:01.837306 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f3e12493-a333-4997-95e5-d7bde76e3152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:01.857372 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 77/156] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:01.864611 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb3141-d121-47a7-af92-5f00bd86282c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.008222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 80/157] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:02.038342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb3141-d121-47a7-af92-5f00bd86282c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:02.044956 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:02.064075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e12493-a333-4997-95e5-d7bde76e3152 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.103259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552c475-a902-4335-8ed5-038516f87366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:02.103556 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c475-a902-4335-8ed5-038516f87366 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.103697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c475-a902-4335-8ed5-038516f87366 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.104372 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c475-a902-4335-8ed5-038516f87366 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.105772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552c475-a902-4335-8ed5-038516f87366 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:02.108431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552c475-a902-4335-8ed5-038516f87366 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:02.111220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 78/158] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:01 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:02.116810 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552c475-a902-4335-8ed5-038516f87366 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.158795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:02.206489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e12493-a333-4997-95e5-d7bde76e3152 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:02.243967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a26f19-6cc1-4ff9-88d3-567900acf502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:02.245101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a26f19-6cc1-4ff9-88d3-567900acf502 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.245726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a26f19-6cc1-4ff9-88d3-567900acf502 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.245924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a26f19-6cc1-4ff9-88d3-567900acf502 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.248838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552c475-a902-4335-8ed5-038516f87366 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:02.261792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a26f19-6cc1-4ff9-88d3-567900acf502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-57a26f19-6cc1-4ff9-88d3-567900acf502 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:02.266876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:02.268063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 81/159] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:02.272213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf08d4-64ee-4066-84f6-2713826c3339 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.273842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf08d4-64ee-4066-84f6-2713826c3339 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.273842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf08d4-64ee-4066-84f6-2713826c3339 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.273842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 82/160] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:02.276924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13143-e1fa-4014-bd73-51f45472c42e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.277204 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13143-e1fa-4014-bd73-51f45472c42e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.277470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13143-e1fa-4014-bd73-51f45472c42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.297218 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e8c13143-e1fa-4014-bd73-51f45472c42e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:02.298473 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e8c13143-e1fa-4014-bd73-51f45472c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:02.311515 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:02.312621 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.313131 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.313669 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.314855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] RBAC: Adding query filter params (role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6, user.id=0ed09815b5214d75993e263bb3fd05c9) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:02.315407 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:02.318049 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:02.331414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 79/161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:02 2022] GET /identity/v3/role_assignments?user.id=0ed09815b5214d75993e263bb3fd05c9&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:02.332353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.395070 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 83/162] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:02.397651 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e753c-ceef-4f95-bd51-1718139f732a None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.398186 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e753c-ceef-4f95-bd51-1718139f732a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.398601 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e753c-ceef-4f95-bd51-1718139f732a None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.401453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 84/163] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:02.407083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a26a-226c-43b6-90e0-17a4c2c64d6f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.407589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a26a-226c-43b6-90e0-17a4c2c64d6f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.408038 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a26a-226c-43b6-90e0-17a4c2c64d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.414917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:02.436264 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a8e2a26a-226c-43b6-90e0-17a4c2c64d6f None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:02.438180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a8e2a26a-226c-43b6-90e0-17a4c2c64d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:02.508487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba48b09-eb10-4075-93a5-78e38902b0ea None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:02.524584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 85/164] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:02.525005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:02.526734 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2a26a-226c-43b6-90e0-17a4c2c64d6f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.547537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:02.548243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.548528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.548799 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.549594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:02.549935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:02.551449 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:02.565366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 80/165] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:02.565366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.679130 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2a26a-226c-43b6-90e0-17a4c2c64d6f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:02.705482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:02.715129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5347e7a-c8b9-4c53-95d5-8d4bfb0b2066 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:02.727437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c511b8f-8cf2-44c0-87f0-0d90bd14c87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:02.728107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c511b8f-8cf2-44c0-87f0-0d90bd14c87d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.728385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c511b8f-8cf2-44c0-87f0-0d90bd14c87d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.728653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c511b8f-8cf2-44c0-87f0-0d90bd14c87d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.744671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c511b8f-8cf2-44c0-87f0-0d90bd14c87d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3c511b8f-8cf2-44c0-87f0-0d90bd14c87d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:02.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:02.750679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:02.751441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 86/166] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:02.752315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c511b8f-8cf2-44c0-87f0-0d90bd14c87d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.762964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97a5a1-f406-41a1-a98d-98ac417d0eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:02.763631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97a5a1-f406-41a1-a98d-98ac417d0eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.763905 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97a5a1-f406-41a1-a98d-98ac417d0eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.764174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97a5a1-f406-41a1-a98d-98ac417d0eb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.778512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97a5a1-f406-41a1-a98d-98ac417d0eb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6e97a5a1-f406-41a1-a98d-98ac417d0eb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:02.784101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:02.785111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:02.786149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 81/167] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:02.786768 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97a5a1-f406-41a1-a98d-98ac417d0eb0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.820575 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:02.902582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c511b8f-8cf2-44c0-87f0-0d90bd14c87d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:02.944076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97a5a1-f406-41a1-a98d-98ac417d0eb0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:02.945248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68752022-5e9f-46c4-9d5f-04a89db50351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:02.945931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68752022-5e9f-46c4-9d5f-04a89db50351 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.946254 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68752022-5e9f-46c4-9d5f-04a89db50351 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.946532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68752022-5e9f-46c4-9d5f-04a89db50351 None admin] PATH_INFO: `/v3/users/acf187c6216740528137e4e8f0aad7b7` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.947800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68752022-5e9f-46c4-9d5f-04a89db50351 None admin] RBAC: Authorizing `identity:get_user(user_id=acf187c6216740528137e4e8f0aad7b7)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:02.949306 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68752022-5e9f-46c4-9d5f-04a89db50351 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:02.951439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 87/168] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:02 2022] GET /identity/v3/users/acf187c6216740528137e4e8f0aad7b7 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:02.956974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68752022-5e9f-46c4-9d5f-04a89db50351 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:02.989781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:02.990515 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:02.990794 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:02.991067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:02.991810 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:02.992117 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:02.993538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:03.019710 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 82/169] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:03.022646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:03.057053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:03.102463 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68752022-5e9f-46c4-9d5f-04a89db50351 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:03.122150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:03.140590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:03.140590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:03.140590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:03.140590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:03.140990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:03.140990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:03.141306 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:03.151850 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 88/170] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:03.152694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:03.241230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf7b0a-2d48-474c-8b78-9ca1ae38da49 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:03.276731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aedcf-40c1-4140-9234-f60c243bbfd9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:03.311163 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc1079-093f-4594-a7ef-0d3903656b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:03.311720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc1079-093f-4594-a7ef-0d3903656b32 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:03.312002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc1079-093f-4594-a7ef-0d3903656b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:03.312277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc1079-093f-4594-a7ef-0d3903656b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:03.344348 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f514582-5d32-4090-a0be-00f2c17f73dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:03.345012 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f514582-5d32-4090-a0be-00f2c17f73dd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:03.345291 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f514582-5d32-4090-a0be-00f2c17f73dd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:03.345573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f514582-5d32-4090-a0be-00f2c17f73dd None admin] PATH_INFO: `/v3/users/81d01c6d0b764715b1c523925b46b296` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:03.346986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f514582-5d32-4090-a0be-00f2c17f73dd None admin] RBAC: Authorizing `identity:get_user(user_id=81d01c6d0b764715b1c523925b46b296)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:03.348519 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f514582-5d32-4090-a0be-00f2c17f73dd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:03.351254 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 89/171] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:03 2022] GET /identity/v3/users/81d01c6d0b764715b1c523925b46b296 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:03.356512 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f514582-5d32-4090-a0be-00f2c17f73dd None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:03.438766 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:03.457291 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc1079-093f-4594-a7ef-0d3903656b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:03.458897 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc1079-093f-4594-a7ef-0d3903656b32 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:03.500400 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:03.533774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f514582-5d32-4090-a0be-00f2c17f73dd None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:03.606792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6c6f6-596a-4ca2-b77c-2ae79abe0eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:03.607390 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c6f6-596a-4ca2-b77c-2ae79abe0eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:03.607672 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c6f6-596a-4ca2-b77c-2ae79abe0eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:03.607949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c6f6-596a-4ca2-b77c-2ae79abe0eb3 None admin] PATH_INFO: `/v3/users/0ed09815b5214d75993e263bb3fd05c9` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:03.609155 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6c6f6-596a-4ca2-b77c-2ae79abe0eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed09815b5214d75993e263bb3fd05c9)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:03.610645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6c6f6-596a-4ca2-b77c-2ae79abe0eb3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:03.619316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 90/172] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:03 2022] GET /identity/v3/users/0ed09815b5214d75993e263bb3fd05c9 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:03.620293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6c6f6-596a-4ca2-b77c-2ae79abe0eb3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:03.627410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 83/173] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:03.629294 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc1079-093f-4594-a7ef-0d3903656b32 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:03.742302 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc1079-093f-4594-a7ef-0d3903656b32 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:03.764546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:03.781501 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf698a-f66b-46ba-8333-2a74a0d39e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:03.782165 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf698a-f66b-46ba-8333-2a74a0d39e3f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:03.782452 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf698a-f66b-46ba-8333-2a74a0d39e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:03.785129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf698a-f66b-46ba-8333-2a74a0d39e3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:03.880514 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf698a-f66b-46ba-8333-2a74a0d39e3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:03.882287 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf698a-f66b-46ba-8333-2a74a0d39e3f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:03.892202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6c6f6-596a-4ca2-b77c-2ae79abe0eb3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:03.909816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:03.934579 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df4f95-dc53-4f49-b529-c4c024d434aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:03.935634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df4f95-dc53-4f49-b529-c4c024d434aa None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:03.936106 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df4f95-dc53-4f49-b529-c4c024d434aa None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:03.936565 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df4f95-dc53-4f49-b529-c4c024d434aa None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:03.937935 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df4f95-dc53-4f49-b529-c4c024d434aa None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:03.939592 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df4f95-dc53-4f49-b529-c4c024d434aa None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:03.941496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 91/174] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:03 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:03.942559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df4f95-dc53-4f49-b529-c4c024d434aa None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:04.051775 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df4f95-dc53-4f49-b529-c4c024d434aa None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:04.087390 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129563d-1b65-466d-a745-9b86a2027570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:04.088402 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129563d-1b65-466d-a745-9b86a2027570 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:04.088886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129563d-1b65-466d-a745-9b86a2027570 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:04.089342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129563d-1b65-466d-a745-9b86a2027570 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:04.090706 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129563d-1b65-466d-a745-9b86a2027570 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:04.092342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129563d-1b65-466d-a745-9b86a2027570 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:04.094576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 92/175] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:03 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:04.095609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129563d-1b65-466d-a745-9b86a2027570 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:04.122782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 84/176] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:04.127390 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf698a-f66b-46ba-8333-2a74a0d39e3f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:04.147792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:04.174487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:04.205543 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129563d-1b65-466d-a745-9b86a2027570 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:04.241339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8656b-494f-4c03-8935-fd2c77de9984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:04.242447 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8656b-494f-4c03-8935-fd2c77de9984 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:04.242916 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8656b-494f-4c03-8935-fd2c77de9984 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:04.243418 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8656b-494f-4c03-8935-fd2c77de9984 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/users/81d01c6d0b764715b1c523925b46b296/roles/becb35407660454b9982c279fd4ee506` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:04.245610 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8656b-494f-4c03-8935-fd2c77de9984 None admin] RBAC: Authorizing `identity:create_grant(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6, user_id=81d01c6d0b764715b1c523925b46b296, role_id=becb35407660454b9982c279fd4ee506)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:04.247320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8656b-494f-4c03-8935-fd2c77de9984 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:04.267427 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 93/177] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:04 2022] PUT /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/users/81d01c6d0b764715b1c523925b46b296/roles/becb35407660454b9982c279fd4ee506 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:04.270342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8656b-494f-4c03-8935-fd2c77de9984 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:04.312373 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf698a-f66b-46ba-8333-2a74a0d39e3f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:04.401479 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8656b-494f-4c03-8935-fd2c77de9984 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:04.407003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:04.407671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:04.407949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:04.408214 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:04.409042 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6, user.id=acf187c6216740528137e4e8f0aad7b7) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:04.409349 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:04.410976 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:04.414840 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:04.426380 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 85/178] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:04 2022] GET /identity/v3/role_assignments?user.id=acf187c6216740528137e4e8f0aad7b7&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=65cd37a73e4f498482fefabb8374006c => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:04.447496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81d401-2137-461f-b9f2-c7e1f1194933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:04.448557 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81d401-2137-461f-b9f2-c7e1f1194933 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:04.449028 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81d401-2137-461f-b9f2-c7e1f1194933 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:04.449493 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81d401-2137-461f-b9f2-c7e1f1194933 None admin] PATH_INFO: `/v3/projects/865a7ea820ca4699a96b9cc1a0e8879f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:04.451477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81d401-2137-461f-b9f2-c7e1f1194933 None admin] RBAC: Authorizing `identity:get_project(project_id=865a7ea820ca4699a96b9cc1a0e8879f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:04.453172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81d401-2137-461f-b9f2-c7e1f1194933 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:04.455223 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 94/179] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:04 2022] GET /identity/v3/projects/865a7ea820ca4699a96b9cc1a0e8879f => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:04.461395 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:04.604467 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:04.621321 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a5df-f286-4196-a0d4-1cfeb76ce277 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:04.661825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330331a-c530-4eeb-a536-ab572105216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:04.662440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330331a-c530-4eeb-a536-ab572105216c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:04.662717 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330331a-c530-4eeb-a536-ab572105216c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:04.662987 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330331a-c530-4eeb-a536-ab572105216c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:04.663785 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330331a-c530-4eeb-a536-ab572105216c None admin] RBAC: Adding query filter params (role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=865a7ea820ca4699a96b9cc1a0e8879f, user.id=0ed09815b5214d75993e263bb3fd05c9) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:04.664091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330331a-c530-4eeb-a536-ab572105216c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:04.665538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330331a-c530-4eeb-a536-ab572105216c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:04.672468 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 86/180] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:04 2022] GET /identity/v3/role_assignments?user.id=0ed09815b5214d75993e263bb3fd05c9&scope.project.id=865a7ea820ca4699a96b9cc1a0e8879f&role.id=06907320d4bc47b3ba3375df728a605a => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:04.930187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e301-bdb3-4297-a5cd-149d64493922 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:04.930672 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e301-bdb3-4297-a5cd-149d64493922 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:04.930672 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e301-bdb3-4297-a5cd-149d64493922 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:04.932193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 95/181] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:04.939051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f3cd7-29a1-4b22-8400-87f67675c1ce None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:04.939575 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f3cd7-29a1-4b22-8400-87f67675c1ce None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:04.939575 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f3cd7-29a1-4b22-8400-87f67675c1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:04.966865 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-615f3cd7-29a1-4b22-8400-87f67675c1ce None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:04.968159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-615f3cd7-29a1-4b22-8400-87f67675c1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:05.013753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d63ef-6f86-4ffb-a933-4fa8f28a74b3 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.014528 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d63ef-6f86-4ffb-a933-4fa8f28a74b3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.014998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d63ef-6f86-4ffb-a933-4fa8f28a74b3 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.017036 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 96/182] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:05.023181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1966b69-d174-4d14-9f5d-205a7cf3c3dc None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.023722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1966b69-d174-4d14-9f5d-205a7cf3c3dc None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.024191 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1966b69-d174-4d14-9f5d-205a7cf3c3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.048425 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b1966b69-d174-4d14-9f5d-205a7cf3c3dc None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:05.051020 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b1966b69-d174-4d14-9f5d-205a7cf3c3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:05.064265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 87/183] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:05.064265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4f407-ca43-4360-8332-3109823b0cfb None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.064265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4f407-ca43-4360-8332-3109823b0cfb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.064265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4f407-ca43-4360-8332-3109823b0cfb None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.064265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 88/184] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:05.070608 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e6db7-a30a-46c2-a7e3-9f2645096abe None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.070889 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e6db7-a30a-46c2-a7e3-9f2645096abe None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.071324 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e6db7-a30a-46c2-a7e3-9f2645096abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.094279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c10e6db7-a30a-46c2-a7e3-9f2645096abe None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:05.095743 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c10e6db7-a30a-46c2-a7e3-9f2645096abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:05.152574 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 97/185] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:05.152574 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1966b69-d174-4d14-9f5d-205a7cf3c3dc None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.184351 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 89/186] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:05.185151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e6db7-a30a-46c2-a7e3-9f2645096abe None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.196960 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:05.274718 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1966b69-d174-4d14-9f5d-205a7cf3c3dc None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:05.284316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:05.307081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e6db7-a30a-46c2-a7e3-9f2645096abe None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:05.315481 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d576f9-c428-433c-875f-547e98bd7b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:05.316159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d576f9-c428-433c-875f-547e98bd7b7c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.316439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d576f9-c428-433c-875f-547e98bd7b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.316757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d576f9-c428-433c-875f-547e98bd7b7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.328780 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d576f9-c428-433c-875f-547e98bd7b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-64d576f9-c428-433c-875f-547e98bd7b7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:05.333895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:05.334867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:05.336064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 98/187] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:05.341150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d576f9-c428-433c-875f-547e98bd7b7c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.350531 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1f7d-0dd7-4fb0-aaa6-0bc97359626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:05.351703 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1f7d-0dd7-4fb0-aaa6-0bc97359626b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.351977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1f7d-0dd7-4fb0-aaa6-0bc97359626b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.352270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1f7d-0dd7-4fb0-aaa6-0bc97359626b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.365265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1f7d-0dd7-4fb0-aaa6-0bc97359626b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1c9e1f7d-0dd7-4fb0-aaa6-0bc97359626b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:05.370305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:05.371252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 90/188] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:05.377092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1f7d-0dd7-4fb0-aaa6-0bc97359626b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.457282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:05.466015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d576f9-c428-433c-875f-547e98bd7b7c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:05.493974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1f7d-0dd7-4fb0-aaa6-0bc97359626b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:05.502432 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccef333-30ed-4308-9163-216442ab9df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:05.503103 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccef333-30ed-4308-9163-216442ab9df1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.503385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccef333-30ed-4308-9163-216442ab9df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.503664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccef333-30ed-4308-9163-216442ab9df1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.504957 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccef333-30ed-4308-9163-216442ab9df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:05.506447 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccef333-30ed-4308-9163-216442ab9df1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:05.508087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 99/189] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:05.508087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccef333-30ed-4308-9163-216442ab9df1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.534791 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:05.535806 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.536276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.536704 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.537582 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:05.538048 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:05.539660 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:05.545277 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:05.549283 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 91/190] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:05.554700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.614902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccef333-30ed-4308-9163-216442ab9df1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:05.650041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:05.650703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.650986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.651260 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.651994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:05.652301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:05.653724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:05.655879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04568a40-3b68-4591-bee0-f3ab7f759b61 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:05.662026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 100/191] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:05.663104 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.689141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:05.696526 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456f029-8e89-4ba7-a32c-3a41a52cb6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:05.697451 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456f029-8e89-4ba7-a32c-3a41a52cb6ce None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.697794 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456f029-8e89-4ba7-a32c-3a41a52cb6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.698065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456f029-8e89-4ba7-a32c-3a41a52cb6ce None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.698785 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456f029-8e89-4ba7-a32c-3a41a52cb6ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:05.700590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456f029-8e89-4ba7-a32c-3a41a52cb6ce None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:05.729083 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 92/192] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:05 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:05.729987 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456f029-8e89-4ba7-a32c-3a41a52cb6ce None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.798787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:05.810719 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b7078-b65a-48c6-bcbb-a30fb331f171 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:05.850428 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456f029-8e89-4ba7-a32c-3a41a52cb6ce None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:05.852701 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e176d-3795-4586-8d54-7672160a5532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:05.853732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e176d-3795-4586-8d54-7672160a5532 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.854077 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e176d-3795-4586-8d54-7672160a5532 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.854441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e176d-3795-4586-8d54-7672160a5532 None admin] PATH_INFO: `/v3/users/68f1e96b2f744fb89457571c62cfeb35` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.872327 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e176d-3795-4586-8d54-7672160a5532 None admin] RBAC: Authorizing `identity:get_user(user_id=68f1e96b2f744fb89457571c62cfeb35)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:05.873989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e176d-3795-4586-8d54-7672160a5532 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:05.880783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 101/193] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/users/68f1e96b2f744fb89457571c62cfeb35 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:05.883240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e176d-3795-4586-8d54-7672160a5532 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.900172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271f0c-95ad-441b-8f04-49c2b80ea180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:05.900871 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271f0c-95ad-441b-8f04-49c2b80ea180 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:05.901120 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271f0c-95ad-441b-8f04-49c2b80ea180 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:05.901394 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271f0c-95ad-441b-8f04-49c2b80ea180 None admin] PATH_INFO: `/v3/users/68f1e96b2f744fb89457571c62cfeb35` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:05.902629 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8271f0c-95ad-441b-8f04-49c2b80ea180 None admin] RBAC: Authorizing `identity:get_user(user_id=68f1e96b2f744fb89457571c62cfeb35)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:05.904088 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8271f0c-95ad-441b-8f04-49c2b80ea180 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:05.910793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 93/194] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/users/68f1e96b2f744fb89457571c62cfeb35 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:05.911739 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271f0c-95ad-441b-8f04-49c2b80ea180 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:05.985991 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:06.016679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e176d-3795-4586-8d54-7672160a5532 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:06.055035 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271f0c-95ad-441b-8f04-49c2b80ea180 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:06.055226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd51eb-3017-452b-969b-a1f8f5bb2344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:06.055818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd51eb-3017-452b-969b-a1f8f5bb2344 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:06.056092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd51eb-3017-452b-969b-a1f8f5bb2344 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:06.056361 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd51eb-3017-452b-969b-a1f8f5bb2344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:06.089855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:06.102479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226be8a9-ef7e-487b-aeb3-9c7d376374c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:06.103122 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226be8a9-ef7e-487b-aeb3-9c7d376374c4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:06.103398 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226be8a9-ef7e-487b-aeb3-9c7d376374c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:06.103673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226be8a9-ef7e-487b-aeb3-9c7d376374c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:06.154637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd51eb-3017-452b-969b-a1f8f5bb2344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:06.156349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd51eb-3017-452b-969b-a1f8f5bb2344 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:06.198587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226be8a9-ef7e-487b-aeb3-9c7d376374c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:06.200975 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226be8a9-ef7e-487b-aeb3-9c7d376374c4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:06.282790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 102/195] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:06.284057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd51eb-3017-452b-969b-a1f8f5bb2344 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:06.285439 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:06.346579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 94/196] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:06.356230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:06.365204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226be8a9-ef7e-487b-aeb3-9c7d376374c4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:06.412295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd51eb-3017-452b-969b-a1f8f5bb2344 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:06.458445 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931a8df-ae90-4483-8418-f14d72c450ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:06.459103 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931a8df-ae90-4483-8418-f14d72c450ed None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:06.459381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931a8df-ae90-4483-8418-f14d72c450ed None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:06.459652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931a8df-ae90-4483-8418-f14d72c450ed None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:06.463592 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931a8df-ae90-4483-8418-f14d72c450ed None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:06.465067 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931a8df-ae90-4483-8418-f14d72c450ed None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:06.466791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 103/197] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:06 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:06.472853 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931a8df-ae90-4483-8418-f14d72c450ed None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:06.593087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931a8df-ae90-4483-8418-f14d72c450ed None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:06.599519 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226be8a9-ef7e-487b-aeb3-9c7d376374c4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:06.619443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:06.636838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:06.637467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:06.637744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:06.638015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:06.638839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=232c57cd96c243f6b181496002d8a724, user.id=68f1e96b2f744fb89457571c62cfeb35) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:06.639146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:06.640607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:06.661024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 104/198] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:06 2022] GET /identity/v3/role_assignments?user.id=68f1e96b2f744fb89457571c62cfeb35&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=65cd37a73e4f498482fefabb8374006c => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:06.663126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995685a9-d0b1-41a3-b43a-5b3be9ec0b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:06.663796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995685a9-d0b1-41a3-b43a-5b3be9ec0b42 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:06.664109 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995685a9-d0b1-41a3-b43a-5b3be9ec0b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:06.664375 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995685a9-d0b1-41a3-b43a-5b3be9ec0b42 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:06.671172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995685a9-d0b1-41a3-b43a-5b3be9ec0b42 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:06.672685 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995685a9-d0b1-41a3-b43a-5b3be9ec0b42 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:06.674861 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 95/199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:06 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:06.681605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:06.826896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed49ae3-5b14-43c9-a9f1-dbfbce4dafd8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:06.835844 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8096e-bf49-4b57-afbd-c3f70b8ffe8b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:06.836198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8096e-bf49-4b57-afbd-c3f70b8ffe8b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:06.836400 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8096e-bf49-4b57-afbd-c3f70b8ffe8b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:06.837737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 96/200] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:06.847821 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396912-1f82-4446-ba56-c3ae81056e0b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:06.848102 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396912-1f82-4446-ba56-c3ae81056e0b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:06.848287 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396912-1f82-4446-ba56-c3ae81056e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:06.873358 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:06.874005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:06.874322 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:06.874599 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:06.875410 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] RBAC: Adding query filter params (role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=232c57cd96c243f6b181496002d8a724, user.id=68f1e96b2f744fb89457571c62cfeb35) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:06.875723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:06.877226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:06.880565 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-00396912-1f82-4446-ba56-c3ae81056e0b None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:06.881763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-00396912-1f82-4446-ba56-c3ae81056e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:06.883761 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 105/201] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:06 2022] GET /identity/v3/role_assignments?user.id=68f1e96b2f744fb89457571c62cfeb35&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:06.911472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:06.999739 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 97/202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:07.032482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:07.110775 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:07.160963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b378e4-9aaf-49ee-ae82-f0a530d6c095 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:07.196924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60f841-7e9f-414b-b2f4-be14d2637ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:07.197614 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60f841-7e9f-414b-b2f4-be14d2637ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:07.197900 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60f841-7e9f-414b-b2f4-be14d2637ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:07.198205 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60f841-7e9f-414b-b2f4-be14d2637ff7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:07.211058 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60f841-7e9f-414b-b2f4-be14d2637ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8c60f841-7e9f-414b-b2f4-be14d2637ff7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:07.216781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:07.219057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:07.220927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 106/203] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:07.231322 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396912-1f82-4446-ba56-c3ae81056e0b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:07.375135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396912-1f82-4446-ba56-c3ae81056e0b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:07.417585 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:07.418295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:07.418573 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:07.418840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:07.419643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:07.419944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:07.421471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:07.431577 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 98/204] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:07.437261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60f841-7e9f-414b-b2f4-be14d2637ff7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:07.581357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60f841-7e9f-414b-b2f4-be14d2637ff7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:07.598534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:07.626718 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc445977-c263-406c-9b20-2095eff7f4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:07.627300 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc445977-c263-406c-9b20-2095eff7f4cf None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:07.627584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc445977-c263-406c-9b20-2095eff7f4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:07.627855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc445977-c263-406c-9b20-2095eff7f4cf None admin] PATH_INFO: `/v3/users/81d01c6d0b764715b1c523925b46b296` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:07.633262 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc445977-c263-406c-9b20-2095eff7f4cf None admin] RBAC: Authorizing `identity:get_user(user_id=81d01c6d0b764715b1c523925b46b296)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:07.634842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc445977-c263-406c-9b20-2095eff7f4cf None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:07.636713 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 107/205] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:07 2022] GET /identity/v3/users/81d01c6d0b764715b1c523925b46b296 => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:07.642589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:07.669861 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:07.782824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ecb34-3e37-46f0-9eb6-5b07995230e0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:07.829550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f13e1-20ab-456f-b3ae-2f1d8ac80953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:07.830243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f13e1-20ab-456f-b3ae-2f1d8ac80953 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:07.830527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f13e1-20ab-456f-b3ae-2f1d8ac80953 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:07.830795 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f13e1-20ab-456f-b3ae-2f1d8ac80953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:07.932379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f13e1-20ab-456f-b3ae-2f1d8ac80953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:07.933796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f13e1-20ab-456f-b3ae-2f1d8ac80953 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:07.951363 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:08.045272 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 99/206] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:08.046975 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc445977-c263-406c-9b20-2095eff7f4cf None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:08.159191 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc445977-c263-406c-9b20-2095eff7f4cf None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:08.215984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3b568-39c8-4589-a616-5334d2f4f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:08.216770 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b568-39c8-4589-a616-5334d2f4f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:08.216950 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b568-39c8-4589-a616-5334d2f4f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:08.217216 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b568-39c8-4589-a616-5334d2f4f4a7 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:08.218475 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3b568-39c8-4589-a616-5334d2f4f4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:08.220365 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3b568-39c8-4589-a616-5334d2f4f4a7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:08.222405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 108/207] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:08 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:08.230382 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f13e1-20ab-456f-b3ae-2f1d8ac80953 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:08.382148 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f13e1-20ab-456f-b3ae-2f1d8ac80953 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:08.396568 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:08.416801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9f81a-3993-4e09-96c2-8b72545c4d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:08.417439 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9f81a-3993-4e09-96c2-8b72545c4d1a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:08.417720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9f81a-3993-4e09-96c2-8b72545c4d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:08.417990 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9f81a-3993-4e09-96c2-8b72545c4d1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:08.418824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9f81a-3993-4e09-96c2-8b72545c4d1a None admin] RBAC: Adding query filter params (role.id=becb35407660454b9982c279fd4ee506, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6, user.id=81d01c6d0b764715b1c523925b46b296) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:08.419136 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9f81a-3993-4e09-96c2-8b72545c4d1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:08.420600 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9f81a-3993-4e09-96c2-8b72545c4d1a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:08.427075 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 100/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:08 2022] GET /identity/v3/role_assignments?user.id=81d01c6d0b764715b1c523925b46b296&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=becb35407660454b9982c279fd4ee506 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:08.558424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ba35-9fc1-4ba4-a0c4-316cfe6fdaf5 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:08.559052 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ba35-9fc1-4ba4-a0c4-316cfe6fdaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:08.559555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ba35-9fc1-4ba4-a0c4-316cfe6fdaf5 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:08.561187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 109/209] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:08.567278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e83b-d438-415f-bdc4-ba5274779780 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:08.567574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e83b-d438-415f-bdc4-ba5274779780 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:08.567838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e83b-d438-415f-bdc4-ba5274779780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:08.588724 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-eac8e83b-d438-415f-bdc4-ba5274779780 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:08.589934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-eac8e83b-d438-415f-bdc4-ba5274779780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:08.640842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b2b90-f195-489f-a986-7d94034d2041 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:08.641147 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b2b90-f195-489f-a986-7d94034d2041 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:08.641393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b2b90-f195-489f-a986-7d94034d2041 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:08.642947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 110/210] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:08.647857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a5e2-ee5e-497a-8858-a3f269ac35a5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:08.648136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a5e2-ee5e-497a-8858-a3f269ac35a5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:08.648408 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a5e2-ee5e-497a-8858-a3f269ac35a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:08.666796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 101/211] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:08.667121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1275a5e2-ee5e-497a-8858-a3f269ac35a5 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:08.668309 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1275a5e2-ee5e-497a-8858-a3f269ac35a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:08.691949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e83b-d438-415f-bdc4-ba5274779780 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:08.729223 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:08.764391 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 111/212] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:08.788566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a5e2-ee5e-497a-8858-a3f269ac35a5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:08.837244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e83b-d438-415f-bdc4-ba5274779780 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:08.849224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:08.881578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61156f74-23fa-41dc-b54a-9d2bddd2d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:08.882330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61156f74-23fa-41dc-b54a-9d2bddd2d5fb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:08.882612 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61156f74-23fa-41dc-b54a-9d2bddd2d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:08.882918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61156f74-23fa-41dc-b54a-9d2bddd2d5fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:08.884368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61156f74-23fa-41dc-b54a-9d2bddd2d5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:08.886906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61156f74-23fa-41dc-b54a-9d2bddd2d5fb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:08.892610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 102/213] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:08.893926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61156f74-23fa-41dc-b54a-9d2bddd2d5fb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:08.916277 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a5e2-ee5e-497a-8858-a3f269ac35a5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:08.954703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcb9da-f8a7-4a47-a143-996ec92a8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:08.955370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcb9da-f8a7-4a47-a143-996ec92a8ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:08.955657 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcb9da-f8a7-4a47-a143-996ec92a8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:08.955925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcb9da-f8a7-4a47-a143-996ec92a8ae8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:08.967001 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcb9da-f8a7-4a47-a143-996ec92a8ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e9dcb9da-f8a7-4a47-a143-996ec92a8ae8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:08.971998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:08.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:08.973710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 112/214] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:08.979293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcb9da-f8a7-4a47-a143-996ec92a8ae8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:09.014297 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61156f74-23fa-41dc-b54a-9d2bddd2d5fb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:09.020787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:09.061715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb03fa-a74d-44ac-9359-a2b48c9f4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:09.063077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb03fa-a74d-44ac-9359-a2b48c9f4c7f None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.063555 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb03fa-a74d-44ac-9359-a2b48c9f4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb03fa-a74d-44ac-9359-a2b48c9f4c7f None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.064908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb03fa-a74d-44ac-9359-a2b48c9f4c7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:09.066691 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb03fa-a74d-44ac-9359-a2b48c9f4c7f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:09.098634 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 103/215] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:08 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:09.108319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcb9da-f8a7-4a47-a143-996ec92a8ae8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:09.148374 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8a4fe-7e75-4ff9-8259-49e6183955a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:09.149122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8a4fe-7e75-4ff9-8259-49e6183955a2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.149407 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8a4fe-7e75-4ff9-8259-49e6183955a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.149735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8a4fe-7e75-4ff9-8259-49e6183955a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.150629 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8a4fe-7e75-4ff9-8259-49e6183955a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:09.151021 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8a4fe-7e75-4ff9-8259-49e6183955a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:09.152618 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8a4fe-7e75-4ff9-8259-49e6183955a2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:09.163296 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 113/216] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:09.169348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb03fa-a74d-44ac-9359-a2b48c9f4c7f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:09.206346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eefa20-c356-4da3-bfe8-47d3bd2ad4d8 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.206669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eefa20-c356-4da3-bfe8-47d3bd2ad4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.206904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eefa20-c356-4da3-bfe8-47d3bd2ad4d8 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.208581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 114/217] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:09.215900 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d2625-75b2-4eb3-9abd-8406a40772a3 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.216271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d2625-75b2-4eb3-9abd-8406a40772a3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.216684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:09.217368 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d2625-75b2-4eb3-9abd-8406a40772a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.238702 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1a0d2625-75b2-4eb3-9abd-8406a40772a3 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:09.239942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1a0d2625-75b2-4eb3-9abd-8406a40772a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:09.302662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb03fa-a74d-44ac-9359-a2b48c9f4c7f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:09.326359 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 115/218] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:09.348335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc8d11-139d-4538-b064-b4bdb46d9e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:09.348724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d2625-75b2-4eb3-9abd-8406a40772a3 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:09.349015 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc8d11-139d-4538-b064-b4bdb46d9e93 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.349292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc8d11-139d-4538-b064-b4bdb46d9e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.349566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc8d11-139d-4538-b064-b4bdb46d9e93 None admin] PATH_INFO: `/v3/users/1855d230f2c64251a39a5263e6529849` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.368211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc8d11-139d-4538-b064-b4bdb46d9e93 None admin] RBAC: Authorizing `identity:get_user(user_id=1855d230f2c64251a39a5263e6529849)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:09.370295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc8d11-139d-4538-b064-b4bdb46d9e93 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:09.372628 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 104/219] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:09 2022] GET /identity/v3/users/1855d230f2c64251a39a5263e6529849 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:09.378674 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc8d11-139d-4538-b064-b4bdb46d9e93 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:09.389357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:09.488876 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:09.492532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d2625-75b2-4eb3-9abd-8406a40772a3 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:09.501670 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc8d11-139d-4538-b064-b4bdb46d9e93 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:09.534033 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf3378-9407-43f5-8eaf-37e76943d6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:09.535078 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf3378-9407-43f5-8eaf-37e76943d6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.535453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf3378-9407-43f5-8eaf-37e76943d6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.535943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf3378-9407-43f5-8eaf-37e76943d6c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.546678 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b376f-d4c6-41b4-932b-d8b68ae4b176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:09.547329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b376f-d4c6-41b4-932b-d8b68ae4b176 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.547665 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b376f-d4c6-41b4-932b-d8b68ae4b176 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.547944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b376f-d4c6-41b4-932b-d8b68ae4b176 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.553224 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf3378-9407-43f5-8eaf-37e76943d6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-adbf3378-9407-43f5-8eaf-37e76943d6c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:09.558385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:09.559226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:09.560019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 116/220] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:09.561499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85831-c35f-4fa3-b20e-d5bdbc5467d9 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.561787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85831-c35f-4fa3-b20e-d5bdbc5467d9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.562054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85831-c35f-4fa3-b20e-d5bdbc5467d9 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.567951 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 117/221] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:09.568895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad85831-c35f-4fa3-b20e-d5bdbc5467d9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:09.625843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b376f-d4c6-41b4-932b-d8b68ae4b176 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:09.627599 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b376f-d4c6-41b4-932b-d8b68ae4b176 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:09.707070 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:09.711998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad85831-c35f-4fa3-b20e-d5bdbc5467d9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:09.726352 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:09.742736 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 105/222] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:09.744107 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf2a1-8bcc-4c2b-aa12-4d1126c3bbed None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.744392 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf2a1-8bcc-4c2b-aa12-4d1126c3bbed None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.744676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf2a1-8bcc-4c2b-aa12-4d1126c3bbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.763090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7daaf2a1-8bcc-4c2b-aa12-4d1126c3bbed None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:09.763283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:09.764099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.764477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.764679 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7daaf2a1-8bcc-4c2b-aa12-4d1126c3bbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:09.764958 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.765791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:09.766221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:09.768996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:09.780927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 118/223] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:09.782188 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:09.841024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 106/224] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:09.847090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf2a1-8bcc-4c2b-aa12-4d1126c3bbed None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:09.921994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d009b87-a986-476c-87c3-ac09c68431d5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:09.970808 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6df74b-10f7-43f2-af8c-715bdbfcee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:09.972016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6df74b-10f7-43f2-af8c-715bdbfcee9d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:09.972470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6df74b-10f7-43f2-af8c-715bdbfcee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:09.973125 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6df74b-10f7-43f2-af8c-715bdbfcee9d None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:09.975201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6df74b-10f7-43f2-af8c-715bdbfcee9d None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:09.977829 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6df74b-10f7-43f2-af8c-715bdbfcee9d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:09.979858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf2a1-8bcc-4c2b-aa12-4d1126c3bbed None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:09.984845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 119/225] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:09 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:09.984845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6df74b-10f7-43f2-af8c-715bdbfcee9d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:10.009307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:10.014977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:10.027622 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815647e-5073-4df3-9ab4-6df90a213e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:10.028290 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815647e-5073-4df3-9ab4-6df90a213e50 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:10.028565 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815647e-5073-4df3-9ab4-6df90a213e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:10.028833 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815647e-5073-4df3-9ab4-6df90a213e50 None admin] PATH_INFO: `/v3/users/68f1e96b2f744fb89457571c62cfeb35` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:10.030134 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815647e-5073-4df3-9ab4-6df90a213e50 None admin] RBAC: Authorizing `identity:get_user(user_id=68f1e96b2f744fb89457571c62cfeb35)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:10.031595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815647e-5073-4df3-9ab4-6df90a213e50 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:10.037911 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 107/226] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:09 2022] GET /identity/v3/users/68f1e96b2f744fb89457571c62cfeb35 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:10.037911 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815647e-5073-4df3-9ab4-6df90a213e50 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:10.130515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6df74b-10f7-43f2-af8c-715bdbfcee9d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:10.158704 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815647e-5073-4df3-9ab4-6df90a213e50 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:10.167589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727238f-6da3-4c15-9e5f-ba75f5ce14d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:10.168403 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727238f-6da3-4c15-9e5f-ba75f5ce14d8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:10.168773 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727238f-6da3-4c15-9e5f-ba75f5ce14d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:10.169141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727238f-6da3-4c15-9e5f-ba75f5ce14d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:10.181138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727238f-6da3-4c15-9e5f-ba75f5ce14d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f727238f-6da3-4c15-9e5f-ba75f5ce14d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:10.186656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:10.187711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:10.188677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:10.195289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 120/227] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:10.196315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727238f-6da3-4c15-9e5f-ba75f5ce14d8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:10.204642 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:10.205513 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:10.205930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:10.206788 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:10.207825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] RBAC: Adding query filter params (role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=232c57cd96c243f6b181496002d8a724, user.id=1855d230f2c64251a39a5263e6529849) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:10.208295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:10.210296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:10.226909 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 108/228] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:10 2022] GET /identity/v3/role_assignments?user.id=1855d230f2c64251a39a5263e6529849&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=06907320d4bc47b3ba3375df728a605a => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:10.226909 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:10.338532 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:10.347921 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:10.421828 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727238f-6da3-4c15-9e5f-ba75f5ce14d8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:10.424377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add72d20-5271-452c-b116-1a70a1a2cf6d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:10.471378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:10.472612 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:10.473153 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:10.473674 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:10.475150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:10.475883 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:10.477663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:10.493164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73017d62-810b-410c-bff7-386c8f1f0732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:10.493476 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 109/229] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:10.494703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017d62-810b-410c-bff7-386c8f1f0732 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:10.495270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:10.495377 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017d62-810b-410c-bff7-386c8f1f0732 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:10.495836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017d62-810b-410c-bff7-386c8f1f0732 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:10.497437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73017d62-810b-410c-bff7-386c8f1f0732 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:10.499636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73017d62-810b-410c-bff7-386c8f1f0732 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:10.502936 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 121/230] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:10 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:10.509894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73017d62-810b-410c-bff7-386c8f1f0732 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:10.630039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73017d62-810b-410c-bff7-386c8f1f0732 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:10.674974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09802bb-fa28-41d3-a8a9-9c80c62ef810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:10.676111 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09802bb-fa28-41d3-a8a9-9c80c62ef810 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:10.678394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09802bb-fa28-41d3-a8a9-9c80c62ef810 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:10.678876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09802bb-fa28-41d3-a8a9-9c80c62ef810 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724/users/68f1e96b2f744fb89457571c62cfeb35/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:10.682686 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09802bb-fa28-41d3-a8a9-9c80c62ef810 None admin] RBAC: Authorizing `identity:create_grant(project_id=232c57cd96c243f6b181496002d8a724, user_id=68f1e96b2f744fb89457571c62cfeb35, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:10.685224 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09802bb-fa28-41d3-a8a9-9c80c62ef810 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:10.718680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 122/231] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:10 2022] PUT /identity/v3/projects/232c57cd96c243f6b181496002d8a724/users/68f1e96b2f744fb89457571c62cfeb35/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:10.743580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8075444-1d82-482c-9075-deaf436b7635 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:10.760580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:10.784453 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb498d3-ab64-4011-9048-665630445f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:10.785578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb498d3-ab64-4011-9048-665630445f57 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:10.786055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb498d3-ab64-4011-9048-665630445f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:10.786570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb498d3-ab64-4011-9048-665630445f57 None admin] PATH_INFO: `/v3/users/68f1e96b2f744fb89457571c62cfeb35` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:10.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb498d3-ab64-4011-9048-665630445f57 None admin] RBAC: Authorizing `identity:get_user(user_id=68f1e96b2f744fb89457571c62cfeb35)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:10.789796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb498d3-ab64-4011-9048-665630445f57 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:10.792096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 110/232] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:10 2022] GET /identity/v3/users/68f1e96b2f744fb89457571c62cfeb35 => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:10.799514 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09802bb-fa28-41d3-a8a9-9c80c62ef810 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:10.813939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:10.962719 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09802bb-fa28-41d3-a8a9-9c80c62ef810 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:11.004739 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ce040-17fc-4b1b-8143-fd46a9c30fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:11.005764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ce040-17fc-4b1b-8143-fd46a9c30fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:11.006261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ce040-17fc-4b1b-8143-fd46a9c30fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:11.006719 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ce040-17fc-4b1b-8143-fd46a9c30fe6 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:11.008708 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ce040-17fc-4b1b-8143-fd46a9c30fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:11.010407 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ce040-17fc-4b1b-8143-fd46a9c30fe6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:11.012323 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 123/233] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:10 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:11.019055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb498d3-ab64-4011-9048-665630445f57 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:11.133366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb498d3-ab64-4011-9048-665630445f57 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:11.177784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ea48-2ef2-4ee8-89ea-14efbcaecf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:11.178656 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ea48-2ef2-4ee8-89ea-14efbcaecf30 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:11.178813 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ea48-2ef2-4ee8-89ea-14efbcaecf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:11.179047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ea48-2ef2-4ee8-89ea-14efbcaecf30 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724/users/68f1e96b2f744fb89457571c62cfeb35/roles/94aa0ce403304ce8a21ef20db556d534` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:11.181267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ea48-2ef2-4ee8-89ea-14efbcaecf30 None admin] RBAC: Authorizing `identity:create_grant(project_id=232c57cd96c243f6b181496002d8a724, user_id=68f1e96b2f744fb89457571c62cfeb35, role_id=94aa0ce403304ce8a21ef20db556d534)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:11.182782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ea48-2ef2-4ee8-89ea-14efbcaecf30 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:11.211214 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 111/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:11 2022] PUT /identity/v3/projects/232c57cd96c243f6b181496002d8a724/users/68f1e96b2f744fb89457571c62cfeb35/roles/94aa0ce403304ce8a21ef20db556d534 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:01:11.461522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3990410-5c82-4077-95bf-19761fa64bae None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:11.462447 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3990410-5c82-4077-95bf-19761fa64bae None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:11.462721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3990410-5c82-4077-95bf-19761fa64bae None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:11.464265 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 124/235] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:11.469968 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ff7e2-f0fa-436f-82a4-985ed015ebb8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:11.470321 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ff7e2-f0fa-436f-82a4-985ed015ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:11.470598 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ff7e2-f0fa-436f-82a4-985ed015ebb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:11.490708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e75ff7e2-f0fa-436f-82a4-985ed015ebb8 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:11.491931 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e75ff7e2-f0fa-436f-82a4-985ed015ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:11.535688 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:11.576426 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d6d6e-53be-4dca-87c9-afb041612b46 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:11.576620 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 112/236] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:11.576978 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d6d6e-53be-4dca-87c9-afb041612b46 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:11.577258 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d6d6e-53be-4dca-87c9-afb041612b46 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:11.578803 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 125/237] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:11.584143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd756b3f-618a-4f2d-98bb-8e3e77b31a07 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:11.584438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd756b3f-618a-4f2d-98bb-8e3e77b31a07 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:11.584714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd756b3f-618a-4f2d-98bb-8e3e77b31a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:11.602967 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-dd756b3f-618a-4f2d-98bb-8e3e77b31a07 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:11.604174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dd756b3f-618a-4f2d-98bb-8e3e77b31a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:11.621190 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d6d6e-53be-4dca-87c9-afb041612b46 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:11.682093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 113/238] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:11.710757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd756b3f-618a-4f2d-98bb-8e3e77b31a07 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:11.784507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:11.820098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d6d6e-53be-4dca-87c9-afb041612b46 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:11.864512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd756b3f-618a-4f2d-98bb-8e3e77b31a07 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:11.866953 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:11.886154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3a97e-29d9-47ed-8a1b-6565b9e43e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:11.886931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3a97e-29d9-47ed-8a1b-6565b9e43e63 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:11.887205 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3a97e-29d9-47ed-8a1b-6565b9e43e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:11.887471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3a97e-29d9-47ed-8a1b-6565b9e43e63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:11.889786 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3a97e-29d9-47ed-8a1b-6565b9e43e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:11.891478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3a97e-29d9-47ed-8a1b-6565b9e43e63 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:11.893369 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 126/239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:11.899465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3a97e-29d9-47ed-8a1b-6565b9e43e63 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:11.902963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51e0a5-3727-44c9-8c0b-c2440c0f6402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:11.903648 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51e0a5-3727-44c9-8c0b-c2440c0f6402 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:11.903930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51e0a5-3727-44c9-8c0b-c2440c0f6402 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:11.904339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51e0a5-3727-44c9-8c0b-c2440c0f6402 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:11.916072 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51e0a5-3727-44c9-8c0b-c2440c0f6402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9c51e0a5-3727-44c9-8c0b-c2440c0f6402 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:11.922162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:11.923055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:11.923882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:11.925084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 114/240] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:11.930611 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51e0a5-3727-44c9-8c0b-c2440c0f6402 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:12.041077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51e0a5-3727-44c9-8c0b-c2440c0f6402 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:12.077827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:12.078503 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:12.078783 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:12.079117 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:12.079857 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:12.080212 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:12.081634 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:12.091737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 115/241] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:12.097860 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:12.126483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:12.208517 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd86e04-a2b3-4304-93b7-a0ca393c7ef4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:12.222434 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3a97e-29d9-47ed-8a1b-6565b9e43e63 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:12.244423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be912f08-dcc0-492a-8d58-50ce2c398a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:12.245403 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be912f08-dcc0-492a-8d58-50ce2c398a54 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:12.245888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be912f08-dcc0-492a-8d58-50ce2c398a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:12.246378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be912f08-dcc0-492a-8d58-50ce2c398a54 None admin] PATH_INFO: `/v3/users/992bea1c4835493b9106be04761c400c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:12.262052 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be912f08-dcc0-492a-8d58-50ce2c398a54 None admin] RBAC: Authorizing `identity:get_user(user_id=992bea1c4835493b9106be04761c400c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:12.263983 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be912f08-dcc0-492a-8d58-50ce2c398a54 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:12.266035 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 116/242] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:12 2022] GET /identity/v3/users/992bea1c4835493b9106be04761c400c => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:12.271938 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be912f08-dcc0-492a-8d58-50ce2c398a54 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:12.334994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599507-6b8e-407b-8a38-505499fe5af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:12.335560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599507-6b8e-407b-8a38-505499fe5af9 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:12.335843 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599507-6b8e-407b-8a38-505499fe5af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:12.336118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599507-6b8e-407b-8a38-505499fe5af9 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:12.336838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599507-6b8e-407b-8a38-505499fe5af9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:12.338346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599507-6b8e-407b-8a38-505499fe5af9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:12.378357 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:12.380831 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be912f08-dcc0-492a-8d58-50ce2c398a54 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:12.387860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:12.408880 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 127/243] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:11 2022] POST /identity/v3/users => generated 293 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:01:12.425504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca3084-4bfa-458b-8ada-10556fe1a26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:12.426590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca3084-4bfa-458b-8ada-10556fe1a26f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:12.427056 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca3084-4bfa-458b-8ada-10556fe1a26f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:12.427628 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca3084-4bfa-458b-8ada-10556fe1a26f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:12.506876 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca3084-4bfa-458b-8ada-10556fe1a26f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:12.508760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca3084-4bfa-458b-8ada-10556fe1a26f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:12.598235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:12.620817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 117/244] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:01:12.626779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599507-6b8e-407b-8a38-505499fe5af9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:12.758936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22bbf9-57ab-4f05-87db-175ce10a0d72 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:12.759215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22bbf9-57ab-4f05-87db-175ce10a0d72 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:12.759488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22bbf9-57ab-4f05-87db-175ce10a0d72 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:12.762574 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 118/245] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec14ba-d507-4370-b233-ecdcedb717c2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:12.768217 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec14ba-d507-4370-b233-ecdcedb717c2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:12.768486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec14ba-d507-4370-b233-ecdcedb717c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:12.786309 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599507-6b8e-407b-8a38-505499fe5af9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:12.796773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a5ec14ba-d507-4370-b233-ecdcedb717c2 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:12.798885 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a5ec14ba-d507-4370-b233-ecdcedb717c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:12.830942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9da175-4cf6-4b59-b6b6-04d9d8775fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:12.832101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9da175-4cf6-4b59-b6b6-04d9d8775fdb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:12.832617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9da175-4cf6-4b59-b6b6-04d9d8775fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:12.833063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9da175-4cf6-4b59-b6b6-04d9d8775fdb None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:12.834717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9da175-4cf6-4b59-b6b6-04d9d8775fdb None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:12.836514 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9da175-4cf6-4b59-b6b6-04d9d8775fdb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:12.838761 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 128/246] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:12 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:12.845656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9da175-4cf6-4b59-b6b6-04d9d8775fdb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:12.877526 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 119/247] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:12.899786 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec14ba-d507-4370-b233-ecdcedb717c2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:12.969452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:13.006603 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec14ba-d507-4370-b233-ecdcedb717c2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:13.020590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:13.030847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9da175-4cf6-4b59-b6b6-04d9d8775fdb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:13.050587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8e270-4802-4db7-81ac-caf5a5ba8650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:13.051325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8e270-4802-4db7-81ac-caf5a5ba8650 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:13.051603 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8e270-4802-4db7-81ac-caf5a5ba8650 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:13.051877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8e270-4802-4db7-81ac-caf5a5ba8650 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:13.063972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8e270-4802-4db7-81ac-caf5a5ba8650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-50e8e270-4802-4db7-81ac-caf5a5ba8650 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:13.069168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:13.070258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:13.071202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 120/248] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:13.079295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8e270-4802-4db7-81ac-caf5a5ba8650 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:13.096061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:13.097070 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:13.097544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:13.098173 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:13.099340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] RBAC: Adding query filter params (role.id=becb35407660454b9982c279fd4ee506, scope.project.id=232c57cd96c243f6b181496002d8a724, user.id=992bea1c4835493b9106be04761c400c) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:13.099946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:13.101726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:13.109170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 129/249] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:12 2022] GET /identity/v3/role_assignments?user.id=992bea1c4835493b9106be04761c400c&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=becb35407660454b9982c279fd4ee506 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:13.205119 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8e270-4802-4db7-81ac-caf5a5ba8650 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:13.247908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:13.248832 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:13.249267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:13.249733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:13.251137 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:13.251678 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:13.254470 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:13.268279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 121/250] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:13.273850 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:13.411832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0a77a-e546-45a0-93db-be96d3bccc3a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:13.442989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:13.453001 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e147e4c-28f8-4117-b844-1ab1f0286890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:13.453674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e147e4c-28f8-4117-b844-1ab1f0286890 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:13.453952 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e147e4c-28f8-4117-b844-1ab1f0286890 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:13.454251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e147e4c-28f8-4117-b844-1ab1f0286890 None admin] PATH_INFO: `/v3/users/1855d230f2c64251a39a5263e6529849` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:13.455503 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e147e4c-28f8-4117-b844-1ab1f0286890 None admin] RBAC: Authorizing `identity:get_user(user_id=1855d230f2c64251a39a5263e6529849)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:13.456920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e147e4c-28f8-4117-b844-1ab1f0286890 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:13.458753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 130/251] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:13 2022] GET /identity/v3/users/1855d230f2c64251a39a5263e6529849 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:13.464631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:13.467048 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a971b-8ecb-44de-994e-4ca6c3a72067 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:13.467332 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a971b-8ecb-44de-994e-4ca6c3a72067 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:13.467635 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a971b-8ecb-44de-994e-4ca6c3a72067 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:13.469379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 131/252] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:13.483006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb7d4-dfae-49ee-a664-20463aaf138e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:13.483326 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb7d4-dfae-49ee-a664-20463aaf138e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:13.483561 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb7d4-dfae-49ee-a664-20463aaf138e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:13.506143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8f6cb7d4-dfae-49ee-a664-20463aaf138e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:13.507690 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8f6cb7d4-dfae-49ee-a664-20463aaf138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:13.517043 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:13.581920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 132/253] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:13.619835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb7d4-dfae-49ee-a664-20463aaf138e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:13.644463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41914ec2-4343-4724-9e7f-ab4e03522d31 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:13.703860 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1a261-4dd6-4a3d-aa34-d566ce851fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:13.704606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1a261-4dd6-4a3d-aa34-d566ce851fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:13.704893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1a261-4dd6-4a3d-aa34-d566ce851fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:13.705217 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1a261-4dd6-4a3d-aa34-d566ce851fc4 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:13.706555 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1a261-4dd6-4a3d-aa34-d566ce851fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:13.710129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1a261-4dd6-4a3d-aa34-d566ce851fc4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:13.710129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 122/254] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:13 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:13.720081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1a261-4dd6-4a3d-aa34-d566ce851fc4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:13.843572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb7d4-dfae-49ee-a664-20463aaf138e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:13.890138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:13.895639 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b1302-257d-4702-ba17-0b12608d015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:13.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b1302-257d-4702-ba17-0b12608d015d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:13.897324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b1302-257d-4702-ba17-0b12608d015d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:13.898722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b1302-257d-4702-ba17-0b12608d015d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:13.926991 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b1302-257d-4702-ba17-0b12608d015d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-cd6b1302-257d-4702-ba17-0b12608d015d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:13.932758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:13.933680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:13.934484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:13.943496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 133/255] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:13.951234 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b1302-257d-4702-ba17-0b12608d015d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:13.996481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1a261-4dd6-4a3d-aa34-d566ce851fc4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:14.002129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:14.087486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b1302-257d-4702-ba17-0b12608d015d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:14.103079 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4929b7-09b9-4046-97a9-c8be0e665dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:14.104071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4929b7-09b9-4046-97a9-c8be0e665dda None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:14.104519 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4929b7-09b9-4046-97a9-c8be0e665dda None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:14.104928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4929b7-09b9-4046-97a9-c8be0e665dda None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724/users/1855d230f2c64251a39a5263e6529849/roles/06907320d4bc47b3ba3375df728a605a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:14.107190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4929b7-09b9-4046-97a9-c8be0e665dda None admin] RBAC: Authorizing `identity:create_grant(project_id=232c57cd96c243f6b181496002d8a724, user_id=1855d230f2c64251a39a5263e6529849, role_id=06907320d4bc47b3ba3375df728a605a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:14.108822 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4929b7-09b9-4046-97a9-c8be0e665dda None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:14.126948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916e7ec-da6b-4097-97f3-5f416ebecd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:14.127685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916e7ec-da6b-4097-97f3-5f416ebecd95 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:14.127976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916e7ec-da6b-4097-97f3-5f416ebecd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:14.128342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916e7ec-da6b-4097-97f3-5f416ebecd95 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:14.129194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916e7ec-da6b-4097-97f3-5f416ebecd95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:14.129683 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916e7ec-da6b-4097-97f3-5f416ebecd95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:14.131475 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916e7ec-da6b-4097-97f3-5f416ebecd95 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:14.142128 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 134/256] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:14.148248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5db59-9136-429b-b7ca-e337b3c4de6b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:14.148614 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5db59-9136-429b-b7ca-e337b3c4de6b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:14.148908 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5db59-9136-429b-b7ca-e337b3c4de6b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:14.149054 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 123/257] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:13 2022] PUT /identity/v3/projects/232c57cd96c243f6b181496002d8a724/users/1855d230f2c64251a39a5263e6529849/roles/06907320d4bc47b3ba3375df728a605a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:14.150664 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4929b7-09b9-4046-97a9-c8be0e665dda None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:14.150819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 135/258] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:14.157119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a26f8c-829d-4a8b-8075-436813dfacb4 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:14.157517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a26f8c-829d-4a8b-8075-436813dfacb4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:14.157912 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a26f8c-829d-4a8b-8075-436813dfacb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:14.178953 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-12a26f8c-829d-4a8b-8075-436813dfacb4 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:14.180211 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-12a26f8c-829d-4a8b-8075-436813dfacb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:14.233656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:14.267858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 136/259] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:14.303961 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a26f8c-829d-4a8b-8075-436813dfacb4 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:14.397179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4929b7-09b9-4046-97a9-c8be0e665dda None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:14.434704 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956ded2-b864-4d04-9dac-d37c45e84545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:14.435503 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956ded2-b864-4d04-9dac-d37c45e84545 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:14.435768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a26f8c-829d-4a8b-8075-436813dfacb4 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:14.436090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956ded2-b864-4d04-9dac-d37c45e84545 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:14.436457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956ded2-b864-4d04-9dac-d37c45e84545 None admin] PATH_INFO: `/v3/users/68f1e96b2f744fb89457571c62cfeb35` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:14.438031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956ded2-b864-4d04-9dac-d37c45e84545 None admin] RBAC: Authorizing `identity:get_user(user_id=68f1e96b2f744fb89457571c62cfeb35)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:14.440551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956ded2-b864-4d04-9dac-d37c45e84545 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:14.442734 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 124/260] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:14 2022] GET /identity/v3/users/68f1e96b2f744fb89457571c62cfeb35 => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:14.454384 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956ded2-b864-4d04-9dac-d37c45e84545 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:14.497863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8012b-d444-47da-ad13-c0848c33014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:14.498859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8012b-d444-47da-ad13-c0848c33014d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:14.499315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8012b-d444-47da-ad13-c0848c33014d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:14.499759 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8012b-d444-47da-ad13-c0848c33014d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:14.525987 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:14.535809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8012b-d444-47da-ad13-c0848c33014d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f0e8012b-d444-47da-ad13-c0848c33014d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:14.544860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:14.546181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:14.547018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:14.548791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 137/261] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:14.558645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8012b-d444-47da-ad13-c0848c33014d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:14.644400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:14.690850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956ded2-b864-4d04-9dac-d37c45e84545 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:14.774311 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa07537-2e7a-4be1-8aab-8d82d5f7d214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:14.774962 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa07537-2e7a-4be1-8aab-8d82d5f7d214 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:14.775244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa07537-2e7a-4be1-8aab-8d82d5f7d214 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:14.775519 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa07537-2e7a-4be1-8aab-8d82d5f7d214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:14.798188 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8012b-d444-47da-ad13-c0848c33014d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:14.872368 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:14.873000 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:14.873279 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:14.873548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:14.874308 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:14.874614 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:14.875997 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:14.894008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 138/262] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:14.903527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:14.995227 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa07537-2e7a-4be1-8aab-8d82d5f7d214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:14.996799 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa07537-2e7a-4be1-8aab-8d82d5f7d214 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:15.064573 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:15.144223 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72bfef-9a35-42d0-ab77-2b8791464e8a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:15.146227 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:15.203021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 125/263] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:01:15.209366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa07537-2e7a-4be1-8aab-8d82d5f7d214 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:15.212856 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62b4a1-d18a-47ce-bdce-71a31a20418b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:15.213531 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b4a1-d18a-47ce-bdce-71a31a20418b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:15.213814 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b4a1-d18a-47ce-bdce-71a31a20418b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:15.214090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b4a1-d18a-47ce-bdce-71a31a20418b None admin] PATH_INFO: `/v3/users/68f1e96b2f744fb89457571c62cfeb35` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:15.215330 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62b4a1-d18a-47ce-bdce-71a31a20418b None admin] RBAC: Authorizing `identity:get_user(user_id=68f1e96b2f744fb89457571c62cfeb35)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:15.216792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62b4a1-d18a-47ce-bdce-71a31a20418b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:15.219466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 139/264] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:14 2022] GET /identity/v3/users/68f1e96b2f744fb89457571c62cfeb35 => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:15.224190 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62b4a1-d18a-47ce-bdce-71a31a20418b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:15.447370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62b4a1-d18a-47ce-bdce-71a31a20418b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:15.450422 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa07537-2e7a-4be1-8aab-8d82d5f7d214 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:15.500300 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:15.515106 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52dbc6-7171-4ebe-8d86-db2467029f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:15.516156 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52dbc6-7171-4ebe-8d86-db2467029f02 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:15.516683 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52dbc6-7171-4ebe-8d86-db2467029f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:15.517138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52dbc6-7171-4ebe-8d86-db2467029f02 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:15.519464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52dbc6-7171-4ebe-8d86-db2467029f02 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:15.526468 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52dbc6-7171-4ebe-8d86-db2467029f02 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:15.528892 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 126/265] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:15 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:15.534871 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52dbc6-7171-4ebe-8d86-db2467029f02 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:15.536818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90012c76-59b6-4202-8a1c-c008031afa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:15.537522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90012c76-59b6-4202-8a1c-c008031afa02 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:15.537808 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90012c76-59b6-4202-8a1c-c008031afa02 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:15.538086 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90012c76-59b6-4202-8a1c-c008031afa02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:15.618141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90012c76-59b6-4202-8a1c-c008031afa02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:15.618906 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90012c76-59b6-4202-8a1c-c008031afa02 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:15.622508 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:15.721819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52dbc6-7171-4ebe-8d86-db2467029f02 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:15.796340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b2ef-48d7-49be-b456-e5dd37176715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:15.796999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b2ef-48d7-49be-b456-e5dd37176715 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:15.797280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b2ef-48d7-49be-b456-e5dd37176715 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:15.797558 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b2ef-48d7-49be-b456-e5dd37176715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:15.798488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b2ef-48d7-49be-b456-e5dd37176715 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=232c57cd96c243f6b181496002d8a724, user.id=68f1e96b2f744fb89457571c62cfeb35) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:15.798802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b2ef-48d7-49be-b456-e5dd37176715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:15.800268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b2ef-48d7-49be-b456-e5dd37176715 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:15.818486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 127/266] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:15 2022] GET /identity/v3/role_assignments?user.id=68f1e96b2f744fb89457571c62cfeb35&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=65cd37a73e4f498482fefabb8374006c => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:15.820074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2cb9-f1f8-43ba-890a-b73e8840ad74 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:15.820368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2cb9-f1f8-43ba-890a-b73e8840ad74 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:15.820647 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2cb9-f1f8-43ba-890a-b73e8840ad74 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:15.822186 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 128/267] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:15.828043 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aca25-b80d-4138-96fd-a41209dd73e3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:15.828216 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 140/268] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:15.828293 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aca25-b80d-4138-96fd-a41209dd73e3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:15.828563 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aca25-b80d-4138-96fd-a41209dd73e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:15.833828 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90012c76-59b6-4202-8a1c-c008031afa02 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:15.867606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0e7aca25-b80d-4138-96fd-a41209dd73e3 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:15.869533 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0e7aca25-b80d-4138-96fd-a41209dd73e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:15.977639 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:16.029340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90012c76-59b6-4202-8a1c-c008031afa02 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:16.030151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:16.037409 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 129/269] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:16.046857 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77661ad0-5603-463c-8ebd-65f7c3f4a5d5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.047167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77661ad0-5603-463c-8ebd-65f7c3f4a5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.047446 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77661ad0-5603-463c-8ebd-65f7c3f4a5d5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.050260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 130/270] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:16.060651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3663b-5fde-489a-a59a-59355efbf8ff None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.060986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3663b-5fde-489a-a59a-59355efbf8ff None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.061207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3663b-5fde-489a-a59a-59355efbf8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.088974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ffa3663b-5fde-489a-a59a-59355efbf8ff None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:16.090236 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ffa3663b-5fde-489a-a59a-59355efbf8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:16.105522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a8ec0-0b41-4868-aeee-86e4e6617bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:16.106297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a8ec0-0b41-4868-aeee-86e4e6617bfe None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.106583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a8ec0-0b41-4868-aeee-86e4e6617bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.106862 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a8ec0-0b41-4868-aeee-86e4e6617bfe None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.108743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a8ec0-0b41-4868-aeee-86e4e6617bfe None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:16.110337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a8ec0-0b41-4868-aeee-86e4e6617bfe None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:16.112192 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 141/271] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:15 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:16.113015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a8ec0-0b41-4868-aeee-86e4e6617bfe None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:16.217523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 131/272] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:16.219224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3663b-5fde-489a-a59a-59355efbf8ff None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:16.261298 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a8ec0-0b41-4868-aeee-86e4e6617bfe None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:16.308288 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241dc1d-6342-461e-bf68-6055fded9b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:16.308966 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241dc1d-6342-461e-bf68-6055fded9b0d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.309421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241dc1d-6342-461e-bf68-6055fded9b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.309709 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241dc1d-6342-461e-bf68-6055fded9b0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.320694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:16.326499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241dc1d-6342-461e-bf68-6055fded9b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-a241dc1d-6342-461e-bf68-6055fded9b0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:16.332409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:16.338074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 142/273] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:16.339095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5abf6-a26b-4c45-bfb4-a21e18d361f7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.339895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5abf6-a26b-4c45-bfb4-a21e18d361f7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.339895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5abf6-a26b-4c45-bfb4-a21e18d361f7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.339895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 143/274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:16.341706 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5abf6-a26b-4c45-bfb4-a21e18d361f7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:16.351979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3663b-5fde-489a-a59a-59355efbf8ff None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:16.377564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:16.393134 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:16.393788 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.394068 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.394774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.395295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] RBAC: Adding query filter params (role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=232c57cd96c243f6b181496002d8a724, user.id=68f1e96b2f744fb89457571c62cfeb35) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:16.395609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:16.397097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:16.403512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 132/275] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:16 2022] GET /identity/v3/role_assignments?user.id=68f1e96b2f744fb89457571c62cfeb35&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:16.404341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:16.504407 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5abf6-a26b-4c45-bfb4-a21e18d361f7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:16.538289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19c666-127c-408f-ab65-cf1df612ab88 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:16.563150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ace35-e7e4-4cba-bdc5-d9b69963c53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:16.563872 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ace35-e7e4-4cba-bdc5-d9b69963c53d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.564147 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ace35-e7e4-4cba-bdc5-d9b69963c53d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.564419 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ace35-e7e4-4cba-bdc5-d9b69963c53d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.565806 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ace35-e7e4-4cba-bdc5-d9b69963c53d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:16.568216 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ace35-e7e4-4cba-bdc5-d9b69963c53d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:16.570420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 144/276] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:16.572847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5c779-707b-48a7-a4ab-7f608fa4a653 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.573807 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5c779-707b-48a7-a4ab-7f608fa4a653 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.574252 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5c779-707b-48a7-a4ab-7f608fa4a653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.585260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:16.586327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.586829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.587412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.588342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:16.589592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:16.591358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:16.597282 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f3e5c779-707b-48a7-a4ab-7f608fa4a653 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:16.598548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f3e5c779-707b-48a7-a4ab-7f608fa4a653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:16.603019 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 133/277] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:16.604151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:16.668251 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:16.681421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:16.694309 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 145/278] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:16.695666 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5c779-707b-48a7-a4ab-7f608fa4a653 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:16.747719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44ac1f-e0e7-4f9b-a07c-b3b7d05f9b76 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:16.784937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038263d-a87f-4e06-bc93-5a17762873d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:16.785740 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038263d-a87f-4e06-bc93-5a17762873d4 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.786016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038263d-a87f-4e06-bc93-5a17762873d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.786315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038263d-a87f-4e06-bc93-5a17762873d4 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.787046 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038263d-a87f-4e06-bc93-5a17762873d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:16.788555 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038263d-a87f-4e06-bc93-5a17762873d4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:16.813922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 134/279] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:16 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:16.815751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038263d-a87f-4e06-bc93-5a17762873d4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:16.825293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5c779-707b-48a7-a4ab-7f608fa4a653 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:16.867987 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6197970-2de5-4edb-91f2-d6013f4a1b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:16.868754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197970-2de5-4edb-91f2-d6013f4a1b56 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.869040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197970-2de5-4edb-91f2-d6013f4a1b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.869310 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197970-2de5-4edb-91f2-d6013f4a1b56 None admin] PATH_INFO: `/v3/users/8a7dee022b81459b97dd7f0e9d34b069` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.887214 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6197970-2de5-4edb-91f2-d6013f4a1b56 None admin] RBAC: Authorizing `identity:get_user(user_id=8a7dee022b81459b97dd7f0e9d34b069)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:16.888874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6197970-2de5-4edb-91f2-d6013f4a1b56 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:16.891120 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 146/280] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:16 2022] GET /identity/v3/users/8a7dee022b81459b97dd7f0e9d34b069 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:16.896130 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92044401-fdbb-48eb-aefd-45593003968b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:16.896608 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92044401-fdbb-48eb-aefd-45593003968b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:16.897071 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92044401-fdbb-48eb-aefd-45593003968b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:16.899583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 147/281] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:16.900826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92044401-fdbb-48eb-aefd-45593003968b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:16.966925 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038263d-a87f-4e06-bc93-5a17762873d4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:16.970398 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:16.974201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:17.011971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fed93-4026-4697-b3a2-c59a51470bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.012654 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fed93-4026-4697-b3a2-c59a51470bfa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.012928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fed93-4026-4697-b3a2-c59a51470bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.013198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fed93-4026-4697-b3a2-c59a51470bfa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.026749 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fed93-4026-4697-b3a2-c59a51470bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b41fed93-4026-4697-b3a2-c59a51470bfa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:17.032330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:17.033261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:17.034241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 135/282] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:17.036801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73247c37-3219-4221-9e08-a9bea5ab3e3c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.037079 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73247c37-3219-4221-9e08-a9bea5ab3e3c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.037348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73247c37-3219-4221-9e08-a9bea5ab3e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.041720 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92044401-fdbb-48eb-aefd-45593003968b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:17.061505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-73247c37-3219-4221-9e08-a9bea5ab3e3c None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:17.062757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-73247c37-3219-4221-9e08-a9bea5ab3e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:17.087189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.087961 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.088360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.088633 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.089603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] RBAC: Adding query filter params (role.id=06907320d4bc47b3ba3375df728a605a, user.id=8a7dee022b81459b97dd7f0e9d34b069, scope.system=all) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:17.089922 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:17.091799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:17.103638 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 148/283] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:01:16 2022] GET /identity/v3/role_assignments?user.id=8a7dee022b81459b97dd7f0e9d34b069&scope.system=all&role.id=06907320d4bc47b3ba3375df728a605a => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:17.104537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:17.168637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 136/284] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:17.191813 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73247c37-3219-4221-9e08-a9bea5ab3e3c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:17.242284 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0341b9-57e1-429f-a33d-ee5876aae6d5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:17.264833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:17.285467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.286139 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.286442 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.287456 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:17.287757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:17.290610 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:17.303782 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 149/285] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:17.311398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:17.335831 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:17.344002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73247c37-3219-4221-9e08-a9bea5ab3e3c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:17.382804 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ee927-9bc2-43e0-880f-195c74c06813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.383578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ee927-9bc2-43e0-880f-195c74c06813 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.383862 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ee927-9bc2-43e0-880f-195c74c06813 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.384232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ee927-9bc2-43e0-880f-195c74c06813 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.396203 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ee927-9bc2-43e0-880f-195c74c06813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-335ee927-9bc2-43e0-880f-195c74c06813 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:17.401588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:17.402512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:17.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 137/286] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:17.408841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ee927-9bc2-43e0-880f-195c74c06813 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:17.423884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a3f48-7ae0-41fd-be15-17427561e51a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:17.465122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120799bb-9fb8-40a7-b86e-c5ef98de1c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.465807 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120799bb-9fb8-40a7-b86e-c5ef98de1c19 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.466083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120799bb-9fb8-40a7-b86e-c5ef98de1c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.466393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120799bb-9fb8-40a7-b86e-c5ef98de1c19 None admin] PATH_INFO: `/v3/users/992bea1c4835493b9106be04761c400c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.467569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120799bb-9fb8-40a7-b86e-c5ef98de1c19 None admin] RBAC: Authorizing `identity:get_user(user_id=992bea1c4835493b9106be04761c400c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:17.469087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120799bb-9fb8-40a7-b86e-c5ef98de1c19 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:17.470751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 150/287] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:17 2022] GET /identity/v3/users/992bea1c4835493b9106be04761c400c => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:17.476636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120799bb-9fb8-40a7-b86e-c5ef98de1c19 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:17.531644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:17.589335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120799bb-9fb8-40a7-b86e-c5ef98de1c19 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:17.599659 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ee927-9bc2-43e0-880f-195c74c06813 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:17.623948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c6f66-a0ca-40dd-8bae-81213f7461a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.624933 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c6f66-a0ca-40dd-8bae-81213f7461a5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.625371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c6f66-a0ca-40dd-8bae-81213f7461a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.625734 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c6f66-a0ca-40dd-8bae-81213f7461a5 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.627212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c6f66-a0ca-40dd-8bae-81213f7461a5 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:17.629220 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c6f66-a0ca-40dd-8bae-81213f7461a5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:17.634762 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 151/288] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:17 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:17.638171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c6f66-a0ca-40dd-8bae-81213f7461a5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:17.663140 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.663787 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.664062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.664330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.665062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:17.665364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:17.666808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:17.687887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 138/289] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:17.692153 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:17.716680 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:17.798902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaf89f-fa27-468c-9561-27c9103b2b4d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:17.833255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d978c76-7fe5-404e-b229-152e46c6354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.833917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978c76-7fe5-404e-b229-152e46c6354a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.834272 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978c76-7fe5-404e-b229-152e46c6354a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.834544 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978c76-7fe5-404e-b229-152e46c6354a None admin] PATH_INFO: `/v3/users/1855d230f2c64251a39a5263e6529849` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.835722 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d978c76-7fe5-404e-b229-152e46c6354a None admin] RBAC: Authorizing `identity:get_user(user_id=1855d230f2c64251a39a5263e6529849)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:17.837211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d978c76-7fe5-404e-b229-152e46c6354a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:17.839240 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 139/290] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:17 2022] GET /identity/v3/users/1855d230f2c64251a39a5263e6529849 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:17.841715 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c6f66-a0ca-40dd-8bae-81213f7461a5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:17.845793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d978c76-7fe5-404e-b229-152e46c6354a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:17.890477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:17.921336 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7d024-b4f4-4cf3-9e70-d02cc125ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:17.922059 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d024-b4f4-4cf3-9e70-d02cc125ace1 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:17.922388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d024-b4f4-4cf3-9e70-d02cc125ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:17.922657 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d024-b4f4-4cf3-9e70-d02cc125ace1 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724/users/992bea1c4835493b9106be04761c400c/roles/becb35407660454b9982c279fd4ee506` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:17.926762 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7d024-b4f4-4cf3-9e70-d02cc125ace1 None admin] RBAC: Authorizing `identity:create_grant(project_id=232c57cd96c243f6b181496002d8a724, user_id=992bea1c4835493b9106be04761c400c, role_id=becb35407660454b9982c279fd4ee506)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:17.930283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7d024-b4f4-4cf3-9e70-d02cc125ace1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:17.960304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 152/291] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:17 2022] PUT /identity/v3/projects/232c57cd96c243f6b181496002d8a724/users/992bea1c4835493b9106be04761c400c/roles/becb35407660454b9982c279fd4ee506 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:18.020773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:18.045445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d978c76-7fe5-404e-b229-152e46c6354a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:18.096178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8213-d54b-4dd4-8a7f-fff634f05501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:18.097186 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8213-d54b-4dd4-8a7f-fff634f05501 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:18.097726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8213-d54b-4dd4-8a7f-fff634f05501 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:18.098241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8213-d54b-4dd4-8a7f-fff634f05501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:18.180578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a8213-d54b-4dd4-8a7f-fff634f05501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:18.182195 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a8213-d54b-4dd4-8a7f-fff634f05501 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:18.270934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:18.288993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 140/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:01:18.294682 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7d024-b4f4-4cf3-9e70-d02cc125ace1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:18.414721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7d024-b4f4-4cf3-9e70-d02cc125ace1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:18.465767 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74a9fc-1223-4e01-bebe-18e652d2b8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:18.466505 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74a9fc-1223-4e01-bebe-18e652d2b8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:18.466793 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74a9fc-1223-4e01-bebe-18e652d2b8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:18.467068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74a9fc-1223-4e01-bebe-18e652d2b8c2 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:18.468261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74a9fc-1223-4e01-bebe-18e652d2b8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:18.469722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74a9fc-1223-4e01-bebe-18e652d2b8c2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:18.471756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 153/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:18 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:18.477884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8213-d54b-4dd4-8a7f-fff634f05501 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:18.604001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8213-d54b-4dd4-8a7f-fff634f05501 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:18.639111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe32de0-71a3-4ec3-be21-7d5bf7f5db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:18.639748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe32de0-71a3-4ec3-be21-7d5bf7f5db91 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:18.640038 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe32de0-71a3-4ec3-be21-7d5bf7f5db91 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:18.640297 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe32de0-71a3-4ec3-be21-7d5bf7f5db91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:18.641107 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe32de0-71a3-4ec3-be21-7d5bf7f5db91 None admin] RBAC: Adding query filter params (role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=232c57cd96c243f6b181496002d8a724, user.id=1855d230f2c64251a39a5263e6529849) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:18.641424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe32de0-71a3-4ec3-be21-7d5bf7f5db91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:18.642933 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe32de0-71a3-4ec3-be21-7d5bf7f5db91 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:18.653491 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 141/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:18 2022] GET /identity/v3/role_assignments?user.id=1855d230f2c64251a39a5263e6529849&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=06907320d4bc47b3ba3375df728a605a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:19.347417 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650525f9-9f5b-44b0-959c-cbc977cda1dc None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.348276 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650525f9-9f5b-44b0-959c-cbc977cda1dc None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.348276 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650525f9-9f5b-44b0-959c-cbc977cda1dc None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.349663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 154/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:19.360828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce9fff-8d6d-4afd-bdb4-97df14dad514 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.360828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce9fff-8d6d-4afd-bdb4-97df14dad514 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.360828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce9fff-8d6d-4afd-bdb4-97df14dad514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.380645 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-65ce9fff-8d6d-4afd-bdb4-97df14dad514 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:19.382249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-65ce9fff-8d6d-4afd-bdb4-97df14dad514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:19.444485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecacd0-6213-4c43-989d-7f8f90d4e339 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.444799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecacd0-6213-4c43-989d-7f8f90d4e339 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.445051 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecacd0-6213-4c43-989d-7f8f90d4e339 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.446538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:19.447869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 155/296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:19.454047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570992-5b1d-4cac-b90d-bd1cb082d719 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.454846 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570992-5b1d-4cac-b90d-bd1cb082d719 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.455317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570992-5b1d-4cac-b90d-bd1cb082d719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.475717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ee570992-5b1d-4cac-b90d-bd1cb082d719 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:19.477375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ee570992-5b1d-4cac-b90d-bd1cb082d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:19.489486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 142/297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:19.510261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce9fff-8d6d-4afd-bdb4-97df14dad514 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:19.537911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:19.578529 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 156/298] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:19.601894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee570992-5b1d-4cac-b90d-bd1cb082d719 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:19.620705 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce9fff-8d6d-4afd-bdb4-97df14dad514 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:19.658001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc598d1-4b72-4925-8dba-91017c560e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:19.659084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc598d1-4b72-4925-8dba-91017c560e08 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.659505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc598d1-4b72-4925-8dba-91017c560e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.659952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc598d1-4b72-4925-8dba-91017c560e08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.672410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc598d1-4b72-4925-8dba-91017c560e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ecc598d1-4b72-4925-8dba-91017c560e08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:19.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:19.679110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:19.680064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:19.682322 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 143/299] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:19.694178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc598d1-4b72-4925-8dba-91017c560e08 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:19.715446 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee570992-5b1d-4cac-b90d-bd1cb082d719 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:19.758009 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:19.759133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e1e3b-1091-4d7e-b418-58e343f2c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:19.759816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e1e3b-1091-4d7e-b418-58e343f2c845 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.760106 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e1e3b-1091-4d7e-b418-58e343f2c845 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.760342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e1e3b-1091-4d7e-b418-58e343f2c845 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.763546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e1e3b-1091-4d7e-b418-58e343f2c845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:19.765512 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e1e3b-1091-4d7e-b418-58e343f2c845 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:19.773201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 157/300] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:19.774582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e1e3b-1091-4d7e-b418-58e343f2c845 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:19.829468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:19.832324 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc598d1-4b72-4925-8dba-91017c560e08 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:19.872802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe537dc3-d7b5-4314-be8e-2b63e6775df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:19.873280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe537dc3-d7b5-4314-be8e-2b63e6775df6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.873824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe537dc3-d7b5-4314-be8e-2b63e6775df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.873824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe537dc3-d7b5-4314-be8e-2b63e6775df6 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.874615 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe537dc3-d7b5-4314-be8e-2b63e6775df6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:19.874921 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe537dc3-d7b5-4314-be8e-2b63e6775df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:19.876306 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe537dc3-d7b5-4314-be8e-2b63e6775df6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:19.885790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 144/301] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:19.886924 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da2680-0eee-4437-b2e2-7d6a1815c9a7 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.887207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da2680-0eee-4437-b2e2-7d6a1815c9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.887477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da2680-0eee-4437-b2e2-7d6a1815c9a7 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.888697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 145/302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:19.894462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf953e-677d-4787-9bb6-7e72b42607a4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.894732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf953e-677d-4787-9bb6-7e72b42607a4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.895004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf953e-677d-4787-9bb6-7e72b42607a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.908693 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e1e3b-1091-4d7e-b418-58e343f2c845 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:19.913657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f3bf953e-677d-4787-9bb6-7e72b42607a4 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:19.914888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f3bf953e-677d-4787-9bb6-7e72b42607a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:19.953607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72892cb7-de58-4fa7-9512-7805951ec2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:19.954549 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72892cb7-de58-4fa7-9512-7805951ec2f8 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:19.955003 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72892cb7-de58-4fa7-9512-7805951ec2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:19.955420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72892cb7-de58-4fa7-9512-7805951ec2f8 None admin] PATH_INFO: `/v3/groups` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:19.956263 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72892cb7-de58-4fa7-9512-7805951ec2f8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:19.957807 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72892cb7-de58-4fa7-9512-7805951ec2f8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:19.978911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 158/303] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:01:19 2022] POST /identity/v3/groups => generated 229 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:19.979954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72892cb7-de58-4fa7-9512-7805951ec2f8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:20.008261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 146/304] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:20.035482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf953e-677d-4787-9bb6-7e72b42607a4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:20.081397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:20.148238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72892cb7-de58-4fa7-9512-7805951ec2f8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:20.151529 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:20.176762 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf953e-677d-4787-9bb6-7e72b42607a4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:20.197854 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee822b70-0485-438a-bfe9-07c54bc96cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:20.198562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee822b70-0485-438a-bfe9-07c54bc96cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.198839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee822b70-0485-438a-bfe9-07c54bc96cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.199107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee822b70-0485-438a-bfe9-07c54bc96cf2 None admin] PATH_INFO: `/v3/users/ae3f3c4436b844bb9d6914f03205baa5` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.218507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee822b70-0485-438a-bfe9-07c54bc96cf2 None admin] RBAC: Authorizing `identity:get_user(user_id=ae3f3c4436b844bb9d6914f03205baa5)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:20.219988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee822b70-0485-438a-bfe9-07c54bc96cf2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:20.222052 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 159/305] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:19 2022] GET /identity/v3/users/ae3f3c4436b844bb9d6914f03205baa5 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:20.224894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924243f3-1dcd-461b-acdf-ade3c3384b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924243f3-1dcd-461b-acdf-ade3c3384b54 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.225839 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924243f3-1dcd-461b-acdf-ade3c3384b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.226129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924243f3-1dcd-461b-acdf-ade3c3384b54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.227989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee822b70-0485-438a-bfe9-07c54bc96cf2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:20.242404 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924243f3-1dcd-461b-acdf-ade3c3384b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-924243f3-1dcd-461b-acdf-ade3c3384b54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:20.248721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:20.249697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:20.250498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 147/306] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:20.256006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924243f3-1dcd-461b-acdf-ade3c3384b54 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:20.358049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee822b70-0485-438a-bfe9-07c54bc96cf2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:20.367821 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:20.379052 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924243f3-1dcd-461b-acdf-ade3c3384b54 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:20.398711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef14e88-4a32-454a-913c-6fc991a27ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:20.399280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14e88-4a32-454a-913c-6fc991a27ecb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.399562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14e88-4a32-454a-913c-6fc991a27ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.399951 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14e88-4a32-454a-913c-6fc991a27ecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.400789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef14e88-4a32-454a-913c-6fc991a27ecb None admin] RBAC: Adding query filter params (role.id=becb35407660454b9982c279fd4ee506, user.id=ae3f3c4436b844bb9d6914f03205baa5, scope.system=all) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:20.401104 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef14e88-4a32-454a-913c-6fc991a27ecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:20.402649 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef14e88-4a32-454a-913c-6fc991a27ecb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:20.414789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 160/307] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:01:20 2022] GET /identity/v3/role_assignments?user.id=ae3f3c4436b844bb9d6914f03205baa5&scope.system=all&role.id=becb35407660454b9982c279fd4ee506 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:20.417263 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409e3c-cbf1-41bd-9f9f-5b01ad9e3496 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.417636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409e3c-cbf1-41bd-9f9f-5b01ad9e3496 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.417984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409e3c-cbf1-41bd-9f9f-5b01ad9e3496 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.420010 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 161/308] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:20.424491 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:20.424954 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.425237 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.425568 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.427930 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc4ead-b236-459c-865e-3cdaa944a312 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.428219 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc4ead-b236-459c-865e-3cdaa944a312 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.428469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc4ead-b236-459c-865e-3cdaa944a312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.431436 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:20.431914 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:20.433436 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:20.447287 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 148/309] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:20.452587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:20.455328 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-41fc4ead-b236-459c-865e-3cdaa944a312 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:20.456567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-41fc4ead-b236-459c-865e-3cdaa944a312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:20.478961 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:20.570425 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 162/310] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:20.588434 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc4ead-b236-459c-865e-3cdaa944a312 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:20.593020 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cd4e6-6944-44ca-b5c1-9d7a31211747 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:20.639168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220ad48-128f-40a8-8c67-83cac1ea0533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:20.639828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220ad48-128f-40a8-8c67-83cac1ea0533 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.640219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220ad48-128f-40a8-8c67-83cac1ea0533 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.640570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220ad48-128f-40a8-8c67-83cac1ea0533 None admin] PATH_INFO: `/v3/users/8a7dee022b81459b97dd7f0e9d34b069` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.642365 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220ad48-128f-40a8-8c67-83cac1ea0533 None admin] RBAC: Authorizing `identity:get_user(user_id=8a7dee022b81459b97dd7f0e9d34b069)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:20.644238 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220ad48-128f-40a8-8c67-83cac1ea0533 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:20.647139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 149/311] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:20 2022] GET /identity/v3/users/8a7dee022b81459b97dd7f0e9d34b069 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:20.652122 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220ad48-128f-40a8-8c67-83cac1ea0533 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:20.695857 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:20.734809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc4ead-b236-459c-865e-3cdaa944a312 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:20.782053 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7e6d2-b484-4e48-a2dc-f63d6b5e3097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:20.783171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7e6d2-b484-4e48-a2dc-f63d6b5e3097 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.783698 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7e6d2-b484-4e48-a2dc-f63d6b5e3097 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.784258 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7e6d2-b484-4e48-a2dc-f63d6b5e3097 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.797718 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7e6d2-b484-4e48-a2dc-f63d6b5e3097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0ab7e6d2-b484-4e48-a2dc-f63d6b5e3097 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:20.803486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:20.804343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:20.805118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:20.807513 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220ad48-128f-40a8-8c67-83cac1ea0533 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:20.808636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 163/312] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:20.814158 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7e6d2-b484-4e48-a2dc-f63d6b5e3097 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:20.836868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:20.856558 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c57e12-12b8-412e-9e86-4ebe35e1e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:20.857257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57e12-12b8-412e-9e86-4ebe35e1e387 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:20.857536 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57e12-12b8-412e-9e86-4ebe35e1e387 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:20.857869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57e12-12b8-412e-9e86-4ebe35e1e387 None admin] PATH_INFO: `/v3/system/users/8a7dee022b81459b97dd7f0e9d34b069/roles/06907320d4bc47b3ba3375df728a605a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:20.860929 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c57e12-12b8-412e-9e86-4ebe35e1e387 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8a7dee022b81459b97dd7f0e9d34b069, role_id=06907320d4bc47b3ba3375df728a605a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:20.863776 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c57e12-12b8-412e-9e86-4ebe35e1e387 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:20.882211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 150/313] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:01:20 2022] PUT /identity/v3/system/users/8a7dee022b81459b97dd7f0e9d34b069/roles/06907320d4bc47b3ba3375df728a605a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:20.966774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7e6d2-b484-4e48-a2dc-f63d6b5e3097 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:21.019347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:21.020029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:21.020320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:21.020571 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:21.021315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:21.021653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:21.023132 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:21.033528 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 164/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:21.039714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c57e12-12b8-412e-9e86-4ebe35e1e387 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:21.134084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:21.152652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c57e12-12b8-412e-9e86-4ebe35e1e387 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:21.190181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b406a1-b7af-4414-a9b5-a570f668f031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:21.191262 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b406a1-b7af-4414-a9b5-a570f668f031 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:21.191765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b406a1-b7af-4414-a9b5-a570f668f031 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:21.192250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b406a1-b7af-4414-a9b5-a570f668f031 None admin] PATH_INFO: `/v3/users/992bea1c4835493b9106be04761c400c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:21.193693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b406a1-b7af-4414-a9b5-a570f668f031 None admin] RBAC: Authorizing `identity:get_user(user_id=992bea1c4835493b9106be04761c400c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:21.195439 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b406a1-b7af-4414-a9b5-a570f668f031 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:21.197686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 151/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:21 2022] GET /identity/v3/users/992bea1c4835493b9106be04761c400c => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:21.204036 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:21.292753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:21.323622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c38bf1-36e0-4b81-ba8c-c863c7cd3197 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:21.358424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a642b-d6c8-471b-9b41-1e6a2f778ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:21.359052 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a642b-d6c8-471b-9b41-1e6a2f778ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:21.359332 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a642b-d6c8-471b-9b41-1e6a2f778ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:21.359605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a642b-d6c8-471b-9b41-1e6a2f778ca4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:21.432345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a642b-d6c8-471b-9b41-1e6a2f778ca4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:21.434314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a642b-d6c8-471b-9b41-1e6a2f778ca4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:21.521911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:21.547007 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 165/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:21.552620 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b406a1-b7af-4414-a9b5-a570f668f031 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:21.665278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b406a1-b7af-4414-a9b5-a570f668f031 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:21.701889 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbea073-3f60-4c38-b34d-5c6f5b0083a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:21.703112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbea073-3f60-4c38-b34d-5c6f5b0083a0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:21.703595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbea073-3f60-4c38-b34d-5c6f5b0083a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:21.704066 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbea073-3f60-4c38-b34d-5c6f5b0083a0 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:21.705515 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbea073-3f60-4c38-b34d-5c6f5b0083a0 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:21.707347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbea073-3f60-4c38-b34d-5c6f5b0083a0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:21.710219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 152/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:21 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:21.716582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a642b-d6c8-471b-9b41-1e6a2f778ca4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:21.822842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a642b-d6c8-471b-9b41-1e6a2f778ca4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:21.858079 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807767b-d6af-44f5-84cf-91926a616623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:21.858735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807767b-d6af-44f5-84cf-91926a616623 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:21.859011 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807767b-d6af-44f5-84cf-91926a616623 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:21.859279 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807767b-d6af-44f5-84cf-91926a616623 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:21.860071 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807767b-d6af-44f5-84cf-91926a616623 None admin] RBAC: Adding query filter params (role.id=becb35407660454b9982c279fd4ee506, scope.project.id=232c57cd96c243f6b181496002d8a724, user.id=992bea1c4835493b9106be04761c400c) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:21.860408 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807767b-d6af-44f5-84cf-91926a616623 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:21.861857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807767b-d6af-44f5-84cf-91926a616623 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:21.868499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 166/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:21 2022] GET /identity/v3/role_assignments?user.id=992bea1c4835493b9106be04761c400c&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=becb35407660454b9982c279fd4ee506 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:22.201439 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c78bf9-8a61-47c8-bdf6-be2d9868b65d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:22.201669 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c78bf9-8a61-47c8-bdf6-be2d9868b65d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:22.201949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c78bf9-8a61-47c8-bdf6-be2d9868b65d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:22.203546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 153/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:22.208745 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff412b5-c149-4f6b-be13-7e8aeb0e6ae6 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:22.208994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff412b5-c149-4f6b-be13-7e8aeb0e6ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:22.209303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff412b5-c149-4f6b-be13-7e8aeb0e6ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:22.227862 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-fff412b5-c149-4f6b-be13-7e8aeb0e6ae6 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:22.229042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fff412b5-c149-4f6b-be13-7e8aeb0e6ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:22.266521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:22.305563 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 167/320] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:22.327626 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c78bf9-8a61-47c8-bdf6-be2d9868b65d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:22.355810 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:22.441963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c78bf9-8a61-47c8-bdf6-be2d9868b65d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:22.480496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53493263-5d7f-4f23-8a36-b0bec84bc3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:22.481089 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493263-5d7f-4f23-8a36-b0bec84bc3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:22.481367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493263-5d7f-4f23-8a36-b0bec84bc3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:22.481624 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493263-5d7f-4f23-8a36-b0bec84bc3e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:22.482917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53493263-5d7f-4f23-8a36-b0bec84bc3e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:22.484457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53493263-5d7f-4f23-8a36-b0bec84bc3e7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:22.486500 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 154/321] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:22.492247 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff412b5-c149-4f6b-be13-7e8aeb0e6ae6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:22.492948 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbb8a-cc8c-4dbe-86d6-b2cc6db82295 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:22.493249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbb8a-cc8c-4dbe-86d6-b2cc6db82295 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:22.493530 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbb8a-cc8c-4dbe-86d6-b2cc6db82295 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:22.495045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 155/322] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:22.500864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafaf4c-c4c3-4587-8158-509a7a43aba0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:22.501159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafaf4c-c4c3-4587-8158-509a7a43aba0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:22.501422 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafaf4c-c4c3-4587-8158-509a7a43aba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:22.522942 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0dafaf4c-c4c3-4587-8158-509a7a43aba0 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:22.524486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0dafaf4c-c4c3-4587-8158-509a7a43aba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:22.598157 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 156/323] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:22.598349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff412b5-c149-4f6b-be13-7e8aeb0e6ae6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:22.619152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafaf4c-c4c3-4587-8158-509a7a43aba0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:22.637636 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:22.642073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f7c03-2fc2-48bb-93a6-0b0ccc606dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:22.642814 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7c03-2fc2-48bb-93a6-0b0ccc606dff None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:22.643135 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7c03-2fc2-48bb-93a6-0b0ccc606dff None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:22.643538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7c03-2fc2-48bb-93a6-0b0ccc606dff None admin] PATH_INFO: `/v3/groups` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:22.644082 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f7c03-2fc2-48bb-93a6-0b0ccc606dff None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:22.645662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f7c03-2fc2-48bb-93a6-0b0ccc606dff None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:22.665222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 168/324] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:01:22 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:22.744681 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafaf4c-c4c3-4587-8158-509a7a43aba0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:22.785393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aee66-ee28-4d50-9836-0ae88bc911f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:22.786075 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aee66-ee28-4d50-9836-0ae88bc911f7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:22.786408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aee66-ee28-4d50-9836-0ae88bc911f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:22.786681 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aee66-ee28-4d50-9836-0ae88bc911f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:22.799288 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aee66-ee28-4d50-9836-0ae88bc911f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-024aee66-ee28-4d50-9836-0ae88bc911f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:22.804566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:22.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:22.806584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 157/325] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:22.815515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f7c03-2fc2-48bb-93a6-0b0ccc606dff None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:22.852101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:22.937005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f7c03-2fc2-48bb-93a6-0b0ccc606dff None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:22.976310 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c60410-ad08-4565-b7b3-ea072679f87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:22.977210 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60410-ad08-4565-b7b3-ea072679f87a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:22.977634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60410-ad08-4565-b7b3-ea072679f87a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:22.978049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60410-ad08-4565-b7b3-ea072679f87a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:22.978993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c60410-ad08-4565-b7b3-ea072679f87a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:22.979471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c60410-ad08-4565-b7b3-ea072679f87a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:22.981013 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c60410-ad08-4565-b7b3-ea072679f87a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:22.991894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 169/326] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:22.997530 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aee66-ee28-4d50-9836-0ae88bc911f7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:23.051829 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8c0d-f0a0-4a3b-929d-40fbc614f2f3 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:23.052165 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8c0d-f0a0-4a3b-929d-40fbc614f2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:23.052416 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8c0d-f0a0-4a3b-929d-40fbc614f2f3 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:23.054039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 170/327] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:23.060479 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6da0b8-e0b1-448c-a9b7-aafee7f5fea8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:23.060921 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6da0b8-e0b1-448c-a9b7-aafee7f5fea8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:23.061299 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6da0b8-e0b1-448c-a9b7-aafee7f5fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:23.081985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3c6da0b8-e0b1-448c-a9b7-aafee7f5fea8 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:23.083998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3c6da0b8-e0b1-448c-a9b7-aafee7f5fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:23.107812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:23.136929 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aee66-ee28-4d50-9836-0ae88bc911f7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:23.175317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 171/328] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:23.191498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f4eb2-2f3c-4d81-837e-081908fdf227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:23.192415 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f4eb2-2f3c-4d81-837e-081908fdf227 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:23.192855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f4eb2-2f3c-4d81-837e-081908fdf227 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:23.193263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f4eb2-2f3c-4d81-837e-081908fdf227 None admin] PATH_INFO: `/v3/users/ae3f3c4436b844bb9d6914f03205baa5` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:23.194698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f4eb2-2f3c-4d81-837e-081908fdf227 None admin] RBAC: Authorizing `identity:get_user(user_id=ae3f3c4436b844bb9d6914f03205baa5)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:23.196354 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f4eb2-2f3c-4d81-837e-081908fdf227 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:23.198505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 158/329] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:22 2022] GET /identity/v3/users/ae3f3c4436b844bb9d6914f03205baa5 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:23.203357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6da0b8-e0b1-448c-a9b7-aafee7f5fea8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:23.205074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f4eb2-2f3c-4d81-837e-081908fdf227 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:23.233231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:23.361613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f4eb2-2f3c-4d81-837e-081908fdf227 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:23.371914 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6da0b8-e0b1-448c-a9b7-aafee7f5fea8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:23.420328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb831e0-487d-4105-b290-9ca9aeb62821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:23.421514 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb831e0-487d-4105-b290-9ca9aeb62821 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:23.422050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb831e0-487d-4105-b290-9ca9aeb62821 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:23.422599 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb831e0-487d-4105-b290-9ca9aeb62821 None admin] PATH_INFO: `/v3/system/users/ae3f3c4436b844bb9d6914f03205baa5/roles/becb35407660454b9982c279fd4ee506` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:23.426377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb831e0-487d-4105-b290-9ca9aeb62821 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ae3f3c4436b844bb9d6914f03205baa5, role_id=becb35407660454b9982c279fd4ee506)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:23.428196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb831e0-487d-4105-b290-9ca9aeb62821 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:23.442706 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccad8e3-dd77-48cd-9f83-1dae43177228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:23.443466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccad8e3-dd77-48cd-9f83-1dae43177228 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:23.443754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccad8e3-dd77-48cd-9f83-1dae43177228 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:23.444030 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccad8e3-dd77-48cd-9f83-1dae43177228 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:23.445029 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 159/330] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:01:23 2022] PUT /identity/v3/system/users/ae3f3c4436b844bb9d6914f03205baa5/roles/becb35407660454b9982c279fd4ee506 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:23.463797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccad8e3-dd77-48cd-9f83-1dae43177228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-7ccad8e3-dd77-48cd-9f83-1dae43177228 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:23.469424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:23.470581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:23.471364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 172/331] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:23.477856 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb831e0-487d-4105-b290-9ca9aeb62821 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:23.499239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:23.644450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb831e0-487d-4105-b290-9ca9aeb62821 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:23.703731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:23.704714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:23.705210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:23.705657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:23.707110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:23.707643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:23.709589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:23.723778 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 160/332] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:23.731110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccad8e3-dd77-48cd-9f83-1dae43177228 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:23.859515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:23.893792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccad8e3-dd77-48cd-9f83-1dae43177228 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:23.950578 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26510b53-1326-4e5a-86d0-115c9255dca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:23.951668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26510b53-1326-4e5a-86d0-115c9255dca1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:23.952146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26510b53-1326-4e5a-86d0-115c9255dca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:23.952637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26510b53-1326-4e5a-86d0-115c9255dca1 None admin] PATH_INFO: `/v3/users/8a7dee022b81459b97dd7f0e9d34b069` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:23.954315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26510b53-1326-4e5a-86d0-115c9255dca1 None admin] RBAC: Authorizing `identity:get_user(user_id=8a7dee022b81459b97dd7f0e9d34b069)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:23.956128 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26510b53-1326-4e5a-86d0-115c9255dca1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:23.958489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 173/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:23 2022] GET /identity/v3/users/8a7dee022b81459b97dd7f0e9d34b069 => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:23.966264 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:24.140523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:24.183463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432a149-90a0-41e3-ba55-ee65ffa9c2aa None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:24.224312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08eaed6-1589-4bbe-9454-880de99c5fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:24.224837 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eaed6-1589-4bbe-9454-880de99c5fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:24.225124 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eaed6-1589-4bbe-9454-880de99c5fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:24.225397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eaed6-1589-4bbe-9454-880de99c5fc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:24.226343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08eaed6-1589-4bbe-9454-880de99c5fc0 None admin] RBAC: Adding query filter params (role.id=06907320d4bc47b3ba3375df728a605a, user.id=8a7dee022b81459b97dd7f0e9d34b069, scope.system=all) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:24.226756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08eaed6-1589-4bbe-9454-880de99c5fc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:24.228232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08eaed6-1589-4bbe-9454-880de99c5fc0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:24.241937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 161/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:01:23 2022] GET /identity/v3/role_assignments?user.id=8a7dee022b81459b97dd7f0e9d34b069&scope.system=all&role.id=06907320d4bc47b3ba3375df728a605a => generated 510 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:25.246498 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a207d4-34ed-44da-af06-56cdd0a95f45 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:25.247695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a207d4-34ed-44da-af06-56cdd0a95f45 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:25.248221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a207d4-34ed-44da-af06-56cdd0a95f45 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:25.250026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 174/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:25.257372 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800041-2ee1-4dff-8a70-a7c5ac9fb38f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:25.258640 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800041-2ee1-4dff-8a70-a7c5ac9fb38f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:25.259366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800041-2ee1-4dff-8a70-a7c5ac9fb38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:25.290807 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-58800041-2ee1-4dff-8a70-a7c5ac9fb38f None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:25.292388 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-58800041-2ee1-4dff-8a70-a7c5ac9fb38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:25.382787 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 162/336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:25.430402 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a207d4-34ed-44da-af06-56cdd0a95f45 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:25.583964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a207d4-34ed-44da-af06-56cdd0a95f45 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:25.630199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3c16b-a064-4215-a682-c67e8d036d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:25.631382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3c16b-a064-4215-a682-c67e8d036d6b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:25.631864 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3c16b-a064-4215-a682-c67e8d036d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:25.632372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3c16b-a064-4215-a682-c67e8d036d6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:25.650013 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3c16b-a064-4215-a682-c67e8d036d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-85d3c16b-a064-4215-a682-c67e8d036d6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:25.656486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:25.657520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:25.658303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:25.661069 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 175/337] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:25.668087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58800041-2ee1-4dff-8a70-a7c5ac9fb38f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:25.731122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740020e6-86bb-4314-b32a-9f90a677877e None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:25.731407 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740020e6-86bb-4314-b32a-9f90a677877e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:25.731673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740020e6-86bb-4314-b32a-9f90a677877e None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:25.733068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 176/338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:25.739330 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8fe9e-7be2-4435-b64d-ae9ef4f896c1 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:25.739617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8fe9e-7be2-4435-b64d-ae9ef4f896c1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:25.739883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8fe9e-7be2-4435-b64d-ae9ef4f896c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:25.776299 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-02d8fe9e-7be2-4435-b64d-ae9ef4f896c1 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:25.777830 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-02d8fe9e-7be2-4435-b64d-ae9ef4f896c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:25.782479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:25.788008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:25.865724 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58800041-2ee1-4dff-8a70-a7c5ac9fb38f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:25.901731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 177/339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:25.919091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:25.919811 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:25.920095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:25.920543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:25.921346 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:25.921655 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:25.923104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:25.932369 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8fe9e-7be2-4435-b64d-ae9ef4f896c1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:25.950907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 163/340] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:25.976177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:26.155558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8fe9e-7be2-4435-b64d-ae9ef4f896c1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:26.197840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46965b2-c3af-442f-930d-9c4404c50d4b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:26.231894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0797cc-8ecf-4d22-9c92-9a305d741efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:26.233413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0797cc-8ecf-4d22-9c92-9a305d741efc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:26.233950 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0797cc-8ecf-4d22-9c92-9a305d741efc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:26.235092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0797cc-8ecf-4d22-9c92-9a305d741efc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:26.239531 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:26.256144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:26.266992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0797cc-8ecf-4d22-9c92-9a305d741efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6c0797cc-8ecf-4d22-9c92-9a305d741efc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:26.272972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:26.273902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:26.274704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:26.277450 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 178/341] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:26.282937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318714-f48a-4890-9440-c9d21bd35da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:26.283669 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318714-f48a-4890-9440-c9d21bd35da8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:26.283954 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318714-f48a-4890-9440-c9d21bd35da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:26.284280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318714-f48a-4890-9440-c9d21bd35da8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:26.285122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0797cc-8ecf-4d22-9c92-9a305d741efc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:26.429469 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df318714-f48a-4890-9440-c9d21bd35da8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:26.430954 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df318714-f48a-4890-9440-c9d21bd35da8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:26.501436 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0797cc-8ecf-4d22-9c92-9a305d741efc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:26.579591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:26.580313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:26.580650 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:26.580945 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:26.581822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:26.582226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:26.583662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:26.609637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 179/342] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:26.616441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:26.664425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 164/343] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:01:26.672144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318714-f48a-4890-9440-c9d21bd35da8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:26.736379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:26.749576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:26.825228 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103b9eb-7326-416e-9ea6-695d71617d70 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:26.881903 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318714-f48a-4890-9440-c9d21bd35da8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:26.886982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ab265-09ad-40e7-b908-7730a1f6642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:26.887652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ab265-09ad-40e7-b908-7730a1f6642d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:26.887933 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ab265-09ad-40e7-b908-7730a1f6642d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:26.888207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ab265-09ad-40e7-b908-7730a1f6642d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:26.974877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114eb88-d5af-4b99-9123-4232ad9f5664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:26.975947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114eb88-d5af-4b99-9123-4232ad9f5664 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:26.976431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114eb88-d5af-4b99-9123-4232ad9f5664 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:26.976984 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114eb88-d5af-4b99-9123-4232ad9f5664 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:26.978529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114eb88-d5af-4b99-9123-4232ad9f5664 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:26.980240 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114eb88-d5af-4b99-9123-4232ad9f5664 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:26.982549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 165/344] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:26 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:26.987824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114eb88-d5af-4b99-9123-4232ad9f5664 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:27.012008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ab265-09ad-40e7-b908-7730a1f6642d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:27.020014 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ab265-09ad-40e7-b908-7730a1f6642d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:27.095747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:27.152258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:27.176126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114eb88-d5af-4b99-9123-4232ad9f5664 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:27.193974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 180/345] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:01:27.200466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3047f7f-6b75-4d45-b22b-f3d4698a5cbc None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.203622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3047f7f-6b75-4d45-b22b-f3d4698a5cbc None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.204193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3047f7f-6b75-4d45-b22b-f3d4698a5cbc None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.208495 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 181/346] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:27.216062 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e80e8-2be2-4b39-a991-7c4fc7ea6834 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.218847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e80e8-2be2-4b39-a991-7c4fc7ea6834 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.219465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e80e8-2be2-4b39-a991-7c4fc7ea6834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.262617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-091e80e8-2be2-4b39-a991-7c4fc7ea6834 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:27.264793 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-091e80e8-2be2-4b39-a991-7c4fc7ea6834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:27.281450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345f8f8-acb3-442c-93eb-90cf67634252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:27.282546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345f8f8-acb3-442c-93eb-90cf67634252 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.283054 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345f8f8-acb3-442c-93eb-90cf67634252 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.284496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345f8f8-acb3-442c-93eb-90cf67634252 None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.295112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345f8f8-acb3-442c-93eb-90cf67634252 None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:27.300982 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345f8f8-acb3-442c-93eb-90cf67634252 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:27.303370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 166/347] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:26 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:27.304406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345f8f8-acb3-442c-93eb-90cf67634252 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:27.357274 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 182/348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:27.358858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e84604-01ab-4dc0-b9a3-b078465b7957 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.359276 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e84604-01ab-4dc0-b9a3-b078465b7957 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.359675 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e84604-01ab-4dc0-b9a3-b078465b7957 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.361433 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 183/349] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:27.366171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e84604-01ab-4dc0-b9a3-b078465b7957 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:27.445454 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345f8f8-acb3-442c-93eb-90cf67634252 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:27.474651 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:27.486822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e84604-01ab-4dc0-b9a3-b078465b7957 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:27.487349 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985c8e1-d16a-4b4e-8754-a5e43c35b5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:27.488188 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c8e1-d16a-4b4e-8754-a5e43c35b5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.488567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c8e1-d16a-4b4e-8754-a5e43c35b5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.488946 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c8e1-d16a-4b4e-8754-a5e43c35b5a8 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.490480 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985c8e1-d16a-4b4e-8754-a5e43c35b5a8 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:27.492171 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985c8e1-d16a-4b4e-8754-a5e43c35b5a8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:27.494352 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 167/350] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:27 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:27.495654 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eee5aa-c66b-4cf0-8960-1fc4e4340b79 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.496042 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eee5aa-c66b-4cf0-8960-1fc4e4340b79 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.496417 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eee5aa-c66b-4cf0-8960-1fc4e4340b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.522365 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-60eee5aa-c66b-4cf0-8960-1fc4e4340b79 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:27.523673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-60eee5aa-c66b-4cf0-8960-1fc4e4340b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:27.530791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:27.531670 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.532092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.532640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.533674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] RBAC: Adding query filter params (group.id=e49329afffe341a995a2683f95608645, role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:27.534237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:27.536122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:27.543160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:27.543610 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 184/351] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:27 2022] GET /identity/v3/role_assignments?group.id=e49329afffe341a995a2683f95608645&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=06907320d4bc47b3ba3375df728a605a => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:27.545026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:27.606156 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 168/352] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:27.607196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eee5aa-c66b-4cf0-8960-1fc4e4340b79 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:27.665805 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820c02-3bad-46a7-b1af-decb689cba58 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:27.706790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84669ccb-c092-4e42-9bf7-f3c08a4c28d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:27.707864 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84669ccb-c092-4e42-9bf7-f3c08a4c28d4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.708348 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84669ccb-c092-4e42-9bf7-f3c08a4c28d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.708819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84669ccb-c092-4e42-9bf7-f3c08a4c28d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.722394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84669ccb-c092-4e42-9bf7-f3c08a4c28d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-84669ccb-c092-4e42-9bf7-f3c08a4c28d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:27.728947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:27.729865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:27.730740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:27.733558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 185/353] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:27.735233 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3324f-5bac-45e5-a4bc-1ed8cef2e99a None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.735755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3324f-5bac-45e5-a4bc-1ed8cef2e99a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.736240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3324f-5bac-45e5-a4bc-1ed8cef2e99a None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.737962 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 186/354] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:27.739029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3324f-5bac-45e5-a4bc-1ed8cef2e99a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:27.746457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eee5aa-c66b-4cf0-8960-1fc4e4340b79 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:27.762156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:27.789131 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc9c64-5bd7-4bcb-a42f-d7ffc52da003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:27.790126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc9c64-5bd7-4bcb-a42f-d7ffc52da003 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.790126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc9c64-5bd7-4bcb-a42f-d7ffc52da003 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.790398 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc9c64-5bd7-4bcb-a42f-d7ffc52da003 None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.791643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc9c64-5bd7-4bcb-a42f-d7ffc52da003 None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:27.793124 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc9c64-5bd7-4bcb-a42f-d7ffc52da003 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:27.795186 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 169/355] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:27 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:27.795939 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc9c64-5bd7-4bcb-a42f-d7ffc52da003 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:27.825778 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:27.859029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3324f-5bac-45e5-a4bc-1ed8cef2e99a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:27.904225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb379d86-5633-412d-9aa7-341211d80630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:27.904997 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb379d86-5633-412d-9aa7-341211d80630 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.905196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb379d86-5633-412d-9aa7-341211d80630 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.905468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb379d86-5633-412d-9aa7-341211d80630 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.923858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb379d86-5633-412d-9aa7-341211d80630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-bb379d86-5633-412d-9aa7-341211d80630 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:27.929372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:27.930357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:27.931161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 187/356] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:27.934045 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c9d7a-dcf6-41d8-9972-5756633ffdbc None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:27.934045 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c9d7a-dcf6-41d8-9972-5756633ffdbc None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:27.934045 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c9d7a-dcf6-41d8-9972-5756633ffdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:27.944262 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc9c64-5bd7-4bcb-a42f-d7ffc52da003 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:27.960607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-bd4c9d7a-dcf6-41d8-9972-5756633ffdbc None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:27.961844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bd4c9d7a-dcf6-41d8-9972-5756633ffdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:28.004864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:28.005532 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.005814 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.006087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.006970 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:28.007234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:28.008670 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:28.021219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 170/357] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:28.027245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:28.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 188/358] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:28.090290 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c9d7a-dcf6-41d8-9972-5756633ffdbc None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:28.103898 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:28.145586 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:28.172314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579957a-e9e6-4140-bddf-85c94a37dff2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:28.216239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:28.218061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.218926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.219405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.221666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] RBAC: Adding query filter params (group.id=e49329afffe341a995a2683f95608645, role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:28.223235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:28.225300 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:28.233304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c9d7a-dcf6-41d8-9972-5756633ffdbc None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:28.236346 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 171/359] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/role_assignments?group.id=e49329afffe341a995a2683f95608645&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:28.237708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:28.308207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d8220-dbb7-4dcb-b8c2-593f036cc32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:28.308902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d8220-dbb7-4dcb-b8c2-593f036cc32c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.309148 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d8220-dbb7-4dcb-b8c2-593f036cc32c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.309415 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d8220-dbb7-4dcb-b8c2-593f036cc32c None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.310331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d8220-dbb7-4dcb-b8c2-593f036cc32c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:28.310641 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d8220-dbb7-4dcb-b8c2-593f036cc32c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:28.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d8220-dbb7-4dcb-b8c2-593f036cc32c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:28.322220 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 189/360] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:28.323427 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996a254-921b-4a95-9a82-9e67b8bc2f40 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.323716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996a254-921b-4a95-9a82-9e67b8bc2f40 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.323988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996a254-921b-4a95-9a82-9e67b8bc2f40 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.326008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 190/361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:28.326899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996a254-921b-4a95-9a82-9e67b8bc2f40 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:28.381727 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f4c3b-ce88-4ba3-bc8b-a5a72ccb236a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:28.412304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:28.427893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5225f-e6ef-4412-83b0-ef3f3cb093f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:28.428597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5225f-e6ef-4412-83b0-ef3f3cb093f0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.428884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5225f-e6ef-4412-83b0-ef3f3cb093f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.429166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5225f-e6ef-4412-83b0-ef3f3cb093f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.448521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:28.461086 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996a254-921b-4a95-9a82-9e67b8bc2f40 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:28.501901 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200597-f661-418f-a8ff-d3217d683c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:28.502616 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200597-f661-418f-a8ff-d3217d683c68 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.502897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200597-f661-418f-a8ff-d3217d683c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.503170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200597-f661-418f-a8ff-d3217d683c68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.509239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5225f-e6ef-4412-83b0-ef3f3cb093f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:28.510872 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5225f-e6ef-4412-83b0-ef3f3cb093f0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:28.514840 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72200597-f661-418f-a8ff-d3217d683c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-72200597-f661-418f-a8ff-d3217d683c68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:28.520057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:28.520919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:28.521732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 191/362] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:28.523176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d1acc-a850-41c7-afa5-e831c30f8b34 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.523176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d1acc-a850-41c7-afa5-e831c30f8b34 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.523176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d1acc-a850-41c7-afa5-e831c30f8b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.542799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-177d1acc-a850-41c7-afa5-e831c30f8b34 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:28.544044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-177d1acc-a850-41c7-afa5-e831c30f8b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:28.619850 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 192/363] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:28.620756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d1acc-a850-41c7-afa5-e831c30f8b34 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:28.624204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 172/364] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:28.625067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5225f-e6ef-4412-83b0-ef3f3cb093f0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:28.680925 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:28.691855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:28.746700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d1acc-a850-41c7-afa5-e831c30f8b34 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:28.746929 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5225f-e6ef-4412-83b0-ef3f3cb093f0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:28.788342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c41ce3-c663-4744-820a-84ae34f59ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:28.789019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c41ce3-c663-4744-820a-84ae34f59ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.789307 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c41ce3-c663-4744-820a-84ae34f59ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.789585 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c41ce3-c663-4744-820a-84ae34f59ae3 None admin] PATH_INFO: `/v3/users/ae3f3c4436b844bb9d6914f03205baa5` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.791176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c41ce3-c663-4744-820a-84ae34f59ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=ae3f3c4436b844bb9d6914f03205baa5)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:28.792807 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c41ce3-c663-4744-820a-84ae34f59ae3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:28.792918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:28.793521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.793833 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.794792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 193/365] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/users/ae3f3c4436b844bb9d6914f03205baa5 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:28.795373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c41ce3-c663-4744-820a-84ae34f59ae3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:28.797326 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.798194 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:28.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:28.800006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:28.832250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 173/366] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:28.833198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:28.920958 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c41ce3-c663-4744-820a-84ae34f59ae3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:28.955072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:28.970192 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369956b-2e50-4e7a-a88d-ae6e447bca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:28.970952 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369956b-2e50-4e7a-a88d-ae6e447bca96 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:28.971237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369956b-2e50-4e7a-a88d-ae6e447bca96 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:28.971508 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369956b-2e50-4e7a-a88d-ae6e447bca96 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:28.972797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369956b-2e50-4e7a-a88d-ae6e447bca96 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:28.974325 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369956b-2e50-4e7a-a88d-ae6e447bca96 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:28.976054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 194/367] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:28.976844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369956b-2e50-4e7a-a88d-ae6e447bca96 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:29.025389 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb572111-9bc4-48b3-be30-8873cc70c4d9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:29.037092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:29.073371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d77032-8e24-49bf-9563-2538d9e0d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:29.074891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d77032-8e24-49bf-9563-2538d9e0d0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:29.075366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d77032-8e24-49bf-9563-2538d9e0d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:29.075843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d77032-8e24-49bf-9563-2538d9e0d0c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:29.090844 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d77032-8e24-49bf-9563-2538d9e0d0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-66d77032-8e24-49bf-9563-2538d9e0d0c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:29.097812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:29.098732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:29.099477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:29.106358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 174/368] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:29.107242 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d77032-8e24-49bf-9563-2538d9e0d0c9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:29.130547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369956b-2e50-4e7a-a88d-ae6e447bca96 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:29.175477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:29.176499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:29.177421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:29.177899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:29.179225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] RBAC: Adding query filter params (role.id=becb35407660454b9982c279fd4ee506, user.id=ae3f3c4436b844bb9d6914f03205baa5, scope.system=all) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:29.179755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:29.181571 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:29.197993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 195/369] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:01:28 2022] GET /identity/v3/role_assignments?user.id=ae3f3c4436b844bb9d6914f03205baa5&scope.system=all&role.id=becb35407660454b9982c279fd4ee506 => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:29.199448 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:29.267433 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:29.301372 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d77032-8e24-49bf-9563-2538d9e0d0c9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:29.331630 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c82c4-3dd0-4d82-8f5e-963e7b95d84a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:29.346571 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5e8e6-2d91-496e-a3ad-dcb479823f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:29.347545 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5e8e6-2d91-496e-a3ad-dcb479823f55 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:29.347975 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5e8e6-2d91-496e-a3ad-dcb479823f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:29.348399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5e8e6-2d91-496e-a3ad-dcb479823f55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:29.378882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778672d-1a50-4873-b40f-ba74ee0e4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:29.379781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778672d-1a50-4873-b40f-ba74ee0e4995 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:29.380345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778672d-1a50-4873-b40f-ba74ee0e4995 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:29.380736 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778672d-1a50-4873-b40f-ba74ee0e4995 None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:29.382723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778672d-1a50-4873-b40f-ba74ee0e4995 None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:29.384513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778672d-1a50-4873-b40f-ba74ee0e4995 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:29.387652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 196/370] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:29 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:29.389030 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778672d-1a50-4873-b40f-ba74ee0e4995 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:29.391915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:29.435871 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5e8e6-2d91-496e-a3ad-dcb479823f55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:29.437700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5e8e6-2d91-496e-a3ad-dcb479823f55 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:29.540074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778672d-1a50-4873-b40f-ba74ee0e4995 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:29.570874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:29.600091 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:29.602218 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:29.602988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:29.603583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:29.604668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:29.605198 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:29.607201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:29.615986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 175/371] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:29.617289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5e8e6-2d91-496e-a3ad-dcb479823f55 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:29.622415 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 197/372] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:29.626248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:29.712846 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:29.760250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5e8e6-2d91-496e-a3ad-dcb479823f55 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:29.784096 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a9f0c-d01e-49f7-a558-40ed7af39de5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:29.804595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:29.805474 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:29.805886 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:29.806380 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:29.807336 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] RBAC: Adding query filter params (group.id=e49329afffe341a995a2683f95608645, role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=232c57cd96c243f6b181496002d8a724) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:29.807800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:29.809399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:29.816084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 176/373] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:29 2022] GET /identity/v3/role_assignments?group.id=e49329afffe341a995a2683f95608645&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=06907320d4bc47b3ba3375df728a605a => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:29.817278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:29.849482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df49bb5-7f38-4a2c-af8c-7e4d596a5d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:29.850613 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df49bb5-7f38-4a2c-af8c-7e4d596a5d94 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:29.851197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df49bb5-7f38-4a2c-af8c-7e4d596a5d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:29.851727 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df49bb5-7f38-4a2c-af8c-7e4d596a5d94 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:29.853586 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df49bb5-7f38-4a2c-af8c-7e4d596a5d94 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:29.855813 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df49bb5-7f38-4a2c-af8c-7e4d596a5d94 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:29.858515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 198/374] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:29 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:29.867538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df49bb5-7f38-4a2c-af8c-7e4d596a5d94 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:29.952018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:29.955755 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abdb5d-b2e5-4a4d-b199-a3ae1b43d031 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:29.995979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:30.005604 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd10b6-0e9f-4bec-83d8-85d99e86f5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:30.006336 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd10b6-0e9f-4bec-83d8-85d99e86f5ec None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.006625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd10b6-0e9f-4bec-83d8-85d99e86f5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.006895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd10b6-0e9f-4bec-83d8-85d99e86f5ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.039970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df49bb5-7f38-4a2c-af8c-7e4d596a5d94 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:30.090602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ee34a-2422-4fc6-930f-57f101693905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:30.091726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ee34a-2422-4fc6-930f-57f101693905 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.092269 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ee34a-2422-4fc6-930f-57f101693905 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.092702 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ee34a-2422-4fc6-930f-57f101693905 None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.095201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ee34a-2422-4fc6-930f-57f101693905 None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:30.095539 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd10b6-0e9f-4bec-83d8-85d99e86f5ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:30.097035 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd10b6-0e9f-4bec-83d8-85d99e86f5ec None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:30.097322 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ee34a-2422-4fc6-930f-57f101693905 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:30.099836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 199/375] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:29 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:30.108652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ee34a-2422-4fc6-930f-57f101693905 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:30.226251 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 177/376] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:30.228422 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8641ab1-0ffa-418b-8533-4fd5617853f4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.230535 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8641ab1-0ffa-418b-8533-4fd5617853f4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.230966 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8641ab1-0ffa-418b-8533-4fd5617853f4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.232974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 178/377] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:30.233852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8641ab1-0ffa-418b-8533-4fd5617853f4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:30.275592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:30.304396 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ee34a-2422-4fc6-930f-57f101693905 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:30.334226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:30.356252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8641ab1-0ffa-418b-8533-4fd5617853f4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:30.362571 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a6aff-6db1-48d3-8f46-24dd21cf7dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:30.363627 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a6aff-6db1-48d3-8f46-24dd21cf7dde None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.364076 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a6aff-6db1-48d3-8f46-24dd21cf7dde None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.364549 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a6aff-6db1-48d3-8f46-24dd21cf7dde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.365569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a6aff-6db1-48d3-8f46-24dd21cf7dde None admin] RBAC: Adding query filter params (group.id=e49329afffe341a995a2683f95608645, role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=232c57cd96c243f6b181496002d8a724) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:30.366178 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a6aff-6db1-48d3-8f46-24dd21cf7dde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:30.367868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a6aff-6db1-48d3-8f46-24dd21cf7dde None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:30.376494 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 200/378] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:30 2022] GET /identity/v3/role_assignments?group.id=e49329afffe341a995a2683f95608645&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 260 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:30.378678 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435411c7-a775-4927-9b15-cf86133ebe9e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.379244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435411c7-a775-4927-9b15-cf86133ebe9e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.379820 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435411c7-a775-4927-9b15-cf86133ebe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.407823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-435411c7-a775-4927-9b15-cf86133ebe9e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:30.409716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-435411c7-a775-4927-9b15-cf86133ebe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:30.419157 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91788989-2603-4aea-a46a-5080b97ff79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:30.425338 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91788989-2603-4aea-a46a-5080b97ff79c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.427102 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91788989-2603-4aea-a46a-5080b97ff79c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.428689 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91788989-2603-4aea-a46a-5080b97ff79c None admin] PATH_INFO: `/v3/projects/bed3d6665f7c40158ba248253deda4a1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.432033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91788989-2603-4aea-a46a-5080b97ff79c None admin] RBAC: Authorizing `identity:get_project(project_id=bed3d6665f7c40158ba248253deda4a1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:30.435985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91788989-2603-4aea-a46a-5080b97ff79c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:30.438956 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 179/379] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:30 2022] GET /identity/v3/projects/bed3d6665f7c40158ba248253deda4a1 => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:30.450165 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91788989-2603-4aea-a46a-5080b97ff79c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:30.580645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 201/380] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:30.632883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435411c7-a775-4927-9b15-cf86133ebe9e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:30.665093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:30.668477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91788989-2603-4aea-a46a-5080b97ff79c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:30.735965 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e4b10-f628-4dec-87fd-a7dc0e66932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:30.737175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e4b10-f628-4dec-87fd-a7dc0e66932c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.737676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e4b10-f628-4dec-87fd-a7dc0e66932c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.738280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e4b10-f628-4dec-87fd-a7dc0e66932c None admin] PATH_INFO: `/v3/groups/f1a440ea9f8a4888a558b7add9b8fd58` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.745655 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e4b10-f628-4dec-87fd-a7dc0e66932c None admin] RBAC: Authorizing `identity:get_group(group_id=f1a440ea9f8a4888a558b7add9b8fd58)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:30.747677 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e4b10-f628-4dec-87fd-a7dc0e66932c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:30.750862 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 180/381] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:30 2022] GET /identity/v3/groups/f1a440ea9f8a4888a558b7add9b8fd58 => generated 229 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:30.760438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e4b10-f628-4dec-87fd-a7dc0e66932c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:30.897202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435411c7-a775-4927-9b15-cf86133ebe9e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:30.930133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:30.940993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e4b10-f628-4dec-87fd-a7dc0e66932c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:30.977507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75706ac7-682e-4dfa-a5a1-578f2c795631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:30.978691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75706ac7-682e-4dfa-a5a1-578f2c795631 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.978980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75706ac7-682e-4dfa-a5a1-578f2c795631 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.979281 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75706ac7-682e-4dfa-a5a1-578f2c795631 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.991412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accd057-55bc-4e38-97ba-39bad123aeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:30.992463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accd057-55bc-4e38-97ba-39bad123aeeb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:30.992948 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accd057-55bc-4e38-97ba-39bad123aeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:30.993507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accd057-55bc-4e38-97ba-39bad123aeeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:30.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75706ac7-682e-4dfa-a5a1-578f2c795631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:30.994867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accd057-55bc-4e38-97ba-39bad123aeeb None admin] RBAC: Adding query filter params (group.id=f1a440ea9f8a4888a558b7add9b8fd58, role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=bed3d6665f7c40158ba248253deda4a1) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:30.995410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accd057-55bc-4e38-97ba-39bad123aeeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:30.997145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accd057-55bc-4e38-97ba-39bad123aeeb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-75706ac7-682e-4dfa-a5a1-578f2c795631 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:31.001860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:31.002990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:31.003980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 202/382] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:31.005239 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca05b0-f5b8-461d-9748-ec96ede6adf2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:31.005407 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca05b0-f5b8-461d-9748-ec96ede6adf2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:31.005671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca05b0-f5b8-461d-9748-ec96ede6adf2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:31.011523 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 203/383] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:31.013704 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 181/384] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:30 2022] GET /identity/v3/role_assignments?group.id=f1a440ea9f8a4888a558b7add9b8fd58&scope.project.id=bed3d6665f7c40158ba248253deda4a1&role.id=65cd37a73e4f498482fefabb8374006c => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:31.014996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca05b0-f5b8-461d-9748-ec96ede6adf2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:31.017291 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa7210-3ef1-40eb-ba71-1d2049631928 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:31.018507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa7210-3ef1-40eb-ba71-1d2049631928 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:31.021772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa7210-3ef1-40eb-ba71-1d2049631928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:31.052830 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bcaa7210-3ef1-40eb-ba71-1d2049631928 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:31.054359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bcaa7210-3ef1-40eb-ba71-1d2049631928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:31.085205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:31.153917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 182/385] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:31.183187 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa7210-3ef1-40eb-ba71-1d2049631928 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:31.192959 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca05b0-f5b8-461d-9748-ec96ede6adf2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:31.244015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:31.244943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:31.245369 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:31.245845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:31.246759 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:31.247295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:31.248871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:31.260836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 204/386] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:31.267579 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:31.310725 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa7210-3ef1-40eb-ba71-1d2049631928 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:31.311999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:31.364514 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9dee9-bf8d-4dc0-8e9d-b986b067f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:31.365248 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9dee9-bf8d-4dc0-8e9d-b986b067f483 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:31.365531 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9dee9-bf8d-4dc0-8e9d-b986b067f483 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:31.365750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9dee9-bf8d-4dc0-8e9d-b986b067f483 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:31.396693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9dee9-bf8d-4dc0-8e9d-b986b067f483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-77b9dee9-bf8d-4dc0-8e9d-b986b067f483 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:31.403712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:31.405087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:31.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 183/387] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:31.413353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9dee9-bf8d-4dc0-8e9d-b986b067f483 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:31.427103 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1afed-ed6f-448c-a664-75461e305464 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:31.436360 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:31.474020 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c41d4-0339-4971-aa50-d0fb598ddf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:31.475144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c41d4-0339-4971-aa50-d0fb598ddf9f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:31.475645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c41d4-0339-4971-aa50-d0fb598ddf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:31.476115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c41d4-0339-4971-aa50-d0fb598ddf9f None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:31.478231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c41d4-0339-4971-aa50-d0fb598ddf9f None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:31.480099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c41d4-0339-4971-aa50-d0fb598ddf9f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:31.482533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 205/388] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:31 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:31.489060 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c41d4-0339-4971-aa50-d0fb598ddf9f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:31.553353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9dee9-bf8d-4dc0-8e9d-b986b067f483 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:31.600563 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:31.601838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:31.601978 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:31.602249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:31.603083 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:31.603394 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:31.605054 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:31.618209 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c41d4-0339-4971-aa50-d0fb598ddf9f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:31.618433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 184/389] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:31.620568 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:31.624571 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:31.711795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c322-283f-4682-9bc6-cae1c440f4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:31.711795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c322-283f-4682-9bc6-cae1c440f4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:31.712071 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c322-283f-4682-9bc6-cae1c440f4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:31.712233 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c322-283f-4682-9bc6-cae1c440f4b7 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:31.716392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7c322-283f-4682-9bc6-cae1c440f4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:31.717790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7c322-283f-4682-9bc6-cae1c440f4b7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:31.720097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 206/390] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:31 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:31.751203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c322-283f-4682-9bc6-cae1c440f4b7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:31.776108 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:31.803053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba862-f683-427c-b323-87233e9f6bc5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:31.848518 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db10162-81c1-4ab9-b07e-900b5b237139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:31.849201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db10162-81c1-4ab9-b07e-900b5b237139 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:31.849484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db10162-81c1-4ab9-b07e-900b5b237139 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:31.849763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db10162-81c1-4ab9-b07e-900b5b237139 None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:31.851013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db10162-81c1-4ab9-b07e-900b5b237139 None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:31.853704 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db10162-81c1-4ab9-b07e-900b5b237139 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:31.856397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 185/391] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:31 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:31.862813 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db10162-81c1-4ab9-b07e-900b5b237139 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:32.002006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db10162-81c1-4ab9-b07e-900b5b237139 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:32.050505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498d036-80c3-48ca-8d84-fc0e0f99306b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:32.052034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498d036-80c3-48ca-8d84-fc0e0f99306b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:32.052373 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498d036-80c3-48ca-8d84-fc0e0f99306b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:32.052671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498d036-80c3-48ca-8d84-fc0e0f99306b None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:32.054176 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498d036-80c3-48ca-8d84-fc0e0f99306b None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:32.056272 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498d036-80c3-48ca-8d84-fc0e0f99306b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:32.058731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 186/392] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:31 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:32.064825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498d036-80c3-48ca-8d84-fc0e0f99306b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:32.068406 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c322-283f-4682-9bc6-cae1c440f4b7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:32.090607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:32.194990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e23394-1f26-4473-836a-2152a09b50c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:32.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e23394-1f26-4473-836a-2152a09b50c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:32.196149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e23394-1f26-4473-836a-2152a09b50c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:32.196526 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e23394-1f26-4473-836a-2152a09b50c6 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/groups/e49329afffe341a995a2683f95608645/roles/06907320d4bc47b3ba3375df728a605a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:32.199341 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e23394-1f26-4473-836a-2152a09b50c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6, group_id=e49329afffe341a995a2683f95608645, role_id=06907320d4bc47b3ba3375df728a605a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:32.200085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498d036-80c3-48ca-8d84-fc0e0f99306b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:32.201608 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e23394-1f26-4473-836a-2152a09b50c6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:32.226685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 207/393] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:01:31 2022] PUT /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/groups/e49329afffe341a995a2683f95608645/roles/06907320d4bc47b3ba3375df728a605a => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:32.249944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3150159-aadf-46a0-a571-24bda119b5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:32.255710 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150159-aadf-46a0-a571-24bda119b5f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:32.256118 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150159-aadf-46a0-a571-24bda119b5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:32.256597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150159-aadf-46a0-a571-24bda119b5f0 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/groups/e49329afffe341a995a2683f95608645/roles/94aa0ce403304ce8a21ef20db556d534` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:32.260922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3150159-aadf-46a0-a571-24bda119b5f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6, group_id=e49329afffe341a995a2683f95608645, role_id=94aa0ce403304ce8a21ef20db556d534)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:32.262786 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3150159-aadf-46a0-a571-24bda119b5f0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:32.281630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 187/394] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:01:32 2022] PUT /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6/groups/e49329afffe341a995a2683f95608645/roles/94aa0ce403304ce8a21ef20db556d534 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:33.103486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eb0eb-345a-429d-a82e-2cff814a34a7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:33.103982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eb0eb-345a-429d-a82e-2cff814a34a7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:33.103982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eb0eb-345a-429d-a82e-2cff814a34a7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:33.105649 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 208/395] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:33.110866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02037dc9-4c56-46a4-992e-3330b3c52da9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:33.111308 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02037dc9-4c56-46a4-992e-3330b3c52da9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:33.111406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02037dc9-4c56-46a4-992e-3330b3c52da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:33.133936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-02037dc9-4c56-46a4-992e-3330b3c52da9 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:33.135162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-02037dc9-4c56-46a4-992e-3330b3c52da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:33.222024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 188/396] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:33.251828 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5eb0eb-345a-429d-a82e-2cff814a34a7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:33.289082 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:33.372864 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5eb0eb-345a-429d-a82e-2cff814a34a7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:33.386353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7150a-bb9a-41d5-bf59-6150bcfff94d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:33.386967 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7150a-bb9a-41d5-bf59-6150bcfff94d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:33.387461 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7150a-bb9a-41d5-bf59-6150bcfff94d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:33.389277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 189/397] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:33.399665 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10265805-7d54-4d43-90b0-d2bdde2566b4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:33.400259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10265805-7d54-4d43-90b0-d2bdde2566b4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:33.400756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10265805-7d54-4d43-90b0-d2bdde2566b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:33.418385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4777-298f-4d08-9fef-1acd0db56f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:33.419065 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4777-298f-4d08-9fef-1acd0db56f1d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:33.419339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4777-298f-4d08-9fef-1acd0db56f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:33.419605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4777-298f-4d08-9fef-1acd0db56f1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:33.430353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:33.443498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-10265805-7d54-4d43-90b0-d2bdde2566b4 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:33.444335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c4777-298f-4d08-9fef-1acd0db56f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:33.444703 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-10265805-7d54-4d43-90b0-d2bdde2566b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-be9c4777-298f-4d08-9fef-1acd0db56f1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:33.449405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:33.450700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:33.451551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 209/398] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:33.459337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4777-298f-4d08-9fef-1acd0db56f1d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:33.588769 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 190/399] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:33.605600 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:33.615215 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4777-298f-4d08-9fef-1acd0db56f1d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:33.622975 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10265805-7d54-4d43-90b0-d2bdde2566b4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:33.655411 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:33.656047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:33.656327 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:33.656600 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:33.657331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:33.657638 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:33.661236 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:33.670434 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 210/400] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:33.676198 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:33.793144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6127-cdf1-4100-87a8-7582d240b54f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:33.819944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10265805-7d54-4d43-90b0-d2bdde2566b4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:33.829698 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8d391-a3e0-42ed-acf2-8fc51e9f96ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:33.830437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8d391-a3e0-42ed-acf2-8fc51e9f96ea None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:33.830714 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8d391-a3e0-42ed-acf2-8fc51e9f96ea None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:33.830989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8d391-a3e0-42ed-acf2-8fc51e9f96ea None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:33.832531 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8d391-a3e0-42ed-acf2-8fc51e9f96ea None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:33.834066 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8d391-a3e0-42ed-acf2-8fc51e9f96ea None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:33.836083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 211/401] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:33 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:33.843299 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8d391-a3e0-42ed-acf2-8fc51e9f96ea None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:33.879144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:33.896257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c72f8-094a-429f-827f-27ee5f2abdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:33.897178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c72f8-094a-429f-827f-27ee5f2abdfd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:33.897620 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c72f8-094a-429f-827f-27ee5f2abdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:33.898209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c72f8-094a-429f-827f-27ee5f2abdfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:33.903778 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:33.923861 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c72f8-094a-429f-827f-27ee5f2abdfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bc8c72f8-094a-429f-827f-27ee5f2abdfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:33.929687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:33.930761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:33.931770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:33.942565 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 191/402] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:33.948719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c72f8-094a-429f-827f-27ee5f2abdfd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:33.974723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8d391-a3e0-42ed-acf2-8fc51e9f96ea None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:34.019745 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed7d90-86a4-4718-a4b7-e6db2c5b03e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:34.020417 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed7d90-86a4-4718-a4b7-e6db2c5b03e6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.020694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed7d90-86a4-4718-a4b7-e6db2c5b03e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.020964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed7d90-86a4-4718-a4b7-e6db2c5b03e6 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.022561 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed7d90-86a4-4718-a4b7-e6db2c5b03e6 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:34.024044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed7d90-86a4-4718-a4b7-e6db2c5b03e6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:34.026335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 212/403] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:33 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:34.027826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe555dd3-aef3-4c8a-b797-f225d0e74d80 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.028107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe555dd3-aef3-4c8a-b797-f225d0e74d80 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.028376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe555dd3-aef3-4c8a-b797-f225d0e74d80 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.030242 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 213/404] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:34.031777 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe555dd3-aef3-4c8a-b797-f225d0e74d80 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:34.111638 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c72f8-094a-429f-827f-27ee5f2abdfd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:34.154596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611450-1091-49c7-abd3-be0e3d474675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:34.155412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611450-1091-49c7-abd3-be0e3d474675 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.155707 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611450-1091-49c7-abd3-be0e3d474675 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.155976 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611450-1091-49c7-abd3-be0e3d474675 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.156700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00611450-1091-49c7-abd3-be0e3d474675 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:34.157004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00611450-1091-49c7-abd3-be0e3d474675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:34.158936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00611450-1091-49c7-abd3-be0e3d474675 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:34.165496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:34.172844 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 192/405] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:34.177275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17852e26-8d2f-4f02-b2ef-8cfcc4d7027d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.178445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17852e26-8d2f-4f02-b2ef-8cfcc4d7027d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.178936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17852e26-8d2f-4f02-b2ef-8cfcc4d7027d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.186182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe555dd3-aef3-4c8a-b797-f225d0e74d80 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:34.198937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-17852e26-8d2f-4f02-b2ef-8cfcc4d7027d None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:34.200721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-17852e26-8d2f-4f02-b2ef-8cfcc4d7027d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:34.222428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5d847-dd0d-45cd-929f-70e5ab211894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:34.223119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5d847-dd0d-45cd-929f-70e5ab211894 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.223394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5d847-dd0d-45cd-929f-70e5ab211894 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.223663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5d847-dd0d-45cd-929f-70e5ab211894 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724/groups/e49329afffe341a995a2683f95608645/roles/94aa0ce403304ce8a21ef20db556d534` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.226122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5d847-dd0d-45cd-929f-70e5ab211894 None admin] RBAC: Authorizing `identity:create_grant(project_id=232c57cd96c243f6b181496002d8a724, group_id=e49329afffe341a995a2683f95608645, role_id=94aa0ce403304ce8a21ef20db556d534)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:34.227577 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5d847-dd0d-45cd-929f-70e5ab211894 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:34.244258 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 214/406] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:01:34 2022] PUT /identity/v3/projects/232c57cd96c243f6b181496002d8a724/groups/e49329afffe341a995a2683f95608645/roles/94aa0ce403304ce8a21ef20db556d534 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:01:34.245319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5d847-dd0d-45cd-929f-70e5ab211894 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:34.264757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:34.283788 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 193/407] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:34.308628 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17852e26-8d2f-4f02-b2ef-8cfcc4d7027d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:34.376588 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5d847-dd0d-45cd-929f-70e5ab211894 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:34.416904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a2268-9fa8-48f1-a23f-3cd70d601ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:34.417495 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a2268-9fa8-48f1-a23f-3cd70d601ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.417772 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a2268-9fa8-48f1-a23f-3cd70d601ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.418043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a2268-9fa8-48f1-a23f-3cd70d601ab5 None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.419383 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a2268-9fa8-48f1-a23f-3cd70d601ab5 None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:34.421487 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a2268-9fa8-48f1-a23f-3cd70d601ab5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:34.424172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 215/408] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:34 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:34.432857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a2268-9fa8-48f1-a23f-3cd70d601ab5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:34.474291 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:34.490411 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17852e26-8d2f-4f02-b2ef-8cfcc4d7027d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:34.527629 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ed21-76d2-4b9a-aecb-df77252ffe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:34.528296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ed21-76d2-4b9a-aecb-df77252ffe36 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.528570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ed21-76d2-4b9a-aecb-df77252ffe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.528841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ed21-76d2-4b9a-aecb-df77252ffe36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.539958 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3ed21-76d2-4b9a-aecb-df77252ffe36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1de3ed21-76d2-4b9a-aecb-df77252ffe36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:34.544928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:34.546888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:34.548730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 194/409] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:34.555296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ed21-76d2-4b9a-aecb-df77252ffe36 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:34.634134 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:34.702971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ed21-76d2-4b9a-aecb-df77252ffe36 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:34.705210 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a2268-9fa8-48f1-a23f-3cd70d601ab5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:34.779916 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:34.780562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.781094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.781664 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.782758 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:34.782981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9372-88e8-42a6-b386-615898ffcba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:34.783486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9372-88e8-42a6-b386-615898ffcba7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:34.783722 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:34.783830 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9372-88e8-42a6-b386-615898ffcba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:34.784029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9372-88e8-42a6-b386-615898ffcba7 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:34.785782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:34.791103 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af9372-88e8-42a6-b386-615898ffcba7 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:34.792624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af9372-88e8-42a6-b386-615898ffcba7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:34.794834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 216/410] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:34 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:34.796497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 195/411] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:34.804150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9372-88e8-42a6-b386-615898ffcba7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:34.805403 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:35.002569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:35.007699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e2870-3143-4378-aae0-2b7c04d2b26e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:35.019460 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:35.021258 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9372-88e8-42a6-b386-615898ffcba7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:35.054294 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ca2b6-7bd9-455f-b83d-d6a2a430ae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:35.054991 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ca2b6-7bd9-455f-b83d-d6a2a430ae93 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.055274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ca2b6-7bd9-455f-b83d-d6a2a430ae93 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.055545 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ca2b6-7bd9-455f-b83d-d6a2a430ae93 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724/groups/e49329afffe341a995a2683f95608645/roles/06907320d4bc47b3ba3375df728a605a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.058777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ca2b6-7bd9-455f-b83d-d6a2a430ae93 None admin] RBAC: Authorizing `identity:create_grant(project_id=232c57cd96c243f6b181496002d8a724, group_id=e49329afffe341a995a2683f95608645, role_id=06907320d4bc47b3ba3375df728a605a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:35.060246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ca2b6-7bd9-455f-b83d-d6a2a430ae93 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:35.061935 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2623d1-a31c-4b3c-9f49-dfcf798c351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:35.062635 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2623d1-a31c-4b3c-9f49-dfcf798c351d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.062910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2623d1-a31c-4b3c-9f49-dfcf798c351d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.063351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2623d1-a31c-4b3c-9f49-dfcf798c351d None admin] PATH_INFO: `/v3/groups/f1a440ea9f8a4888a558b7add9b8fd58` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.064682 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2623d1-a31c-4b3c-9f49-dfcf798c351d None admin] RBAC: Authorizing `identity:get_group(group_id=f1a440ea9f8a4888a558b7add9b8fd58)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:35.066431 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2623d1-a31c-4b3c-9f49-dfcf798c351d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:35.068190 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 217/412] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:34 2022] GET /identity/v3/groups/f1a440ea9f8a4888a558b7add9b8fd58 => generated 229 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:35.073924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ee50c-693f-45fa-bde5-952013032951 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.074932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ee50c-693f-45fa-bde5-952013032951 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.075398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ee50c-693f-45fa-bde5-952013032951 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.076184 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 196/413] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:01:34 2022] PUT /identity/v3/projects/232c57cd96c243f6b181496002d8a724/groups/e49329afffe341a995a2683f95608645/roles/06907320d4bc47b3ba3375df728a605a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:35.076762 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 218/414] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:35.077489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ee50c-693f-45fa-bde5-952013032951 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:35.077933 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082170-c855-419f-bc09-ef5b4d84469d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.078387 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082170-c855-419f-bc09-ef5b4d84469d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.078804 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082170-c855-419f-bc09-ef5b4d84469d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.080483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 197/415] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:35.082683 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b7378-7bf9-45d2-8d16-67c5eb591e6b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.083244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b7378-7bf9-45d2-8d16-67c5eb591e6b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.083703 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b7378-7bf9-45d2-8d16-67c5eb591e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.110279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-776b7378-7bf9-45d2-8d16-67c5eb591e6b None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:35.111922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-776b7378-7bf9-45d2-8d16-67c5eb591e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:35.218970 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 198/416] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:35.221170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38a51d-f4cd-497e-98ab-9ed38f273516 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.221819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38a51d-f4cd-497e-98ab-9ed38f273516 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.222159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38a51d-f4cd-497e-98ab-9ed38f273516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.227891 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ee50c-693f-45fa-bde5-952013032951 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:35.253800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-df38a51d-f4cd-497e-98ab-9ed38f273516 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:35.256000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-df38a51d-f4cd-497e-98ab-9ed38f273516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:35.281368 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4577-3c03-4e72-8f7a-2b37d2c16e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:35.282426 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4577-3c03-4e72-8f7a-2b37d2c16e71 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.282944 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4577-3c03-4e72-8f7a-2b37d2c16e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.283404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4577-3c03-4e72-8f7a-2b37d2c16e71 None admin] PATH_INFO: `/v3/projects/bed3d6665f7c40158ba248253deda4a1` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.285349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b4577-3c03-4e72-8f7a-2b37d2c16e71 None admin] RBAC: Authorizing `identity:get_project(project_id=bed3d6665f7c40158ba248253deda4a1)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:35.287213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b4577-3c03-4e72-8f7a-2b37d2c16e71 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:35.289366 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 219/417] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:35 2022] GET /identity/v3/projects/bed3d6665f7c40158ba248253deda4a1 => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:35.290707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4577-3c03-4e72-8f7a-2b37d2c16e71 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:35.317080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:35.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 199/418] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:35.343070 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38a51d-f4cd-497e-98ab-9ed38f273516 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:35.378557 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:35.429766 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4577-3c03-4e72-8f7a-2b37d2c16e71 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:35.454645 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38a51d-f4cd-497e-98ab-9ed38f273516 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:35.470149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0631abb-e169-4bad-8da6-4c97fbfd50c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:35.471136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0631abb-e169-4bad-8da6-4c97fbfd50c1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.471549 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0631abb-e169-4bad-8da6-4c97fbfd50c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.471992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0631abb-e169-4bad-8da6-4c97fbfd50c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.483806 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0631abb-e169-4bad-8da6-4c97fbfd50c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b0631abb-e169-4bad-8da6-4c97fbfd50c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:35.489484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:35.490375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:35.491122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:35.491696 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 220/419] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:35.492412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0631abb-e169-4bad-8da6-4c97fbfd50c1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:35.499117 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfda04e-b4f5-4d17-aee1-f8b61aee2ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:35.500155 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfda04e-b4f5-4d17-aee1-f8b61aee2ce8 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.500894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfda04e-b4f5-4d17-aee1-f8b61aee2ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.502274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfda04e-b4f5-4d17-aee1-f8b61aee2ce8 None admin] PATH_INFO: `/v3/projects/bed3d6665f7c40158ba248253deda4a1/groups/f1a440ea9f8a4888a558b7add9b8fd58/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.507446 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfda04e-b4f5-4d17-aee1-f8b61aee2ce8 None admin] RBAC: Authorizing `identity:create_grant(project_id=bed3d6665f7c40158ba248253deda4a1, group_id=f1a440ea9f8a4888a558b7add9b8fd58, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:35.510006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfda04e-b4f5-4d17-aee1-f8b61aee2ce8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:35.530662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 200/420] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:01:35 2022] PUT /identity/v3/projects/bed3d6665f7c40158ba248253deda4a1/groups/f1a440ea9f8a4888a558b7add9b8fd58/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:35.532897 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfda04e-b4f5-4d17-aee1-f8b61aee2ce8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:35.642468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:35.685114 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0631abb-e169-4bad-8da6-4c97fbfd50c1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:35.705741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:35.727538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfda04e-b4f5-4d17-aee1-f8b61aee2ce8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:35.756224 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c4968-0e4f-4099-8c4c-d49b8bec1188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:35.757265 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c4968-0e4f-4099-8c4c-d49b8bec1188 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.758047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c4968-0e4f-4099-8c4c-d49b8bec1188 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.758550 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c4968-0e4f-4099-8c4c-d49b8bec1188 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.772868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c4968-0e4f-4099-8c4c-d49b8bec1188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:35.773063 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:35.773532 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:35.773812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:35.774084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:35.774863 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:35.775171 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:35.776558 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ad6c4968-0e4f-4099-8c4c-d49b8bec1188 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:35.780567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:35.781499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:35.782355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:35.783881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 221/421] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:35.790663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c4968-0e4f-4099-8c4c-d49b8bec1188 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:35.791952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 201/422] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:35.806067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:35.966858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914029-91da-4c24-a01d-7f56dd621386 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:35.991304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c4968-0e4f-4099-8c4c-d49b8bec1188 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:36.016475 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993216a2-c09a-45e5-a6a2-fe9d539fa922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:36.017464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993216a2-c09a-45e5-a6a2-fe9d539fa922 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.017937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993216a2-c09a-45e5-a6a2-fe9d539fa922 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.018434 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993216a2-c09a-45e5-a6a2-fe9d539fa922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.054042 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:36.067380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:36.068018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.068304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.068580 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.069319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:36.069629 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:36.071059 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:36.089857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 222/423] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:36.096080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:36.106345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:36.133431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993216a2-c09a-45e5-a6a2-fe9d539fa922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:36.135701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993216a2-c09a-45e5-a6a2-fe9d539fa922 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:36.265857 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 202/424] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:36.271958 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993216a2-c09a-45e5-a6a2-fe9d539fa922 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:36.316198 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a2791-257d-4e78-b3ad-a45d8b8f63d2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:36.357256 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a0373-8c88-4935-a7d0-fdf3bc050c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:36.358301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a0373-8c88-4935-a7d0-fdf3bc050c8e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.358771 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a0373-8c88-4935-a7d0-fdf3bc050c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.359224 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a0373-8c88-4935-a7d0-fdf3bc050c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.406404 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:36.436207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a0373-8c88-4935-a7d0-fdf3bc050c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:36.438253 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a0373-8c88-4935-a7d0-fdf3bc050c8e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:36.440837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:36.522169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993216a2-c09a-45e5-a6a2-fe9d539fa922 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:36.554479 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 223/425] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:01:36.556123 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51070a94-ebbf-4ee4-b184-fe6344814543 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.556507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51070a94-ebbf-4ee4-b184-fe6344814543 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.556877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51070a94-ebbf-4ee4-b184-fe6344814543 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.562295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 224/426] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:36.562295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51070a94-ebbf-4ee4-b184-fe6344814543 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:36.576379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b90b6-429b-4004-980f-16d71b48bef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:36.577646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b90b6-429b-4004-980f-16d71b48bef3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.578181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b90b6-429b-4004-980f-16d71b48bef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.578881 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b90b6-429b-4004-980f-16d71b48bef3 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.580842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b90b6-429b-4004-980f-16d71b48bef3 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:36.582911 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b90b6-429b-4004-980f-16d71b48bef3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:36.585469 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 203/427] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:36 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:36.586871 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6112fd-7b61-46df-a9e8-b1e5e837774c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.587301 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6112fd-7b61-46df-a9e8-b1e5e837774c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.587719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6112fd-7b61-46df-a9e8-b1e5e837774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.611581 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cb6112fd-7b61-46df-a9e8-b1e5e837774c None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:36.613178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cb6112fd-7b61-46df-a9e8-b1e5e837774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:36.690094 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51070a94-ebbf-4ee4-b184-fe6344814543 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:36.692128 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:36.714129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 204/428] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:36.714129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6112fd-7b61-46df-a9e8-b1e5e837774c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:36.754556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500073de-15a8-4466-a7de-344f4738fcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:36.755233 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500073de-15a8-4466-a7de-344f4738fcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.755516 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500073de-15a8-4466-a7de-344f4738fcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.755793 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500073de-15a8-4466-a7de-344f4738fcd0 None admin] PATH_INFO: `/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.757688 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500073de-15a8-4466-a7de-344f4738fcd0 None admin] RBAC: Authorizing `identity:get_project(project_id=045abcc1dcee4aae9d1d4e4106e5e3c6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:36.759520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500073de-15a8-4466-a7de-344f4738fcd0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:36.762500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 225/429] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:36 2022] GET /identity/v3/projects/045abcc1dcee4aae9d1d4e4106e5e3c6 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:36.763612 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500073de-15a8-4466-a7de-344f4738fcd0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:36.785926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:36.850818 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6112fd-7b61-46df-a9e8-b1e5e837774c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:36.898316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500073de-15a8-4466-a7de-344f4738fcd0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:36.901018 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cb0f3-f5d1-4c7e-881b-9a04618beb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:36.902960 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cb0f3-f5d1-4c7e-881b-9a04618beb26 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.906363 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cb0f3-f5d1-4c7e-881b-9a04618beb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.907004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cb0f3-f5d1-4c7e-881b-9a04618beb26 None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.909037 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cb0f3-f5d1-4c7e-881b-9a04618beb26 None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:36.910810 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cb0f3-f5d1-4c7e-881b-9a04618beb26 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:36.913860 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 205/430] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:36 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:36.915247 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cb0f3-f5d1-4c7e-881b-9a04618beb26 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:36.948017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e6c39-03b9-4fde-82c7-c3c067f5a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:36.949449 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e6c39-03b9-4fde-82c7-c3c067f5a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:36.950294 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e6c39-03b9-4fde-82c7-c3c067f5a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:36.950759 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e6c39-03b9-4fde-82c7-c3c067f5a5e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:36.965745 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e6c39-03b9-4fde-82c7-c3c067f5a5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-9d3e6c39-03b9-4fde-82c7-c3c067f5a5e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:36.972399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:36.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:36.974212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:36.976744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 226/431] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:36.977833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e6c39-03b9-4fde-82c7-c3c067f5a5e9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:37.019294 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:37.052510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cb0f3-f5d1-4c7e-881b-9a04618beb26 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:37.075693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:37.100108 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f82b4-63fd-46a0-b2da-8a3b72ccad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:37.100780 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f82b4-63fd-46a0-b2da-8a3b72ccad6c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.101060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f82b4-63fd-46a0-b2da-8a3b72ccad6c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.101332 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f82b4-63fd-46a0-b2da-8a3b72ccad6c None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.102467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e6c39-03b9-4fde-82c7-c3c067f5a5e9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:37.102575 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f82b4-63fd-46a0-b2da-8a3b72ccad6c None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:37.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f82b4-63fd-46a0-b2da-8a3b72ccad6c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:37.111477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 206/432] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:36 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:37.112741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f82b4-63fd-46a0-b2da-8a3b72ccad6c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:37.143955 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:37.145107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.145617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.146079 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.147159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] RBAC: Adding query filter params (group.id=e49329afffe341a995a2683f95608645, role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:37.147665 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:37.149335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:37.156521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 227/433] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:36 2022] GET /identity/v3/role_assignments?group.id=e49329afffe341a995a2683f95608645&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=06907320d4bc47b3ba3375df728a605a => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:37.157819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:37.254436 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f82b4-63fd-46a0-b2da-8a3b72ccad6c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:37.271507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:37.281860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c6b33-2a2f-4420-959f-0f1e519c5871 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:37.308821 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:37.309610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.309890 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.310180 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.310909 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:37.311212 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:37.313330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:37.325378 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7053e33-75bb-494c-a0a5-193196370a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:37.326213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 207/434] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:37.326988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7053e33-75bb-494c-a0a5-193196370a2f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.327552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7053e33-75bb-494c-a0a5-193196370a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.328016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7053e33-75bb-494c-a0a5-193196370a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.329025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7053e33-75bb-494c-a0a5-193196370a2f None admin] RBAC: Adding query filter params (group.id=e49329afffe341a995a2683f95608645, role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:37.329551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7053e33-75bb-494c-a0a5-193196370a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:37.331511 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7053e33-75bb-494c-a0a5-193196370a2f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:37.332226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:37.347780 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 228/435] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:37 2022] GET /identity/v3/role_assignments?group.id=e49329afffe341a995a2683f95608645&scope.project.id=045abcc1dcee4aae9d1d4e4106e5e3c6&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:37.398485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2987d-6d56-42a5-abab-bec4615e7545 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.398799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2987d-6d56-42a5-abab-bec4615e7545 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.399049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2987d-6d56-42a5-abab-bec4615e7545 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.400740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 229/436] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:37.406583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf46d-b1a4-4c6d-b8ca-8f1d7c49b81f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.406962 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf46d-b1a4-4c6d-b8ca-8f1d7c49b81f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.407147 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf46d-b1a4-4c6d-b8ca-8f1d7c49b81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.413141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:37.430238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-606bf46d-b1a4-4c6d-b8ca-8f1d7c49b81f None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:37.431547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-606bf46d-b1a4-4c6d-b8ca-8f1d7c49b81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:37.476488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5543a7-d2b9-46a5-958f-26df65e521d4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:37.517880 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 230/437] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:37.526283 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab63a9e-9c77-436e-b06a-93c9e457bb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:37.527334 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63a9e-9c77-436e-b06a-93c9e457bb2c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.527829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63a9e-9c77-436e-b06a-93c9e457bb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.528292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63a9e-9c77-436e-b06a-93c9e457bb2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.544281 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bf46d-b1a4-4c6d-b8ca-8f1d7c49b81f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:37.640170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab63a9e-9c77-436e-b06a-93c9e457bb2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:37.641679 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab63a9e-9c77-436e-b06a-93c9e457bb2c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:37.672781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:37.693961 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bf46d-b1a4-4c6d-b8ca-8f1d7c49b81f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:37.705717 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:37.743749 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7726611-85bb-4f73-b28b-d8ad4655c359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:37.744409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7726611-85bb-4f73-b28b-d8ad4655c359 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.744697 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7726611-85bb-4f73-b28b-d8ad4655c359 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.744966 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7726611-85bb-4f73-b28b-d8ad4655c359 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.759049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7726611-85bb-4f73-b28b-d8ad4655c359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e7726611-85bb-4f73-b28b-d8ad4655c359 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:37.764621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:37.765502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:37.766380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 231/438] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:37.772148 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7726611-85bb-4f73-b28b-d8ad4655c359 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:37.787059 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 208/439] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:37.793027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab63a9e-9c77-436e-b06a-93c9e457bb2c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:37.905160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7726611-85bb-4f73-b28b-d8ad4655c359 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:37.935191 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab63a9e-9c77-436e-b06a-93c9e457bb2c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:37.947553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb646f-23bf-4bfb-b9c8-18962cfd9d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:37.948293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb646f-23bf-4bfb-b9c8-18962cfd9d2c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.948632 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb646f-23bf-4bfb-b9c8-18962cfd9d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.948906 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb646f-23bf-4bfb-b9c8-18962cfd9d2c None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.950023 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb646f-23bf-4bfb-b9c8-18962cfd9d2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:37.950419 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb646f-23bf-4bfb-b9c8-18962cfd9d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:37.951862 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb646f-23bf-4bfb-b9c8-18962cfd9d2c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:37.962155 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 232/440] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:01:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:37.964715 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe88fe-d105-44eb-ab0a-61d6e13d49a2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.964997 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe88fe-d105-44eb-ab0a-61d6e13d49a2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.965268 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe88fe-d105-44eb-ab0a-61d6e13d49a2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.967899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 233/441] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:37.968705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe88fe-d105-44eb-ab0a-61d6e13d49a2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:37.983895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:37.988724 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d57163-d57f-4efc-b176-a1bbb5f595b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:37.989334 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57163-d57f-4efc-b176-a1bbb5f595b3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.989641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57163-d57f-4efc-b176-a1bbb5f595b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.989915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57163-d57f-4efc-b176-a1bbb5f595b3 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:37.991706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d57163-d57f-4efc-b176-a1bbb5f595b3 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:37.993207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d57163-d57f-4efc-b176-a1bbb5f595b3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:37.995371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 209/442] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:37 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:37.996643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55756034-e7a3-493f-87b7-655281b2cefe None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:37.996926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55756034-e7a3-493f-87b7-655281b2cefe None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:37.997198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55756034-e7a3-493f-87b7-655281b2cefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.019337 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-55756034-e7a3-493f-87b7-655281b2cefe None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:38.020563 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-55756034-e7a3-493f-87b7-655281b2cefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:38.036237 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:38.101559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 210/443] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:38.102572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55756034-e7a3-493f-87b7-655281b2cefe None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:38.105767 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe88fe-d105-44eb-ab0a-61d6e13d49a2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.142517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c188107-ad19-4e6d-abf5-0cb93e532685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.143152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c188107-ad19-4e6d-abf5-0cb93e532685 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.143426 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c188107-ad19-4e6d-abf5-0cb93e532685 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.143733 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c188107-ad19-4e6d-abf5-0cb93e532685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.208678 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55756034-e7a3-493f-87b7-655281b2cefe None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.219059 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c188107-ad19-4e6d-abf5-0cb93e532685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:38.220550 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c188107-ad19-4e6d-abf5-0cb93e532685 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:38.232381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:38.245247 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd65273-dd49-49ca-b832-81593ae9bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.245908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd65273-dd49-49ca-b832-81593ae9bd3a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.246221 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd65273-dd49-49ca-b832-81593ae9bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.246497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd65273-dd49-49ca-b832-81593ae9bd3a None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.247657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd65273-dd49-49ca-b832-81593ae9bd3a None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:38.249106 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd65273-dd49-49ca-b832-81593ae9bd3a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:38.250778 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 211/444] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:38.251531 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd65273-dd49-49ca-b832-81593ae9bd3a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:38.271122 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:38.321791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 234/445] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:38.322695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c188107-ad19-4e6d-abf5-0cb93e532685 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:38.360241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd65273-dd49-49ca-b832-81593ae9bd3a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.395048 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84bdfc-4493-4bf5-a0ec-aad32268af4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.395687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84bdfc-4493-4bf5-a0ec-aad32268af4f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.395967 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84bdfc-4493-4bf5-a0ec-aad32268af4f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.396228 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84bdfc-4493-4bf5-a0ec-aad32268af4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.406668 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84bdfc-4493-4bf5-a0ec-aad32268af4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-de84bdfc-4493-4bf5-a0ec-aad32268af4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:38.411741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:38.412602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:38.413399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 212/446] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:38.414030 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84bdfc-4493-4bf5-a0ec-aad32268af4f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:38.429547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c188107-ad19-4e6d-abf5-0cb93e532685 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.440462 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:38.467082 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.467676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.467953 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.468217 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.469008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] RBAC: Adding query filter params (group.id=e49329afffe341a995a2683f95608645, role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=232c57cd96c243f6b181496002d8a724) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:38.469311 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:38.470796 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:38.476857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 235/447] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/role_assignments?group.id=e49329afffe341a995a2683f95608645&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:38.477617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:38.496566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:38.527367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84bdfc-4493-4bf5-a0ec-aad32268af4f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.567557 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a432b5-88f8-4a5c-a9ab-1d059b759874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.568728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a432b5-88f8-4a5c-a9ab-1d059b759874 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.568728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a432b5-88f8-4a5c-a9ab-1d059b759874 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.568998 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a432b5-88f8-4a5c-a9ab-1d059b759874 None admin] PATH_INFO: `/v3/projects/232c57cd96c243f6b181496002d8a724` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.570272 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a432b5-88f8-4a5c-a9ab-1d059b759874 None admin] RBAC: Authorizing `identity:get_project(project_id=232c57cd96c243f6b181496002d8a724)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:38.571759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a432b5-88f8-4a5c-a9ab-1d059b759874 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:38.573515 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 213/448] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/projects/232c57cd96c243f6b181496002d8a724 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:38.579607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a432b5-88f8-4a5c-a9ab-1d059b759874 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:38.601593 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a983d5b-edaf-4db7-a251-880eeb869920 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.641271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.641911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.642230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.642509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.643248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:38.643560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:38.645087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:38.654631 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 236/449] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:38.660658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:38.689744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a432b5-88f8-4a5c-a9ab-1d059b759874 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.701950 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:38.724920 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835c842-9ce1-4d4c-afbb-e41edb88bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.725505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c842-9ce1-4d4c-afbb-e41edb88bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.725787 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c842-9ce1-4d4c-afbb-e41edb88bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.726075 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c842-9ce1-4d4c-afbb-e41edb88bc38 None admin] PATH_INFO: `/v3/groups/e49329afffe341a995a2683f95608645` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.727271 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835c842-9ce1-4d4c-afbb-e41edb88bc38 None admin] RBAC: Authorizing `identity:get_group(group_id=e49329afffe341a995a2683f95608645)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:38.728727 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835c842-9ce1-4d4c-afbb-e41edb88bc38 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:38.730404 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 214/450] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/groups/e49329afffe341a995a2683f95608645 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:38.736152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835c842-9ce1-4d4c-afbb-e41edb88bc38 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:38.748349 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:38.772965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a633015-e493-4094-b271-6970287b7e39 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.810090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5caac-c645-495e-a714-b98a2705db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.810766 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5caac-c645-495e-a714-b98a2705db8c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.811042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5caac-c645-495e-a714-b98a2705db8c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.811305 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5caac-c645-495e-a714-b98a2705db8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.840468 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835c842-9ce1-4d4c-afbb-e41edb88bc38 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:38.874562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:38.875204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:38.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:38.875745 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:38.876555 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] RBAC: Adding query filter params (group.id=e49329afffe341a995a2683f95608645, role.id=06907320d4bc47b3ba3375df728a605a, scope.project.id=232c57cd96c243f6b181496002d8a724) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:38.876850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:38.878376 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:38.879724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5caac-c645-495e-a714-b98a2705db8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:38.881218 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5caac-c645-495e-a714-b98a2705db8c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:38.885081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 215/451] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/role_assignments?group.id=e49329afffe341a995a2683f95608645&scope.project.id=232c57cd96c243f6b181496002d8a724&role.id=06907320d4bc47b3ba3375df728a605a => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:38.918700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:38.994135 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 237/452] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:01:38.999997 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:39.085354 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:39.110742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4b843-4e36-4336-87e1-cf76e17080f1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:39.145887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c72f7-1665-432b-a727-93257f4339c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:39.146606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c72f7-1665-432b-a727-93257f4339c1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:39.146884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c72f7-1665-432b-a727-93257f4339c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:39.147140 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c72f7-1665-432b-a727-93257f4339c1 None admin] PATH_INFO: `/v3/projects/bed3d6665f7c40158ba248253deda4a1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:39.148303 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c72f7-1665-432b-a727-93257f4339c1 None admin] RBAC: Authorizing `identity:get_project(project_id=bed3d6665f7c40158ba248253deda4a1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:39.149751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c72f7-1665-432b-a727-93257f4339c1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:39.151800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 216/453] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:01:38 2022] GET /identity/v3/projects/bed3d6665f7c40158ba248253deda4a1 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:39.157268 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5caac-c645-495e-a714-b98a2705db8c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:39.261720 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5caac-c645-495e-a714-b98a2705db8c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:39.296349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff561-4c59-4737-b795-b4b1dcd6eabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:39.296996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff561-4c59-4737-b795-b4b1dcd6eabb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:39.301442 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff561-4c59-4737-b795-b4b1dcd6eabb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:39.301442 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff561-4c59-4737-b795-b4b1dcd6eabb None admin] PATH_INFO: `/v3/groups/f1a440ea9f8a4888a558b7add9b8fd58` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:39.301442 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff561-4c59-4737-b795-b4b1dcd6eabb None admin] RBAC: Authorizing `identity:get_group(group_id=f1a440ea9f8a4888a558b7add9b8fd58)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:39.301442 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff561-4c59-4737-b795-b4b1dcd6eabb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:39.302032 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 238/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:01:39 2022] GET /identity/v3/groups/f1a440ea9f8a4888a558b7add9b8fd58 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:39.307684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c72f7-1665-432b-a727-93257f4339c1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:39.413478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c72f7-1665-432b-a727-93257f4339c1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:39.448736 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfdadd-b3e0-4187-86b1-8df98b3c8d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:39.449371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdadd-b3e0-4187-86b1-8df98b3c8d89 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:39.450022 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdadd-b3e0-4187-86b1-8df98b3c8d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:39.450327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdadd-b3e0-4187-86b1-8df98b3c8d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:39.451132 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfdadd-b3e0-4187-86b1-8df98b3c8d89 None admin] RBAC: Adding query filter params (group.id=f1a440ea9f8a4888a558b7add9b8fd58, role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=bed3d6665f7c40158ba248253deda4a1) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:39.451442 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfdadd-b3e0-4187-86b1-8df98b3c8d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:39.452896 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfdadd-b3e0-4187-86b1-8df98b3c8d89 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:39.459063 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 217/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:01:39 2022] GET /identity/v3/role_assignments?group.id=f1a440ea9f8a4888a558b7add9b8fd58&scope.project.id=bed3d6665f7c40158ba248253deda4a1&role.id=65cd37a73e4f498482fefabb8374006c => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:41.811893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffcfde-b819-4793-b01a-c9872fe87284 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:41.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffcfde-b819-4793-b01a-c9872fe87284 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:41.812509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffcfde-b819-4793-b01a-c9872fe87284 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:41.814098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 239/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:41.819584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0447abe-ac95-4888-87b8-33ffa17b7285 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:41.820455 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0447abe-ac95-4888-87b8-33ffa17b7285 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:41.820942 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0447abe-ac95-4888-87b8-33ffa17b7285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:41.841446 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:41.847712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d0447abe-ac95-4888-87b8-33ffa17b7285 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:41.848913 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d0447abe-ac95-4888-87b8-33ffa17b7285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:41.952981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 218/457] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:41.996553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffcfde-b819-4793-b01a-c9872fe87284 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:42.000555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:42.105753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffcfde-b819-4793-b01a-c9872fe87284 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:42.141833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d38d1-de35-416a-a69e-054928043dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:42.142528 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d38d1-de35-416a-a69e-054928043dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.142810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d38d1-de35-416a-a69e-054928043dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.143083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d38d1-de35-416a-a69e-054928043dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.152360 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9263c-0fe6-4590-b56f-15003bebe0c6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.152997 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9263c-0fe6-4590-b56f-15003bebe0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.153862 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9263c-0fe6-4590-b56f-15003bebe0c6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.155702 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d38d1-de35-416a-a69e-054928043dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:42.156324 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 219/458] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:42.160563 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d38d1-de35-416a-a69e-054928043dd8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:42.162041 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfa46e-bf5d-426e-8c32-cb47488e6ecd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.162373 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfa46e-bf5d-426e-8c32-cb47488e6ecd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.162645 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfa46e-bf5d-426e-8c32-cb47488e6ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-320d38d1-de35-416a-a69e-054928043dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:42.176482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:42.177394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:42.178283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:42.178283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:42.178283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:42.178283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:42.182219 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 240/459] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:42.183643 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7be685-3942-4b75-a51a-ef01bc0a57fc None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.183940 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7be685-3942-4b75-a51a-ef01bc0a57fc None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.184215 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7be685-3942-4b75-a51a-ef01bc0a57fc None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.185885 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 241/460] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:42.187575 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7be685-3942-4b75-a51a-ef01bc0a57fc None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:42.191437 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4adfa46e-bf5d-426e-8c32-cb47488e6ecd None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:42.192848 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4adfa46e-bf5d-426e-8c32-cb47488e6ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:42.263172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:42.269926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 220/461] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:42.271629 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256c4a-1264-4cb2-b003-839d95a8979e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.272064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256c4a-1264-4cb2-b003-839d95a8979e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.272479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256c4a-1264-4cb2-b003-839d95a8979e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.298698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a9256c4a-1264-4cb2-b003-839d95a8979e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:42.300379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a9256c4a-1264-4cb2-b003-839d95a8979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:42.325003 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7be685-3942-4b75-a51a-ef01bc0a57fc None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:42.362716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:42.363355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.363636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.363913 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.364737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:42.365050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:42.366483 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:42.368255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:42.377012 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 221/462] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:42.378288 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9256c4a-1264-4cb2-b003-839d95a8979e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:42.381888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 242/463] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:42.387751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:42.518190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9256c4a-1264-4cb2-b003-839d95a8979e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:42.519474 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150cfbc-d1e0-40cf-9e0d-ea04d5853b01 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:42.540317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:42.559665 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:42.560848 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.561390 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.561833 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.563453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d3859-cf17-4180-a68b-aef6e3fec8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:42.564342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d3859-cf17-4180-a68b-aef6e3fec8e9 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.564826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d3859-cf17-4180-a68b-aef6e3fec8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.565290 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d3859-cf17-4180-a68b-aef6e3fec8e9 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.566333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d3859-cf17-4180-a68b-aef6e3fec8e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:42.568059 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d3859-cf17-4180-a68b-aef6e3fec8e9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:42.574001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:42.575918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:42.593329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:42.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:42.595164 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:42.595164 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:42.595164 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:42.595164 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:42.597063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 243/464] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:42 2022] POST /identity/v3/users => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:01:42.598212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d3859-cf17-4180-a68b-aef6e3fec8e9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:42.601176 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 222/465] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:42.602242 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:42.694703 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:42.731097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d3859-cf17-4180-a68b-aef6e3fec8e9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:42.772196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59105b-35c2-4ae6-83c7-a2e0d9ea80fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:42.773231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59105b-35c2-4ae6-83c7-a2e0d9ea80fb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.773738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59105b-35c2-4ae6-83c7-a2e0d9ea80fb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.774244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59105b-35c2-4ae6-83c7-a2e0d9ea80fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.775125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4c089-8fce-4e40-b027-c12fd96df6e3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:42.788320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59105b-35c2-4ae6-83c7-a2e0d9ea80fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:42.790227 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59105b-35c2-4ae6-83c7-a2e0d9ea80fb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1b59105b-35c2-4ae6-83c7-a2e0d9ea80fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:42.807584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:42.808467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:42.809280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:42.809280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:42.809280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:42.809280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:42.812093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 244/466] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:42.814361 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a295941-fb35-4cc9-82fe-cb430f93253f None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.814891 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a295941-fb35-4cc9-82fe-cb430f93253f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.815333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a295941-fb35-4cc9-82fe-cb430f93253f None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.815490 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5b61e-f789-4557-b1bd-6111103baac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:42.816303 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5b61e-f789-4557-b1bd-6111103baac3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.816718 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5b61e-f789-4557-b1bd-6111103baac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.817090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5b61e-f789-4557-b1bd-6111103baac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.817601 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 245/467] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:42.817999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5b61e-f789-4557-b1bd-6111103baac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:42.818433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5b61e-f789-4557-b1bd-6111103baac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:42.819008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a295941-fb35-4cc9-82fe-cb430f93253f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:42.820105 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5b61e-f789-4557-b1bd-6111103baac3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:42.836332 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 223/468] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:42.842493 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62569a6-ddef-46e7-8db2-c9378147c640 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.842928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62569a6-ddef-46e7-8db2-c9378147c640 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.843310 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62569a6-ddef-46e7-8db2-c9378147c640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.852544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:42.865945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a62569a6-ddef-46e7-8db2-c9378147c640 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:42.867179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a62569a6-ddef-46e7-8db2-c9378147c640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:42.948312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a295941-fb35-4cc9-82fe-cb430f93253f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:42.953273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 224/469] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:42.954310 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62569a6-ddef-46e7-8db2-c9378147c640 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:42.995412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:42.996055 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:42.996333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:42.996603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:42.997313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:42.997617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:42.999055 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:43.003322 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:43.013833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 246/470] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:43.014931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:43.086207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62569a6-ddef-46e7-8db2-c9378147c640 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:43.106404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:43.125371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718dc8-f8cd-46fb-bd2f-849532d3495a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:43.131051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fa966-9ae2-4c12-b4b4-74c954daca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:43.131991 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fa966-9ae2-4c12-b4b4-74c954daca42 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:43.132491 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fa966-9ae2-4c12-b4b4-74c954daca42 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:43.132932 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fa966-9ae2-4c12-b4b4-74c954daca42 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:43.134238 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fa966-9ae2-4c12-b4b4-74c954daca42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:43.137419 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fa966-9ae2-4c12-b4b4-74c954daca42 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:43.170713 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 225/471] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:42 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:43.170713 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fa966-9ae2-4c12-b4b4-74c954daca42 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:43.183041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:43.184229 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:43.184516 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:43.184786 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:43.214517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:43.216910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:43.255677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:43.256589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:43.257401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:43.257401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:43.257401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:43.257401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:43.260961 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 247/472] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:43.267119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:43.368151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:43.378869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fa966-9ae2-4c12-b4b4-74c954daca42 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:43.416439 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c4d55-e06d-49b7-9e4b-946199db76f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:43.416886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfc9d1-5435-4304-a260-11b76399572e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:43.417062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c4d55-e06d-49b7-9e4b-946199db76f9 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:43.417345 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c4d55-e06d-49b7-9e4b-946199db76f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:43.417609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c4d55-e06d-49b7-9e4b-946199db76f9 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:43.418316 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c4d55-e06d-49b7-9e4b-946199db76f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:43.419832 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c4d55-e06d-49b7-9e4b-946199db76f9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:43.448162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 226/473] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:43 2022] POST /identity/v3/users => generated 248 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:43.457793 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25a783-05a8-455a-8b33-4627690636c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:43.458897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25a783-05a8-455a-8b33-4627690636c6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:43.459373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25a783-05a8-455a-8b33-4627690636c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:43.459839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25a783-05a8-455a-8b33-4627690636c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:43.460771 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25a783-05a8-455a-8b33-4627690636c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:43.461280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25a783-05a8-455a-8b33-4627690636c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:43.462971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25a783-05a8-455a-8b33-4627690636c6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:43.466256 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:43.478697 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 248/474] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:43.484209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c4d55-e06d-49b7-9e4b-946199db76f9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:43.492137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce6020-bd35-4c54-9696-6105fc051a47 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:43.492751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce6020-bd35-4c54-9696-6105fc051a47 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:43.493246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce6020-bd35-4c54-9696-6105fc051a47 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:43.495290 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 249/475] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:43.500613 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd931b07-a738-4593-9aed-f079f9222853 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:43.501244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd931b07-a738-4593-9aed-f079f9222853 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:43.501705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd931b07-a738-4593-9aed-f079f9222853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:43.523094 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dd931b07-a738-4593-9aed-f079f9222853 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:43.524981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dd931b07-a738-4593-9aed-f079f9222853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:43.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c4d55-e06d-49b7-9e4b-946199db76f9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:43.610024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 250/476] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:43.632364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd931b07-a738-4593-9aed-f079f9222853 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:43.654941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1765f6-499f-4612-b67e-a7882de3c0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:43.655643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1765f6-499f-4612-b67e-a7882de3c0c4 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:43.656445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1765f6-499f-4612-b67e-a7882de3c0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:43.656720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1765f6-499f-4612-b67e-a7882de3c0c4 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:43.657624 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1765f6-499f-4612-b67e-a7882de3c0c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:43.659398 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1765f6-499f-4612-b67e-a7882de3c0c4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:43.689755 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 227/477] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:43 2022] POST /identity/v3/users => generated 249 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:43.775140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd931b07-a738-4593-9aed-f079f9222853 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:43.808026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:43.819844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:43.820634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:43.820847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:43.821120 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:43.822007 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:43.823552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:43.834146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.sql.core [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:01:43.834146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:01:43.834146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [parameters: {'id': '17624dad933d40bab2082834a88ea14c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:01:43.834146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80369) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 28 08:01:43.852122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:01:43.853026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application yield resource Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise self.value Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application session.commit() Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:01:43.853902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:01:43.854776 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:01:43.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:01:43.857459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [parameters: {'id': '17624dad933d40bab2082834a88ea14c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:43.858667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:01:43.859981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:43.867703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 251/478] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:01:43 2022] POST /identity/v3/roles => generated 144 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:43.867851 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1765f6-499f-4612-b67e-a7882de3c0c4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:43.878828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:43.996092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1765f6-499f-4612-b67e-a7882de3c0c4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:44.034053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17ecc3-bf98-42a0-86a9-f9c563e9f5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:44.034911 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17ecc3-bf98-42a0-86a9-f9c563e9f5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:44.035188 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17ecc3-bf98-42a0-86a9-f9c563e9f5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:44.035458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17ecc3-bf98-42a0-86a9-f9c563e9f5a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:44.049267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17ecc3-bf98-42a0-86a9-f9c563e9f5a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6c17ecc3-bf98-42a0-86a9-f9c563e9f5a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:44.055177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:44.056093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:44.056838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 228/479] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:01:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:44.062256 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:44.174330 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb183414-a4b5-47c7-8435-4b5a5f181dd9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:44.211971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32a0ea-8f2b-4cb5-a5c9-d2eb520eee9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:44.212600 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32a0ea-8f2b-4cb5-a5c9-d2eb520eee9f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:44.212878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32a0ea-8f2b-4cb5-a5c9-d2eb520eee9f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:44.213141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32a0ea-8f2b-4cb5-a5c9-d2eb520eee9f None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:44.213869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32a0ea-8f2b-4cb5-a5c9-d2eb520eee9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:44.214233 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32a0ea-8f2b-4cb5-a5c9-d2eb520eee9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:44.215605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32a0ea-8f2b-4cb5-a5c9-d2eb520eee9f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:44.224824 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 252/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:45.178483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d7492-7159-435c-9400-d6413eb62ff6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:45.179190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d7492-7159-435c-9400-d6413eb62ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:45.179190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d7492-7159-435c-9400-d6413eb62ff6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:45.180712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 229/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:45.186750 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17297d-fd47-427c-abc1-6f277bc993ce None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:45.187679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17297d-fd47-427c-abc1-6f277bc993ce None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:45.188146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17297d-fd47-427c-abc1-6f277bc993ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:45.223732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8e17297d-fd47-427c-abc1-6f277bc993ce None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:45.224944 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8e17297d-fd47-427c-abc1-6f277bc993ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:45.297246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:45.325191 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 253/482] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:45.359484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d7492-7159-435c-9400-d6413eb62ff6 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:45.443645 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:45.481279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d7492-7159-435c-9400-d6413eb62ff6 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:45.520089 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e44da8-d66f-4cbe-b6c7-83231b9360f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:45.520765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e44da8-d66f-4cbe-b6c7-83231b9360f3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:45.521045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e44da8-d66f-4cbe-b6c7-83231b9360f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:45.521321 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e44da8-d66f-4cbe-b6c7-83231b9360f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:45.537679 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e44da8-d66f-4cbe-b6c7-83231b9360f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-78e44da8-d66f-4cbe-b6c7-83231b9360f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:45.542988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:45.543993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:45.544760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 230/483] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:45.552011 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17297d-fd47-427c-abc1-6f277bc993ce None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:45.674497 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17297d-fd47-427c-abc1-6f277bc993ce None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:45.727526 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4d0c0-e429-4708-9406-5d9a6c33dd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:45.728169 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4d0c0-e429-4708-9406-5d9a6c33dd23 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:45.728447 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4d0c0-e429-4708-9406-5d9a6c33dd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:45.728715 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4d0c0-e429-4708-9406-5d9a6c33dd23 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:45.729455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4d0c0-e429-4708-9406-5d9a6c33dd23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:45.729758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4d0c0-e429-4708-9406-5d9a6c33dd23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:45.731160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4d0c0-e429-4708-9406-5d9a6c33dd23 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:45.745818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 254/484] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:45.751706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e44da8-d66f-4cbe-b6c7-83231b9360f3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:45.947311 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e44da8-d66f-4cbe-b6c7-83231b9360f3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:45.983143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:45.995759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:45.996442 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:45.996726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:45.997004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.012985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86575a9a-2926-4acb-b73e-8ddb97850cbb None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.013290 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86575a9a-2926-4acb-b73e-8ddb97850cbb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.013532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86575a9a-2926-4acb-b73e-8ddb97850cbb None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.014926 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 255/485] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:46.016358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:46.017875 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:46.020739 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4b2c7-c5db-4d36-a482-fc6e6d02d797 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.021317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4b2c7-c5db-4d36-a482-fc6e6d02d797 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.021795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4b2c7-c5db-4d36-a482-fc6e6d02d797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:46.033671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:46.034651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:46.035458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:46.035458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:46.035458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:46.035458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:46.035458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 231/486] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:46.047885 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.063296 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d4a4b2c7-c5db-4d36-a482-fc6e6d02d797 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:46.064507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d4a4b2c7-c5db-4d36-a482-fc6e6d02d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:46.156332 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e565545-9877-4261-be6e-88c8963e296f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:46.170922 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:46.195947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:46.197545 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.197995 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.198439 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.199447 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:46.200255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:46.201054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 256/487] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:46.202399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13eda-be67-4661-9c1e-743a7cc99b62 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.202565 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:46.202683 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13eda-be67-4661-9c1e-743a7cc99b62 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.202951 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13eda-be67-4661-9c1e-743a7cc99b62 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.204451 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 257/488] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:46.209536 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77b9c-265a-4d57-906e-e635ac8fe0e7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.210060 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77b9c-265a-4d57-906e-e635ac8fe0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.210700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77b9c-265a-4d57-906e-e635ac8fe0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.216873 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 232/489] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:46.222350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.231279 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-08e77b9c-265a-4d57-906e-e635ac8fe0e7 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:46.232802 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-08e77b9c-265a-4d57-906e-e635ac8fe0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:46.268608 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:46.317940 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 258/490] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:46.319190 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e77b9c-265a-4d57-906e-e635ac8fe0e7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.353682 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661d0e2-f90f-4d99-801c-614dfe98d931 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:46.400581 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:46.401549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.401989 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.402423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.411259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:46.413033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:46.428001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:46.429021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:01:46.430085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:46.431471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 233/491] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:46.436352 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e77b9c-265a-4d57-906e-e635ac8fe0e7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:46.436433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.436499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:46.472431 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae06fc9-efde-43b2-8d30-6e6b20b2b2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:46.473554 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06fc9-efde-43b2-8d30-6e6b20b2b2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.473554 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06fc9-efde-43b2-8d30-6e6b20b2b2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.473717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06fc9-efde-43b2-8d30-6e6b20b2b2e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.484378 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae06fc9-efde-43b2-8d30-6e6b20b2b2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-5ae06fc9-efde-43b2-8d30-6e6b20b2b2e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:46.489376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:46.490261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:46.491022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 259/492] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:46.492332 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e8261-6ed5-44e6-9d87-85d352afe133 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.492758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e8261-6ed5-44e6-9d87-85d352afe133 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.492758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e8261-6ed5-44e6-9d87-85d352afe133 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.498027 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 260/493] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:46.498027 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e8261-6ed5-44e6-9d87-85d352afe133 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.542199 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:46.546847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36e471-1133-43fb-a064-f83d44723652 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:46.586402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270cdea-bba2-4af3-b205-7b49738a8db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:46.587233 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270cdea-bba2-4af3-b205-7b49738a8db3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.587530 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270cdea-bba2-4af3-b205-7b49738a8db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.588039 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270cdea-bba2-4af3-b205-7b49738a8db3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.600224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270cdea-bba2-4af3-b205-7b49738a8db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f270cdea-bba2-4af3-b205-7b49738a8db3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:46.606006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:46.606977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:46.607756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 234/494] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:46.608389 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270cdea-bba2-4af3-b205-7b49738a8db3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.613399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e8261-6ed5-44e6-9d87-85d352afe133 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:46.650000 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604be2e-bf02-4bcf-8c18-cfee6a8acc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:46.650662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604be2e-bf02-4bcf-8c18-cfee6a8acc0d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.650940 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604be2e-bf02-4bcf-8c18-cfee6a8acc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.651206 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604be2e-bf02-4bcf-8c18-cfee6a8acc0d None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.652437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604be2e-bf02-4bcf-8c18-cfee6a8acc0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:46.652744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604be2e-bf02-4bcf-8c18-cfee6a8acc0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:46.654598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604be2e-bf02-4bcf-8c18-cfee6a8acc0d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:46.694379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 261/495] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:46.696133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378bb4a-d3a8-4056-966d-71a11d557f4e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.696486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378bb4a-d3a8-4056-966d-71a11d557f4e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.696828 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378bb4a-d3a8-4056-966d-71a11d557f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.715567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a378bb4a-d3a8-4056-966d-71a11d557f4e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:46.716754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a378bb4a-d3a8-4056-966d-71a11d557f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:46.723984 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270cdea-bba2-4af3-b205-7b49738a8db3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:46.734958 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:46.766504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:46.767510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.768040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.768370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.769280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:46.769641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:46.771380 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:46.780128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 235/496] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:46.780969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.802212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 262/497] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:46.802212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378bb4a-d3a8-4056-966d-71a11d557f4e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.814316 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:46.906914 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fed58a-2801-4c9c-b50a-2041b998f46d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:46.912191 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378bb4a-d3a8-4056-966d-71a11d557f4e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:46.944553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:46.945312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.945658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.946000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.946783 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:46.947119 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:46.947737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddc769-c329-43f9-9375-350b54185259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:46.948367 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddc769-c329-43f9-9375-350b54185259 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:46.948532 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:46.948677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddc769-c329-43f9-9375-350b54185259 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:46.948906 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddc769-c329-43f9-9375-350b54185259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:46.958348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 236/498] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:46.959255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:46.978197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.028944 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddc769-c329-43f9-9375-350b54185259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.030481 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddc769-c329-43f9-9375-350b54185259 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.045383 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.064692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080908f2-ffc4-45b8-b511-15bd250154f5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.100954 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.101695 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.101985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.102290 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.114265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.115741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:47.130732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:47.131570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:47.132402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:47.132402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:47.132402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:47.132402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:47.132402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 237/499] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:47.132923 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.137164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 263/500] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:47.138139 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddc769-c329-43f9-9375-350b54185259 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.192886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.234250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df288a7-e4a2-47c0-b2b3-66e672669798 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.243285 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddc769-c329-43f9-9375-350b54185259 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.269011 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1eaada-f263-4137-8848-09d4bcb665a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.269671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eaada-f263-4137-8848-09d4bcb665a5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.269953 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eaada-f263-4137-8848-09d4bcb665a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.270275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eaada-f263-4137-8848-09d4bcb665a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.272170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.279994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.280649 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.280926 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.281197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.281783 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1eaada-f263-4137-8848-09d4bcb665a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9d1eaada-f263-4137-8848-09d4bcb665a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:47.287208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:47.288244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:47.289047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 238/501] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:47.289910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1b608-4a49-493d-ad1f-961e186a0d1d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.290169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1b608-4a49-493d-ad1f-961e186a0d1d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.290432 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1b608-4a49-493d-ad1f-961e186a0d1d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.291672 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 239/502] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:47.292350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1b608-4a49-493d-ad1f-961e186a0d1d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.295882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.297390 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:47.312662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:47.313731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:47.314671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:47.314671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:47.314671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:47.314671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:47.314671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 264/503] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:47.314671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.400325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1b608-4a49-493d-ad1f-961e186a0d1d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.418457 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a92424-44bd-4101-b603-5fe04afa1df9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.421560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.435819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.436513 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.436790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.437063 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.449106 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.450979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.454747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6859a58-8170-42ea-95a8-aef52c317c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.455389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6859a58-8170-42ea-95a8-aef52c317c05 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.455666 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6859a58-8170-42ea-95a8-aef52c317c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.455928 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6859a58-8170-42ea-95a8-aef52c317c05 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.456768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6859a58-8170-42ea-95a8-aef52c317c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:47.457068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6859a58-8170-42ea-95a8-aef52c317c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.458446 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6859a58-8170-42ea-95a8-aef52c317c05 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:47.466060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:47.466908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:47.467723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:47.467723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:47.467723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:47.467723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:47.467723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 240/504] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:47.468482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.471608 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 265/505] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:47.473186 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74789adc-2962-4e7b-ab7e-8c8f18f1fb1f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.473474 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74789adc-2962-4e7b-ab7e-8c8f18f1fb1f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.473760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74789adc-2962-4e7b-ab7e-8c8f18f1fb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.491874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-74789adc-2962-4e7b-ab7e-8c8f18f1fb1f None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:47.493095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-74789adc-2962-4e7b-ab7e-8c8f18f1fb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:47.516551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.562782 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 266/506] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:47.563558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74789adc-2962-4e7b-ab7e-8c8f18f1fb1f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.571169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac27100-3ed8-45f8-8e87-742e0f6869e9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.609979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.610679 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.610955 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.611229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.611963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:47.612279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.613742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.622668 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 241/507] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:47.623546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.664939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.671835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74789adc-2962-4e7b-ab7e-8c8f18f1fb1f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.712798 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.713443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.713730 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.714015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.714753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:47.715063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.716480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.729996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 267/508] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:47.731515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.735487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19184ab0-d2a2-4e97-8039-c4573c6835f0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.744803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.770033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0799316-2186-42df-82b0-199075674944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.770723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0799316-2186-42df-82b0-199075674944 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.771005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0799316-2186-42df-82b0-199075674944 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.771280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0799316-2186-42df-82b0-199075674944 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.771995 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0799316-2186-42df-82b0-199075674944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:47.772309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0799316-2186-42df-82b0-199075674944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.773728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0799316-2186-42df-82b0-199075674944 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.786947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 242/509] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:47.787852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0799316-2186-42df-82b0-199075674944 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.841230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d4be-b547-4488-b58b-7e5f50e90f93 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.878889 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.879560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.879840 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.880097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.889601 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.891235 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.897097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0799316-2186-42df-82b0-199075674944 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:47.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:47.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:47.907201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 268/510] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:47.907745 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.926324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:47.940358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:47.941028 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:47.941288 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:47.941556 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:47.953052 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:47.954690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:47.970840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:47.971726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:47.972572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:47.972572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:47.972572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:47.972572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:47.972572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 243/511] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:47.972935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:47.998467 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:48.021183 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd529a86-a1e0-43d7-9b16-0038b5f569f4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.059926 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.060583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.060864 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.061126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.077872 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.079422 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.088708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c7bc8-8bfd-46ad-afca-c5a86f6f3389 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:48.100825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:48.101747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:48.102963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:48.102963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:48.102963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:48.102963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:48.102963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 269/512] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:48.102963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.129251 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.129908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.130220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.130495 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.138303 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.139760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:48.151269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:48.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:48.153069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 244/513] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:48.153602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.176792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:48.211895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ac5d4-aa72-4685-8319-cd6fb3ad7002 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.235242 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:48.245956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.246644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.246919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.247189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.254602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.256049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.256228 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcac736-6eae-4b5c-9515-9356cf5b7d3e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:48.268581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:48.269485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:48.270351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 270/514] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:48.270816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.295740 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.296369 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.296645 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.296927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.297725 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:48.298030 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.299511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.335651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 245/515] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:48.337671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.371937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60feffe-4777-4a31-a7a2-eca4da1a3b98 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.398375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:48.408515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.409151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.409490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.409767 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.410514 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:48.410832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.412207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.424410 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 271/516] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:48.425126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.461126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf529289-f056-4728-a6a2-1b192281921f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.495585 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.496214 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.496493 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.496761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.497457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:48.497759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.499183 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.505923 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:48.511849 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 246/517] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:48.512655 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.527253 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372ff57-4c69-40b6-aeea-3c94336aa5ff None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.562600 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.563231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.563511 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.563787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.564600 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:48.564909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.566491 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.584976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 272/518] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:48.585825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.618922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf46ed-752f-40a5-aaff-06f2a856a4f1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.630830 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:48.653410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.654049 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.654362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.654634 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.655397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:48.655697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.657190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.675868 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 247/519] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:48.676701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.690142 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37545750-ec5b-4e7e-8206-0466d4e48e1d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.732091 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5bce8-1c26-427f-acae-5dfd54025bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.732733 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5bce8-1c26-427f-acae-5dfd54025bbb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.733019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5bce8-1c26-427f-acae-5dfd54025bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.733324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5bce8-1c26-427f-acae-5dfd54025bbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.738903 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:48.780418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0074f-5f8c-4899-8bdc-31842436f145 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.804555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5bce8-1c26-427f-acae-5dfd54025bbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.806068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5bce8-1c26-427f-acae-5dfd54025bbb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.816398 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328cd9-3860-4cce-85d0-d6c3d50796dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:48.817033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328cd9-3860-4cce-85d0-d6c3d50796dc None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:48.817313 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328cd9-3860-4cce-85d0-d6c3d50796dc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:48.817588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328cd9-3860-4cce-85d0-d6c3d50796dc None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:48.818304 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae328cd9-3860-4cce-85d0-d6c3d50796dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:48.819750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae328cd9-3860-4cce-85d0-d6c3d50796dc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:48.843868 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 248/520] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:01:48 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:01:48.844747 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328cd9-3860-4cce-85d0-d6c3d50796dc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.864566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:48.934531 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 273/521] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:48.935580 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5bce8-1c26-427f-acae-5dfd54025bbb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:48.981742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328cd9-3860-4cce-85d0-d6c3d50796dc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:48.997986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:49.023617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.024602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.024602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.024796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.033245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.034830 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:49.047589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:49.048524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 249/522] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:49.050325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:49.066151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5bce8-1c26-427f-acae-5dfd54025bbb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:49.103471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd06d0-1027-4992-9c91-7dccf991426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.104099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd06d0-1027-4992-9c91-7dccf991426c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.104500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd06d0-1027-4992-9c91-7dccf991426c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.104743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd06d0-1027-4992-9c91-7dccf991426c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.120332 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:49.157131 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaad971-9ba6-4c4f-a3ad-667a43837f18 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:49.179108 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd06d0-1027-4992-9c91-7dccf991426c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.180945 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd06d0-1027-4992-9c91-7dccf991426c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.193821 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.194488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.194765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.195035 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.195834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=4527e4500f984b1a81214c3656e7947f) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:49.196148 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.197587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.203798 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 250/523] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:49 2022] GET /identity/v3/role_assignments?user.id=4527e4500f984b1a81214c3656e7947f&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:49.204625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:49.262658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:49.320089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 274/524] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:49.321793 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd06d0-1027-4992-9c91-7dccf991426c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:49.341129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e20f-e830-4486-91c0-2510bb9dd247 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:49.380500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:49.383993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.384675 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.384963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.385239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.400780 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.402213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:49.418096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:49.419002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:49.419902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:49.419902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:49.419902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:49.419902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:49.419902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 251/525] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:49.421286 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:49.455376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd06d0-1027-4992-9c91-7dccf991426c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:49.494093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.494998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.495420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.495832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.496913 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:49.497503 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.499003 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.519178 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 275/526] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:01:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:49.520122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:49.523207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:49.536246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ea2d3-7f98-4a31-a875-7109232c9332 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:49.585118 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.586465 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.587198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.588057 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.603445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.605816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:49.627344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:49.628309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:49.629107 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:49.629107 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:49.629107 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:49.629107 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:49.632319 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 252/527] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:49.633406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:49.633848 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8877171-d2ec-4fc9-9bfc-95ae32168a25 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:49.639822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:49.677683 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.678347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.678625 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.678978 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.679585 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:49.679887 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.681265 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.694800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 276/528] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:49.695870 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:49.765073 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd3e10-d3f2-449f-af9d-79727f9b5278 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:49.803138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce49c6-85d9-4a5c-a041-1e1cc2b285fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.803753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce49c6-85d9-4a5c-a041-1e1cc2b285fe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.804105 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce49c6-85d9-4a5c-a041-1e1cc2b285fe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.804445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce49c6-85d9-4a5c-a041-1e1cc2b285fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.818289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:49.830392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d0e0b-02ad-4d76-87a5-e47b5ecde11b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:49.873896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:49.874752 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:49.875135 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:49.875506 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:49.876329 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:49.876728 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.878384 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.880697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce49c6-85d9-4a5c-a041-1e1cc2b285fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:49.882377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce49c6-85d9-4a5c-a041-1e1cc2b285fe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:49.896009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 277/529] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:49.896009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:49.919218 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:49.999222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135858a-8e71-490c-8c75-3f060278e0ba None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.002875 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 253/530] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:50.004193 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce49c6-85d9-4a5c-a041-1e1cc2b285fe None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.038645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.039464 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.039573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.039856 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.048216 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.049700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:50.062089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:50.062974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:50.064320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:50.064320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:50.064320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:50.064320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:50.064320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:50.064320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:50.064320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:50.064320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:50.067658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:50.068155 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 278/531] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:50.069146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.132118 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce49c6-85d9-4a5c-a041-1e1cc2b285fe None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.154144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:50.173172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.173870 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdb10-bfc2-4ed1-8e22-2bc8ccd35ddd None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.174478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.175081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.175649 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.185662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:50.199002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:50.200252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:50.201067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:50.201067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:50.201067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:50.201067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:50.201067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:50.201067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:50.201067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:50.201067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:50.208650 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 254/532] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:50.210232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.211836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.212529 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.212812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.213090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.229844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.231800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:50.248133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:50.249057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:50.250467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:50.250467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:50.250467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:50.250467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:50.250467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 279/533] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:50.250467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.326851 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1904fc-a61a-4a4c-b42c-92612f529652 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.329058 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:50.357879 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbda14-cbf7-435b-8e44-cfe782f82051 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.368045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.369060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.369583 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.370049 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.371031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:50.371550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.373359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.387512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 255/534] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:50.388850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.402843 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.404436 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.404918 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.405420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.406879 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:50.407803 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.409904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.413150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:50.444847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 280/535] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:50.444847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.523650 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c313d65-4bc1-4373-a1d3-12e69647a6f5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.560858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.561522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.561805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.562083 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.562994 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:50.563305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.565026 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.581325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 256/536] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:50.582157 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.583360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7609e5f-7b7b-4595-8469-2ecdab1232ae None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.612971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:50.631302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.632851 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.633328 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.633789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.635231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=fd2964a3f5aa41ce8c40945a529deb5c) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:50.636193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.638197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.646873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 281/537] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/role_assignments?user.id=fd2964a3f5aa41ce8c40945a529deb5c&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:50.648011 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.722512 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:50.774385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713abb-ff05-4103-9998-2a187298e524 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.777708 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21524c0-ad2a-43c2-ac3a-b450c45573d5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:50.835675 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.836689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.837183 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.837646 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.851184 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.854022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:50.875884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:50.876778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:50.877576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:50.877576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:50.877576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:50.877576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:50.877576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:50.877576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:50.877576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:50.877576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:50.881507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 282/538] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:50.883224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e15e10-72ed-4bf4-b45e-dc2f77503ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:50.883581 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e15e10-72ed-4bf4-b45e-dc2f77503ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.883862 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e15e10-72ed-4bf4-b45e-dc2f77503ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.884231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e15e10-72ed-4bf4-b45e-dc2f77503ff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.884942 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e15e10-72ed-4bf4-b45e-dc2f77503ff7 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=c96a4fc7279a44259990c423c5c35dea) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:50.885255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e15e10-72ed-4bf4-b45e-dc2f77503ff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:50.886728 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:50.889211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e15e10-72ed-4bf4-b45e-dc2f77503ff7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:50.896765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 257/539] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/role_assignments?user.id=c96a4fc7279a44259990c423c5c35dea&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:50.957593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17566f-ba8a-40f8-9eaf-8479b309e7e4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.957828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17566f-ba8a-40f8-9eaf-8479b309e7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.958057 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17566f-ba8a-40f8-9eaf-8479b309e7e4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.959700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 258/540] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:01:50.965328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62d10a-cf61-4e19-98b3-f1f09644cd69 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:50.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62d10a-cf61-4e19-98b3-f1f09644cd69 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:50.965870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62d10a-cf61-4e19-98b3-f1f09644cd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:50.986555 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0c62d10a-cf61-4e19-98b3-f1f09644cd69 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:50.987865 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0c62d10a-cf61-4e19-98b3-f1f09644cd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:51.030689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f18b7d-a268-4990-87fc-25b15089bf11 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:51.056863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:51.064428 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 259/541] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:01:51.085922 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:51.087065 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.087543 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.088005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.088115 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62d10a-cf61-4e19-98b3-f1f09644cd69 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:51.089118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:51.089639 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:51.095040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:51.116323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:51.118163 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 283/542] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:51.126347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:51.275716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62d10a-cf61-4e19-98b3-f1f09644cd69 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:51.338360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d3c0-007a-4564-8e9a-82063da22c7d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:51.348528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f7ea1-0612-44c4-9bda-663f2ab3dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:51.349253 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f7ea1-0612-44c4-9bda-663f2ab3dd57 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.349529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f7ea1-0612-44c4-9bda-663f2ab3dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.349850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f7ea1-0612-44c4-9bda-663f2ab3dd57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.364260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f7ea1-0612-44c4-9bda-663f2ab3dd57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f33f7ea1-0612-44c4-9bda-663f2ab3dd57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:51.370657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:51.371559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:51.372327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 260/543] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:51.373433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b75d8e-1eed-4677-a69a-ec85bbeae032 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.373697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b75d8e-1eed-4677-a69a-ec85bbeae032 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.373973 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b75d8e-1eed-4677-a69a-ec85bbeae032 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.379605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 261/544] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:51.381490 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b75d8e-1eed-4677-a69a-ec85bbeae032 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:51.395004 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce640227-a506-44ed-b79f-dc9f9d92b11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:51.396808 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640227-a506-44ed-b79f-dc9f9d92b11a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.397292 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640227-a506-44ed-b79f-dc9f9d92b11a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.397814 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640227-a506-44ed-b79f-dc9f9d92b11a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.399213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce640227-a506-44ed-b79f-dc9f9d92b11a None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=2ae2e70a28824ac58182627a8c7bf702) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:51.399757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce640227-a506-44ed-b79f-dc9f9d92b11a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:51.401968 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce640227-a506-44ed-b79f-dc9f9d92b11a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:51.411034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 284/545] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:51 2022] GET /identity/v3/role_assignments?user.id=2ae2e70a28824ac58182627a8c7bf702&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:51.414479 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b66cfe-78d4-41fb-88d9-4707a273a75e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.415347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b66cfe-78d4-41fb-88d9-4707a273a75e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.415824 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b66cfe-78d4-41fb-88d9-4707a273a75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.440316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f6b66cfe-78d4-41fb-88d9-4707a273a75e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:51.442967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f6b66cfe-78d4-41fb-88d9-4707a273a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:51.499128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:51.510530 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:51.531577 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b75d8e-1eed-4677-a69a-ec85bbeae032 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:51.580927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 285/546] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:51.594049 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:51.594644 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.594967 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.595244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.596002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:51.596346 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:51.598268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:51.608556 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 262/547] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:51.618351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b66cfe-78d4-41fb-88d9-4707a273a75e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:51.619498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:51.746985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d5f0a-0b26-4f3f-ac81-1fa55c87c37f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:51.749082 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b66cfe-78d4-41fb-88d9-4707a273a75e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:51.786772 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:51.787731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.788133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.788528 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.799037 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82823-46e7-4acd-9c84-72cc4597a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:51.800353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82823-46e7-4acd-9c84-72cc4597a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.800627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82823-46e7-4acd-9c84-72cc4597a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.800995 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82823-46e7-4acd-9c84-72cc4597a5a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.803263 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:51.805144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:51.820129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:51.823444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:51.827034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:51.827875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:51.827875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:51.827875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:51.827875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:51.827875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 286/548] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:51.828838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf82823-46e7-4acd-9c84-72cc4597a5a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:51.831906 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2cf82823-46e7-4acd-9c84-72cc4597a5a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:51.836374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:51.837280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:51.838646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:51.841904 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 263/549] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:51.851441 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82823-46e7-4acd-9c84-72cc4597a5a6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:51.874554 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:51.951567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fd60c-323e-4f1b-9fca-afead8ba39ee None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:51.989535 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:51.990209 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:51.990491 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:51.990842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:51.991475 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:51.991780 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:52.001115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:52.006710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 287/550] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:52.011981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:52.047772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82823-46e7-4acd-9c84-72cc4597a5a6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:52.114708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:52.115713 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:52.116206 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:52.116669 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:52.117598 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:52.118097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:52.119702 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:52.135450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 264/551] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:52.137728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:52.190538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:52.227073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b2923-85fc-4802-abee-71f6d80b2a79 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:52.260977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:52.298090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:52.299199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:52.299675 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:52.300179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:52.312168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:52.315564 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:52.337844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:52.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:01:52.339873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:52.342848 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 288/552] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:52.348557 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:52.354182 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad727da-c928-4c61-b65e-9c2650cb481b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:52.417065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:52.422354 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:52.422844 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:52.423310 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:52.441230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:52.444757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:52.476549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:52.477462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:52.478317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:52.478317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:52.478317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:52.478317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:52.488690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 265/553] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:52.493151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:52.556071 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed050-160e-418e-9957-09f6dece4a96 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:52.626143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:52.627172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:52.627670 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:52.628179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:52.629184 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:52.629685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:52.631336 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:52.656658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd3e94-37d5-4c4e-b674-6aac442eb481 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:52.668814 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 289/554] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:01:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:52.676444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:52.679871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:52.685643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:52.753780 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:52.754948 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:52.756528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:52.757967 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:52.759979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:52.766501 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:52.768163 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:52.837340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 266/555] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:52.852404 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:52.985153 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4611262-94a0-45ee-bbe9-0593f6900212 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:53.070902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be2a01-9b9e-468c-a9f2-fa666a022dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:53.071560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be2a01-9b9e-468c-a9f2-fa666a022dee None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.071830 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be2a01-9b9e-468c-a9f2-fa666a022dee None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.072102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be2a01-9b9e-468c-a9f2-fa666a022dee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.124339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0a04-15f9-487a-adf4-2ca394658bc8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:53.230883 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f628a8-f9bc-464c-8bd3-65c99e8bdc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:53.231561 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f628a8-f9bc-464c-8bd3-65c99e8bdc99 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.231843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f628a8-f9bc-464c-8bd3-65c99e8bdc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.232387 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f628a8-f9bc-464c-8bd3-65c99e8bdc99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.243794 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f628a8-f9bc-464c-8bd3-65c99e8bdc99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:53.248298 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be2a01-9b9e-468c-a9f2-fa666a022dee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:53.249646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f628a8-f9bc-464c-8bd3-65c99e8bdc99 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:53.249758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be2a01-9b9e-468c-a9f2-fa666a022dee None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:53.252411 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f5f628a8-f9bc-464c-8bd3-65c99e8bdc99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:53.267840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:53.269438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:01:53.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:53.274265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 267/556] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:01:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:53.276666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520024-75ad-414f-8428-254ac4a89520 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.276954 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520024-75ad-414f-8428-254ac4a89520 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.279913 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520024-75ad-414f-8428-254ac4a89520 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.283465 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 268/557] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:53.284680 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74520024-75ad-414f-8428-254ac4a89520 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:53.324144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:53.391355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 290/558] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:53.397531 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157b14-eb39-4e54-a9d2-1abaa3515137 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.397837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157b14-eb39-4e54-a9d2-1abaa3515137 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.398094 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157b14-eb39-4e54-a9d2-1abaa3515137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.410129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74520024-75ad-414f-8428-254ac4a89520 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:53.420289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0d157b14-eb39-4e54-a9d2-1abaa3515137 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:53.421954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0d157b14-eb39-4e54-a9d2-1abaa3515137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:53.453323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c77682-f955-4587-a732-115675b3e707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:53.453892 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c77682-f955-4587-a732-115675b3e707 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.454202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c77682-f955-4587-a732-115675b3e707 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.454477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c77682-f955-4587-a732-115675b3e707 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.455290 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c77682-f955-4587-a732-115675b3e707 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:53.455599 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c77682-f955-4587-a732-115675b3e707 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:53.457197 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c77682-f955-4587-a732-115675b3e707 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:53.478222 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 269/559] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:01:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:53.479585 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2c495-9d6a-4156-802e-57008a4c4e36 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.479873 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2c495-9d6a-4156-802e-57008a4c4e36 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.480131 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2c495-9d6a-4156-802e-57008a4c4e36 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.481490 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 270/560] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:53.482859 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2c495-9d6a-4156-802e-57008a4c4e36 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:53.504053 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 291/561] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:53.505241 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d157b14-eb39-4e54-a9d2-1abaa3515137 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:53.544293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:53.600302 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:53.612286 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2c495-9d6a-4156-802e-57008a4c4e36 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:53.620717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d157b14-eb39-4e54-a9d2-1abaa3515137 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:53.646642 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359121a-2da5-423c-954f-f45021fe5635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:53.647296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359121a-2da5-423c-954f-f45021fe5635 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.647576 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359121a-2da5-423c-954f-f45021fe5635 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.647843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359121a-2da5-423c-954f-f45021fe5635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.658029 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359121a-2da5-423c-954f-f45021fe5635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:53.659607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359121a-2da5-423c-954f-f45021fe5635 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:53.660068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:53.660972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.661500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.661930 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.673211 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2359121a-2da5-423c-954f-f45021fe5635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:53.674635 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:53.675618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:53.676505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:53.676505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:53.676505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:53.676505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:53.676505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 271/562] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:53.677095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:53.677425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e223c6-63b1-405b-8fa2-d77aac14a7ff None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.677705 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e223c6-63b1-405b-8fa2-d77aac14a7ff None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.677973 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e223c6-63b1-405b-8fa2-d77aac14a7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:53.696024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:53.696993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:53.697859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:53.697859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:53.697859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:53.697859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:53.697859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 292/563] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:53.698196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:53.700179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-08e223c6-63b1-405b-8fa2-d77aac14a7ff None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:53.701387 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-08e223c6-63b1-405b-8fa2-d77aac14a7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:53.776630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 272/564] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:53.785253 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e223c6-63b1-405b-8fa2-d77aac14a7ff None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:53.805358 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:53.822025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912d9f1-835a-4155-b812-ad6b712cd938 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:53.858466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4b037-4641-4d85-979b-c0010b39bac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:53.859133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4b037-4641-4d85-979b-c0010b39bac3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.859412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4b037-4641-4d85-979b-c0010b39bac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.859683 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4b037-4641-4d85-979b-c0010b39bac3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.861939 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:53.872201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4b037-4641-4d85-979b-c0010b39bac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e0d4b037-4641-4d85-979b-c0010b39bac3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:53.877838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:53.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:53.879704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 293/565] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:53.880754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4b037-4641-4d85-979b-c0010b39bac3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:53.900739 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e223c6-63b1-405b-8fa2-d77aac14a7ff None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:53.938460 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39059340-4f02-46b6-9571-a93eef6279cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:53.939203 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059340-4f02-46b6-9571-a93eef6279cc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.939480 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059340-4f02-46b6-9571-a93eef6279cc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.939747 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059340-4f02-46b6-9571-a93eef6279cc None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.940502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39059340-4f02-46b6-9571-a93eef6279cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:53.940803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39059340-4f02-46b6-9571-a93eef6279cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:53.942251 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39059340-4f02-46b6-9571-a93eef6279cc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:53.954656 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 273/566] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:53.955938 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea936346-7467-4921-9f45-ae32ba9a2f1b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:53.956321 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea936346-7467-4921-9f45-ae32ba9a2f1b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:53.956597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea936346-7467-4921-9f45-ae32ba9a2f1b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:53.961699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 274/567] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:53.961699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea936346-7467-4921-9f45-ae32ba9a2f1b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:53.993199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4b037-4641-4d85-979b-c0010b39bac3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.025946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.026595 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.026930 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.027140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.027836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:54.028131 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.029495 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.041394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 294/568] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:54.042241 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.045560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:54.069850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea936346-7467-4921-9f45-ae32ba9a2f1b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.100433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:54.106796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d997c-7df7-437b-be8e-4df9f4c10b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.107444 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d997c-7df7-437b-be8e-4df9f4c10b62 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.107735 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d997c-7df7-437b-be8e-4df9f4c10b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.108007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d997c-7df7-437b-be8e-4df9f4c10b62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.118523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d997c-7df7-437b-be8e-4df9f4c10b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-346d997c-7df7-437b-be8e-4df9f4c10b62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:54.123957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:54.125006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:54.125817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 275/569] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:54.126529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d997c-7df7-437b-be8e-4df9f4c10b62 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.147433 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60c6f2-ce97-4143-8042-bb06f84e089e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.181641 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b1a89-f8ad-4a92-af56-ad2e419b79f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.182398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b1a89-f8ad-4a92-af56-ad2e419b79f4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.182759 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b1a89-f8ad-4a92-af56-ad2e419b79f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.183031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b1a89-f8ad-4a92-af56-ad2e419b79f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.183758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b1a89-f8ad-4a92-af56-ad2e419b79f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:54.184063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b1a89-f8ad-4a92-af56-ad2e419b79f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.185449 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b1a89-f8ad-4a92-af56-ad2e419b79f4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.193900 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 295/570] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:54.197252 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328abd05-f5ef-45d4-837e-2882388ca9fa None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.197567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328abd05-f5ef-45d4-837e-2882388ca9fa None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.200662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328abd05-f5ef-45d4-837e-2882388ca9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.218967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-328abd05-f5ef-45d4-837e-2882388ca9fa None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:54.220204 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-328abd05-f5ef-45d4-837e-2882388ca9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:54.233774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d997c-7df7-437b-be8e-4df9f4c10b62 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.271193 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.271837 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.272118 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.272385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.279175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.280578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.283280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:54.293613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:54.294684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:54.295917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:54.295917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:54.295917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:54.295917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:54.295917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:54.295917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:54.295917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:54.295917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:54.296439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 296/571] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:54.296439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328abd05-f5ef-45d4-837e-2882388ca9fa None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.298364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 276/572] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:54.299064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.365537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:54.403656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328abd05-f5ef-45d4-837e-2882388ca9fa None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.414830 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac28112-c02a-40d5-9bfb-2e7944bcbe10 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.440346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.441115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.441395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.441663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.448535 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.448784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.449169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.449452 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.450095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.450649 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.450851 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:54.451163 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.452524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.460820 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 277/573] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:54.461632 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:54.462226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:54.463039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:54.463876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 297/574] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:54.464640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.525874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:54.571043 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09502a1-0a7f-4067-ba5a-1e835dc025f0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.572339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6be6a6-68f1-4389-8a53-75202961ec5e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.587728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:54.609537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.610206 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.610486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.610758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.611525 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:54.611833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.611983 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.612770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.613052 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.613323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.613412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.624553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.625857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 298/575] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:54.626339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.626790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:54.644145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:54.645133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:54.645972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:54.645972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:54.645972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:54.645972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:54.645972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 278/576] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:54.646292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.731925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160392c-b313-47a6-a3d2-5a080791d705 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.748753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:54.749475 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ee48f-2c96-44cf-986f-dc4a4365b9f2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.766886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7d157-3170-4772-b0ac-ead526338838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.767542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7d157-3170-4772-b0ac-ead526338838 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.767822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7d157-3170-4772-b0ac-ead526338838 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7d157-3170-4772-b0ac-ead526338838 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.778632 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7d157-3170-4772-b0ac-ead526338838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:54.783208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-edc7d157-3170-4772-b0ac-ead526338838 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:54.783668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:54.784546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:54.785647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:54.786492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.786492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.786492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.786492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:54.786744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 299/577] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:54.786819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.786893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7d157-3170-4772-b0ac-ead526338838 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.786960 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.799201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 279/578] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:54.800099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:54.895333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7d157-3170-4772-b0ac-ead526338838 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.900952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d753e24-0b4f-4f35-a6c0-4615f44d88ae None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:54.931517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.932167 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.932448 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.932718 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.935497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:54.935941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:54.936275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:54.936548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:54.937347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=f285251a0e0a49849d071155d2d3e478) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:54.937658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.939137 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.943160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:54.944915 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:54.945039 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 280/579] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/role_assignments?user.id=f285251a0e0a49849d071155d2d3e478&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:54.945483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:54.970533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:54.971721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:54.972594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:54.972594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:54.972594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:54.972594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:54.972594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 300/580] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:54.972875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.005280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:55.050851 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:55.060972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9fbf2-0daf-4654-a0b1-5e9aa5fa5347 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.092644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751b96e-2410-4e44-8cec-1c8f2bcb6d51 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.097325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.097967 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.098282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.098565 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.099266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:55.099577 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.100977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.113525 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 281/581] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:55.114390 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.130152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.130781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.131057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.131322 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.132139 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:55.132453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.133816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.141925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 301/582] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:55.142805 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.226745 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c14f0-071c-46de-97ab-7323be2081e3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.248662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:55.261531 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a0c57-022a-49e6-a5fc-ba7d6cb9c44e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.266505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c082b72-6e2f-443d-8cb4-76f1ba855c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.267198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082b72-6e2f-443d-8cb4-76f1ba855c30 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.267481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082b72-6e2f-443d-8cb4-76f1ba855c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.267757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082b72-6e2f-443d-8cb4-76f1ba855c30 None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/4527e4500f984b1a81214c3656e7947f/roles/b5c148877d4a489f85a6d554d44dba3f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.283123 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c082b72-6e2f-443d-8cb4-76f1ba855c30 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=4527e4500f984b1a81214c3656e7947f, role_id=b5c148877d4a489f85a6d554d44dba3f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.284666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c082b72-6e2f-443d-8cb4-76f1ba855c30 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.295880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:55.298387 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.299018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.299289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.299559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.300284 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:55.300587 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.300883 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 282/583] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:55 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/4527e4500f984b1a81214c3656e7947f/roles/b5c148877d4a489f85a6d554d44dba3f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:55.301963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c082b72-6e2f-443d-8cb4-76f1ba855c30 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.304269 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.325798 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 302/584] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:55.327398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.434037 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c082b72-6e2f-443d-8cb4-76f1ba855c30 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.476793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.477458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.477743 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.478017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.478257 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c1462-0583-48bc-a032-efd0a96dc1d0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.491981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.493458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:55.506099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:55.507080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:55.507981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 283/585] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:55.508543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.535815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.536825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.537105 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.537624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.551356 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.552881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:55.568615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:55.569534 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:55.570414 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:55.570414 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:55.570414 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:55.570414 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:55.570644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 303/586] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:55.571266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.574461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:55.582481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:55.636873 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08550-7297-433e-8d51-1bd67d930b08 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.686827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.687406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.687681 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.687953 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] PATH_INFO: `/v3/users/glance` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.689278 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93138e81-9a5b-49c5-8725-a2e6f8af5d07 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.698413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.700300 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:55.716869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:55.717841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:01:55.718827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:55.721764 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 284/587] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:55.722978 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.742192 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.743248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.743537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.743807 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.744774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:55.745181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.747100 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.774193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 304/588] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:55.775607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.841898 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:55.860340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a5381-700b-4c92-a354-53a344ed59ec None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.885509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:55.886040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763e4ca-f074-4a83-af88-b94fda398d16 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:55.903621 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.904406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.904804 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.905085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.905787 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:55.906087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.907893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.928704 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 285/589] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:55.928704 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:55.932371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:55.933261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:55.933764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:55.934275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:55.935350 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:55.935858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:55.937689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:55.958857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 305/590] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:55.960343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.041292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65760958-d78f-4f9e-8e08-00c85c382bf7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.082856 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.084017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.084499 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.084995 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.086964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520b769-63e3-43d7-9624-833eba30bc31 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.096885 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.098850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.112233 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:56.117003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:56.117894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:56.118816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:56.118816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:56.118816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:56.118816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:56.121307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 286/591] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:56.122553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.134840 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.135511 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.135799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.136063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.147899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.149429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.162842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:56.170180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:56.171193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:01:56.172101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:56.176508 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 306/592] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:56.177547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.245622 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cd1cc-3df2-4dff-a98c-b6aeb2773957 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.283707 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.284414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.284693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.284973 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.300185 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.301949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.313261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bd0df-ffa6-4f59-bb7a-2b3c13df05d1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:56.321641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:56.322748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:56.323593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:56.323593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:56.323593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:56.323593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:56.323593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 287/593] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:56.324178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.353381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.354466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.354950 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.355423 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.356403 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:56.356910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.358550 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.371832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 307/594] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:56.373040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.414988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:56.439364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:56.462012 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518bb5-ac3f-4779-8caf-4ff6310d3d71 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.496694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def6bc-19ee-412a-b13c-e6f67cd9e390 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.500344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.500687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.501112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.501484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.502530 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:56.503155 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.504918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.529766 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 288/595] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:56.529766 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.550780 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.552265 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.552695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.553361 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.554497 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:56.554970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.557226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.577596 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 308/596] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:56.578728 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.644187 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b227a-cc8c-41e4-b9b4-3787a0b1c94f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.680676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.681250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.681627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.682002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.689243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.690908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:56.703125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:56.704175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:56.705266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:56.705266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:56.705266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:56.705266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:56.705266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:56.705266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:56.705266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:56.705266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:56.707092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 289/597] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:56.708094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.708993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:56.711152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:56.716504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a2fa4-4b65-4955-a731-1d3e6ed139cc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.762029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.762659 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.762933 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.763200 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.782148 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.784282 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:56.802144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:56.803179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:56.804046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:56.804046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:56.804046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:56.804046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:56.808336 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 309/598] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:56.809807 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.848206 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8144a-9101-4063-ab55-b6440ddc1d40 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.896609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.897976 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.898510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.898982 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.909115 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:56.911512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:56.928074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:56.929299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:56.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:56.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:56.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:56.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:56.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:56.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:56.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:56.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:56.932926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 290/599] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:56.934166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:56.946223 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9516d-d59c-4f2d-96c8-55fee1c1a0a6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:56.995997 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:56.996818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:56.997204 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:56.997579 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:56.998938 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:56.999251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:57.000726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:57.015466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 310/600] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:57.016675 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:57.030091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:57.031429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:57.082135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69f720-f284-4905-9bfd-ed2db23b9a71 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:57.123544 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accc4bb-c827-4647-8203-8b287d9c8ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:57.124291 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accc4bb-c827-4647-8203-8b287d9c8ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.124569 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accc4bb-c827-4647-8203-8b287d9c8ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.124938 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accc4bb-c827-4647-8203-8b287d9c8ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.125656 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accc4bb-c827-4647-8203-8b287d9c8ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:57.125945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accc4bb-c827-4647-8203-8b287d9c8ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:57.127505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accc4bb-c827-4647-8203-8b287d9c8ad7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:57.134469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fa951-fea4-4446-b8f3-756b145f4034 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:57.145893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 291/601] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:57.145893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58c588-a252-4969-a1c8-fe483c283956 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.145893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58c588-a252-4969-a1c8-fe483c283956 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.145893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58c588-a252-4969-a1c8-fe483c283956 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.145893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 292/602] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:57.145893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58c588-a252-4969-a1c8-fe483c283956 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:57.180637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:57.181270 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.181550 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.181822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.182630 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:57.183047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:57.184518 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:57.197672 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 311/603] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:57.198323 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:57.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58c588-a252-4969-a1c8-fe483c283956 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:57.281520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:57.308183 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:57.321590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6734cf4-58d3-4015-8d1d-f30830065af7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:57.323418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625ac8d-f2e0-4a2f-986b-cc55090e0291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:57.324102 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ac8d-f2e0-4a2f-986b-cc55090e0291 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.324375 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ac8d-f2e0-4a2f-986b-cc55090e0291 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.325016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ac8d-f2e0-4a2f-986b-cc55090e0291 None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/c96a4fc7279a44259990c423c5c35dea/roles/b5c148877d4a489f85a6d554d44dba3f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.342033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625ac8d-f2e0-4a2f-986b-cc55090e0291 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=c96a4fc7279a44259990c423c5c35dea, role_id=b5c148877d4a489f85a6d554d44dba3f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:57.343641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625ac8d-f2e0-4a2f-986b-cc55090e0291 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:57.365239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 293/604] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:57 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/c96a4fc7279a44259990c423c5c35dea/roles/b5c148877d4a489f85a6d554d44dba3f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:57.367033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1fb76-dd99-4134-8ee1-98cc673fe2a5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.367941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1fb76-dd99-4134-8ee1-98cc673fe2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.368702 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1fb76-dd99-4134-8ee1-98cc673fe2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.372973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:57.373663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.373954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.374267 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.384586 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:57.386919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:57.394067 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4ca1fb76-dd99-4134-8ee1-98cc673fe2a5 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:57.396203 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4ca1fb76-dd99-4134-8ee1-98cc673fe2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:57.402726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:57.403676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:57.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:57.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:57.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:57.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:57.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:57.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:57.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:57.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:57.409737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 312/605] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:57.410758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:57.506139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 294/606] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:57.506139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1fb76-dd99-4134-8ee1-98cc673fe2a5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:57.551865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44f7d8-da0d-4da0-a0a4-99a7ffd21d5f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:57.587138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:57.595937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e85ff-78e5-4cae-bebf-c7bbe11a8704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:57.597099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e85ff-78e5-4cae-bebf-c7bbe11a8704 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.597921 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e85ff-78e5-4cae-bebf-c7bbe11a8704 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.598222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e85ff-78e5-4cae-bebf-c7bbe11a8704 None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/fd2964a3f5aa41ce8c40945a529deb5c/roles/b5c148877d4a489f85a6d554d44dba3f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.616394 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1fb76-dd99-4134-8ee1-98cc673fe2a5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:57.616713 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e85ff-78e5-4cae-bebf-c7bbe11a8704 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=fd2964a3f5aa41ce8c40945a529deb5c, role_id=b5c148877d4a489f85a6d554d44dba3f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:57.618155 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:57.618285 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e85ff-78e5-4cae-bebf-c7bbe11a8704 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:57.642137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 313/607] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:57 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/fd2964a3f5aa41ce8c40945a529deb5c/roles/b5c148877d4a489f85a6d554d44dba3f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:57.642137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e211d3-3f4c-4965-9d5a-74059c36faec None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.642137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e211d3-3f4c-4965-9d5a-74059c36faec None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.642137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e211d3-3f4c-4965-9d5a-74059c36faec None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.642137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 314/608] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:57.642137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e211d3-3f4c-4965-9d5a-74059c36faec None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:57.682499 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79990985-2db3-4fe1-9383-1983cf3ffa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:57.683476 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79990985-2db3-4fe1-9383-1983cf3ffa3b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.683762 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79990985-2db3-4fe1-9383-1983cf3ffa3b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.684040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79990985-2db3-4fe1-9383-1983cf3ffa3b None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.684842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79990985-2db3-4fe1-9383-1983cf3ffa3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:57.685461 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79990985-2db3-4fe1-9383-1983cf3ffa3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:57.687733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79990985-2db3-4fe1-9383-1983cf3ffa3b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:57.711402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 295/609] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:57.713013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629713-af29-42da-a6c4-e558f9d1a587 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.713664 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629713-af29-42da-a6c4-e558f9d1a587 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.713963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629713-af29-42da-a6c4-e558f9d1a587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.738232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2b629713-af29-42da-a6c4-e558f9d1a587 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:57.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2b629713-af29-42da-a6c4-e558f9d1a587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:57.794481 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e211d3-3f4c-4965-9d5a-74059c36faec None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:57.850308 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 296/610] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:01:57.851211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b629713-af29-42da-a6c4-e558f9d1a587 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:57.853346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea8927-e68c-46a5-b61e-2948b170e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:57.854072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea8927-e68c-46a5-b61e-2948b170e7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:57.854967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea8927-e68c-46a5-b61e-2948b170e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:57.857293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea8927-e68c-46a5-b61e-2948b170e7c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:57.871092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea8927-e68c-46a5-b61e-2948b170e7c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-43ea8927-e68c-46a5-b61e-2948b170e7c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:57.877521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:57.879320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:57.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 315/611] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:57.881285 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea8927-e68c-46a5-b61e-2948b170e7c5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:57.948619 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:57.968795 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:57.982122 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b629713-af29-42da-a6c4-e558f9d1a587 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:58.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4908f24-6200-448d-aef7-950fb34f6983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:58.026863 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4908f24-6200-448d-aef7-950fb34f6983 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:58.027204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4908f24-6200-448d-aef7-950fb34f6983 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:58.027538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4908f24-6200-448d-aef7-950fb34f6983 None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/2ae2e70a28824ac58182627a8c7bf702/roles/b5c148877d4a489f85a6d554d44dba3f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:58.028052 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea8927-e68c-46a5-b61e-2948b170e7c5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:58.048420 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4908f24-6200-448d-aef7-950fb34f6983 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=2ae2e70a28824ac58182627a8c7bf702, role_id=b5c148877d4a489f85a6d554d44dba3f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:58.050187 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4908f24-6200-448d-aef7-950fb34f6983 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:58.068281 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 297/612] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:01:57 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/2ae2e70a28824ac58182627a8c7bf702/roles/b5c148877d4a489f85a6d554d44dba3f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:01:58.069273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4908f24-6200-448d-aef7-950fb34f6983 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:58.079029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b86a-4629-4843-b033-beb36d96eeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:58.079801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b86a-4629-4843-b033-beb36d96eeff None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:58.080097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b86a-4629-4843-b033-beb36d96eeff None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:58.080411 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b86a-4629-4843-b033-beb36d96eeff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:58.100251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159b86a-4629-4843-b033-beb36d96eeff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1159b86a-4629-4843-b033-beb36d96eeff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:01:58.111954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:58.114189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:58.116140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 316/613] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:01:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:58.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b86a-4629-4843-b033-beb36d96eeff None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:58.188538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4908f24-6200-448d-aef7-950fb34f6983 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:58.230252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:58.230933 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:58.231212 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:58.231486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:58.232347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:58.232659 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:58.234423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:58.244652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 298/614] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:58.250481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:58.266534 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:58.312705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:58.320234 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b86a-4629-4843-b033-beb36d96eeff None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:58.387495 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66270e02-a020-408e-8dbb-deebc23c5ab3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:58.397098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:58.397098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:58.397098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:58.397098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:58.397872 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:58.397872 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:58.398515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:58.410079 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 317/615] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:01:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:58.423231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:58.446457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:58.447144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:58.447422 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:58.447689 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:58.469899 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:58.472728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:58.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:58.505076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:58.505910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:58.505910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:58.505910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:58.505910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:58.505910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 299/616] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:58.512710 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:58.622795 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:58.630350 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a5622-91ad-4bdb-81f7-a03e01f0b8ec None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:58.693227 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:58.693811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:58.694090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:58.694389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:58.712516 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c7b6e-9158-4a72-928d-4e5887440d6e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:58.716307 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:58.717812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:58.735582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:58.736490 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:58.737429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:58.737429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:58.737429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:58.737429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:58.741544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 318/617] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:58.742887 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:58.759858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:58.800713 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:58.801363 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:58.801647 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:58.801919 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:58.802655 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:58.802962 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:58.804353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:58.840757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 300/618] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:58.851160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:58.853157 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e1c6c-270f-45f6-af36-4d3db3ec9bcb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:58.895451 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:58.896098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:58.896375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:58.896643 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:58.897354 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:58.897658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:58.900994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:58.928485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 319/619] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:58.932735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:59.021678 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6580a-969e-47e5-ba83-329363b89d16 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:59.043673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:59.066143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:59.066822 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.067101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.067372 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.076977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:59.078492 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:59.079099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:59.093213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:59.095377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:59.096478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 301/620] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:01:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:01:59.103390 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:59.121752 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2e8ca-6375-48d4-a043-879917692c59 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:59.158085 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:59.158962 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.159484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.159823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.168554 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:59.170184 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:01:59.182917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:59.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:01:59.186965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:01:59.194131 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 320/621] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:01:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:59.194131 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:59.257502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d63297-a624-4172-8efb-c4672718b6c5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:59.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:59.300566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.300825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.301103 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.301934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:59.302267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:59.306482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:59.326132 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 302/622] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:01:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:01:59.333637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:59.371176 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:59.436398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2efc-e7ec-4714-9c6e-f4339c785982 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:59.444604 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002de98-10b3-4e65-87db-0b0176244001 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:59.482584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:59.483365 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.483642 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.483915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.495447 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:59.496933 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:01:59.513229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:59.514617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:59.515440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:01:59.515440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:01:59.515440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:01:59.515440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:59.515440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 303/623] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:59.520507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:59.526954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:59.527599 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.527879 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.528150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.528972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:59.529278 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:59.537871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:59.542585 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:59.574232 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 321/624] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:01:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:59.581230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:59.647726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:59.664262 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0aa5aa-c0c6-4288-bf3f-6b21d41b966d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:59.696176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76b6fd-223f-4804-98c8-6e5efc789eda None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:59.700327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e32e-b5c0-4100-a6a7-3045bf1ff5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:59.701280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e32e-b5c0-4100-a6a7-3045bf1ff5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.701763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e32e-b5c0-4100-a6a7-3045bf1ff5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.702258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e32e-b5c0-4100-a6a7-3045bf1ff5b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.703172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e32e-b5c0-4100-a6a7-3045bf1ff5b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:01:59.703682 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e32e-b5c0-4100-a6a7-3045bf1ff5b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:59.705272 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e32e-b5c0-4100-a6a7-3045bf1ff5b1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:59.718860 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 304/625] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:01:59.720503 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e92dd-8728-48c6-8a9c-fdd4f92c8071 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.720986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e92dd-8728-48c6-8a9c-fdd4f92c8071 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.721456 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e92dd-8728-48c6-8a9c-fdd4f92c8071 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.723431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 305/626] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:01:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:01:59.724417 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e92dd-8728-48c6-8a9c-fdd4f92c8071 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:01:59.745565 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcbbe8-dfaa-4a36-a2b4-8d6ffb7d0fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:59.746246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcbbe8-dfaa-4a36-a2b4-8d6ffb7d0fbf None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.746531 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcbbe8-dfaa-4a36-a2b4-8d6ffb7d0fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.746806 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcbbe8-dfaa-4a36-a2b4-8d6ffb7d0fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.816406 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:59.830255 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcbbe8-dfaa-4a36-a2b4-8d6ffb7d0fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:59.832386 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcbbe8-dfaa-4a36-a2b4-8d6ffb7d0fbf None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:59.851210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e92dd-8728-48c6-8a9c-fdd4f92c8071 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:01:59.888280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc263b4-c178-42be-b2ef-795bbc2c23c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:01:59.888948 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc263b4-c178-42be-b2ef-795bbc2c23c1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.889231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc263b4-c178-42be-b2ef-795bbc2c23c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.889503 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc263b4-c178-42be-b2ef-795bbc2c23c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.897248 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc263b4-c178-42be-b2ef-795bbc2c23c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:01:59.898964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc263b4-c178-42be-b2ef-795bbc2c23c1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cdc263b4-c178-42be-b2ef-795bbc2c23c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:01:59.914676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:01:59.915605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:01:59.916527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 306/627] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:01:59.917610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b0a32-681a-4b02-a14d-cfe22b51e9fd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:01:59.917895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b0a32-681a-4b02-a14d-cfe22b51e9fd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:01:59.918200 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b0a32-681a-4b02-a14d-cfe22b51e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:01:59.943325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:01:59.956018 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-477b0a32-681a-4b02-a14d-cfe22b51e9fd None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:01:59.958472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-477b0a32-681a-4b02-a14d-cfe22b51e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:01:59.968577 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 322/628] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:01:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:01:59.968577 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcbbe8-dfaa-4a36-a2b4-8d6ffb7d0fbf None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.054242 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 307/629] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:00.055479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b0a32-681a-4b02-a14d-cfe22b51e9fd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.065732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:00.076140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcbbe8-dfaa-4a36-a2b4-8d6ffb7d0fbf None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:00.112955 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5999468-2271-4a3e-a8d5-d1b30766c5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:00.113529 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5999468-2271-4a3e-a8d5-d1b30766c5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.113812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5999468-2271-4a3e-a8d5-d1b30766c5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.114074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5999468-2271-4a3e-a8d5-d1b30766c5e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.114857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5999468-2271-4a3e-a8d5-d1b30766c5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:00.115154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5999468-2271-4a3e-a8d5-d1b30766c5e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:00.116582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5999468-2271-4a3e-a8d5-d1b30766c5e8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:00.128898 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 323/630] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:00.130326 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da820362-e018-4327-8fc5-5f70ebef23ea None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.130445 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da820362-e018-4327-8fc5-5f70ebef23ea None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.130688 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da820362-e018-4327-8fc5-5f70ebef23ea None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.131953 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 324/631] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:00.132577 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da820362-e018-4327-8fc5-5f70ebef23ea None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.182031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b0a32-681a-4b02-a14d-cfe22b51e9fd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:00.219214 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:00.219879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.220162 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.220494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.223461 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:00.232247 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:00.233742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:00.242806 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da820362-e018-4327-8fc5-5f70ebef23ea None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:00.250429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:00.251463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:00.252463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:00.252463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:00.252463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:00.252463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:00.256818 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 308/632] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:00.257682 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.282486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578d730-cb31-49b6-86da-483b455c2ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:00.283498 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578d730-cb31-49b6-86da-483b455c2ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.284016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578d730-cb31-49b6-86da-483b455c2ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.284302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578d730-cb31-49b6-86da-483b455c2ab4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.295145 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578d730-cb31-49b6-86da-483b455c2ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b578d730-cb31-49b6-86da-483b455c2ab4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:00.300016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:00.300932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:00.301784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 325/633] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:02:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:00.303420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a916-18b0-46fc-a6a7-c6c2cc9df4d7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.303420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a916-18b0-46fc-a6a7-c6c2cc9df4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.303420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a916-18b0-46fc-a6a7-c6c2cc9df4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.321495 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-db91a916-18b0-46fc-a6a7-c6c2cc9df4d7 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:00.322865 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-db91a916-18b0-46fc-a6a7-c6c2cc9df4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:00.332134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:00.390532 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71784-1ce2-43fd-812e-b275725e1341 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:00.422716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 326/634] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:00.424570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a916-18b0-46fc-a6a7-c6c2cc9df4d7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.450494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8895e8-dc4f-4a82-a646-5ffe5c2ed12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:00.451567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8895e8-dc4f-4a82-a646-5ffe5c2ed12a None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.452036 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8895e8-dc4f-4a82-a646-5ffe5c2ed12a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.452487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8895e8-dc4f-4a82-a646-5ffe5c2ed12a None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/f285251a0e0a49849d071155d2d3e478/roles/b5c148877d4a489f85a6d554d44dba3f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.476386 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8895e8-dc4f-4a82-a646-5ffe5c2ed12a None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=f285251a0e0a49849d071155d2d3e478, role_id=b5c148877d4a489f85a6d554d44dba3f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:00.478414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8895e8-dc4f-4a82-a646-5ffe5c2ed12a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:00.494530 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 309/635] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:02:00 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/f285251a0e0a49849d071155d2d3e478/roles/b5c148877d4a489f85a6d554d44dba3f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:02:00.495383 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8895e8-dc4f-4a82-a646-5ffe5c2ed12a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.542440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:00.584175 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a916-18b0-46fc-a6a7-c6c2cc9df4d7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:00.623294 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8895e8-dc4f-4a82-a646-5ffe5c2ed12a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:00.634390 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:00.635019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.635299 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.635577 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.636301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:00.636606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:00.638060 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:00.651089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 327/636] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:00.651977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.658478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:00.673949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c2d03-5fad-487f-9fa3-d422665c536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:00.674894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2d03-5fad-487f-9fa3-d422665c536e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.674894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2d03-5fad-487f-9fa3-d422665c536e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.675301 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2d03-5fad-487f-9fa3-d422665c536e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.675919 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c2d03-5fad-487f-9fa3-d422665c536e None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:00.676224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c2d03-5fad-487f-9fa3-d422665c536e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:00.677618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c2d03-5fad-487f-9fa3-d422665c536e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:00.689433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 310/637] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:02:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:00.690646 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05f0c9-ba9b-4cc2-8036-65f1a271b0c5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.690924 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05f0c9-ba9b-4cc2-8036-65f1a271b0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.691196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05f0c9-ba9b-4cc2-8036-65f1a271b0c5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.692427 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 311/638] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:00.693118 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05f0c9-ba9b-4cc2-8036-65f1a271b0c5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.773721 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e280e5-1d9a-420c-bcca-0712dc3883b6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:00.810732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91283b9-62e4-4893-98e7-80e5228eba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:00.811388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91283b9-62e4-4893-98e7-80e5228eba35 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.811670 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91283b9-62e4-4893-98e7-80e5228eba35 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.811941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91283b9-62e4-4893-98e7-80e5228eba35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.813161 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:00.818604 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05f0c9-ba9b-4cc2-8036-65f1a271b0c5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:00.825150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91283b9-62e4-4893-98e7-80e5228eba35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b91283b9-62e4-4893-98e7-80e5228eba35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:00.830569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:00.831603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:00.832413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 328/639] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:02:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:00.833671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbacb9-f5e6-4744-bdbc-40091e43b6ae None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.833671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbacb9-f5e6-4744-bdbc-40091e43b6ae None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.833671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbacb9-f5e6-4744-bdbc-40091e43b6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.851988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-29bbacb9-f5e6-4744-bdbc-40091e43b6ae None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:00.853204 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-29bbacb9-f5e6-4744-bdbc-40091e43b6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:00.864350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:00.864997 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:00.865271 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:00.865543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:00.873125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:00.874606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:00.886091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:00.887053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:00.887915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 312/640] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:00.889245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:00.918298 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:00.930985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 329/641] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:00.931833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbacb9-f5e6-4744-bdbc-40091e43b6ae None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.017714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644a869-5ec1-44a7-9118-20aec210338c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.042615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbacb9-f5e6-4744-bdbc-40091e43b6ae None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.053612 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.054415 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.054689 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.054944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.067138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.068663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.079061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.079678 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.079974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.080227 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.080996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:01.081301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.082703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:01.084275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:01.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:01.086502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:01.086502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:01.086502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:01.086502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:01.086502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 313/642] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:01.086502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.097005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 330/643] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:02:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:01.097983 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.112610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:01.162304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:01.211820 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec0507-a5e1-492f-af2b-7f0bed8f3f19 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.212231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6aea0-a04b-4dcc-a75c-d242e31cb410 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.248339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad019b5e-6c64-43c7-9482-a6a0b26d574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.248749 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.249155 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad019b5e-6c64-43c7-9482-a6a0b26d574f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.249380 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.249566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad019b5e-6c64-43c7-9482-a6a0b26d574f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.249664 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.249909 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.250019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad019b5e-6c64-43c7-9482-a6a0b26d574f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.250716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:01.250995 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.252453 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.261763 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad019b5e-6c64-43c7-9482-a6a0b26d574f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:01.270871 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 314/644] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:01.270871 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ad019b5e-6c64-43c7-9482-a6a0b26d574f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:01.271174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:01.272194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 331/645] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:01.273321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad019b5e-6c64-43c7-9482-a6a0b26d574f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.350027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:01.380478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad019b5e-6c64-43c7-9482-a6a0b26d574f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.383692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f6149-3439-4c30-a212-b569bc37b023 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.395462 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:01.418797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.419743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.420020 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.420303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.420894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.421526 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.421806 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.422135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.422804 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:01.423108 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.424497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.431740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.433187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.441373 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 315/646] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:01.441373 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:01.450875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:01.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:01.452986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:01.452986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:01.452986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:01.452986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:01.453634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 332/647] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:01.454386 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.553393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819443b-ff82-4c8b-8319-3cbaa02ec07f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.561964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2aa30-e5a8-4d8f-b26b-9e0e5ab3b25b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.584651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:01.590340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.590977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.591258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.591528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.592327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=4527e4500f984b1a81214c3656e7947f) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:01.592637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.594078 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.599698 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.600514 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 316/648] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/role_assignments?user.id=4527e4500f984b1a81214c3656e7947f&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:01.600638 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.600779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.601147 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.601987 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:01.602316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.603713 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.614162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 333/649] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:01.615195 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.615644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.651014 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:01.739088 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8b87b-316f-4046-aa47-018e02e5ccb7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.746444 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0401b7-1357-4345-ae76-5d9054e928cf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.778800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.779671 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.779926 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.780980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.781783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:01.782178 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.783631 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.785700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.786936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.787220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.787497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.797464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:01.798520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 334/650] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:01.798974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:01.799543 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:01.816472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:01.817408 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:01.818418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 317/651] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:01.819014 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:01.891104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:01.922602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:01.946616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5ced3-24d4-4f48-ae9f-ca4531236024 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.947761 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d5e84-37f8-47a0-b942-78944b593087 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:01.989552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.990245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.990524 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.990794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:01.992440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:01.994002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:01.994539 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:01.995003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.002339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.003816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.007732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.009503 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:02.019896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:02.020818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:02.021674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:02.021674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:02.021674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:02.021674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:02.021674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 335/652] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:02.022314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:02.026910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:02.028135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 318/653] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:02.029034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.146445 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5c51f-955e-4dcb-a9a2-c533f5562c52 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:02.150859 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364d300-aa6d-4471-b3d4-5e375c1078a5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:02.168158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:02.188930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:02.189570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.189855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.190143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.190902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:02.191201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.192591 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.197890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:02.198805 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.199291 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.200681 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.201689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:02.202765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.204731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.206007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 319/654] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:02.206935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.222160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:02.231151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 336/655] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:02.232073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.340633 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1c26e-54bd-4be2-9473-0c867f85a190 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:02.342513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337823a-27c1-41c7-804b-b7ec202aed54 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:02.380717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:02.381732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.382222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.382680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.384482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:02.385410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.385697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.385974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.387463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:02.387775 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.389313 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.391955 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.393623 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:02.408054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:02.409072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:02:02.409999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:02.411557 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 337/656] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:02.411557 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.415724 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 320/657] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:02.417796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.455736 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:02.496809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:02.526456 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d459dcd-ad0a-401e-8ae3-64aa1931beb3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:02.557511 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0ab4-9e47-4f88-9e3b-bf368ad2bdd3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:02.561888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:02.562567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.562841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.563106 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.563907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:02.564207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.565773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.584619 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 321/658] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:02.585474 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.600859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b6c03-a736-495c-8dc4-da4c61015ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:02.601501 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6c03-a736-495c-8dc4-da4c61015ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.601761 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6c03-a736-495c-8dc4-da4c61015ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.602031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6c03-a736-495c-8dc4-da4c61015ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.672028 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6c03-a736-495c-8dc4-da4c61015ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.673402 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6c03-a736-495c-8dc4-da4c61015ab2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.685094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:02.697193 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e01a6-1724-470a-8917-dcb4254fad7c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:02.740978 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6bcb8-dbc7-49ab-9cb9-d67b79dedceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:02.741619 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6bcb8-dbc7-49ab-9cb9-d67b79dedceb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.741937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6bcb8-dbc7-49ab-9cb9-d67b79dedceb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.742215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6bcb8-dbc7-49ab-9cb9-d67b79dedceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.743571 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:02.788044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 338/659] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:02:02.789751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307aec3-067c-4953-a9a0-cc6443fd21b1 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.790151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307aec3-067c-4953-a9a0-cc6443fd21b1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.790499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307aec3-067c-4953-a9a0-cc6443fd21b1 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.791982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 339/660] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:02.796900 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307aec3-067c-4953-a9a0-cc6443fd21b1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.821179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6bcb8-dbc7-49ab-9cb9-d67b79dedceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:02.822745 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6bcb8-dbc7-49ab-9cb9-d67b79dedceb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:02.911069 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307aec3-067c-4953-a9a0-cc6443fd21b1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:02.921042 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:02.938134 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 322/661] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:02:02.939076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6bcb8-dbc7-49ab-9cb9-d67b79dedceb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:02.948774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23703cf1-8ae7-431b-9506-17d39bc53907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:02.949835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23703cf1-8ae7-431b-9506-17d39bc53907 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:02.950162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23703cf1-8ae7-431b-9506-17d39bc53907 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:02.950437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23703cf1-8ae7-431b-9506-17d39bc53907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:02.979166 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:03.032326 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23703cf1-8ae7-431b-9506-17d39bc53907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.034582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23703cf1-8ae7-431b-9506-17d39bc53907 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.041495 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6bcb8-dbc7-49ab-9cb9-d67b79dedceb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.078690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9383f3-1bd8-41af-a94c-b5b2cddbaded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.079391 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9383f3-1bd8-41af-a94c-b5b2cddbaded None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.079674 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9383f3-1bd8-41af-a94c-b5b2cddbaded None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.079895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9383f3-1bd8-41af-a94c-b5b2cddbaded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.092521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9383f3-1bd8-41af-a94c-b5b2cddbaded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.094033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9383f3-1bd8-41af-a94c-b5b2cddbaded None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1c9383f3-1bd8-41af-a94c-b5b2cddbaded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:03.111667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:03.112760 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:03.114596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:03.114596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:03.114596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:03.114596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:03.114596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 323/662] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:03.119088 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b2b11-271f-4d25-bc12-bb7586edc6d0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.119418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b2b11-271f-4d25-bc12-bb7586edc6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.119616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b2b11-271f-4d25-bc12-bb7586edc6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.139372 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-730b2b11-271f-4d25-bc12-bb7586edc6d0 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:03.140622 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-730b2b11-271f-4d25-bc12-bb7586edc6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:03.186670 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:03.221130 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 324/663] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:03.222381 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b2b11-271f-4d25-bc12-bb7586edc6d0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.282701 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 340/664] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:02:03.283571 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23703cf1-8ae7-431b-9506-17d39bc53907 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.345642 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:03.349428 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b2b11-271f-4d25-bc12-bb7586edc6d0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.384741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.385393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.385675 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.385934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.395247 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23703cf1-8ae7-431b-9506-17d39bc53907 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.396719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.398213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:03.413774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:03.414738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:03.415591 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:03.415591 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:03.415591 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:03.415591 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:03.419719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 325/665] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:03.420605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.430785 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.431563 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.431794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.432016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.432632 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:03.432932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.435744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.447596 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 341/666] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:03.448573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.463937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:03.545009 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da59e2-b830-4a50-b944-4bb22b64aff1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.561647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950c982-ebe1-48c6-9483-6ce26df1a4fb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.580448 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02285ff5-259c-4b5c-a436-0966145f0304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.581138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02285ff5-259c-4b5c-a436-0966145f0304 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.581414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02285ff5-259c-4b5c-a436-0966145f0304 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.581626 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:03.581737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02285ff5-259c-4b5c-a436-0966145f0304 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.593527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02285ff5-259c-4b5c-a436-0966145f0304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-02285ff5-259c-4b5c-a436-0966145f0304 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:03.598980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:03.599866 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:03.600686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:03.602048 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.602733 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.603013 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.603283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.605167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 326/667] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:03.606017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02285ff5-259c-4b5c-a436-0966145f0304 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.614788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.616278 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:03.631949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:03.633042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:03.633909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:03.633909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:03.633909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:03.633909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:03.633909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 342/668] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:03.633909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.712814 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:03.731377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02285ff5-259c-4b5c-a436-0966145f0304 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.752785 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2112fb6-7af3-41bb-8a1b-f38b338c225a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.770650 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.771284 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.771568 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.771842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.772539 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:03.772843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.774260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.790894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 327/669] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:03.790894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.791189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.791189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.791189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.791189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.797704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.799120 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:03.810444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:03.811537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 343/670] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:03.812404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.848664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:03.899571 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cbddc-2ec9-4280-ac5f-2992e189dc2d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.926452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe73f7-7fa2-4a9c-9411-63238c6a8977 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:03.935945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9c2da-a23c-4a2e-b9b7-130e5a781476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.936623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c2da-a23c-4a2e-b9b7-130e5a781476 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.936920 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c2da-a23c-4a2e-b9b7-130e5a781476 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.937188 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c2da-a23c-4a2e-b9b7-130e5a781476 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.937931 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9c2da-a23c-4a2e-b9b7-130e5a781476 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:03.938260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9c2da-a23c-4a2e-b9b7-130e5a781476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.939653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9c2da-a23c-4a2e-b9b7-130e5a781476 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.948957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 328/671] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:03.950737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd34d-9fc9-43bb-b175-1c5115ad429e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.951024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd34d-9fc9-43bb-b175-1c5115ad429e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.951292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd34d-9fc9-43bb-b175-1c5115ad429e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.952557 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 329/672] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:03.953260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fd34d-9fc9-43bb-b175-1c5115ad429e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:03.965299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:03.972406 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:03.973044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:03.973324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:03.973584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:03.974343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:03.974650 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:03.976035 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:03.990687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 344/673] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:03.991559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.063570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fd34d-9fc9-43bb-b175-1c5115ad429e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.085290 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:04.094917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433b5f9-06f2-4ab5-bf26-bdb1a909608f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.101757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.102445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.102721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.102987 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.109897 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.111406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:04.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:04.123562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:04.124551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 330/674] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:04.125085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.132555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09b483-c564-4e8e-b9d8-d4b06076f4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.133187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09b483-c564-4e8e-b9d8-d4b06076f4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.133509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09b483-c564-4e8e-b9d8-d4b06076f4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.133821 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09b483-c564-4e8e-b9d8-d4b06076f4c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.134628 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09b483-c564-4e8e-b9d8-d4b06076f4c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:04.134939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09b483-c564-4e8e-b9d8-d4b06076f4c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.136384 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09b483-c564-4e8e-b9d8-d4b06076f4c7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:04.150200 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 345/675] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:04.151553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca344e9-c914-4f94-8206-38df8c7fb0a5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.151828 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca344e9-c914-4f94-8206-38df8c7fb0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.152103 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca344e9-c914-4f94-8206-38df8c7fb0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.172781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dca344e9-c914-4f94-8206-38df8c7fb0a5 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:04.174009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dca344e9-c914-4f94-8206-38df8c7fb0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:04.201141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:04.238285 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf1954-0636-44d3-a4cf-2cc4f48b923e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.243317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 346/676] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:04.244162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca344e9-c914-4f94-8206-38df8c7fb0a5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.273983 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.274658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.274940 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.275211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.285679 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.287155 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:04.301040 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:04.302174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:04.303266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:04.303266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:04.303266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:04.303266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:04.303266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 331/677] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:04.303266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.327688 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:04.348691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca344e9-c914-4f94-8206-38df8c7fb0a5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.388376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.389054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.389338 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.389616 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.397374 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.398910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:04.411890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:04.412847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:04.413956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 347/678] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:04.414845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.419930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a4dd6-0394-466b-88c9-9bbef5a6b31c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.445247 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:04.463169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.463829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.464111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.464382 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.465151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:04.465460 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.466970 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:04.480258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 332/679] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:04.481184 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.526547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10ea33-f128-4648-9bd3-5dad7cdfa725 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.562192 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.563149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.563429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.563698 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.564527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=c96a4fc7279a44259990c423c5c35dea) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:04.564831 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.566361 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:04.573164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 348/680] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/role_assignments?user.id=c96a4fc7279a44259990c423c5c35dea&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:04.574381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.580018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:04.599665 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca17540-407b-465f-b4d8-91b0db2c4cce None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.640377 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34db438-1569-4a6e-8f11-f67cef4a564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.641076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34db438-1569-4a6e-8f11-f67cef4a564a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.641355 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34db438-1569-4a6e-8f11-f67cef4a564a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.641624 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34db438-1569-4a6e-8f11-f67cef4a564a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.654125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34db438-1569-4a6e-8f11-f67cef4a564a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a34db438-1569-4a6e-8f11-f67cef4a564a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:04.659278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:04.660277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:04.661104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 333/681] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:04.661880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34db438-1569-4a6e-8f11-f67cef4a564a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.717386 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e2ec8-056b-4496-82ff-61af7f9f2682 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.719496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:04.763051 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.763707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.763986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.764263 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.765015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:04.766159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.766837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:04.782023 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 349/682] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:04.783437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.835009 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34db438-1569-4a6e-8f11-f67cef4a564a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.888273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.889244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.889712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.890171 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.891451 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:04.891968 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.894324 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:04.894495 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:04.913592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 334/683] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:04.914976 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:04.952505 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41957f7-dfd7-413a-a9c4-7f632cb7b3a4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:04.990924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:04.991553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:04.991832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:04.992138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:04.992911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=fd2964a3f5aa41ce8c40945a529deb5c) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:04.993225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:04.994724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:05.001372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 350/684] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/role_assignments?user.id=fd2964a3f5aa41ce8c40945a529deb5c&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:05.007118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:05.045973 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:05.050243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972cca6-cf69-4a43-836b-aba724415b39 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:05.119027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:05.119601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:05.119884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:05.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:05.120908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:05.121219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:05.122699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:05.136356 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 335/685] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:05.137064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:05.159576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ec642-e8cd-480c-9a88-d3d8d5e9a434 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:05.196944 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:05.208140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bddaf-5ece-479e-af55-29714982104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:05.209146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bddaf-5ece-479e-af55-29714982104b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:05.209433 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bddaf-5ece-479e-af55-29714982104b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:05.210491 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bddaf-5ece-479e-af55-29714982104b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:05.218693 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bddaf-5ece-479e-af55-29714982104b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:05.220835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bddaf-5ece-479e-af55-29714982104b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-282bddaf-5ece-479e-af55-29714982104b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:05.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:05.233967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:05.234970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 351/686] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:05.235551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bddaf-5ece-479e-af55-29714982104b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:05.269252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0e211-0e59-4ca3-98b0-68880748a5d2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:05.319261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:05.320239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:05.320828 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:05.321104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:05.322152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=2ae2e70a28824ac58182627a8c7bf702) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:05.322266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:05.323720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:05.331339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 336/687] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:05 2022] GET /identity/v3/role_assignments?user.id=2ae2e70a28824ac58182627a8c7bf702&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:05.332295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:05.376154 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:05.423203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bddaf-5ece-479e-af55-29714982104b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:05.510573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:05.511248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:05.511518 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:05.511785 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:05.529332 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:05.530918 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:05.538183 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c5e4b-9628-4119-9de5-abb44118304a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:05.548428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:05.549364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:05.550340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:05.550340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:05.550340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:05.550340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:05.550340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 352/688] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:05.557194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:05.586571 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:05.587678 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:05.588294 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:05.588817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:05.589892 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:05.590330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:05.591924 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:05.609838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:05.615818 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 337/689] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:02:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:05.623632 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:05.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e033d6-4e19-453f-af8f-77492f60d8a7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:05.720015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:05.720734 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:05.721020 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:05.726505 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:05.727160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:05.728347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:05.728673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:05.730292 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:05.742933 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835c2d-ebc1-4fb7-8906-de75a34474e4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:05.746222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 353/690] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:05.752176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:05.830026 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5c79c-dd1b-4b00-b501-3df775646165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:05.831033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c79c-dd1b-4b00-b501-3df775646165 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:05.831504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c79c-dd1b-4b00-b501-3df775646165 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:05.831957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c79c-dd1b-4b00-b501-3df775646165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:05.947850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5c79c-dd1b-4b00-b501-3df775646165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:05.950182 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5c79c-dd1b-4b00-b501-3df775646165 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:05.967156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:05.972515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76a383-5c7b-4883-833d-06ff59b80ffe None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:06.035797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:06.042185 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.042470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.042741 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:06.044215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:06.051460 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:06.052917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:06.063891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 338/691] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:02:06.069773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5c79c-dd1b-4b00-b501-3df775646165 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:06.079760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:06.081115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:06.082075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 354/692] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:02:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:06.087444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:06.179243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5c79c-dd1b-4b00-b501-3df775646165 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:06.231065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:06.232106 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.232721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.233986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:06.246578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:06.248414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:06.272268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:06.273219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:06.274085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:06.274085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:06.274085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:06.274085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:06.283941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 339/693] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:06.290991 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:06.307313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d8b33-78ad-4b82-90a5-e92e215d2197 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:06.366530 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:06.373690 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:06.374685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.374685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.374943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:06.375724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:06.376034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:06.377481 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:06.412379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 355/694] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:02:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:06.418878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:06.442793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b0170-1e62-41bc-af98-9965d3b3a78b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:06.459142 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:06.516591 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:06.517147 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.517425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.517698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:06.518483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:06.518791 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:06.522946 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:06.546509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 340/695] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:06.552661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:06.618638 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a4df3-df76-48a7-a917-0f8f44f40a86 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:06.684496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:06.693385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a77d2c-c494-4299-99fa-10711ade4dd5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:06.703565 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cbda9-e4da-4e73-bcd1-69468ae1d89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:06.704602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cbda9-e4da-4e73-bcd1-69468ae1d89d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.705081 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cbda9-e4da-4e73-bcd1-69468ae1d89d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.705547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cbda9-e4da-4e73-bcd1-69468ae1d89d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:06.765112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:06.765788 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.766069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.766366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:06.774335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:06.776492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:06.790062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:06.791002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:06.791880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:06.791880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:06.791880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:06.791880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:06.791880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:06.791880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:06.791880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:06.791880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:06.792858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 341/696] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:06.798650 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:06.878452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159cbda9-e4da-4e73-bcd1-69468ae1d89d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:06.879999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159cbda9-e4da-4e73-bcd1-69468ae1d89d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:06.899321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:06.916132 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1f61-df99-40b6-8a3f-072d6aefb34c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:06.955641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85c20c-08e8-43f5-ad77-812f45002e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:06.956458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c20c-08e8-43f5-ad77-812f45002e71 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.956801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c20c-08e8-43f5-ad77-812f45002e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.957160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c20c-08e8-43f5-ad77-812f45002e71 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:06.957994 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85c20c-08e8-43f5-ad77-812f45002e71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:06.958407 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85c20c-08e8-43f5-ad77-812f45002e71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:06.959941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85c20c-08e8-43f5-ad77-812f45002e71 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:06.978481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 342/697] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:06.978481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b81c2-09b4-4bd0-84d4-05c41d177887 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.978481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b81c2-09b4-4bd0-84d4-05c41d177887 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.978481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b81c2-09b4-4bd0-84d4-05c41d177887 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:06.981782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 343/698] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:06.983356 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b203f-25c8-43ff-abdb-124657cb625a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:06.983642 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b203f-25c8-43ff-abdb-124657cb625a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:06.983921 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b203f-25c8-43ff-abdb-124657cb625a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.005382 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ff7b203f-25c8-43ff-abdb-124657cb625a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:07.006639 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ff7b203f-25c8-43ff-abdb-124657cb625a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:07.018917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:07.028975 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 356/699] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:02:07.030151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cbda9-e4da-4e73-bcd1-69468ae1d89d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.094386 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 344/700] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:07.095303 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b203f-25c8-43ff-abdb-124657cb625a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.161748 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cbda9-e4da-4e73-bcd1-69468ae1d89d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:07.185764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:07.207427 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:07.208377 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.208872 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.209333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.210347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=f285251a0e0a49849d071155d2d3e478) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:07.210836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:07.212538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:07.219602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 357/701] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/role_assignments?user.id=f285251a0e0a49849d071155d2d3e478&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:07.226260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b203f-25c8-43ff-abdb-124657cb625a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:07.227814 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.259482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:07.260242 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.260578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.260958 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.272582 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:07.274205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:07.282191 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:07.292506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:07.293450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:07.294472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:07.294472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:07.294472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:07.294472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:07.294472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 345/702] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:07.299576 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.357262 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c45e1-f3bb-4a27-bbc0-e69d16f40168 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:07.394964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d29e28-d22e-46a2-a6f3-b544429de259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:07.396056 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d29e28-d22e-46a2-a6f3-b544429de259 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.396539 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d29e28-d22e-46a2-a6f3-b544429de259 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.397035 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d29e28-d22e-46a2-a6f3-b544429de259 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.397909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d29e28-d22e-46a2-a6f3-b544429de259 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:07.399576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d29e28-d22e-46a2-a6f3-b544429de259 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-24d29e28-d22e-46a2-a6f3-b544429de259 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:07.412073 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:07.413107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:07.413920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:07.416994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 358/703] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:07.418661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744dfcc-b16b-485d-9a02-2abc50107993 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.419227 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744dfcc-b16b-485d-9a02-2abc50107993 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.419706 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744dfcc-b16b-485d-9a02-2abc50107993 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.421638 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 359/704] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:07.427804 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da0ed-2a20-405d-b7ee-14f2458fe130 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.428141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da0ed-2a20-405d-b7ee-14f2458fe130 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.428423 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da0ed-2a20-405d-b7ee-14f2458fe130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.448535 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-574da0ed-2a20-405d-b7ee-14f2458fe130 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:07.450371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-574da0ed-2a20-405d-b7ee-14f2458fe130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:07.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921a97d-2d2d-4774-bbeb-0504f8e50a2f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:07.491368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:07.492009 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.492285 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.492568 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.493255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:07.493547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:07.494981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:07.506454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:07.508444 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 346/705] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:07.509368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.537820 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 360/706] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:02:07.538539 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574da0ed-2a20-405d-b7ee-14f2458fe130 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.600720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:07.656378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c0a02-2139-48ef-b3b9-fb1ceadaa6db None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:07.677604 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574da0ed-2a20-405d-b7ee-14f2458fe130 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:07.700781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:07.701668 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.701944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.702275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.703199 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:07.703858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:07.705307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:07.724380 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 347/707] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:07.725601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.732753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:07.733472 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.733755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.734024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.743680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:07.745418 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:07.760044 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:07.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:07.761905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 361/708] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:07.762412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.804956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:07.841477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acafa0-237b-4ba0-9639-8e37af2ffa3c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:07.865802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:07.878127 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:07.879160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.879675 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.880190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.880487 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a572d38-f084-4e52-9a56-cc25fd0e6773 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:07.891513 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:07.893382 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:07.909087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:07.910007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:07.910908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:07.910908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:07.910908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:07.910908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:07.913132 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 348/709] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:07.914164 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:07.918275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:07.919552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:07.919552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:07.919552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:07.920251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:07.920777 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:07.922803 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:07.930059 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 362/710] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:07.931173 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:08.029077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89f0bd-82e7-4ad8-a68b-d8aa7ccb3b76 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:08.051027 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:08.055100 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d765985-3fa0-4be6-bfb6-c55f5e002b0a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:08.068779 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1d7af-330c-4dc0-a8cc-da28d0e11e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:08.069820 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1d7af-330c-4dc0-a8cc-da28d0e11e60 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.070352 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1d7af-330c-4dc0-a8cc-da28d0e11e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.070882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1d7af-330c-4dc0-a8cc-da28d0e11e60 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.071877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1d7af-330c-4dc0-a8cc-da28d0e11e60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:08.072397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1d7af-330c-4dc0-a8cc-da28d0e11e60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:08.074075 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1d7af-330c-4dc0-a8cc-da28d0e11e60 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:08.089751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 349/711] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:08.091457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec4281-1802-43b9-903b-bc141a293ae9 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.091963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec4281-1802-43b9-903b-bc141a293ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.092260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec4281-1802-43b9-903b-bc141a293ae9 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.093693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 350/712] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:08.095110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec4281-1802-43b9-903b-bc141a293ae9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:08.096875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b6cb5-a332-48d5-8531-c959b71fb28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:08.097521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b6cb5-a332-48d5-8531-c959b71fb28c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.097827 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b6cb5-a332-48d5-8531-c959b71fb28c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.098199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b6cb5-a332-48d5-8531-c959b71fb28c None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.099007 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b6cb5-a332-48d5-8531-c959b71fb28c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:08.099367 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b6cb5-a332-48d5-8531-c959b71fb28c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:08.101061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b6cb5-a332-48d5-8531-c959b71fb28c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:08.114013 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 363/713] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:08.115119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62210b-9249-49bd-9ac7-c75d3479f374 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.115397 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62210b-9249-49bd-9ac7-c75d3479f374 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.115667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62210b-9249-49bd-9ac7-c75d3479f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.134725 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:08.136312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3b62210b-9249-49bd-9ac7-c75d3479f374 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:08.137525 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3b62210b-9249-49bd-9ac7-c75d3479f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:08.213609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 364/714] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:08.214741 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62210b-9249-49bd-9ac7-c75d3479f374 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:08.219104 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec4281-1802-43b9-903b-bc141a293ae9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:08.257609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:08.258643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.259069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.259510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.260485 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=4527e4500f984b1a81214c3656e7947f) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:08.260992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:08.262735 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:08.269722 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 351/715] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:08 2022] GET /identity/v3/role_assignments?user.id=4527e4500f984b1a81214c3656e7947f&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=65cd37a73e4f498482fefabb8374006c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:08.270789 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:08.316473 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:08.328942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62210b-9249-49bd-9ac7-c75d3479f374 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:08.368593 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c191b-c880-4d6f-8a18-ca9333a7ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:08.369484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c191b-c880-4d6f-8a18-ca9333a7ec96 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.369767 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c191b-c880-4d6f-8a18-ca9333a7ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.370038 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c191b-c880-4d6f-8a18-ca9333a7ec96 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.371198 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c191b-c880-4d6f-8a18-ca9333a7ec96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:08.372655 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c191b-c880-4d6f-8a18-ca9333a7ec96 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:08.383616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:08.389094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72cc45-afb4-4847-9453-d8a3264363a1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:08.402367 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 365/716] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:02:08 2022] POST /identity/v3/users => generated 254 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:08.444580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:08.445589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.446021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.446539 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] PATH_INFO: `/v3/services/network` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.447430 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:08.449039 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:08.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:08.467805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:08.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:08.476028 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 352/717] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:08 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:08.482561 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c191b-c880-4d6f-8a18-ca9333a7ec96 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:08.642498 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c191b-c880-4d6f-8a18-ca9333a7ec96 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:08.683381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaf65e-d31c-4f18-a043-a1618e1ad5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:08.684046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf65e-d31c-4f18-a043-a1618e1ad5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.684312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf65e-d31c-4f18-a043-a1618e1ad5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.684580 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf65e-d31c-4f18-a043-a1618e1ad5b7 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.685287 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf65e-d31c-4f18-a043-a1618e1ad5b7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:08.685598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf65e-d31c-4f18-a043-a1618e1ad5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:08.687132 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf65e-d31c-4f18-a043-a1618e1ad5b7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:08.692741 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 366/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:08.698728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:08.828473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305fe5a-29ec-4e95-b998-c4f54d250608 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:08.871210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8ae12-0b59-4b95-ac6a-c8fc332e77f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:08.871884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ae12-0b59-4b95-ac6a-c8fc332e77f2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:08.872161 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ae12-0b59-4b95-ac6a-c8fc332e77f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:08.872431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ae12-0b59-4b95-ac6a-c8fc332e77f2 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:08.873136 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8ae12-0b59-4b95-ac6a-c8fc332e77f2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:08.873431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8ae12-0b59-4b95-ac6a-c8fc332e77f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:08.874985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8ae12-0b59-4b95-ac6a-c8fc332e77f2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:08.886282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 353/719] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:09.779956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce66d2-6ef3-4c62-a436-c8e8fbf3e87e None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:09.780499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce66d2-6ef3-4c62-a436-c8e8fbf3e87e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:09.780499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce66d2-6ef3-4c62-a436-c8e8fbf3e87e None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:09.782087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 367/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:09.787473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dce7b6-98a6-4ba9-a4e0-917f52f9d125 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:09.787983 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dce7b6-98a6-4ba9-a4e0-917f52f9d125 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:09.788478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dce7b6-98a6-4ba9-a4e0-917f52f9d125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:09.807080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-65dce7b6-98a6-4ba9-a4e0-917f52f9d125 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:09.808549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-65dce7b6-98a6-4ba9-a4e0-917f52f9d125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:09.830524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:09.890387 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 354/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:09.928238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce66d2-6ef3-4c62-a436-c8e8fbf3e87e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:09.942257 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:10.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce66d2-6ef3-4c62-a436-c8e8fbf3e87e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:10.099566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75965249-6e8e-43c5-8917-6bd6b9abc06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:10.100220 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75965249-6e8e-43c5-8917-6bd6b9abc06c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:10.100493 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75965249-6e8e-43c5-8917-6bd6b9abc06c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:10.100757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75965249-6e8e-43c5-8917-6bd6b9abc06c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:10.111179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75965249-6e8e-43c5-8917-6bd6b9abc06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-75965249-6e8e-43c5-8917-6bd6b9abc06c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:10.116095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:10.117002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:10.117816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 368/722] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:10.123522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dce7b6-98a6-4ba9-a4e0-917f52f9d125 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:10.249309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dce7b6-98a6-4ba9-a4e0-917f52f9d125 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:10.284994 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:10.285995 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:10.286533 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:10.287036 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:10.287963 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:10.288470 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:10.290414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:10.300664 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 355/723] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:10.306169 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75965249-6e8e-43c5-8917-6bd6b9abc06c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:10.434619 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:10.477795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75965249-6e8e-43c5-8917-6bd6b9abc06c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:10.521945 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacd333-d893-4d16-8cc1-2d7aa598176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:10.522628 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacd333-d893-4d16-8cc1-2d7aa598176d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:10.522906 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacd333-d893-4d16-8cc1-2d7aa598176d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:10.523176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacd333-d893-4d16-8cc1-2d7aa598176d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:10.540380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacd333-d893-4d16-8cc1-2d7aa598176d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:10.541836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacd333-d893-4d16-8cc1-2d7aa598176d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-afacd333-d893-4d16-8cc1-2d7aa598176d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:10.559617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:10.560547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:10.562149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:10.562149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:10.562149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:10.562149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:10.562149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 369/724] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:10.566796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:10.587510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:10.634765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cebf0b-802b-4e4c-bdee-3b1d28b60a8b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:10.635429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cebf0b-802b-4e4c-bdee-3b1d28b60a8b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:10.635904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cebf0b-802b-4e4c-bdee-3b1d28b60a8b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:10.637695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 370/725] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:10.642931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243444a-e384-4584-af64-66a75f33de54 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:10.643374 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243444a-e384-4584-af64-66a75f33de54 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:10.643882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243444a-e384-4584-af64-66a75f33de54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:10.663834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a243444a-e384-4584-af64-66a75f33de54 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:10.665520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a243444a-e384-4584-af64-66a75f33de54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:10.682950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fddc5-899e-428f-ba97-6673c1c04f3e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:10.718606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad725106-c04c-44c9-99b6-6f26066f35ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:10.719579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725106-c04c-44c9-99b6-6f26066f35ba None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:10.720043 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725106-c04c-44c9-99b6-6f26066f35ba None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:10.720714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725106-c04c-44c9-99b6-6f26066f35ba None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:10.721594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad725106-c04c-44c9-99b6-6f26066f35ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:10.722194 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad725106-c04c-44c9-99b6-6f26066f35ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:10.724781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad725106-c04c-44c9-99b6-6f26066f35ba None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:10.739376 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 356/726] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:10.740921 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fc50-da3a-4488-b629-44b85171c293 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:10.741203 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fc50-da3a-4488-b629-44b85171c293 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:10.741644 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fc50-da3a-4488-b629-44b85171c293 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:10.743324 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 357/727] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:10.748373 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95fc50-da3a-4488-b629-44b85171c293 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:10.763330 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 371/728] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:02:10.767411 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bba7ca-d77e-47b8-8e78-a3d7ae12d1e2 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:10.767832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bba7ca-d77e-47b8-8e78-a3d7ae12d1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:10.768301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bba7ca-d77e-47b8-8e78-a3d7ae12d1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:10.799617 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-69bba7ca-d77e-47b8-8e78-a3d7ae12d1e2 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:10.801458 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-69bba7ca-d77e-47b8-8e78-a3d7ae12d1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:10.851510 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:10.855629 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:10.873412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 372/729] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:10.874516 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bba7ca-d77e-47b8-8e78-a3d7ae12d1e2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:10.881256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95fc50-da3a-4488-b629-44b85171c293 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:10.985492 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bba7ca-d77e-47b8-8e78-a3d7ae12d1e2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:11.024794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edea95-bea0-4091-9f77-e40211a6fe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:11.025707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edea95-bea0-4091-9f77-e40211a6fe5b None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.026134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edea95-bea0-4091-9f77-e40211a6fe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.026537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edea95-bea0-4091-9f77-e40211a6fe5b None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.027379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edea95-bea0-4091-9f77-e40211a6fe5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:11.028874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edea95-bea0-4091-9f77-e40211a6fe5b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:11.049267 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 373/730] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:02:10 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:11.050516 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edea95-bea0-4091-9f77-e40211a6fe5b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:11.088670 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d786f34-e861-4cff-bd45-2c9124ca3431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:11.090318 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d786f34-e861-4cff-bd45-2c9124ca3431 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.091159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d786f34-e861-4cff-bd45-2c9124ca3431 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.092073 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d786f34-e861-4cff-bd45-2c9124ca3431 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.112926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d786f34-e861-4cff-bd45-2c9124ca3431 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:11.116506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d786f34-e861-4cff-bd45-2c9124ca3431 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9d786f34-e861-4cff-bd45-2c9124ca3431 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:11.143353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:11.144522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:02:11.145673 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:11.154094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 358/731] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:11.154094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcbdd2-c962-437e-a5bb-f71f9baac781 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.154094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcbdd2-c962-437e-a5bb-f71f9baac781 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.154094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcbdd2-c962-437e-a5bb-f71f9baac781 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.162497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 359/732] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:11.164409 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcbdd2-c962-437e-a5bb-f71f9baac781 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:11.167054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:11.235353 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edea95-bea0-4091-9f77-e40211a6fe5b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:11.275604 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab24864-5efb-4f2c-be24-9f5db1b7e9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:11.276279 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab24864-5efb-4f2c-be24-9f5db1b7e9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.276560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab24864-5efb-4f2c-be24-9f5db1b7e9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.276833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab24864-5efb-4f2c-be24-9f5db1b7e9e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.289027 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab24864-5efb-4f2c-be24-9f5db1b7e9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3ab24864-5efb-4f2c-be24-9f5db1b7e9e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:11.294532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:11.296400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 374/733] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:11.298245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4e9a-e917-4304-9022-b5cd915459be None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.301018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4e9a-e917-4304-9022-b5cd915459be None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.301018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4e9a-e917-4304-9022-b5cd915459be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.316847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcbdd2-c962-437e-a5bb-f71f9baac781 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:11.323560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c9ce4e9a-e917-4304-9022-b5cd915459be None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:11.325141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c9ce4e9a-e917-4304-9022-b5cd915459be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:11.374736 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:11.375669 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.376532 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.377833 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:11.379334 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:11.381017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:11.383134 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:11.410562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 360/734] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:02:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:11.412531 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:11.428835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 375/735] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:11.430851 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce4e9a-e917-4304-9022-b5cd915459be None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:11.474873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:11.528339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26a82b-c20c-4c6a-8dc0-9dd5bd5af767 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:11.567169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:11.567726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.567989 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.568267 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.569006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:11.569314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:11.572887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:11.582455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce4e9a-e917-4304-9022-b5cd915459be None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:11.584275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 361/736] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:11.585147 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:11.632564 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5812aa3-0b48-4efe-8de5-884c318eeaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:11.633154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812aa3-0b48-4efe-8de5-884c318eeaaa None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.633535 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812aa3-0b48-4efe-8de5-884c318eeaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.633793 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812aa3-0b48-4efe-8de5-884c318eeaaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.652195 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:11.715787 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05b505-d7fa-4157-8b85-d400c0dbd927 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:11.727601 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5812aa3-0b48-4efe-8de5-884c318eeaaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:11.729117 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5812aa3-0b48-4efe-8de5-884c318eeaaa None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:11.759329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b8b47-d088-40da-a909-e0805eb53a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:11.759985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b8b47-d088-40da-a909-e0805eb53a59 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.760262 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b8b47-d088-40da-a909-e0805eb53a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.760536 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b8b47-d088-40da-a909-e0805eb53a59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.771202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:11.772708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b8b47-d088-40da-a909-e0805eb53a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-de8b8b47-d088-40da-a909-e0805eb53a59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:11.778314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:11.779362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:11.780150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 362/737] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:02:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:11.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d612b6-1a5f-4047-8608-e06abc7a693d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d612b6-1a5f-4047-8608-e06abc7a693d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.781627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d612b6-1a5f-4047-8608-e06abc7a693d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.790143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 363/738] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:11.790143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d612b6-1a5f-4047-8608-e06abc7a693d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:11.887985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 376/739] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:11 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:02:11.889652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5812aa3-0b48-4efe-8de5-884c318eeaaa None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:11.928078 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d612b6-1a5f-4047-8608-e06abc7a693d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:11.945446 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:11.972252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d00ca3-0c88-4fe8-b0c2-003bc5343e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:11.972919 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00ca3-0c88-4fe8-b0c2-003bc5343e3a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:11.973190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00ca3-0c88-4fe8-b0c2-003bc5343e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:11.973464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00ca3-0c88-4fe8-b0c2-003bc5343e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:11.987131 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d00ca3-0c88-4fe8-b0c2-003bc5343e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:11.988841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d00ca3-0c88-4fe8-b0c2-003bc5343e3a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-45d00ca3-0c88-4fe8-b0c2-003bc5343e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:12.006260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:12.007424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:12.008962 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:12.008962 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:12.008962 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:12.008962 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:12.012160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 364/740] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:12.013505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029726b-1716-46be-91be-5e8f406e8122 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:12.013774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029726b-1716-46be-91be-5e8f406e8122 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:12.014043 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029726b-1716-46be-91be-5e8f406e8122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:12.032947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c029726b-1716-46be-91be-5e8f406e8122 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:12.033157 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5812aa3-0b48-4efe-8de5-884c318eeaaa None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:12.034238 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c029726b-1716-46be-91be-5e8f406e8122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:12.067334 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:12.077906 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:12.078960 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:12.079403 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:12.079839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:12.081144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:12.081640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:12.083452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:12.100230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 377/741] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:02:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:12.100230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:12.150447 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 365/742] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:02:12.152501 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029726b-1716-46be-91be-5e8f406e8122 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:12.268974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9142a-bf5a-4fd5-8b91-538f3b1101e3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:12.273980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:12.284923 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029726b-1716-46be-91be-5e8f406e8122 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:12.320877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:12.321562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:12.321844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:12.322152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:12.335122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:12.335359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:12.335986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:12.336264 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:12.336537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:12.336676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:12.337259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:12.337564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:12.339365 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:12.353624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:12.354679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:12.355570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:12.355570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:12.355570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:12.355570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:12.360807 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 366/743] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:12.360807 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:12.361041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 378/744] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:12.361041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:12.424492 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:12.513777 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31301c75-ef6b-4761-a496-2152e9c5dd38 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:12.514130 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81bff0-c4df-4d70-9c4d-190258212a5e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:12.571832 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:12.572825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:12.573280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:12.573731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:12.576761 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430788-a3a9-464a-bf71-5a709fd2519e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:12.577683 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430788-a3a9-464a-bf71-5a709fd2519e None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:12.578165 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430788-a3a9-464a-bf71-5a709fd2519e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:12.579104 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430788-a3a9-464a-bf71-5a709fd2519e None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:12.579387 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b430788-a3a9-464a-bf71-5a709fd2519e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:12.581618 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b430788-a3a9-464a-bf71-5a709fd2519e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:12.593111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:12.595326 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:12.607284 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 379/745] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:02:12 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:12.608607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430788-a3a9-464a-bf71-5a709fd2519e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:12.637596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:12.639479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:12.641172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:12.641172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:12.641172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:12.641172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:12.649280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 367/746] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:12.649280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:12.679825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:12.850925 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430788-a3a9-464a-bf71-5a709fd2519e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:12.854714 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:12.856047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1d36a-fcf0-4a00-a1b2-9b705d2a664f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:12.913739 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:12.914853 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:12.915278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:12.915686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:12.916658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:12.917097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:12.918935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:12.928753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:12.930988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:12.931726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:12.932723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:12.943005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 368/747] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:12.944556 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:12.948471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:12.952043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:12.976025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:12.977883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:12.979774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:12.984402 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 380/748] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:02:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:12.990278 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:13.094544 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:13.135092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad0c43-d431-4734-bf91-badbe2ec6d41 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:13.167296 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb64ec-072b-4708-80d3-6111fb7ff355 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:13.201671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:13.202682 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.203183 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.203666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.204959 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:13.205510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:13.206824 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee56382-8a39-43f5-a83d-f746ac78e919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:13.207765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56382-8a39-43f5-a83d-f746ac78e919 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.208125 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56382-8a39-43f5-a83d-f746ac78e919 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.208468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56382-8a39-43f5-a83d-f746ac78e919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.208676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:13.216264 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee56382-8a39-43f5-a83d-f746ac78e919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:13.217742 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee56382-8a39-43f5-a83d-f746ac78e919 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-bee56382-8a39-43f5-a83d-f746ac78e919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:13.230295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:13.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:13.232121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:13.232121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:13.232121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:13.232121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:13.232121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:13.232121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:13.232121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:13.232121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:13.232550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 369/749] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:13.234263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:13.236282 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 381/750] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:13.238159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a622d7-358f-4651-8d7e-f169abe38825 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.238454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a622d7-358f-4651-8d7e-f169abe38825 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.238734 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a622d7-358f-4651-8d7e-f169abe38825 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.240339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 382/751] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:13.241067 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a622d7-358f-4651-8d7e-f169abe38825 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:13.271509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:13.387144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a622d7-358f-4651-8d7e-f169abe38825 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:13.411482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411ac4-f32c-4339-b0fc-8e010d983545 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:13.425738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:13.426412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.426694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.426966 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.427734 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:13.428043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:13.429513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:13.442582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 383/752] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:13.447102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:13.463674 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:13.476798 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcb63d-90cf-4d86-b50b-1a237cd502d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:13.477785 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb63d-90cf-4d86-b50b-1a237cd502d3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.478289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb63d-90cf-4d86-b50b-1a237cd502d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.478728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb63d-90cf-4d86-b50b-1a237cd502d3 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.480065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcb63d-90cf-4d86-b50b-1a237cd502d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:13.480563 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcb63d-90cf-4d86-b50b-1a237cd502d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:13.483161 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcb63d-90cf-4d86-b50b-1a237cd502d3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:13.514794 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 370/753] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:02:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:13.516780 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e39899-7832-4563-92d9-0369385b5092 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.517260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e39899-7832-4563-92d9-0369385b5092 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.517695 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e39899-7832-4563-92d9-0369385b5092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.545859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:13.546670 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-56e39899-7832-4563-92d9-0369385b5092 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:13.548589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-56e39899-7832-4563-92d9-0369385b5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:13.566454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5e99a-a869-4007-8ee8-599041365114 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:13.606831 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:13.608860 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.609194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.609447 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.620716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:13.622253 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:13.635573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:13.637295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 384/754] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:02:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:13.639084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:13.662163 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 371/755] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:13.662163 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e39899-7832-4563-92d9-0369385b5092 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:13.775749 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b939c-406c-41ce-abd6-9ecd26d8a2b4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:13.784689 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e39899-7832-4563-92d9-0369385b5092 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:13.803177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:13.822808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:13.823463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.823942 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.824048 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.833029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:13.834040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:13.834549 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:13.835023 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:13.836330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:13.836449 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=f285251a0e0a49849d071155d2d3e478) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:13.836899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:13.837814 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:13.839399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:13.851460 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 385/756] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:13 2022] GET /identity/v3/role_assignments?user.id=f285251a0e0a49849d071155d2d3e478&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=65cd37a73e4f498482fefabb8374006c => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:13.852717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:13.858397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:13.859347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:13.860312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:13.860312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:13.860312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:13.860312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:13.860312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 372/757] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:13.861008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:13.873280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:13.991118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13653be8-8cb2-4237-a286-4c0fb07f1811 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:14.034528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4c87-abac-42b3-a447-b32bf0e45cdb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:14.042095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:14.043045 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:14.043466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:14.043914 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:14.044889 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:14.045347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:14.047042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:14.069007 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 386/758] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:02:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:14.070327 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:14.105033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:14.106413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:14.107055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:14.107692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:14.108932 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:14.111844 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:14.129952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:14.130969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:14.131753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:14.138240 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 373/759] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:02:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:14.139907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:14.152057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:14.205038 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:14.237990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bfde9-5ebb-4872-8967-1a07f6cdf2db None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:14.304431 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:14.305812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:14.306515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:14.307141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:14.308545 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:14.309319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:14.312011 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:14.345773 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 387/760] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:14.345773 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:14.353917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edaff3-e0d2-46a8-827e-704c4ee2852b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:14.419123 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d915cf-1c4b-4d6b-be30-22512218243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:14.420353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d915cf-1c4b-4d6b-be30-22512218243c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:14.420858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d915cf-1c4b-4d6b-be30-22512218243c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:14.421360 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d915cf-1c4b-4d6b-be30-22512218243c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:14.560212 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d915cf-1c4b-4d6b-be30-22512218243c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:14.563913 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d915cf-1c4b-4d6b-be30-22512218243c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:14.570556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc7e4-4ac0-4955-8546-adc463856f69 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:14.620647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:14.624836 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:14.646724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:14.647808 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:14.648294 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:14.648763 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:14.650006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:14.654407 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:14.658016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:14.669884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 388/761] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:02:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:14.671785 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:14.800043 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 374/762] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:02:14.800043 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d915cf-1c4b-4d6b-be30-22512218243c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:14.913704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304c2f9-85ca-4b52-aae3-dffa96b3f572 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:14.989232 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:14.990303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:14.990911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:14.991505 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.002426 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d915cf-1c4b-4d6b-be30-22512218243c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:15.005649 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:15.009086 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:15.032673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:15.034744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:15.045530 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 389/763] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:15.045530 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:15.082719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:15.094842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:15.096779 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:15.097738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:15.099149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.101156 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:15.101977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:15.105621 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:15.117610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 375/764] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:02:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:15.119503 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:15.155468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:15.285987 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba70a6-2f7c-4cc9-acb7-3e2a835bb663 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:15.340559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:15.341397 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:15.341770 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:15.342189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.356606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:15.358528 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:15.363543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f9dd2-3e24-483c-b4de-a118ad75b33f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:15.376954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:15.379039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:15.381798 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:15.381798 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:15.381798 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:15.381798 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:15.385284 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 390/765] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:15.386790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:15.437319 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:15.438599 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:15.439103 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:15.439640 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.441425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:15.442038 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:15.445164 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:15.474158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 376/766] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:15.474158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:15.507024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:15.510463 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178bccf-ff22-4390-918b-b310c6bfca7d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:15.551943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a2437-a75a-44a3-bbf2-c87cc693b2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:15.552502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a2437-a75a-44a3-bbf2-c87cc693b2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:15.552779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a2437-a75a-44a3-bbf2-c87cc693b2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:15.553047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a2437-a75a-44a3-bbf2-c87cc693b2c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.553826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a2437-a75a-44a3-bbf2-c87cc693b2c3 None admin] RBAC: Adding query filter params (interface=public, service_id=e64ca820a61246c2b1e151c30875a5fe, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:15.554178 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a2437-a75a-44a3-bbf2-c87cc693b2c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:15.555637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a2437-a75a-44a3-bbf2-c87cc693b2c3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:15.569084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 391/767] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:02:15 2022] GET /identity/v3/endpoints?service_id=e64ca820a61246c2b1e151c30875a5fe&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:15.569084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957527b-1dc8-48bb-acd4-784c5ff1be35 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:15.569084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957527b-1dc8-48bb-acd4-784c5ff1be35 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:15.569084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957527b-1dc8-48bb-acd4-784c5ff1be35 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.569084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 392/768] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:15.569084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957527b-1dc8-48bb-acd4-784c5ff1be35 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:15.589187 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:15.651182 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731afa1-aec8-4848-981a-eb9c9b0a3c92 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:15.691590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:15.692241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:15.692522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:15.692793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.693511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:15.693817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:15.695282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:15.717307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 377/769] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:15.717307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:15.726720 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957527b-1dc8-48bb-acd4-784c5ff1be35 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:15.772010 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c8932-10fc-4b5b-b21d-436a6289dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:15.772718 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c8932-10fc-4b5b-b21d-436a6289dac0 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:15.773000 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c8932-10fc-4b5b-b21d-436a6289dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:15.773275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c8932-10fc-4b5b-b21d-436a6289dac0 None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/4527e4500f984b1a81214c3656e7947f/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.775353 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c8932-10fc-4b5b-b21d-436a6289dac0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=4527e4500f984b1a81214c3656e7947f, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:15.776811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c8932-10fc-4b5b-b21d-436a6289dac0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:15.793488 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 393/770] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:02:15 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/4527e4500f984b1a81214c3656e7947f/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:02:15.795536 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623512c5-b4e0-4519-ba3f-8ede3025ffd8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:15.798008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623512c5-b4e0-4519-ba3f-8ede3025ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:15.798589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623512c5-b4e0-4519-ba3f-8ede3025ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:15.821422 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-623512c5-b4e0-4519-ba3f-8ede3025ffd8 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:15.823250 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-623512c5-b4e0-4519-ba3f-8ede3025ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:15.847909 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:15.922636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 394/771] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:15.924070 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623512c5-b4e0-4519-ba3f-8ede3025ffd8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:15.964057 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e1013-a681-4224-9e01-6da86a7004ef None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:16.001342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:16.025712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c6c86-1fbd-4367-a96b-824d4f6e7ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:16.027284 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c6c86-1fbd-4367-a96b-824d4f6e7ecf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.027284 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c6c86-1fbd-4367-a96b-824d4f6e7ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.027284 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c6c86-1fbd-4367-a96b-824d4f6e7ecf None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.028034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c6c86-1fbd-4367-a96b-824d4f6e7ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:16.029625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c6c86-1fbd-4367-a96b-824d4f6e7ecf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:16.036883 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 378/772] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:02:15 2022] GET /identity/v3/services => generated 772 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:16.038060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c6c86-1fbd-4367-a96b-824d4f6e7ecf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:16.096558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623512c5-b4e0-4519-ba3f-8ede3025ffd8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:16.154435 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983f284-8ab0-443e-a2ad-b9f236c00854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:16.155126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983f284-8ab0-443e-a2ad-b9f236c00854 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.155464 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983f284-8ab0-443e-a2ad-b9f236c00854 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.155947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983f284-8ab0-443e-a2ad-b9f236c00854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.165677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983f284-8ab0-443e-a2ad-b9f236c00854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:16.167238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983f284-8ab0-443e-a2ad-b9f236c00854 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-2983f284-8ab0-443e-a2ad-b9f236c00854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:16.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:16.192590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 395/773] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2693-9343-409d-ba9e-6cf7d0ec758c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.194050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2693-9343-409d-ba9e-6cf7d0ec758c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.197633 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2693-9343-409d-ba9e-6cf7d0ec758c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.197633 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 396/774] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:16.202818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf2693-9343-409d-ba9e-6cf7d0ec758c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:16.226170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c6c86-1fbd-4367-a96b-824d4f6e7ecf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:16.246153 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:16.293170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf6c10-9ed0-4b65-b64f-c71dec1a7bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:16.294186 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf6c10-9ed0-4b65-b64f-c71dec1a7bbf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.295019 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf6c10-9ed0-4b65-b64f-c71dec1a7bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.295019 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf6c10-9ed0-4b65-b64f-c71dec1a7bbf None admin] PATH_INFO: `/v3/services/network` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.296127 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf6c10-9ed0-4b65-b64f-c71dec1a7bbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:16.298550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf6c10-9ed0-4b65-b64f-c71dec1a7bbf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-faaf6c10-9ed0-4b65-b64f-c71dec1a7bbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:16.314438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:16.315737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:16.316895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:16.317763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 379/775] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:16 2022] GET /identity/v3/services/network => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:16.319847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561f962-8f21-429f-a615-294eeaf4a8f7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.319847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561f962-8f21-429f-a615-294eeaf4a8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.320156 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561f962-8f21-429f-a615-294eeaf4a8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.357467 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9561f962-8f21-429f-a615-294eeaf4a8f7 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:16.358955 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9561f962-8f21-429f-a615-294eeaf4a8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:16.419998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf2693-9343-409d-ba9e-6cf7d0ec758c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:16.434847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:16.486296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 380/776] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:16.487458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561f962-8f21-429f-a615-294eeaf4a8f7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:16.500080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:16.500726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.501065 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.501272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.502032 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:16.502511 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:16.504111 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:16.529202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 397/777] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:16.531777 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:16.631296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561f962-8f21-429f-a615-294eeaf4a8f7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:16.659988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:16.672822 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:16.673480 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.673761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.674032 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.674893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] RBAC: Adding query filter params (name=network) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:16.675238 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:16.676850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:16.682715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 381/778] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:16.683667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:16.709514 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfafef7-c1ba-4b4f-a121-4197d5534cda None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:16.738672 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:16.783024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bf5b4-ef70-48b4-b05e-99dda6690b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:16.784577 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bf5b4-ef70-48b4-b05e-99dda6690b2e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.785265 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bf5b4-ef70-48b4-b05e-99dda6690b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.785927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bf5b4-ef70-48b4-b05e-99dda6690b2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.808347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bf5b4-ef70-48b4-b05e-99dda6690b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3c5bf5b4-ef70-48b4-b05e-99dda6690b2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:16.819834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:16.821982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:16.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:16.834206 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad1984-01e3-4f15-8ad6-59fa80a2450a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:16.835092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 398/779] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:16.836973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bf5b4-ef70-48b4-b05e-99dda6690b2e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:16.876974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:16.877754 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:16.878056 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:16.878341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:16.879204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=76e8ae1ebdbb4e34bb4b4ef59e70305c) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:16.879478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:16.880970 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:16.892602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 382/780] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:16 2022] GET /identity/v3/role_assignments?user.id=76e8ae1ebdbb4e34bb4b4ef59e70305c&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:16.892602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:17.010588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128c38e-14af-4815-8516-876449a84a54 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:17.012194 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:17.049366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:17.050057 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:17.050410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:17.050688 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:17.051435 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:17.051740 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:17.053729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:17.076618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 383/781] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:17.087081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:17.137595 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bf5b4-ef70-48b4-b05e-99dda6690b2e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:17.214327 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:17.215870 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:17.216625 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:17.217463 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:17.219154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] RBAC: Adding query filter params (type=network) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:17.220116 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:17.223509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:17.226525 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:17.236573 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 399/782] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:17.247098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:17.269044 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044ccd-4034-49b0-a4a4-57dfb9205203 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:17.307820 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:17.308492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:17.308774 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:17.309050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:17.322057 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:17.323951 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:17.340364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:17.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:17.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:17.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:17.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:17.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:17.346247 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 384/783] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:17.347825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:17.368836 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:17.461417 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875f652-8e24-4f14-83bc-8e327c70bb7b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:17.469968 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f509344-06fb-44b5-9aac-cfb0f688a13f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:17.499091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:17.499735 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:17.500011 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:17.500280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:17.500990 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:17.501296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:17.503222 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:17.520479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 385/784] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:17.521941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:17.544002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:17.545643 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:17.546488 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:17.547253 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:17.549272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] RBAC: Adding query filter params (interface=public, service_id=a4913eedba6144b3a922f300422088f2, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:17.550160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:17.553352 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:17.565078 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 400/785] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:02:17 2022] GET /identity/v3/endpoints?service_id=a4913eedba6144b3a922f300422088f2&interface=public®ion_id=RegionOne => generated 194 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:17.575287 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:17.633462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:17.672231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6941ab-8d34-412c-8a2f-2c0d2431d030 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:17.741284 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:17.754247 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:17.756226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:17.756978 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:17.757692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:17.773961 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:17.777463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:17.805405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:17.807230 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:02:17.809004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:17.813803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 386/786] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:17.822970 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:17.870859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380407a7-5c38-48d3-9633-930ca2665c4b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:17.943843 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c23388-419d-4396-b177-76bf324a47a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:17.945408 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c23388-419d-4396-b177-76bf324a47a2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:17.946221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c23388-419d-4396-b177-76bf324a47a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:17.946990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c23388-419d-4396-b177-76bf324a47a2 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:17.948468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c23388-419d-4396-b177-76bf324a47a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:17.958844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c23388-419d-4396-b177-76bf324a47a2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:17.962237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 401/787] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:02:17 2022] GET /identity/v3/services => generated 772 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:18.117867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b405fa-c6e6-4530-9932-01b18009496e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:18.189102 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95390e9-d38e-43f5-a6d3-19af7926d805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:18.190791 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95390e9-d38e-43f5-a6d3-19af7926d805 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:18.191597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95390e9-d38e-43f5-a6d3-19af7926d805 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:18.192349 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95390e9-d38e-43f5-a6d3-19af7926d805 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:18.194207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95390e9-d38e-43f5-a6d3-19af7926d805 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:18.195069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95390e9-d38e-43f5-a6d3-19af7926d805 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:18.198008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95390e9-d38e-43f5-a6d3-19af7926d805 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:18.208425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:18.241809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 387/788] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:02:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:18.255529 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c23388-419d-4396-b177-76bf324a47a2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:18.385655 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c23388-419d-4396-b177-76bf324a47a2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:18.394714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215d690-d462-4d21-a5fc-5d5fa01b6ede None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:18.396166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215d690-d462-4d21-a5fc-5d5fa01b6ede None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:18.396920 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215d690-d462-4d21-a5fc-5d5fa01b6ede None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:18.400008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 388/789] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:18.407784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4fc22-bd51-4c3e-b00b-5e04c659dfc9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:18.408847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4fc22-bd51-4c3e-b00b-5e04c659dfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:18.409518 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4fc22-bd51-4c3e-b00b-5e04c659dfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:18.418720 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:18.425973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:18.426688 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:18.426968 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:18.427232 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:18.443101 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:18.444595 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:18.462566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:18.464152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:18.465754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:18.465754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:18.465754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:18.465754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:18.470143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d9a4fc22-bd51-4c3e-b00b-5e04c659dfc9 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:18.470635 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 402/790] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:18.470854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d9a4fc22-bd51-4c3e-b00b-5e04c659dfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:18.471121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:18.601112 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd14cc-3c71-435d-9d6d-ac048adfbc85 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:18.640484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:18.641020 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:18.641659 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:18.641993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:18.642751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:18.643116 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:18.645598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:18.664127 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 403/791] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:18.666546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:18.718841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:18.768735 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 389/792] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:18.785351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13726e45-6fa6-4256-989d-684d16778a69 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:18.815296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4fc22-bd51-4c3e-b00b-5e04c659dfc9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:18.830992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:18.831928 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:18.832353 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:18.832765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:18.845223 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:18.847196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:18.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:18.861948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:18.862976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:18.862976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:18.862976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:18.862976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:18.862976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:18.862976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:18.862976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:18.862976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:18.869855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 404/793] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:18.872201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:18.983801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2375b-4833-4a2d-8d18-af9a81703e42 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:18.994722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:19.024627 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:19.025581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.026013 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.026467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.027387 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:19.027843 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:19.029470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:19.048648 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 405/794] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:19.050312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:19.103309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4fc22-bd51-4c3e-b00b-5e04c659dfc9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:19.154357 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:19.164939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc0e36-635a-463b-a105-20f14d621d3a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:19.173720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b334c-1626-4f3a-81fa-6b4c936b3837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:19.174498 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b334c-1626-4f3a-81fa-6b4c936b3837 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.174805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b334c-1626-4f3a-81fa-6b4c936b3837 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.175547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b334c-1626-4f3a-81fa-6b4c936b3837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.187490 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b334c-1626-4f3a-81fa-6b4c936b3837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-229b334c-1626-4f3a-81fa-6b4c936b3837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:19.192564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:19.193488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:19.194777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 390/795] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:19.195666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c983c10-b259-4b03-a3d0-b6fc4a5ae5d7 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.195749 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c983c10-b259-4b03-a3d0-b6fc4a5ae5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.195996 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c983c10-b259-4b03-a3d0-b6fc4a5ae5d7 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.197745 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 391/796] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:19.206988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc5868-c227-446f-9929-b965721a4f6d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.207425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc5868-c227-446f-9929-b965721a4f6d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.207906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc5868-c227-446f-9929-b965721a4f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.211901 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391606e-16ef-4127-9a9d-a8e4a3a7133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:19.212806 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391606e-16ef-4127-9a9d-a8e4a3a7133d None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.213230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391606e-16ef-4127-9a9d-a8e4a3a7133d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.213647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391606e-16ef-4127-9a9d-a8e4a3a7133d None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/f285251a0e0a49849d071155d2d3e478/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.216104 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391606e-16ef-4127-9a9d-a8e4a3a7133d None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=f285251a0e0a49849d071155d2d3e478, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:19.217854 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391606e-16ef-4127-9a9d-a8e4a3a7133d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:19.238230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 406/797] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:02:19 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/f285251a0e0a49849d071155d2d3e478/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:02:19.238230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391606e-16ef-4127-9a9d-a8e4a3a7133d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:19.280080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cecc5868-c227-446f-9929-b965721a4f6d None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:19.281612 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:19.282271 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cecc5868-c227-446f-9929-b965721a4f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:19.410321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391606e-16ef-4127-9a9d-a8e4a3a7133d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:19.449178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 392/798] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:19.452627 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:19.453335 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.453619 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.453891 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.454653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:19.454948 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:19.456389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:19.466274 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 407/799] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:19.472936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc5868-c227-446f-9929-b965721a4f6d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:19.479503 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:19.596440 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:19.635999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b680d-7d78-44c3-9a50-499a8e88a074 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:19.656246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc5868-c227-446f-9929-b965721a4f6d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:19.676914 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:19.690802 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342a2f1-c9ef-42e5-9345-efb1dcb97423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:19.691896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a2f1-c9ef-42e5-9345-efb1dcb97423 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.692971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a2f1-c9ef-42e5-9345-efb1dcb97423 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.693483 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a2f1-c9ef-42e5-9345-efb1dcb97423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.716923 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342a2f1-c9ef-42e5-9345-efb1dcb97423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:19.718276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23e333-f5ad-4f97-92b0-165aaad08417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:19.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342a2f1-c9ef-42e5-9345-efb1dcb97423 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:19.719501 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23e333-f5ad-4f97-92b0-165aaad08417 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.720070 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23e333-f5ad-4f97-92b0-165aaad08417 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.720591 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23e333-f5ad-4f97-92b0-165aaad08417 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.723825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23e333-f5ad-4f97-92b0-165aaad08417 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:19.723825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23e333-f5ad-4f97-92b0-165aaad08417 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-fb23e333-f5ad-4f97-92b0-165aaad08417 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:19.738949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:19.740133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:19.741000 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 393/800] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:02:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-4342a2f1-c9ef-42e5-9345-efb1dcb97423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:19.745893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:19.746871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:19.747794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:19.747794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:19.747794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:19.747794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:19.748007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dacf07-4f5b-42d1-b801-2553924c19b2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.748007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dacf07-4f5b-42d1-b801-2553924c19b2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.748007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dacf07-4f5b-42d1-b801-2553924c19b2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.749720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 394/801] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:19.754406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dacf07-4f5b-42d1-b801-2553924c19b2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:19.755388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 408/802] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:19.758341 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187618bd-15ef-4a91-b6ff-c798ef232382 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.758992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187618bd-15ef-4a91-b6ff-c798ef232382 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.759485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187618bd-15ef-4a91-b6ff-c798ef232382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.783370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-187618bd-15ef-4a91-b6ff-c798ef232382 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:19.785164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-187618bd-15ef-4a91-b6ff-c798ef232382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:19.882905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 409/803] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:19.883963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187618bd-15ef-4a91-b6ff-c798ef232382 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:19.917685 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dacf07-4f5b-42d1-b801-2553924c19b2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:19.966968 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:19.967864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:19.968170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:19.968484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:19.969493 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:19.969958 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:19.971835 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:19.983552 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 395/804] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:02:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:19.984620 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:19.999984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:20.051842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:20.073932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187618bd-15ef-4a91-b6ff-c798ef232382 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:20.131488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5feda1-4e29-4c2b-8dda-6fdc5296a32c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:20.137514 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:20.138495 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:20.138972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:20.139430 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:20.140664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:20.141194 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:20.143923 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:20.172982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 410/805] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:20.174816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:20.179506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651142d9-e2c0-4721-adfa-3b21f80a280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:20.180401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651142d9-e2c0-4721-adfa-3b21f80a280f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:20.181082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651142d9-e2c0-4721-adfa-3b21f80a280f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:20.181669 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651142d9-e2c0-4721-adfa-3b21f80a280f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:20.196345 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651142d9-e2c0-4721-adfa-3b21f80a280f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-651142d9-e2c0-4721-adfa-3b21f80a280f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:20.203144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:20.204339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:20.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:20.207829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 396/806] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:02:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:20.209047 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651142d9-e2c0-4721-adfa-3b21f80a280f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:20.341113 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651142d9-e2c0-4721-adfa-3b21f80a280f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:20.343499 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:20.366207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea04967-9270-473e-900b-38b5bf46465f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:20.386889 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:20.388013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:20.388536 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:20.389101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:20.398897 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:20.400796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:20.415249 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:20.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:20.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:20.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:20.421339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 397/807] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:02:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:20.423287 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:20.448122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:20.448804 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:20.449173 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:20.449811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:20.450625 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:20.450898 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:20.452414 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:20.459474 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 411/808] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:02:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:20.460379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:20.611849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d1b5a-f725-431f-8cb8-7fef708cb268 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:20.618552 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9b940-6e52-43ca-a6e2-ee6fd2a8a657 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:20.659437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:20.660300 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:20.660709 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:20.661332 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:20.662071 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:20.662600 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:20.664615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:20.667555 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:20.669363 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:20.669880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:20.670383 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:20.671320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:20.671842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:20.673441 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:20.685809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 398/809] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:02:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:20.687389 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:20.694334 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 412/810] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:02:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:20.696129 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:20.731721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:20.758148 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:20.823055 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066601b2-51ee-4bad-9a76-62fbdeb3a80a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:20.838876 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebad26-05b0-4285-81d1-0216490e30aa None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:20.864605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:20.865538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:20.866145 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:20.866614 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:20.879781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:20.881696 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:20.897813 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e43426-83f1-4123-820e-e19d51ba8b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:20.899024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e43426-83f1-4123-820e-e19d51ba8b5f None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:20.899590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e43426-83f1-4123-820e-e19d51ba8b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:20.900149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e43426-83f1-4123-820e-e19d51ba8b5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:20.901389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:20.902370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:20.903311 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:20.903311 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:20.903311 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:20.903311 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:20.903538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e43426-83f1-4123-820e-e19d51ba8b5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:20.904688 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e43426-83f1-4123-820e-e19d51ba8b5f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:20.906196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 413/811] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:20.907290 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:20.946237 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 399/812] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:02:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:20.946811 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e43426-83f1-4123-820e-e19d51ba8b5f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:21.094976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:21.101269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:21.103325 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a05fc-b6f9-408b-a9dc-7f6dc54f3d08 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:21.134640 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e43426-83f1-4123-820e-e19d51ba8b5f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:21.147798 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89aecb-6768-472d-bde6-205c2fa949d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:21.148156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89aecb-6768-472d-bde6-205c2fa949d4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:21.148582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89aecb-6768-472d-bde6-205c2fa949d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:21.149013 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89aecb-6768-472d-bde6-205c2fa949d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:21.180559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88774ab8-9c96-434f-85ac-5256f01471dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:21.181718 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88774ab8-9c96-434f-85ac-5256f01471dc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:21.182249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88774ab8-9c96-434f-85ac-5256f01471dc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:21.182759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88774ab8-9c96-434f-85ac-5256f01471dc None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:21.183815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88774ab8-9c96-434f-85ac-5256f01471dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:21.184470 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88774ab8-9c96-434f-85ac-5256f01471dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:21.186384 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88774ab8-9c96-434f-85ac-5256f01471dc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:21.204092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 400/813] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:21.208509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2af5c5-a96d-4d1a-906f-57d1937a0b03 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:21.208509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2af5c5-a96d-4d1a-906f-57d1937a0b03 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:21.208509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2af5c5-a96d-4d1a-906f-57d1937a0b03 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:21.208874 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 401/814] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:21.212787 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2af5c5-a96d-4d1a-906f-57d1937a0b03 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:21.243417 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89aecb-6768-472d-bde6-205c2fa949d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:21.244913 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89aecb-6768-472d-bde6-205c2fa949d4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:21.374279 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:21.376110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2af5c5-a96d-4d1a-906f-57d1937a0b03 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:21.398428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 414/815] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:02:21.400629 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e73006-ada0-48d5-90fd-e5c347d3d42b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:21.401015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e73006-ada0-48d5-90fd-e5c347d3d42b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:21.401385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e73006-ada0-48d5-90fd-e5c347d3d42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:21.427232 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f4e73006-ada0-48d5-90fd-e5c347d3d42b None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:21.428833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f4e73006-ada0-48d5-90fd-e5c347d3d42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:21.430682 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:21.431657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:21.432015 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:21.432359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:21.437088 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:21.451902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:21.454584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:21.472116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:21.473676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:21.475370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 402/816] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:02:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:21.476399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:21.566046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 415/817] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:21.566046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e73006-ada0-48d5-90fd-e5c347d3d42b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:21.627695 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc970552-d721-4262-a9dd-f5852bd7a646 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:21.684819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:21.685612 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:21.685889 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:21.686213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:21.703168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:21.705453 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:21.741781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:21.743210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:21.744220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:21.744220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:21.744220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:21.744220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:21.744537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 403/818] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:21.745720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:21.783017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e73006-ada0-48d5-90fd-e5c347d3d42b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:21.818766 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:21.827186 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:21.832116 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:21.833021 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:21.833544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:21.834007 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:21.835109 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:21.836151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:21.838215 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:21.863731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 416/819] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:02:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:21.864960 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:21.915259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d4d5d-5efa-4bc6-aa35-0a745aef30b9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:21.961413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:21.962509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:21.962907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:21.963484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] PATH_INFO: `/v3/services/network` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:21.965182 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:21.966145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:21.978637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:21.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:02:21.982808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:21.987119 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 404/820] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:21 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:21.988466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:22.025323 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82424e1e-ebb8-4dad-a4a5-390254c67601 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:22.068790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:22.069625 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.069903 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.070312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.071067 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:22.071375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:22.072911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:22.095947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 417/821] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:22.095947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:22.154821 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7d51e-75de-4f7c-834d-b98981b03044 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:22.170089 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:22.175975 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:22.206712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c8a36-9c0d-4b50-8e24-1b996e11e0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:22.207788 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c8a36-9c0d-4b50-8e24-1b996e11e0ca None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.208228 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c8a36-9c0d-4b50-8e24-1b996e11e0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.208679 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c8a36-9c0d-4b50-8e24-1b996e11e0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.225628 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c8a36-9c0d-4b50-8e24-1b996e11e0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:22.228332 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c8a36-9c0d-4b50-8e24-1b996e11e0ca None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:22.244256 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da16145-010b-4a98-abab-0c3bc21ebfd8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c46c8a36-9c0d-4b50-8e24-1b996e11e0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:22.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:22.247196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:22.248279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:22.248279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:22.248279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:22.248279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:22.248279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 405/822] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:22.249940 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622b9a1-f05b-4c97-b31e-b0b5b66b25f0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.250398 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622b9a1-f05b-4c97-b31e-b0b5b66b25f0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.250894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622b9a1-f05b-4c97-b31e-b0b5b66b25f0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.254306 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 406/823] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:22.254900 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622b9a1-f05b-4c97-b31e-b0b5b66b25f0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:22.298793 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:22.299525 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.299817 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.300097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.300838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] RBAC: Adding query filter params (name=network) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:22.301154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:22.302634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:22.308154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 418/824] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:22.308992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:22.407287 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622b9a1-f05b-4c97-b31e-b0b5b66b25f0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:22.431857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e183d-2067-481f-a953-5a15f91c599b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:22.443636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:22.448938 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf555675-d151-432c-ba3a-3a4cc19ca0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:22.450323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf555675-d151-432c-ba3a-3a4cc19ca0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.450826 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf555675-d151-432c-ba3a-3a4cc19ca0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.451303 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf555675-d151-432c-ba3a-3a4cc19ca0f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.460189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf555675-d151-432c-ba3a-3a4cc19ca0f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:22.462009 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf555675-d151-432c-ba3a-3a4cc19ca0f5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cf555675-d151-432c-ba3a-3a4cc19ca0f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:22.477091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:22.478315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 407/825] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321c331-ca6a-41b9-aa01-ed8efca9da9e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321c331-ca6a-41b9-aa01-ed8efca9da9e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.486663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321c331-ca6a-41b9-aa01-ed8efca9da9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:22.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.491942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.492433 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.493766 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:22.494346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:22.500376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:22.519352 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b321c331-ca6a-41b9-aa01-ed8efca9da9e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:22.520096 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 419/826] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:22.521248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:22.521638 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b321c331-ca6a-41b9-aa01-ed8efca9da9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:22.553082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:22.634293 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 408/827] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:22.635701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321c331-ca6a-41b9-aa01-ed8efca9da9e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:22.699199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16746772-d163-4d99-9b31-44f1301b02fe None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:22.756462 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:22.757176 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.757449 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.757758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.758586 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] RBAC: Adding query filter params (type=network) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:22.758911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:22.760896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:22.768663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 420/828] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:22.769641 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:22.823098 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321c331-ca6a-41b9-aa01-ed8efca9da9e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:22.833748 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:22.894025 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:22.895060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.895577 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.896043 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:22.897420 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:22.897961 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:22.902597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:22.934165 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 409/829] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:22.934165 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:22.947513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7585c-de4a-4443-9d9b-e9e813caec2c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:22.954967 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:22.997614 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:22.998139 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:22.998437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:22.998727 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.006931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:23.008632 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:23.024272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:23.025755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 421/830] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:23.034388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:23.090374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534307ea-d664-4091-9548-cabb132dfd42 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:23.149952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3154e5-60ce-412c-934c-5bac4f990164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:23.150994 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3154e5-60ce-412c-934c-5bac4f990164 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:23.151425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3154e5-60ce-412c-934c-5bac4f990164 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:23.151861 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3154e5-60ce-412c-934c-5bac4f990164 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.173179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3154e5-60ce-412c-934c-5bac4f990164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9a3154e5-60ce-412c-934c-5bac4f990164 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:23.182181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:23.184330 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:23.185587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 410/831] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:23.186773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3154e5-60ce-412c-934c-5bac4f990164 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:23.194150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:23.209272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adf67a-5bd2-43b4-91d9-c9fdb2d6ef9d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:23.261710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949ba7c-b39a-48f7-8a96-9e4d0b6aec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:23.262723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ba7c-b39a-48f7-8a96-9e4d0b6aec66 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:23.262999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ba7c-b39a-48f7-8a96-9e4d0b6aec66 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:23.263273 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ba7c-b39a-48f7-8a96-9e4d0b6aec66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.264018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949ba7c-b39a-48f7-8a96-9e4d0b6aec66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:23.265585 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949ba7c-b39a-48f7-8a96-9e4d0b6aec66 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:23.293640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 422/832] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:02:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:23.294574 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949ba7c-b39a-48f7-8a96-9e4d0b6aec66 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:23.315694 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:23.367443 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3154e5-60ce-412c-934c-5bac4f990164 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:23.430874 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:23.431664 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:23.431945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:23.432224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.433245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=4527e4500f984b1a81214c3656e7947f) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:23.433597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:23.435276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:23.443508 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 411/833] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:23 2022] GET /identity/v3/role_assignments?user.id=4527e4500f984b1a81214c3656e7947f&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=65cd37a73e4f498482fefabb8374006c => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:23.443836 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:23.482581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949ba7c-b39a-48f7-8a96-9e4d0b6aec66 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:23.560873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992ff46-895e-4b40-a576-b8cd30f38b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:23.562149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ff46-895e-4b40-a576-b8cd30f38b82 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:23.562862 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ff46-895e-4b40-a576-b8cd30f38b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:23.563211 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ff46-895e-4b40-a576-b8cd30f38b82 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.566224 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ff46-895e-4b40-a576-b8cd30f38b82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:23.566224 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ff46-895e-4b40-a576-b8cd30f38b82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:23.567728 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ff46-895e-4b40-a576-b8cd30f38b82 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:23.592982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 423/834] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:23.596260 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd129-86cb-4f96-89e2-51005dc668d0 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:23.596746 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd129-86cb-4f96-89e2-51005dc668d0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:23.597763 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd129-86cb-4f96-89e2-51005dc668d0 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.599904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 424/835] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:23.600974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dd129-86cb-4f96-89e2-51005dc668d0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:23.623967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:23.644122 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788bfeb-d787-4cb1-823e-0d11afa373b4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:23.695006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af45772-5641-475d-82a9-578dfadd613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:23.695668 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45772-5641-475d-82a9-578dfadd613d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:23.695951 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45772-5641-475d-82a9-578dfadd613d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:23.696299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45772-5641-475d-82a9-578dfadd613d None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.697077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af45772-5641-475d-82a9-578dfadd613d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:23.697412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af45772-5641-475d-82a9-578dfadd613d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:23.698890 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af45772-5641-475d-82a9-578dfadd613d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:23.701002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:23.712102 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 412/836] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:23.712980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8141972-5c20-4a73-8217-d3693869eac4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:23.714168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8141972-5c20-4a73-8217-d3693869eac4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:23.714168 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8141972-5c20-4a73-8217-d3693869eac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.738789 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d8141972-5c20-4a73-8217-d3693869eac4 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:23.741158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d8141972-5c20-4a73-8217-d3693869eac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:23.772832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dd129-86cb-4f96-89e2-51005dc668d0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:23.815612 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2148fc4-e586-4431-bf16-af0bd23efac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:23.816237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2148fc4-e586-4431-bf16-af0bd23efac1 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:23.816513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2148fc4-e586-4431-bf16-af0bd23efac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:23.817459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2148fc4-e586-4431-bf16-af0bd23efac1 None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/76e8ae1ebdbb4e34bb4b4ef59e70305c/roles/b5c148877d4a489f85a6d554d44dba3f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:23.838645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2148fc4-e586-4431-bf16-af0bd23efac1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=76e8ae1ebdbb4e34bb4b4ef59e70305c, role_id=b5c148877d4a489f85a6d554d44dba3f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:23.840420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2148fc4-e586-4431-bf16-af0bd23efac1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:23.866198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 413/837] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:23.866448 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 425/838] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:02:23 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/76e8ae1ebdbb4e34bb4b4ef59e70305c/roles/b5c148877d4a489f85a6d554d44dba3f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:02:23.866669 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8141972-5c20-4a73-8217-d3693869eac4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:23.911067 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2148fc4-e586-4431-bf16-af0bd23efac1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:24.040691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:24.049707 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8141972-5c20-4a73-8217-d3693869eac4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:24.089028 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:24.111231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:24.112432 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:24.113023 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:24.113622 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:24.121108 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2148fc4-e586-4431-bf16-af0bd23efac1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:24.133778 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:24.136548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:24.160375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:24.161048 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:24.161324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:24.161595 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:24.162276 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:24.163676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:24.170696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:24.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:24.174249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:24.174249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:24.174249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:24.174249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:24.174542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:24.176161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:24.177857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 426/839] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:24.180780 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 414/840] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:24.186371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:24.186550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:24.301973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031791bd-1aca-4e7f-8b67-f8d2bf2b2bdc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:24.329635 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:24.340158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6550ee-3cb8-4dbb-9375-2b1ddf8f8e0e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:24.350202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:24.351210 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:24.352203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:24.352680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:24.353996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:24.354842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:24.356743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:24.364717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 427/841] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:24.370252 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:24.380691 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:24.381460 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:24.381763 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:24.382099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:24.382890 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:24.383278 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:24.384761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:24.398895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 415/842] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:24.404580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:24.453887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:24.513359 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417e5cd-6b75-4987-81cf-a03b778558c9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:24.532509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b7612-060a-4e58-9dd0-244b4e081fea None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:24.551744 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:24.552423 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:24.552708 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:24.552986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:24.553691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:24.554002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:24.555453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:24.561304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 428/843] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:24.571194 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:24.571877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:24.572155 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:24.572427 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:24.581931 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:24.583532 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:24.599440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:24.600879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:24.602867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 416/844] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:24.607323 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:24.678673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:24.745493 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68acf8-090b-4846-870d-fa06f8b29628 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:24.799912 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:24.800576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:24.800857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:24.801130 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:24.801972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:24.802317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:24.803817 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:24.830413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 429/845] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:02:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:24.832674 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:24.958578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:24.962730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e42ead-2fae-4f4d-ab05-2dd751e7ff7a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:24.999881 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43365b7-4380-4de3-971d-b5bc612ac8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:25.000527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43365b7-4380-4de3-971d-b5bc612ac8bc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:25.000801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43365b7-4380-4de3-971d-b5bc612ac8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:25.001066 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43365b7-4380-4de3-971d-b5bc612ac8bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:25.077711 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43365b7-4380-4de3-971d-b5bc612ac8bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:25.079239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43365b7-4380-4de3-971d-b5bc612ac8bc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:25.189273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:25.194791 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 417/846] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:02:25.201881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:25.363076 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae48b89-faf4-423f-82d8-5eecbc9b4d24 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:25.378897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:25.423302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:25.424420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:25.424904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:25.425400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:25.438553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:25.440442 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:25.458788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:25.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:25.460567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:25.460567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:25.460567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:25.460567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:25.462885 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 430/847] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:25.471351 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43365b7-4380-4de3-971d-b5bc612ac8bc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:25.591367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43365b7-4380-4de3-971d-b5bc612ac8bc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:25.629704 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:25.630381 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:25.630698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:25.630930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:25.631728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:25.632034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:25.633436 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:25.648128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 418/848] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:25.653622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:25.791179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed610e10-3444-4b32-a5e5-829482de1acb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:25.832634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ff9a5-5836-429f-a6a1-d01a2949ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:25.833333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff9a5-5836-429f-a6a1-d01a2949ab91 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:25.833620 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff9a5-5836-429f-a6a1-d01a2949ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:25.833893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff9a5-5836-429f-a6a1-d01a2949ab91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:25.842330 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ff9a5-5836-429f-a6a1-d01a2949ab91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:25.843819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ff9a5-5836-429f-a6a1-d01a2949ab91 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-2b2ff9a5-5836-429f-a6a1-d01a2949ab91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:25.857395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:25.858389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:25.859266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 431/849] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:25.865106 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:25.938340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:25.988710 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf1a53-fee2-4521-b77f-8b9fdcc52f40 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:26.007328 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4c249-6e08-40b8-9ed9-48c947195a14 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.007681 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4c249-6e08-40b8-9ed9-48c947195a14 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.008017 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4c249-6e08-40b8-9ed9-48c947195a14 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.009749 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 432/850] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:26.017005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac6ce-aac3-43d9-b483-96266f43412a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.017672 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac6ce-aac3-43d9-b483-96266f43412a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.018533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac6ce-aac3-43d9-b483-96266f43412a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.037358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:26.038471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.038869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.039147 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.040205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:26.040544 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:26.042029 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:26.043727 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f42ac6ce-aac3-43d9-b483-96266f43412a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:26.046013 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f42ac6ce-aac3-43d9-b483-96266f43412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:26.066172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 419/851] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:26.069277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:26.073383 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:26.159920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 433/852] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:26.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ac6ce-aac3-43d9-b483-96266f43412a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:26.241085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99662bef-2479-447e-a362-0a294e4a3101 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:26.275228 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:26.309371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:26.310614 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.311124 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.311664 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.313146 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=f285251a0e0a49849d071155d2d3e478) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:26.313768 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:26.316947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:26.332874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ac6ce-aac3-43d9-b483-96266f43412a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:26.334580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 420/853] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:26 2022] GET /identity/v3/role_assignments?user.id=f285251a0e0a49849d071155d2d3e478&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=65cd37a73e4f498482fefabb8374006c => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:26.382172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd7052-a684-4599-b6b9-bde351b9dd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:26.382874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd7052-a684-4599-b6b9-bde351b9dd8e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.383157 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd7052-a684-4599-b6b9-bde351b9dd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.383517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd7052-a684-4599-b6b9-bde351b9dd8e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.384182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd7052-a684-4599-b6b9-bde351b9dd8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:26.385839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd7052-a684-4599-b6b9-bde351b9dd8e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-59cd7052-a684-4599-b6b9-bde351b9dd8e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:26.399031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:26.400482 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:26.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 434/854] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:02:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:26.412833 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:26.465850 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a43adc-081b-47a6-9231-24557c2fbcaa None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.466082 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a43adc-081b-47a6-9231-24557c2fbcaa None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.466457 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a43adc-081b-47a6-9231-24557c2fbcaa None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.468024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 435/855] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:26.473877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50b3c0-b51d-4e99-8a8f-aa43a6c01a81 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.474312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50b3c0-b51d-4e99-8a8f-aa43a6c01a81 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.474478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50b3c0-b51d-4e99-8a8f-aa43a6c01a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.494955 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4d50b3c0-b51d-4e99-8a8f-aa43a6c01a81 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:26.496200 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4d50b3c0-b51d-4e99-8a8f-aa43a6c01a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:26.508642 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:26.586983 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 436/856] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:26.597473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a4d91-8d46-4940-ae1f-86939513d215 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:26.609339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50b3c0-b51d-4e99-8a8f-aa43a6c01a81 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:26.655303 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:26.656008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.656286 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.656564 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.657601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:26.657957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:26.660850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:26.672603 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 421/857] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:26.675048 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:26.744084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:26.768740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50b3c0-b51d-4e99-8a8f-aa43a6c01a81 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:26.813506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe9b14-ee10-4889-9ae8-e83223ead080 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:26.838094 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab95d0-524f-4c11-8da8-9940816b6d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:26.839724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab95d0-524f-4c11-8da8-9940816b6d49 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.840598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab95d0-524f-4c11-8da8-9940816b6d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.841374 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab95d0-524f-4c11-8da8-9940816b6d49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.856873 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464eebb-1863-475f-8830-a66cb686c0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:26.857579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464eebb-1863-475f-8830-a66cb686c0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.857860 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464eebb-1863-475f-8830-a66cb686c0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.858156 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464eebb-1863-475f-8830-a66cb686c0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.858868 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464eebb-1863-475f-8830-a66cb686c0f7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:26.859172 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464eebb-1863-475f-8830-a66cb686c0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:26.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464eebb-1863-475f-8830-a66cb686c0f7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:26.874273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 422/858] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:26.876717 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422d8d2-43df-4175-80f1-66e06a4ae312 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.877307 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab95d0-524f-4c11-8da8-9940816b6d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:26.877565 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422d8d2-43df-4175-80f1-66e06a4ae312 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.878225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422d8d2-43df-4175-80f1-66e06a4ae312 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.887634 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 423/859] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:26 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-90ab95d0-524f-4c11-8da8-9940816b6d49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:26.888765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:26.890504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:02:26.892133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:26.893418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d7150-70b9-43e7-960c-cae8fab20e07 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:26.893418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d7150-70b9-43e7-960c-cae8fab20e07 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:26.893418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d7150-70b9-43e7-960c-cae8fab20e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:26.901515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 437/860] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:02:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:26.907723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab95d0-524f-4c11-8da8-9940816b6d49 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:26.915467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:26.922524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-042d7150-70b9-43e7-960c-cae8fab20e07 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:26.924706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-042d7150-70b9-43e7-960c-cae8fab20e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:27.017074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 424/861] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:27.051122 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d7150-70b9-43e7-960c-cae8fab20e07 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:27.107378 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab95d0-524f-4c11-8da8-9940816b6d49 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:27.129850 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:27.151494 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:27.152557 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:27.153034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:27.153543 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:27.154507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:27.155016 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:27.156940 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:27.168218 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 438/862] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:02:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:27.180551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:27.246737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d7150-70b9-43e7-960c-cae8fab20e07 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:27.302998 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef48f09-3e55-46b0-b8c4-f79e28a891c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:27.303558 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef48f09-3e55-46b0-b8c4-f79e28a891c2 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:27.303941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef48f09-3e55-46b0-b8c4-f79e28a891c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:27.304216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef48f09-3e55-46b0-b8c4-f79e28a891c2 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:27.305098 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef48f09-3e55-46b0-b8c4-f79e28a891c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:27.306598 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef48f09-3e55-46b0-b8c4-f79e28a891c2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:27.312513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:27.331571 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 425/863] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:02:27 2022] POST /identity/v3/services => generated 251 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:02:27.362407 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef9d30-62e8-4c00-b877-e711f87e73dc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:27.413230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:27.414309 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:27.414739 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:27.415183 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:27.431546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:27.433867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:27.452668 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:27.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:27.455152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:27.455152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:27.455152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:27.455152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:27.457259 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 439/864] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:27.463859 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef48f09-3e55-46b0-b8c4-f79e28a891c2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:27.577094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:27.582700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef48f09-3e55-46b0-b8c4-f79e28a891c2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:27.620671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:27.621741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:27.622198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:27.622493 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:27.623237 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:27.623585 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:27.625309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:27.643257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 426/865] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:27.649288 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:27.773893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cbe2f-f5f2-46ca-afbb-f4865591077c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:27.789549 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:27.811104 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:27.811802 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:27.812080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:27.812396 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:27.820606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:27.822057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:27.834790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:27.835897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:27.837026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 440/866] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:02:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:27.843416 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:27.968027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997912d-d51f-4b45-be18-612371a9dbd2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:28.013257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:28.014434 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:28.014905 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:28.015380 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:28.016415 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:28.017056 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:28.018798 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:28.039829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 427/867] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:02:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:28.045844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:28.164859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc110d8e-30d9-4000-b185-e2e1a5447080 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:28.204285 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b99252-43e6-4cdf-995b-dcec894fd3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:28.204942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99252-43e6-4cdf-995b-dcec894fd3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:28.205252 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99252-43e6-4cdf-995b-dcec894fd3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:28.205603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99252-43e6-4cdf-995b-dcec894fd3c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:28.254076 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:28.285049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99252-43e6-4cdf-995b-dcec894fd3c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:28.286735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99252-43e6-4cdf-995b-dcec894fd3c7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:28.394677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 441/868] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:28 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:02:28.403343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:28.425212 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:28.524259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46076898-d1f4-46cc-a7fe-7045b3159c99 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:28.552833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770bd75-eb32-461d-a413-b57340d2f6a2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:28.553317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770bd75-eb32-461d-a413-b57340d2f6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:28.553792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770bd75-eb32-461d-a413-b57340d2f6a2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:28.556812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 442/869] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:28.564360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07028851-d21f-4e57-bdfd-0dd52eb77854 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:28.564999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07028851-d21f-4e57-bdfd-0dd52eb77854 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:28.565476 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07028851-d21f-4e57-bdfd-0dd52eb77854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:28.581793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:28.582514 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:28.582794 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:28.583068 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:28.586666 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-07028851-d21f-4e57-bdfd-0dd52eb77854 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:28.588096 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-07028851-d21f-4e57-bdfd-0dd52eb77854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:28.595525 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:28.597536 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:28.617570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:28.618815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:28.619959 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:28.619959 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:28.619959 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:28.619959 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:28.619959 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 428/870] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:28.625416 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:28.681725 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 443/871] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:28.703814 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07028851-d21f-4e57-bdfd-0dd52eb77854 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:28.711162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:28.737393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:28.788252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f27e6d-7a9c-4dd3-9187-0bc1eee93ad4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:28.827356 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:28.828444 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:28.828444 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:28.828618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:28.829221 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:28.829526 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:28.830996 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:28.846306 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 429/872] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:28.855449 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:28.862938 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07028851-d21f-4e57-bdfd-0dd52eb77854 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:28.920465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:28.921197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:28.921483 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:28.921754 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:28.922485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:28.923903 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:28.938594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:28.939841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:28.940789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 444/873] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:28.948667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:29.016649 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01bcd-0c2b-4608-987b-25f9a3d80452 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:29.058879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:29.067787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:29.071315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:29.072675 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.073221 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.074818 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.084602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:29.087348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:29.106579 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:29.107627 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:29.108653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:29.108653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:29.108653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:29.108653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:29.108653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:29.108653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:29.108653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:29.108653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:29.116650 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 430/874] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:29.119573 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:29.124411 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d8da-0c01-4473-91dd-732c0eb9b0c1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:29.167946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bb0f4-e05e-4f8d-9a88-d54337834738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:29.168581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bb0f4-e05e-4f8d-9a88-d54337834738 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.168867 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bb0f4-e05e-4f8d-9a88-d54337834738 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.169142 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bb0f4-e05e-4f8d-9a88-d54337834738 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.169838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bb0f4-e05e-4f8d-9a88-d54337834738 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:29.170172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bb0f4-e05e-4f8d-9a88-d54337834738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:29.171582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bb0f4-e05e-4f8d-9a88-d54337834738 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:29.177446 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 445/875] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:29.178604 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2055e-a0a8-47ce-aa9e-35a51fcf3847 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.178893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2055e-a0a8-47ce-aa9e-35a51fcf3847 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.179163 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2055e-a0a8-47ce-aa9e-35a51fcf3847 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.180667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 446/876] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:29.183610 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2055e-a0a8-47ce-aa9e-35a51fcf3847 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:29.280596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd2d59-9139-4907-bd04-25b721b40c20 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:29.331009 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01397a-564e-4563-a51c-60378dad3ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:29.332407 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01397a-564e-4563-a51c-60378dad3ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.332676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01397a-564e-4563-a51c-60378dad3ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.332996 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01397a-564e-4563-a51c-60378dad3ba4 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.334277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01397a-564e-4563-a51c-60378dad3ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:29.334588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01397a-564e-4563-a51c-60378dad3ba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:29.336282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01397a-564e-4563-a51c-60378dad3ba4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:29.338226 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2055e-a0a8-47ce-aa9e-35a51fcf3847 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:29.356618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 431/877] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:29.358378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d97db-7b2c-48ab-8dd4-9f21808f038b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.358855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d97db-7b2c-48ab-8dd4-9f21808f038b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.359491 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d97db-7b2c-48ab-8dd4-9f21808f038b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.378254 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-774d97db-7b2c-48ab-8dd4-9f21808f038b None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:29.379474 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-774d97db-7b2c-48ab-8dd4-9f21808f038b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:29.386316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:29.393664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a125c-18f9-4794-ac16-25525396725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:29.394392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a125c-18f9-4794-ac16-25525396725f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.394771 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a125c-18f9-4794-ac16-25525396725f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.394958 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a125c-18f9-4794-ac16-25525396725f None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.395720 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a125c-18f9-4794-ac16-25525396725f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:29.396068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a125c-18f9-4794-ac16-25525396725f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:29.397802 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a125c-18f9-4794-ac16-25525396725f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:29.403777 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 447/878] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:29.404731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a125c-18f9-4794-ac16-25525396725f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:29.457715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:29.494740 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 432/879] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:29 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:29.516536 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d97db-7b2c-48ab-8dd4-9f21808f038b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:29.547152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a125c-18f9-4794-ac16-25525396725f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:29.601852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca051f-e90c-4c80-8980-7530085730cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:29.602862 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca051f-e90c-4c80-8980-7530085730cc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.603166 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca051f-e90c-4c80-8980-7530085730cc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.603457 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca051f-e90c-4c80-8980-7530085730cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.604359 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca051f-e90c-4c80-8980-7530085730cc None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=76e8ae1ebdbb4e34bb4b4ef59e70305c) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:29.604693 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca051f-e90c-4c80-8980-7530085730cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:29.608768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca051f-e90c-4c80-8980-7530085730cc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:29.616716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 448/880] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:29 2022] GET /identity/v3/role_assignments?user.id=76e8ae1ebdbb4e34bb4b4ef59e70305c&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:29.652731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d97db-7b2c-48ab-8dd4-9f21808f038b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:29.661681 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0770e1-0f3a-46f2-a117-2dcfea1d5683 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.662055 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0770e1-0f3a-46f2-a117-2dcfea1d5683 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.662382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0770e1-0f3a-46f2-a117-2dcfea1d5683 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.669113 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 449/881] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:29.671235 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eb20b-88ef-4e44-b4b2-b01b0af1792d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.671609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eb20b-88ef-4e44-b4b2-b01b0af1792d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.671886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eb20b-88ef-4e44-b4b2-b01b0af1792d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.694608 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c81eb20b-88ef-4e44-b4b2-b01b0af1792d None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:29.696119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c81eb20b-88ef-4e44-b4b2-b01b0af1792d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:29.700615 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c37cf9-ef78-473b-a186-b8445f383d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:29.701522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37cf9-ef78-473b-a186-b8445f383d12 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:29.702451 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37cf9-ef78-473b-a186-b8445f383d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:29.703127 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37cf9-ef78-473b-a186-b8445f383d12 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:29.704331 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c37cf9-ef78-473b-a186-b8445f383d12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:29.706436 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c37cf9-ef78-473b-a186-b8445f383d12 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:29.743894 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 433/882] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:02:29 2022] POST /identity/v3/services => generated 269 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:02:29.811375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:29.827030 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 450/883] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:29.852178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c37cf9-ef78-473b-a186-b8445f383d12 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:29.932941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:30.010020 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c37cf9-ef78-473b-a186-b8445f383d12 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:30.053864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:30.054700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:30.054988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:30.055332 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:30.056083 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:30.057658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:30.068729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:30.069715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:30.070550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 434/884] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:30.077156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81eb20b-88ef-4e44-b4b2-b01b0af1792d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:30.199974 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81eb20b-88ef-4e44-b4b2-b01b0af1792d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:30.238598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:30.239317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:30.239615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:30.239888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:30.240592 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:30.240899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:30.242423 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:30.253486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 451/885] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:30.256018 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:30.376306 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e8c9f-76ef-4d63-8bcd-b091dc15b83f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:30.379317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:30.418416 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:30.419124 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:30.419463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:30.419777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:30.420540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:30.420893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:30.422374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:30.428553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 435/886] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:30.435443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:30.501292 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:30.553725 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ee67a-2c02-4895-89af-a1eb49644671 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:30.600893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a8c43-b24d-4ac4-a234-8bd7783dce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:30.601973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a8c43-b24d-4ac4-a234-8bd7783dce78 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:30.602504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a8c43-b24d-4ac4-a234-8bd7783dce78 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:30.602864 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a8c43-b24d-4ac4-a234-8bd7783dce78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:30.604138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a8c43-b24d-4ac4-a234-8bd7783dce78 None admin] RBAC: Adding query filter params (interface=public, service_id=abc2a4f29206422c80a643fcc322fd24, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:30.604703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a8c43-b24d-4ac4-a234-8bd7783dce78 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:30.607388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a8c43-b24d-4ac4-a234-8bd7783dce78 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:30.621976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 452/887] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:02:30 2022] GET /identity/v3/endpoints?service_id=abc2a4f29206422c80a643fcc322fd24&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:30.623662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:30.769399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f729133-e3af-4036-a740-7d996bb90608 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:30.811059 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75310e-bd88-4fe0-9a9f-b0c379e8f54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:30.811773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75310e-bd88-4fe0-9a9f-b0c379e8f54b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:30.812090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75310e-bd88-4fe0-9a9f-b0c379e8f54b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:30.812410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75310e-bd88-4fe0-9a9f-b0c379e8f54b None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:30.813160 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75310e-bd88-4fe0-9a9f-b0c379e8f54b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:30.814693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75310e-bd88-4fe0-9a9f-b0c379e8f54b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:30.821869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 436/888] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:02:30 2022] GET /identity/v3/services => generated 1268 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:02:31.704104 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfafcbc-2be5-41ba-9913-8837a8d37b73 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:31.704679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfafcbc-2be5-41ba-9913-8837a8d37b73 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:31.704679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfafcbc-2be5-41ba-9913-8837a8d37b73 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:31.706582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 453/889] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:31.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8369-aaaf-45a5-af24-c32d72df3c03 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:31.713544 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8369-aaaf-45a5-af24-c32d72df3c03 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:31.713625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8369-aaaf-45a5-af24-c32d72df3c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:31.742141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f5fe8369-aaaf-45a5-af24-c32d72df3c03 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:31.743906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f5fe8369-aaaf-45a5-af24-c32d72df3c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:31.781206 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:31.834509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 437/890] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:31.860182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfafcbc-2be5-41ba-9913-8837a8d37b73 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:31.999821 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf878e2-e490-4631-9970-0521d6970296 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.000075 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf878e2-e490-4631-9970-0521d6970296 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.000460 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf878e2-e490-4631-9970-0521d6970296 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.002182 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 438/891] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:32.007443 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f811f6-f5f8-4636-b117-1b53db9d89ab None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.007758 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f811f6-f5f8-4636-b117-1b53db9d89ab None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.007985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f811f6-f5f8-4636-b117-1b53db9d89ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.028164 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-98f811f6-f5f8-4636-b117-1b53db9d89ab None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:32.028839 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-98f811f6-f5f8-4636-b117-1b53db9d89ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:32.035634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfafcbc-2be5-41ba-9913-8837a8d37b73 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:32.063711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:32.094548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dbdf8-9625-4199-9120-6848024b1b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:32.095441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dbdf8-9625-4199-9120-6848024b1b32 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.095965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dbdf8-9625-4199-9120-6848024b1b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.096498 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dbdf8-9625-4199-9120-6848024b1b32 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.097707 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09dbdf8-9625-4199-9120-6848024b1b32 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:32.099221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09dbdf8-9625-4199-9120-6848024b1b32 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:32.118625 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 454/892] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:02:31 2022] POST /identity/v3/services => generated 241 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:32.120752 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a7e5-edc1-4164-b298-de6a9cac26d8 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.121113 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a7e5-edc1-4164-b298-de6a9cac26d8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.121464 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a7e5-edc1-4164-b298-de6a9cac26d8 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.123540 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 455/893] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:32.132847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185db930-d20e-44c5-8529-306cb7e83f32 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.133171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185db930-d20e-44c5-8529-306cb7e83f32 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.133444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185db930-d20e-44c5-8529-306cb7e83f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.142238 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 439/894] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:32.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-185db930-d20e-44c5-8529-306cb7e83f32 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:32.157024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-185db930-d20e-44c5-8529-306cb7e83f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:32.172128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f811f6-f5f8-4636-b117-1b53db9d89ab None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:32.249263 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 456/895] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:32.272114 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185db930-d20e-44c5-8529-306cb7e83f32 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:32.344464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:32.358652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f811f6-f5f8-4636-b117-1b53db9d89ab None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:32.397422 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cdf4d-c191-4ad7-84d8-4c97ed3f4105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:32.398154 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cdf4d-c191-4ad7-84d8-4c97ed3f4105 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.398438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cdf4d-c191-4ad7-84d8-4c97ed3f4105 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.398707 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cdf4d-c191-4ad7-84d8-4c97ed3f4105 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.410316 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cdf4d-c191-4ad7-84d8-4c97ed3f4105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:32.412790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185db930-d20e-44c5-8529-306cb7e83f32 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-069cdf4d-c191-4ad7-84d8-4c97ed3f4105 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:32.415266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:32.416235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:32.417274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 440/896] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:02:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:32.422901 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cdf4d-c191-4ad7-84d8-4c97ed3f4105 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:32.424121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:32.463430 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:32.464450 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.465381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.465715 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.466932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:32.469427 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:32.488717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:32.490984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:32.492691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:32.497535 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 457/897] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:02:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:32.499446 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:32.578343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cdf4d-c191-4ad7-84d8-4c97ed3f4105 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:32.625218 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a0115-ac78-46cb-b028-861fdde3965e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:32.635594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:32.636935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.637456 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.637937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.638928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:32.639453 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:32.641233 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:32.656921 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 441/898] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:02:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:32.659207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:32.664337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:32.664789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.665025 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.665297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.665987 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:32.666322 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:32.667702 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:32.674142 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 458/899] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:32.681363 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:32.684739 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:32.745857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:32.814119 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68197d4-0ec3-4ca6-9891-0c898a6ebea4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:32.828945 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774ffca-1fbb-4b79-be74-05e730883641 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:32.853808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:32.855220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.855825 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.856443 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.866632 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:32.867942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:32.867942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:32.868214 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:32.868905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:32.869213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:32.871550 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:32.873520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:32.875442 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:32.884558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 459/900] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:32.888023 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:32.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:32.900484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:32.901393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:32.901393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:32.901393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:32.901393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:32.908983 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 442/901] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:32.910793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:32.981457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:33.008535 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:33.016321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b65eb-df39-4ee3-a30f-6018f83aac44 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.029277 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af57f5-1a32-4833-8da0-2ca8b02b48e1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.055512 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde8a47-d495-44c1-9809-448ffdca35b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.056268 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde8a47-d495-44c1-9809-448ffdca35b2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.056502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde8a47-d495-44c1-9809-448ffdca35b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.056772 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde8a47-d495-44c1-9809-448ffdca35b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.057468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde8a47-d495-44c1-9809-448ffdca35b2 None admin] RBAC: Adding query filter params (interface=public, service_id=f4f978775de040dd936dee296d36fffa, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:33.057774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde8a47-d495-44c1-9809-448ffdca35b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:33.059203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde8a47-d495-44c1-9809-448ffdca35b2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:33.065075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 460/902] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:02:32 2022] GET /identity/v3/endpoints?service_id=f4f978775de040dd936dee296d36fffa&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:33.070145 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362f8a3-20a3-4e20-8e43-29694f1feb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.070331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44991-898c-4fa5-ae71-0a1b0745bbc9 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.070331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44991-898c-4fa5-ae71-0a1b0745bbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.070331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44991-898c-4fa5-ae71-0a1b0745bbc9 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.070543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f8a3-20a3-4e20-8e43-29694f1feb3a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.070543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f8a3-20a3-4e20-8e43-29694f1feb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.070543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f8a3-20a3-4e20-8e43-29694f1feb3a None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.070543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362f8a3-20a3-4e20-8e43-29694f1feb3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:33.070784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 461/903] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:33.070858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362f8a3-20a3-4e20-8e43-29694f1feb3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:33.070858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362f8a3-20a3-4e20-8e43-29694f1feb3a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:33.074267 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d44991-898c-4fa5-ae71-0a1b0745bbc9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:33.090182 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 443/904] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:33.091566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bb862-aaed-4e9b-9b9a-7fee4ff57689 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.091872 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bb862-aaed-4e9b-9b9a-7fee4ff57689 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.092151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bb862-aaed-4e9b-9b9a-7fee4ff57689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.115602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d69bb862-aaed-4e9b-9b9a-7fee4ff57689 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:33.117114 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d69bb862-aaed-4e9b-9b9a-7fee4ff57689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:33.209627 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d44991-898c-4fa5-ae71-0a1b0745bbc9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.218171 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 444/905] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:33.218171 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bb862-aaed-4e9b-9b9a-7fee4ff57689 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:33.251180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad855ac-e7b9-4754-bc50-92662591f898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.251914 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad855ac-e7b9-4754-bc50-92662591f898 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.252267 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad855ac-e7b9-4754-bc50-92662591f898 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.252535 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad855ac-e7b9-4754-bc50-92662591f898 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.253212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad855ac-e7b9-4754-bc50-92662591f898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:33.254795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad855ac-e7b9-4754-bc50-92662591f898 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:33.261799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 462/906] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:02:33 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:02:33.262753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad855ac-e7b9-4754-bc50-92662591f898 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:33.297648 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:33.300139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:33.367858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bb862-aaed-4e9b-9b9a-7fee4ff57689 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.423176 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.424525 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.424998 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.426024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.434661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad855ac-e7b9-4754-bc50-92662591f898 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.436880 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:33.439935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:33.457341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:33.458917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:33.459854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:33.460397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 445/907] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:02:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:33.468141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:33.488642 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.489312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.489591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.489866 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.490680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:33.492108 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:33.511009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:33.512092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:33.512965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 463/908] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:33.521119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:33.604240 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462454-745e-4d17-b1e2-0ff6cdacfb89 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.625390 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:33.651076 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:33.665150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.666578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.666773 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c98133-dc7f-4d3b-a154-6829fa1e0d8d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.667469 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.668081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.669144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:33.669621 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:33.672528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:33.698580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 446/909] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:02:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:33.710158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:33.722428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.723074 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.723399 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.723675 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.724373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:33.730140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:33.730140 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:33.734220 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 464/910] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:33.739148 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:33.854825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7c1d7-f82a-4142-b0da-b5064e0ea956 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.860985 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aede4-808b-4963-bc70-acc2e396a703 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:33.903275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.904553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.905043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.905589 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.906533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:33.907180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:33.908892 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:33.914913 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 465/911] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:02:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:33.920196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c1e70-c620-48d9-a4ae-925c2697449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:33.920837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:33.921525 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c1e70-c620-48d9-a4ae-925c2697449d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:33.922031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c1e70-c620-48d9-a4ae-925c2697449d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:33.922537 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c1e70-c620-48d9-a4ae-925c2697449d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:33.977561 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:33.979166 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:34.021461 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c1e70-c620-48d9-a4ae-925c2697449d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:34.025548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c1e70-c620-48d9-a4ae-925c2697449d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:34.093117 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ad831-80dd-4ff8-bc66-1f8a0d3d8f48 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:34.133261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20456c2c-b1ea-4231-a3ad-7affcca2950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:34.133931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20456c2c-b1ea-4231-a3ad-7affcca2950f None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:34.134319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20456c2c-b1ea-4231-a3ad-7affcca2950f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:34.134588 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20456c2c-b1ea-4231-a3ad-7affcca2950f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:34.135312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20456c2c-b1ea-4231-a3ad-7affcca2950f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:34.136695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20456c2c-b1ea-4231-a3ad-7affcca2950f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:34.170157 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 447/912] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:33 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:02:34.182493 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 466/913] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:02:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:02:34.182721 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c1e70-c620-48d9-a4ae-925c2697449d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:34.290897 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:34.328631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c1e70-c620-48d9-a4ae-925c2697449d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:34.403464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac8369-5f84-40cc-a3f4-69ed04e39725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:34.404957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac8369-5f84-40cc-a3f4-69ed04e39725 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:34.405835 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac8369-5f84-40cc-a3f4-69ed04e39725 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:34.406738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac8369-5f84-40cc-a3f4-69ed04e39725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:34.428265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac8369-5f84-40cc-a3f4-69ed04e39725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:34.430992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac8369-5f84-40cc-a3f4-69ed04e39725 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5cac8369-5f84-40cc-a3f4-69ed04e39725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:34.451692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:34.452742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:34.453744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:34.453744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:34.453744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:34.453744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:34.458270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 448/914] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:34.463598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20456c2c-b1ea-4231-a3ad-7affcca2950f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:34.529229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95dde9-fda0-4116-b046-6f5d3644c276 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:34.531440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95dde9-fda0-4116-b046-6f5d3644c276 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:34.531440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95dde9-fda0-4116-b046-6f5d3644c276 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:34.532201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 449/915] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:34.540347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadeb5b-162e-4489-9cb8-76cf41d72b6e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:34.540717 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadeb5b-162e-4489-9cb8-76cf41d72b6e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:34.541146 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadeb5b-162e-4489-9cb8-76cf41d72b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:34.567413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1cadeb5b-162e-4489-9cb8-76cf41d72b6e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:34.569169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1cadeb5b-162e-4489-9cb8-76cf41d72b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:34.586614 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:34.621315 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20456c2c-b1ea-4231-a3ad-7affcca2950f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:34.668711 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 450/916] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:34.677049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:34.677956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:34.678465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:34.678687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:34.679975 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:34.680309 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:34.682774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:34.693492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cadeb5b-162e-4489-9cb8-76cf41d72b6e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:34.703015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 467/917] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:34.710989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:34.791070 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:34.859207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cadeb5b-162e-4489-9cb8-76cf41d72b6e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:34.872251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980e73c-4287-419b-a505-92398224ce07 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:34.900042 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:34.900843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:34.901020 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:34.901257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:34.902240 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:34.903553 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:34.914808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:34.922329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:34.924327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 451/918] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:34.938017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:34.953468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:34.962376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:34.964098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:34.964794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:34.965470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:34.979581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:34.982443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:35.005738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:35.007102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:35.008453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:35.008453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:35.008453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:35.008453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:35.008453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:35.008453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:35.008453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:35.008453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:35.017241 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 468/919] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:35.019014 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:35.056882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c36b-dfca-4933-9323-9e81a0ae122b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:35.072011 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:35.097182 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:35.098335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:35.098521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:35.098832 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:35.099630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:35.100032 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:35.101523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:35.112068 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 452/920] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:35.114715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:35.163767 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb12c30-ee0b-48ee-ab08-27de5b0b9d87 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:35.203557 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:35.204464 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:35.205030 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:35.205544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:35.206729 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:35.207273 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:35.208958 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:35.228605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 469/921] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:35.229518 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ed9d-45c7-44c0-9795-d56b9b5d4741 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:35.230737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:35.267363 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:35.268076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:35.268353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:35.268623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:35.269319 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:35.269617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:35.271901 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:35.281708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 453/922] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:35.285629 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:35.300277 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:35.366756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76673e77-a0c5-465e-a619-4b2ba2a4843f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:35.367994 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:35.409404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:35.410145 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:35.410431 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:35.410699 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:35.411644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] RBAC: Adding query filter params (role.id=d8710b1ea79f4ed78a1c4d54ee6e9148, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=76e8ae1ebdbb4e34bb4b4ef59e70305c) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:35.411977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:35.413878 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:35.422156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 470/923] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:35 2022] GET /identity/v3/role_assignments?user.id=76e8ae1ebdbb4e34bb4b4ef59e70305c&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=d8710b1ea79f4ed78a1c4d54ee6e9148 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:35.462722 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272e5d6-c4e5-4a81-9408-e086f2e5280b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:35.505822 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e325a-6ff1-43ec-afea-b27d2b7d2403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:35.506639 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e325a-6ff1-43ec-afea-b27d2b7d2403 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:35.506919 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e325a-6ff1-43ec-afea-b27d2b7d2403 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:35.507198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e325a-6ff1-43ec-afea-b27d2b7d2403 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:35.507990 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e325a-6ff1-43ec-afea-b27d2b7d2403 None admin] RBAC: Adding query filter params (interface=public, service_id=813af9f261974435bdfe183c96aaa4e3, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:35.508300 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e325a-6ff1-43ec-afea-b27d2b7d2403 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:35.509815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e325a-6ff1-43ec-afea-b27d2b7d2403 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:35.519091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 454/924] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:02:35 2022] GET /identity/v3/endpoints?service_id=813af9f261974435bdfe183c96aaa4e3&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:35.524830 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:35.647241 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55478107-d426-4d42-8dd0-2f63f5b5d89f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:35.666454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:35.691337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e58dd-6e76-4f62-8b37-abc5992c0953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:35.691971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e58dd-6e76-4f62-8b37-abc5992c0953 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:35.692311 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e58dd-6e76-4f62-8b37-abc5992c0953 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:35.692575 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e58dd-6e76-4f62-8b37-abc5992c0953 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:35.693291 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e58dd-6e76-4f62-8b37-abc5992c0953 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:35.694726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e58dd-6e76-4f62-8b37-abc5992c0953 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:35.702293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 471/925] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:02:35 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:02:35.801815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620a11-3232-4f91-ba61-9d0eefe53809 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:35.802213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620a11-3232-4f91-ba61-9d0eefe53809 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:35.802540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620a11-3232-4f91-ba61-9d0eefe53809 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:35.804770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 455/926] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:35.811305 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ed030-9a56-4b3f-83d9-7dfc0f2d8f82 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:35.811582 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ed030-9a56-4b3f-83d9-7dfc0f2d8f82 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:35.812022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ed030-9a56-4b3f-83d9-7dfc0f2d8f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:35.832794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-425ed030-9a56-4b3f-83d9-7dfc0f2d8f82 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:35.834170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-425ed030-9a56-4b3f-83d9-7dfc0f2d8f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:35.934601 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 472/927] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:35.961460 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10620a11-3232-4f91-ba61-9d0eefe53809 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:36.094474 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10620a11-3232-4f91-ba61-9d0eefe53809 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:36.097280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:36.133183 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:36.133858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:36.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:36.134434 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:36.135095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:36.136507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:36.146803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:36.147800 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.148870 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 456/928] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:02:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:36.154722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ed030-9a56-4b3f-83d9-7dfc0f2d8f82 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:36.280910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ed030-9a56-4b3f-83d9-7dfc0f2d8f82 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:36.303960 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:36.348221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:36.349185 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:36.349646 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:36.350087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:36.351043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:36.351565 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:36.353130 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:36.359323 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 473/929] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:36.365409 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:36.490895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609279a-cf26-486b-bed9-4847dc95c94c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:36.528526 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fa8d5-972e-4fa6-af26-d6d603ac7211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:36.529173 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa8d5-972e-4fa6-af26-d6d603ac7211 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:36.529454 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa8d5-972e-4fa6-af26-d6d603ac7211 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:36.529733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa8d5-972e-4fa6-af26-d6d603ac7211 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:36.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fa8d5-972e-4fa6-af26-d6d603ac7211 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:36.530784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fa8d5-972e-4fa6-af26-d6d603ac7211 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:36.532290 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fa8d5-972e-4fa6-af26-d6d603ac7211 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:36.538520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 457/930] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:36.544243 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:36.588136 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7305e1-3de1-47ce-909f-8d3c2817eb50 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:36.588505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7305e1-3de1-47ce-909f-8d3c2817eb50 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:36.588696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7305e1-3de1-47ce-909f-8d3c2817eb50 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:36.590398 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 458/931] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:36.596559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957c0b8-6b95-46c4-b404-fe6d5592b355 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:36.596900 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957c0b8-6b95-46c4-b404-fe6d5592b355 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:36.597110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957c0b8-6b95-46c4-b404-fe6d5592b355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:36.617189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f957c0b8-6b95-46c4-b404-fe6d5592b355 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:36.618415 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f957c0b8-6b95-46c4-b404-fe6d5592b355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:36.649499 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:36.663432 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55143178-bb5b-447c-98bb-85b8aa351806 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:36.702541 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 459/932] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:36.720438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957c0b8-6b95-46c4-b404-fe6d5592b355 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:36.728291 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23a716-53ec-4fdc-acff-5119d38fc32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:36.729827 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23a716-53ec-4fdc-acff-5119d38fc32f None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:36.730660 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23a716-53ec-4fdc-acff-5119d38fc32f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:36.731500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23a716-53ec-4fdc-acff-5119d38fc32f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:36.733348 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23a716-53ec-4fdc-acff-5119d38fc32f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:36.736506 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23a716-53ec-4fdc-acff-5119d38fc32f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:36.774915 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 474/933] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:02:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:36.837134 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957c0b8-6b95-46c4-b404-fe6d5592b355 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:36.886523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:36.887275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:36.887577 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:36.887854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:36.888578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:36.890220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:36.893200 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: WARNING keystone.server.flask.common [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 08:02:36.916008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.sql.core [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:02:36.916008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:02:36.916008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [parameters: {'id': 'e7d9b93f30514894991a05f9a9a98531', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:02:36.916008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80368) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e7d9b93f30514894991a05f9a9a98531.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e7d9b93f30514894991a05f9a9a98531. Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:02:36.937412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application result.read() Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 08:02:36.938323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application yield resource Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise self.value Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application session.commit() Jul 28 08:02:36.939246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:02:36.940201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:02:36.941061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application result.read() Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:02:36.941927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [parameters: {'id': 'e7d9b93f30514894991a05f9a9a98531', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:36.944697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:36.946547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e7d9b93f30514894991a05f9a9a98531. Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:36.948878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 460/934] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:02:36 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:36.952298 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23a716-53ec-4fdc-acff-5119d38fc32f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:36.992557 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:37.077699 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23a716-53ec-4fdc-acff-5119d38fc32f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:37.115806 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079f323-3f2f-4b64-a466-ad3b559e9826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:37.116929 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f323-3f2f-4b64-a466-ad3b559e9826 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:37.117407 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f323-3f2f-4b64-a466-ad3b559e9826 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:37.117868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f323-3f2f-4b64-a466-ad3b559e9826 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:37.125939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f323-3f2f-4b64-a466-ad3b559e9826 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:37.127876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f323-3f2f-4b64-a466-ad3b559e9826 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b079f323-3f2f-4b64-a466-ad3b559e9826 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:37.140606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:37.141515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:37.142393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:37.142393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:37.142393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:37.142393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:37.142393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:37.142393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:37.142393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:37.142393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:37.145104 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 475/935] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:37.151276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:37.171250 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:37.262577 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb22ac3-f901-4ff4-9de2-d2c5c71c7b58 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:37.301271 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e386ac-ec22-4d0b-8773-b81b44d0dcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:37.301994 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e386ac-ec22-4d0b-8773-b81b44d0dcdb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:37.302343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e386ac-ec22-4d0b-8773-b81b44d0dcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:37.302666 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e386ac-ec22-4d0b-8773-b81b44d0dcdb None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:37.303486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e386ac-ec22-4d0b-8773-b81b44d0dcdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:37.303847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e386ac-ec22-4d0b-8773-b81b44d0dcdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:37.305652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e386ac-ec22-4d0b-8773-b81b44d0dcdb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:37.328882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 461/936] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:37.754998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfccab-d9b2-47df-8062-c526166afa33 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:37.755371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfccab-d9b2-47df-8062-c526166afa33 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:37.755532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfccab-d9b2-47df-8062-c526166afa33 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:37.757242 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 476/937] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:37.763088 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270f67b-870b-4f8b-93c6-bd551c3dccb4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:37.763472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270f67b-870b-4f8b-93c6-bd551c3dccb4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:37.763661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270f67b-870b-4f8b-93c6-bd551c3dccb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:37.787391 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f270f67b-870b-4f8b-93c6-bd551c3dccb4 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:37.788949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f270f67b-870b-4f8b-93c6-bd551c3dccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:37.874378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 462/938] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:37.902949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfccab-d9b2-47df-8062-c526166afa33 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:38.008614 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:38.022646 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfccab-d9b2-47df-8062-c526166afa33 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:38.065416 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ab12b-fc6c-45a2-a14f-031285cea746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:38.066081 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ab12b-fc6c-45a2-a14f-031285cea746 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:38.066397 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ab12b-fc6c-45a2-a14f-031285cea746 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:38.066673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ab12b-fc6c-45a2-a14f-031285cea746 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:38.079018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ab12b-fc6c-45a2-a14f-031285cea746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-906ab12b-fc6c-45a2-a14f-031285cea746 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:38.084251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:38.085154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:38.085971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:38.090237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 477/939] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:02:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:38.101710 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270f67b-870b-4f8b-93c6-bd551c3dccb4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:38.105921 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:38.234548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270f67b-870b-4f8b-93c6-bd551c3dccb4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:38.275050 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25255033-1ce7-4786-8c2d-e3d2d9a2d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:38.276116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255033-1ce7-4786-8c2d-e3d2d9a2d826 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:38.276644 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255033-1ce7-4786-8c2d-e3d2d9a2d826 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:38.277133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255033-1ce7-4786-8c2d-e3d2d9a2d826 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:38.278071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25255033-1ce7-4786-8c2d-e3d2d9a2d826 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:38.278659 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25255033-1ce7-4786-8c2d-e3d2d9a2d826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:38.280316 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25255033-1ce7-4786-8c2d-e3d2d9a2d826 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:38.290806 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 463/940] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:02:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:38.299810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ab12b-fc6c-45a2-a14f-031285cea746 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:38.423483 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ab12b-fc6c-45a2-a14f-031285cea746 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:38.471132 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:38.471792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:38.472097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:38.472366 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:38.483954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:38.485500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:38.486006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6edac-c011-4aee-b53c-d09b892c9fa6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:38.486470 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6edac-c011-4aee-b53c-d09b892c9fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:38.486757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6edac-c011-4aee-b53c-d09b892c9fa6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:38.488327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 464/941] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:38.496674 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e6147-f62e-46b1-80c4-7f507f4b90b0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:38.497361 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e6147-f62e-46b1-80c4-7f507f4b90b0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:38.497832 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e6147-f62e-46b1-80c4-7f507f4b90b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:38.501591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:38.502809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:38.503915 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:38.503915 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:38.503915 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:38.503915 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:38.503915 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 478/942] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:38.511515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:38.515574 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:38.530054 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2e8e6147-f62e-46b1-80c4-7f507f4b90b0 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:38.531974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2e8e6147-f62e-46b1-80c4-7f507f4b90b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:38.622204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:38.635155 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 465/943] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:38.656559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a0a33-bfb7-4cd1-92f0-744e192b51e1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:38.662988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e6147-f62e-46b1-80c4-7f507f4b90b0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:38.720993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:38.721652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:38.721931 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:38.722237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:38.722982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:38.723283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:38.725431 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:38.747293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 479/944] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:38.749195 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:38.807161 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e6147-f62e-46b1-80c4-7f507f4b90b0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:38.817676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:38.853062 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:38.853855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:38.854092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:38.854455 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:38.855174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:38.856512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:38.863810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a009d5-e875-43cb-9e7e-c12911d06442 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:38.869667 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:38.870950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:38.871964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:38.872727 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 466/945] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:38.883387 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:38.912643 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:38.913725 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:38.914403 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:38.915004 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:38.925567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:38.927703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:38.951134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:38.952888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:38.954768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:38.965373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 480/946] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:02:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:38.966309 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:38.974474 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:39.007676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f291e57-a3b7-4883-972f-db2184d75a1a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:39.046007 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024b45e-0c84-400d-9147-661fc4930349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:39.047177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b45e-0c84-400d-9147-661fc4930349 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.047660 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b45e-0c84-400d-9147-661fc4930349 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.048149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b45e-0c84-400d-9147-661fc4930349 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.049111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024b45e-0c84-400d-9147-661fc4930349 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:39.049705 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024b45e-0c84-400d-9147-661fc4930349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:39.051534 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024b45e-0c84-400d-9147-661fc4930349 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:39.058270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 467/947] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:39.063010 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ff956-4bf4-44b5-b296-49811e577a6a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.063295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ff956-4bf4-44b5-b296-49811e577a6a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.063571 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ff956-4bf4-44b5-b296-49811e577a6a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.064980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 468/948] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:39.065675 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ff956-4bf4-44b5-b296-49811e577a6a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:39.101918 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c12a6-6018-4843-bfaf-7a6d7e988aab None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:39.130241 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:39.144002 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32322623-cad5-49f4-b250-5a0472629018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:39.144653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322623-cad5-49f4-b250-5a0472629018 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.144904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322623-cad5-49f4-b250-5a0472629018 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.145177 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322623-cad5-49f4-b250-5a0472629018 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.145976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32322623-cad5-49f4-b250-5a0472629018 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:39.146317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32322623-cad5-49f4-b250-5a0472629018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:39.147771 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32322623-cad5-49f4-b250-5a0472629018 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:39.168225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 481/949] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:02:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:39.168225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e722-b6a1-4cf0-b00c-c8be5d31e117 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.168225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e722-b6a1-4cf0-b00c-c8be5d31e117 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.168225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e722-b6a1-4cf0-b00c-c8be5d31e117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.189268 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4106e722-b6a1-4cf0-b00c-c8be5d31e117 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:39.190505 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4106e722-b6a1-4cf0-b00c-c8be5d31e117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:39.229374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ff956-4bf4-44b5-b296-49811e577a6a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:39.261056 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:39.274805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:39.275575 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.275966 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.276299 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.277157 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:39.277511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:39.279081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:39.282212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 482/950] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:39.283412 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106e722-b6a1-4cf0-b00c-c8be5d31e117 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:39.292876 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 469/951] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:39.299280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:39.408015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106e722-b6a1-4cf0-b00c-c8be5d31e117 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:39.424168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:39.448563 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d3813-95b0-4544-95ef-43479e6d3186 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:39.458556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:39.459222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.459500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.459769 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.472667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:39.474261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:39.491555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:39.492590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:39.493567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:39.493567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:39.493567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:39.493567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:39.493567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 483/952] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:39.493567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:39.507483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465844f6-e6b5-434a-a8aa-8b1f66a36fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:39.508136 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465844f6-e6b5-434a-a8aa-8b1f66a36fe8 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.508517 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465844f6-e6b5-434a-a8aa-8b1f66a36fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.508655 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465844f6-e6b5-434a-a8aa-8b1f66a36fe8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.509450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465844f6-e6b5-434a-a8aa-8b1f66a36fe8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:39.510781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465844f6-e6b5-434a-a8aa-8b1f66a36fe8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:39.546496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 470/953] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:02:39 2022] POST /identity/v3/endpoints => generated 349 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:39.547438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465844f6-e6b5-434a-a8aa-8b1f66a36fe8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:39.624372 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:39.655496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25574e8a-6c66-4146-b0be-7ea053904dea None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:39.706559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3323c-e529-4d4f-b37b-22c5633e6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:39.707186 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3323c-e529-4d4f-b37b-22c5633e6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.707471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3323c-e529-4d4f-b37b-22c5633e6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.708132 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3323c-e529-4d4f-b37b-22c5633e6fb8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.708465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3323c-e529-4d4f-b37b-22c5633e6fb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:39.709941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3323c-e529-4d4f-b37b-22c5633e6fb8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:39.717241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465844f6-e6b5-434a-a8aa-8b1f66a36fe8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-33e3323c-e529-4d4f-b37b-22c5633e6fb8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:39.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:39.723110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:39.725640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:39.729318 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 484/954] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:39.729318 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fe18a-e5ff-4246-a804-3206bd852c47 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.729318 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fe18a-e5ff-4246-a804-3206bd852c47 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.731038 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fe18a-e5ff-4246-a804-3206bd852c47 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.731344 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 485/955] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:39.731941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fe18a-e5ff-4246-a804-3206bd852c47 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:39.766470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:39.769255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502a80c-7c16-4383-a966-a29d53c65d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:39.769965 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502a80c-7c16-4383-a966-a29d53c65d17 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.770342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502a80c-7c16-4383-a966-a29d53c65d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.770663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502a80c-7c16-4383-a966-a29d53c65d17 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.771728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502a80c-7c16-4383-a966-a29d53c65d17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:39.771838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502a80c-7c16-4383-a966-a29d53c65d17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:39.773280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502a80c-7c16-4383-a966-a29d53c65d17 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:39.787440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 471/956] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:39.790808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e1b05-d221-465f-be57-76cd72d19aca None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.790808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e1b05-d221-465f-be57-76cd72d19aca None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.790808 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e1b05-d221-465f-be57-76cd72d19aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.811205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3f2e1b05-d221-465f-be57-76cd72d19aca None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:39.812524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3f2e1b05-d221-465f-be57-76cd72d19aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:39.870384 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fe18a-e5ff-4246-a804-3206bd852c47 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:39.898488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 472/957] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:39.900493 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e1b05-d221-465f-be57-76cd72d19aca None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:39.923212 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:39.924060 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:39.924405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:39.924829 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:39.925616 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:39.926042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:39.927630 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:39.934281 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 486/958] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:39.935251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:39.966946 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:40.058850 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:40.059561 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e1b05-d221-465f-be57-76cd72d19aca None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:40.080128 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7ebdb-4e68-4933-aa21-7d626946695b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:40.102859 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:40.104085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:40.104496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:40.104905 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:40.115136 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:40.117413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:40.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:40.126379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:40.126769 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:40.127107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:40.128127 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:40.130156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:40.131353 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:40.141979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:40.143652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 473/959] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:40.144406 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: WARNING keystone.server.flask.common [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 08:02:40.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:40.151275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.sql.core [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:02:40.151275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:02:40.151275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [parameters: {'user_id': '4efa195bcf694e6c8dc6595c697250c7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:02:40.151275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80369) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 28 08:02:40.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:02:40.163858 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application yield resource Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise self.value Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application session.commit() Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:02:40.168783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:02:40.171920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:02:40.175480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:02:40.178998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [parameters: {'user_id': '4efa195bcf694e6c8dc6595c697250c7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:40.182371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 08:02:40.184452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 08:02:40.186477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 08:02:40.186477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 08:02:40.186477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:02:40.186477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:02:40.186477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:02:40.186477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:40.186477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 487/960] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:02:39 2022] POST /identity/v3/users => generated 160 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:40.186477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:40.287555 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:40.297536 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0790f-c3d8-4428-b0a6-a1810e815972 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:40.306770 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e796e7f-1197-4362-bd70-1906b857ec0a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:40.346117 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:40.347144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:40.347526 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:40.347957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:40.348873 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:40.349428 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:40.351195 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:40.351677 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:40.351855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:40.352113 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:40.352385 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:40.353153 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:40.353462 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:40.355182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:40.358939 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 474/961] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:40.360824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:40.382639 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 488/962] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:40.384558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:40.402263 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:40.544685 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f20886-8db1-4f64-997e-70bcea15cfcd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:40.576345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11571a9-e3d5-4a35-9371-e1811ef0a3e2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:40.598995 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:40.599670 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:40.599943 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:40.600211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:40.610735 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:40.612287 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:40.624471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7a8d8-833f-4785-9816-a56ba4d8014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:40.625380 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a8d8-833f-4785-9816-a56ba4d8014e None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:40.625750 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a8d8-833f-4785-9816-a56ba4d8014e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:40.626188 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a8d8-833f-4785-9816-a56ba4d8014e None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/76e8ae1ebdbb4e34bb4b4ef59e70305c/roles/d8710b1ea79f4ed78a1c4d54ee6e9148` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:40.628204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:40.629159 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:02:40.630097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:40.630901 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7a8d8-833f-4785-9816-a56ba4d8014e None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=76e8ae1ebdbb4e34bb4b4ef59e70305c, role_id=d8710b1ea79f4ed78a1c4d54ee6e9148)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:40.630991 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 475/963] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:40.635299 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7a8d8-833f-4785-9816-a56ba4d8014e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:40.636650 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:40.667284 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:40.675967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 489/964] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:02:40 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/76e8ae1ebdbb4e34bb4b4ef59e70305c/roles/d8710b1ea79f4ed78a1c4d54ee6e9148 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:02:40.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be750c65-4c0d-4d31-8dad-3d4c56612c0a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:40.856683 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725e976-f52c-4bad-a9b8-19e6520bf5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:40.857657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725e976-f52c-4bad-a9b8-19e6520bf5bb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:40.858208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725e976-f52c-4bad-a9b8-19e6520bf5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:40.858687 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725e976-f52c-4bad-a9b8-19e6520bf5bb None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:40.859613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725e976-f52c-4bad-a9b8-19e6520bf5bb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:40.860029 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725e976-f52c-4bad-a9b8-19e6520bf5bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:40.861824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725e976-f52c-4bad-a9b8-19e6520bf5bb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:40.911584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 476/965] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:02:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:41.671692 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aaf7b-7ef4-46e3-a5b7-c94ac172c712 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:41.672513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aaf7b-7ef4-46e3-a5b7-c94ac172c712 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:41.672513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aaf7b-7ef4-46e3-a5b7-c94ac172c712 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:41.673770 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 490/966] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:41.679507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c47b6-6433-4a4b-bb34-f2cb30ef638e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:41.680126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c47b6-6433-4a4b-bb34-f2cb30ef638e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:41.680126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c47b6-6433-4a4b-bb34-f2cb30ef638e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:41.699034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1a3c47b6-6433-4a4b-bb34-f2cb30ef638e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:41.700608 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1a3c47b6-6433-4a4b-bb34-f2cb30ef638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:41.811599 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 477/967] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:41.835417 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566aaf7b-7ef4-46e3-a5b7-c94ac172c712 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:41.937922 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:41.999602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566aaf7b-7ef4-46e3-a5b7-c94ac172c712 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:42.048313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:42.049033 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.049310 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.049583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.050261 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:42.051647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:42.061461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:42.062985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:42.064019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:42.066485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 491/968] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:42.069088 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c47b6-6433-4a4b-bb34-f2cb30ef638e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:42.081818 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:42.206997 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c47b6-6433-4a4b-bb34-f2cb30ef638e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:42.257745 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:42.258548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.258951 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.259362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.260604 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:42.261146 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:42.263749 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:42.278123 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 478/969] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:42.279381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:42.402097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7208027-259d-4d64-bd77-64b319d5073a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:42.434338 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:42.440665 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc3163-161d-4867-9960-c27af450deaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:42.441313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc3163-161d-4867-9960-c27af450deaa None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.441798 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc3163-161d-4867-9960-c27af450deaa None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.442143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc3163-161d-4867-9960-c27af450deaa None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.442986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc3163-161d-4867-9960-c27af450deaa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:42.443216 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc3163-161d-4867-9960-c27af450deaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:42.444833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc3163-161d-4867-9960-c27af450deaa None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:42.451465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 492/970] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:42.457457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:42.535260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:42.571022 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128c05e-3151-493c-b95a-22e8020dd2df None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:42.608816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:42.609693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.610045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.610365 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.611365 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] RBAC: Adding query filter params (interface=admin, service_id=813af9f261974435bdfe183c96aaa4e3, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:42.611676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:42.613138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:42.614509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dfd14-7b95-463d-9e46-bd7ec8d2cdbc None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.614800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dfd14-7b95-463d-9e46-bd7ec8d2cdbc None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.615077 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dfd14-7b95-463d-9e46-bd7ec8d2cdbc None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.616673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 493/971] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:42.626137 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 479/972] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jul 28 08:02:42 2022] GET /identity/v3/endpoints?service_id=813af9f261974435bdfe183c96aaa4e3&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:42.626345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af177-8794-4dc7-ab51-26ba09a2b58d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.626345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af177-8794-4dc7-ab51-26ba09a2b58d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.626345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af177-8794-4dc7-ab51-26ba09a2b58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.627960 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:42.649963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b11af177-8794-4dc7-ab51-26ba09a2b58d None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:42.652681 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b11af177-8794-4dc7-ab51-26ba09a2b58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:42.765959 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 494/973] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:42.772813 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d15cb4-fd73-40f1-82af-1ebd3f8a5df8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:42.788308 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11af177-8794-4dc7-ab51-26ba09a2b58d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:42.815598 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc769e-56b8-413a-b698-4fd74b00ab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:42.816234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc769e-56b8-413a-b698-4fd74b00ab9c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.816540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc769e-56b8-413a-b698-4fd74b00ab9c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.816778 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc769e-56b8-413a-b698-4fd74b00ab9c None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.817442 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc769e-56b8-413a-b698-4fd74b00ab9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:42.818995 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc769e-56b8-413a-b698-4fd74b00ab9c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:42.820406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:42.829727 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 480/974] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:02:42 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:02:42.905748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014f70-7b0d-4860-8d17-f5302168b7fe None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.906169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014f70-7b0d-4860-8d17-f5302168b7fe None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.906309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014f70-7b0d-4860-8d17-f5302168b7fe None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.910867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 481/975] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:42.914258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea9f26-709a-4fb6-ab6d-ada7f527d7ba None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:42.914813 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea9f26-709a-4fb6-ab6d-ada7f527d7ba None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:42.915183 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea9f26-709a-4fb6-ab6d-ada7f527d7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:42.956382 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4dea9f26-709a-4fb6-ab6d-ada7f527d7ba None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:42.959028 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4dea9f26-709a-4fb6-ab6d-ada7f527d7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:42.974638 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11af177-8794-4dc7-ab51-26ba09a2b58d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:43.009473 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:43.023693 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdd445-d481-47bd-8c2b-33b07e5dcbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:43.024666 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdd445-d481-47bd-8c2b-33b07e5dcbae None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:43.025123 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdd445-d481-47bd-8c2b-33b07e5dcbae None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:43.025529 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdd445-d481-47bd-8c2b-33b07e5dcbae None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:43.026693 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdd445-d481-47bd-8c2b-33b07e5dcbae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:43.028634 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdd445-d481-47bd-8c2b-33b07e5dcbae None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:43.047158 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 495/976] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:02:42 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:02:43.087417 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 482/977] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:43.115769 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdd445-d481-47bd-8c2b-33b07e5dcbae None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:43.279731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdd445-d481-47bd-8c2b-33b07e5dcbae None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:43.334288 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e315d-d513-4c03-af2b-996fb80255c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:43.334959 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e315d-d513-4c03-af2b-996fb80255c8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:43.335238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e315d-d513-4c03-af2b-996fb80255c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:43.335512 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e315d-d513-4c03-af2b-996fb80255c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:43.349305 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e315d-d513-4c03-af2b-996fb80255c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-728e315d-d513-4c03-af2b-996fb80255c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:43.356031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:43.357041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:43.358489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 496/978] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:02:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:43.369506 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea9f26-709a-4fb6-ab6d-ada7f527d7ba None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:43.521992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea9f26-709a-4fb6-ab6d-ada7f527d7ba None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:43.556022 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:43.575487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:43.576169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:43.578549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:43.578549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:43.578549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:43.578982 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:43.580613 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:43.593184 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 483/979] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:02:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:43.599935 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e315d-d513-4c03-af2b-996fb80255c8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:43.664327 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:43.737460 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e315d-d513-4c03-af2b-996fb80255c8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:43.785818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:43.786939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:43.787371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:43.787802 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:43.806316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:43.808708 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:43.833876 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:43.834881 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:43.835778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:43.835778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:43.835778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:43.835778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:43.838515 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 497/980] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:43.843943 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:43.981500 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccf410-5f07-450d-8682-3adcd1ee5ee6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:44.021827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:44.022522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:44.022805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:44.023069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:44.023811 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:44.024116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:44.025572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:44.044207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 484/981] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:44.045604 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:44.161420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:44.169083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ad8f6-2f10-4305-83eb-544dafa44ee5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:44.207975 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:44.208652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:44.208934 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:44.209202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:44.218175 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:44.219720 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:44.233863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:44.235653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:02:44.237758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:44.240525 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 498/982] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:02:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:44.242340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:44.303823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:44.369080 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafd246-2b35-4582-a705-0ba758ac00ff None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:44.416231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:44.417235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:44.417707 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:44.418241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:44.419294 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:44.419819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:44.421593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:44.443298 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 485/983] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:02:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:44.448686 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:44.571627 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7af0b-5895-49f4-81e4-de9bb9fcbfed None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:44.606047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad13ac-2a8e-4982-95b3-d6dd941576fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:44.606715 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad13ac-2a8e-4982-95b3-d6dd941576fd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:44.606992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad13ac-2a8e-4982-95b3-d6dd941576fd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:44.607273 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad13ac-2a8e-4982-95b3-d6dd941576fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:44.635451 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:44.679370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad13ac-2a8e-4982-95b3-d6dd941576fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:44.680726 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad13ac-2a8e-4982-95b3-d6dd941576fd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:44.798145 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 499/984] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:44 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:02:44.804014 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:44.919816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaac2d8-1824-4806-92aa-d07162947f36 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:44.923473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:44.928846 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f326262-331c-4f05-910c-cead603f542c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:44.929494 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f326262-331c-4f05-910c-cead603f542c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:44.929961 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f326262-331c-4f05-910c-cead603f542c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:44.933488 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 500/985] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:44.938367 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b762f-4d7f-4a5e-bee3-19bb6f43e849 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:44.939420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b762f-4d7f-4a5e-bee3-19bb6f43e849 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:44.940179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b762f-4d7f-4a5e-bee3-19bb6f43e849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:44.965618 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-021b762f-4d7f-4a5e-bee3-19bb6f43e849 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:44.966918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:44.967471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-021b762f-4d7f-4a5e-bee3-19bb6f43e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:44.967601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:44.967846 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:44.968114 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:44.979225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:44.980690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:45.001205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:45.002521 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:45.003371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:45.003371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:45.003371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:45.003371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:45.003371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 486/986] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:45.008292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.038689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:45.058519 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 501/987] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:45.077184 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b762f-4d7f-4a5e-bee3-19bb6f43e849 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.120130 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab2a0-77e0-4aa3-8ae7-7d50488b863a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.155113 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a700e-8c98-416d-a01e-af77a9694b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.155804 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a700e-8c98-416d-a01e-af77a9694b87 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.156086 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a700e-8c98-416d-a01e-af77a9694b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.156360 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a700e-8c98-416d-a01e-af77a9694b87 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.157084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a700e-8c98-416d-a01e-af77a9694b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:45.157373 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a700e-8c98-416d-a01e-af77a9694b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.158839 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a700e-8c98-416d-a01e-af77a9694b87 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.171152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 487/988] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:45.172424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b238dd6-2e18-4ce4-acac-c93275a9b0cf None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.172713 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b238dd6-2e18-4ce4-acac-c93275a9b0cf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.172972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b238dd6-2e18-4ce4-acac-c93275a9b0cf None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.174292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 488/989] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:45.177108 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b238dd6-2e18-4ce4-acac-c93275a9b0cf None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.182393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b762f-4d7f-4a5e-bee3-19bb6f43e849 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.195514 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:45.218398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b29778-1029-418a-affa-0a2adf121c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.219071 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b29778-1029-418a-affa-0a2adf121c82 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.219316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b29778-1029-418a-affa-0a2adf121c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.219587 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b29778-1029-418a-affa-0a2adf121c82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.220222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b29778-1029-418a-affa-0a2adf121c82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.221742 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b29778-1029-418a-affa-0a2adf121c82 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-01b29778-1029-418a-affa-0a2adf121c82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:45.231240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:45.232333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:45.233170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 502/990] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:45.233937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507260-008c-431a-b88b-e85212c8bbc6 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.233937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507260-008c-431a-b88b-e85212c8bbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.234187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507260-008c-431a-b88b-e85212c8bbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.252566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0d507260-008c-431a-b88b-e85212c8bbc6 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:45.254036 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0d507260-008c-431a-b88b-e85212c8bbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:45.288675 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b238dd6-2e18-4ce4-acac-c93275a9b0cf None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.303456 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:45.323694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 503/991] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:45.324947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d507260-008c-431a-b88b-e85212c8bbc6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.326561 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.327222 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.327510 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.327781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.342068 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.343559 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:45.355520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:45.356580 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:45.357505 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 489/992] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:45.357996 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.436177 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d507260-008c-431a-b88b-e85212c8bbc6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.437706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:45.461926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445fd68-e40e-45a1-8fe1-b6bdef127b66 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.473231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.473871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.474179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.474453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:45.475509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.476841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.481986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 504/993] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:45.481986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.497831 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.498527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.498809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.499085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.499717 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.501103 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:45.511138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:45.512226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:45.513016 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 490/994] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:45.513943 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.544819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:45.601528 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f75e8-61ed-4ffa-8085-1a68730c2b70 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.615693 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e1ceb-3459-4446-af4d-1f7800d0b306 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.635621 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.636256 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.636538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.636815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.637558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:45.637863 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.639347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.650977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.651610 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.651844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 505/995] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:45.651939 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.652203 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.652669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.652908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:45.653195 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.654742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.660315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 491/996] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:45.661170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.665476 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:45.773177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32775c2e-58e8-43ca-ac41-74e83a95a2f6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.775569 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c3750-de34-4006-9d88-7517e4b0a7e7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.783674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:45.808620 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.809253 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.809538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.809814 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.810642 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] RBAC: Adding query filter params (role.id=d8710b1ea79f4ed78a1c4d54ee6e9148, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=76e8ae1ebdbb4e34bb4b4ef59e70305c) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:45.810955 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.812403 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.812517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:45.813068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:45.813349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:45.813620 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:45.814372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:45.814676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:45.816038 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:45.826014 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 492/997] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/role_assignments?user.id=76e8ae1ebdbb4e34bb4b4ef59e70305c&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=d8710b1ea79f4ed78a1c4d54ee6e9148 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:45.826014 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.826325 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 506/998] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:45.827587 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:45.919003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:45.970150 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8fcd7-1db4-4552-b634-90c2afe7bb49 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:45.985594 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414694a-a5b0-4636-bc2f-51edb8203d1c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:46.015028 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3698d5f-4a67-47cb-848e-d7899af5ace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:46.015671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3698d5f-4a67-47cb-848e-d7899af5ace0 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:46.015949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3698d5f-4a67-47cb-848e-d7899af5ace0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:46.016220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3698d5f-4a67-47cb-848e-d7899af5ace0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:46.017024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3698d5f-4a67-47cb-848e-d7899af5ace0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:46.018500 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3698d5f-4a67-47cb-848e-d7899af5ace0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:46.030628 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:46.031274 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:46.031554 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:46.031853 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:46.033090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:46.033312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:46.034968 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:46.037642 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 493/999] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:02:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:46.042366 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 507/1000] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:46.048805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3698d5f-4a67-47cb-848e-d7899af5ace0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:46.185429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3698d5f-4a67-47cb-848e-d7899af5ace0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:46.220187 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:46.240581 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495195a6-4b6a-4bc0-b728-91a166e07fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:46.241331 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495195a6-4b6a-4bc0-b728-91a166e07fde None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:46.241824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495195a6-4b6a-4bc0-b728-91a166e07fde None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:46.242351 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495195a6-4b6a-4bc0-b728-91a166e07fde None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:46.243802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495195a6-4b6a-4bc0-b728-91a166e07fde None admin] RBAC: Adding query filter params (interface=public, service_id=1272fdebb1a54650880041b62d6b9cfb, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:46.244344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495195a6-4b6a-4bc0-b728-91a166e07fde None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:46.246932 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495195a6-4b6a-4bc0-b728-91a166e07fde None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:46.256809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 494/1001] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:02:46 2022] GET /identity/v3/endpoints?service_id=1272fdebb1a54650880041b62d6b9cfb&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:46.262664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:46.318062 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:46.406833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b28f11-b743-4b7c-964f-10bd8e6ef009 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:46.454256 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d5864-7346-405b-a692-e8f177ba80ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:46.455444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5864-7346-405b-a692-e8f177ba80ac None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:46.455939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5864-7346-405b-a692-e8f177ba80ac None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:46.456421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5864-7346-405b-a692-e8f177ba80ac None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:46.457604 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d5864-7346-405b-a692-e8f177ba80ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:46.460139 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d5864-7346-405b-a692-e8f177ba80ac None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:46.472145 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 508/1002] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:02:46 2022] GET /identity/v3/services => generated 1727 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:02:48.004231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b33ff-47d8-4128-8385-4a76278c01f8 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.004771 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b33ff-47d8-4128-8385-4a76278c01f8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.004771 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b33ff-47d8-4128-8385-4a76278c01f8 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 495/1003] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:48.011734 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5879-4d6d-4928-b126-e338a6af090b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.012195 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5879-4d6d-4928-b126-e338a6af090b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.012275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5879-4d6d-4928-b126-e338a6af090b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.031680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5c3d5879-4d6d-4928-b126-e338a6af090b None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:48.032887 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5c3d5879-4d6d-4928-b126-e338a6af090b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:48.102874 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b023d5-d486-4151-b36d-b725ebcf8030 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.103247 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b023d5-d486-4151-b36d-b725ebcf8030 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.103412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b023d5-d486-4151-b36d-b725ebcf8030 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.104999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 496/1004] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:48.106448 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 509/1005] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:48 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:48.109860 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8a4dd-34a6-4f37-8f59-ea3285c3607f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.110151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8a4dd-34a6-4f37-8f59-ea3285c3607f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.110446 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8a4dd-34a6-4f37-8f59-ea3285c3607f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.127956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d5879-4d6d-4928-b126-e338a6af090b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:48.128840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-04a8a4dd-34a6-4f37-8f59-ea3285c3607f None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:48.130049 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-04a8a4dd-34a6-4f37-8f59-ea3285c3607f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:48.167641 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:48.202812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 497/1006] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:48 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:48.225590 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8a4dd-34a6-4f37-8f59-ea3285c3607f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:48.255583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d5879-4d6d-4928-b126-e338a6af090b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:48.292748 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:48.293397 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.293677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.293965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] PATH_INFO: `/v3/services/image` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.294615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:48.296064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:48.305159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:48.306199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:48.307034 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 510/1007] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:02:48 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:48.312114 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:48.338789 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8a4dd-34a6-4f37-8f59-ea3285c3607f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:48.358454 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:48.378878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee5faf-105a-4add-8dfc-baff6b0734b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:48.379597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee5faf-105a-4add-8dfc-baff6b0734b1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.379814 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee5faf-105a-4add-8dfc-baff6b0734b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.380090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee5faf-105a-4add-8dfc-baff6b0734b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.381245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee5faf-105a-4add-8dfc-baff6b0734b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:48.382723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee5faf-105a-4add-8dfc-baff6b0734b1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:48.386149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 498/1008] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:48.390019 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee5faf-105a-4add-8dfc-baff6b0734b1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:48.411182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:48.420900 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d7ad6-a5e9-43c1-86ed-d58c3cb2f954 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:48.458738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:48.459373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.459654 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.459926 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.460620 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] RBAC: Adding query filter params (name=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:48.460927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:48.462343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:48.467377 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 511/1009] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:48.473014 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:48.512856 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee5faf-105a-4add-8dfc-baff6b0734b1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:48.549678 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd50453-168e-4dc9-bcb6-a80c4474a859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:48.550343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50453-168e-4dc9-bcb6-a80c4474a859 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.550640 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50453-168e-4dc9-bcb6-a80c4474a859 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.551031 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50453-168e-4dc9-bcb6-a80c4474a859 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.552098 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd50453-168e-4dc9-bcb6-a80c4474a859 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:48.553174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd50453-168e-4dc9-bcb6-a80c4474a859 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:48.585777 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 499/1010] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:02:48 2022] POST /identity/v3/projects => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:48.588704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7efb6-e7dc-455c-aa87-641e17f9469e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:48.630063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29a800-6cca-4da4-acf7-4b4ea1921b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:48.631113 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a800-6cca-4da4-acf7-4b4ea1921b12 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.631607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a800-6cca-4da4-acf7-4b4ea1921b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.632078 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a800-6cca-4da4-acf7-4b4ea1921b12 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.632976 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29a800-6cca-4da4-acf7-4b4ea1921b12 None admin] RBAC: Adding query filter params (type=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:48.633593 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29a800-6cca-4da4-acf7-4b4ea1921b12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:48.635258 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29a800-6cca-4da4-acf7-4b4ea1921b12 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:48.641774 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 512/1011] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:48.658511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83546d-ee2b-4a06-bdb7-b32cc62544e7 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.658809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83546d-ee2b-4a06-bdb7-b32cc62544e7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.659231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83546d-ee2b-4a06-bdb7-b32cc62544e7 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.660941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 500/1012] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:48.667574 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0fff4-7cb9-4751-81d5-397e7d2553f3 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.667902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0fff4-7cb9-4751-81d5-397e7d2553f3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.668170 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0fff4-7cb9-4751-81d5-397e7d2553f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.689275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-78e0fff4-7cb9-4751-81d5-397e7d2553f3 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:48.690689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-78e0fff4-7cb9-4751-81d5-397e7d2553f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:48.714355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:48.790283 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 513/1013] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:48 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:48.822440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83546d-ee2b-4a06-bdb7-b32cc62544e7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:48.870991 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:48.945322 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83546d-ee2b-4a06-bdb7-b32cc62544e7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:48.981641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:48.982325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:48.982605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:48.982875 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:48.983519 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:48.984902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:48.994548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:48.995538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:49.001362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 501/1014] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:49.004015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0fff4-7cb9-4751-81d5-397e7d2553f3 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:49.123692 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0fff4-7cb9-4751-81d5-397e7d2553f3 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:49.161090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:49.161739 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:49.162022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:49.162326 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:49.163028 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:49.163334 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:49.164742 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:49.176094 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 514/1015] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:49.177347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:49.293232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6519856-f830-45bf-8324-9b7a4a915150 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:49.294900 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:49.330403 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3804eaf-f8c9-48d6-a66b-2fce5e0adf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:49.331037 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3804eaf-f8c9-48d6-a66b-2fce5e0adf18 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:49.331314 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3804eaf-f8c9-48d6-a66b-2fce5e0adf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:49.331581 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3804eaf-f8c9-48d6-a66b-2fce5e0adf18 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:49.332260 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3804eaf-f8c9-48d6-a66b-2fce5e0adf18 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:49.332565 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3804eaf-f8c9-48d6-a66b-2fce5e0adf18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:49.333931 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3804eaf-f8c9-48d6-a66b-2fce5e0adf18 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:49.339484 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 502/1016] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:02:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:49.345166 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:49.362207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:49.450390 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f9cb-3c6d-4331-beec-565c0cee38fc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:49.490295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917c75f-d1a6-41d1-bbac-5b0d6ec8ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:49.490819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917c75f-d1a6-41d1-bbac-5b0d6ec8ae92 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:49.491103 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917c75f-d1a6-41d1-bbac-5b0d6ec8ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:49.491376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917c75f-d1a6-41d1-bbac-5b0d6ec8ae92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:49.492112 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917c75f-d1a6-41d1-bbac-5b0d6ec8ae92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:49.493498 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917c75f-d1a6-41d1-bbac-5b0d6ec8ae92 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:49.519942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 515/1017] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:02:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:50.676437 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602047-51d4-4f32-9990-18bdbf324faf None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:50.676971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602047-51d4-4f32-9990-18bdbf324faf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:50.676971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602047-51d4-4f32-9990-18bdbf324faf None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:50.678486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 503/1018] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:50.679857 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b6c83-f8a4-4040-8487-3922c8f94031 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:50.680266 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b6c83-f8a4-4040-8487-3922c8f94031 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:50.680432 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b6c83-f8a4-4040-8487-3922c8f94031 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:50.681749 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 516/1019] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:02:50.683573 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba14a7-4e9e-4301-8b1f-a41c9970ebec None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:50.683818 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba14a7-4e9e-4301-8b1f-a41c9970ebec None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:50.684125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba14a7-4e9e-4301-8b1f-a41c9970ebec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:50.687492 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc370-6a72-4127-8cb7-b3170a4aa5e5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:50.687753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc370-6a72-4127-8cb7-b3170a4aa5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:50.688045 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc370-6a72-4127-8cb7-b3170a4aa5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:50.705306 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-efba14a7-4e9e-4301-8b1f-a41c9970ebec None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:50.709358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-efba14a7-4e9e-4301-8b1f-a41c9970ebec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:50.709531 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ed2bc370-6a72-4127-8cb7-b3170a4aa5e5 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:50.709907 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ed2bc370-6a72-4127-8cb7-b3170a4aa5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:50.784310 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:50.794058 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 504/1020] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:50.801295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 517/1021] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:50.818332 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba14a7-4e9e-4301-8b1f-a41c9970ebec None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:50.823639 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bc370-6a72-4127-8cb7-b3170a4aa5e5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:50.923445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:50.934384 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba14a7-4e9e-4301-8b1f-a41c9970ebec None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:50.941372 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bc370-6a72-4127-8cb7-b3170a4aa5e5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:50.974736 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6703a5-3fa2-4e43-b0b2-9abe36684af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:50.975790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6703a5-3fa2-4e43-b0b2-9abe36684af3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:50.976255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6703a5-3fa2-4e43-b0b2-9abe36684af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:50.976730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6703a5-3fa2-4e43-b0b2-9abe36684af3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:50.978728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6703a5-3fa2-4e43-b0b2-9abe36684af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:50.981255 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6703a5-3fa2-4e43-b0b2-9abe36684af3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:50.985570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f2e8-24ca-424d-aed0-8f575119ee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:50.986161 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 505/1022] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:50.986463 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f2e8-24ca-424d-aed0-8f575119ee6b None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:50.986746 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f2e8-24ca-424d-aed0-8f575119ee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:50.987018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f2e8-24ca-424d-aed0-8f575119ee6b None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:50.987717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780f2e8-24ca-424d-aed0-8f575119ee6b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:50.989099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780f2e8-24ca-424d-aed0-8f575119ee6b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:50.993061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6703a5-3fa2-4e43-b0b2-9abe36684af3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:51.012065 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 518/1023] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:02:50 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:02:51.116877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6703a5-3fa2-4e43-b0b2-9abe36684af3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:51.168523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9431c64-9d7f-4aba-b34b-82246145891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:51.169204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9431c64-9d7f-4aba-b34b-82246145891e None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:51.169485 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9431c64-9d7f-4aba-b34b-82246145891e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:51.169757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9431c64-9d7f-4aba-b34b-82246145891e None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:51.170608 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9431c64-9d7f-4aba-b34b-82246145891e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:51.172327 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9431c64-9d7f-4aba-b34b-82246145891e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:51.197904 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 506/1024] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:02:50 2022] POST /identity/v3/users => generated 285 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:02:51.661401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a87029-98c7-48af-8608-aca99d48c56c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:51.661819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a87029-98c7-48af-8608-aca99d48c56c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:51.661932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a87029-98c7-48af-8608-aca99d48c56c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:51.663548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 519/1025] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:51.669033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e95076-7f5d-4726-baf5-4eeb2d9d0a5c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:51.669301 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e95076-7f5d-4726-baf5-4eeb2d9d0a5c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:51.669606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e95076-7f5d-4726-baf5-4eeb2d9d0a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:51.688928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-00e95076-7f5d-4726-baf5-4eeb2d9d0a5c None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:51.690236 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-00e95076-7f5d-4726-baf5-4eeb2d9d0a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:51.699282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:51.777364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 507/1026] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:51.800146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a87029-98c7-48af-8608-aca99d48c56c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:51.865011 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:51.925716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a87029-98c7-48af-8608-aca99d48c56c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:51.965964 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc9143-5d8d-4a38-8fb4-0a677fb52207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:51.966679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9143-5d8d-4a38-8fb4-0a677fb52207 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:51.966967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9143-5d8d-4a38-8fb4-0a677fb52207 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:51.967242 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9143-5d8d-4a38-8fb4-0a677fb52207 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:51.979855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc9143-5d8d-4a38-8fb4-0a677fb52207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6bfc9143-5d8d-4a38-8fb4-0a677fb52207 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:51.984910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:51.985855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:51.987231 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:51.990499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 520/1027] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:02:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:51.992534 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e95076-7f5d-4726-baf5-4eeb2d9d0a5c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:52.101185 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e95076-7f5d-4726-baf5-4eeb2d9d0a5c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:52.137525 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:52.138189 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:52.138464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:52.138731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:52.139473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:52.139772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:52.141191 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:52.151245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 508/1028] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:02:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:52.157094 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc9143-5d8d-4a38-8fb4-0a677fb52207 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:52.262637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc9143-5d8d-4a38-8fb4-0a677fb52207 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:52.280670 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:52.301232 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:52.302181 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:52.302605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:52.303018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:52.314099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:52.315738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:52.331956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:52.332967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:52.333868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:52.333868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:52.333868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:52.333868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:52.338421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 521/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:52.339906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:52.344082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:52.448083 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e127ba0-0741-41cf-9ddc-f13edd0673e0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:52.488822 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:52.489450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:52.489731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:52.489999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:52.490723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:52.491028 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:52.492397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:52.504833 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 509/1030] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:52.510461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:52.618460 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b476921-0a82-418a-9d54-74cc878b8985 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:52.653649 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:52.654394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:52.654727 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:52.654999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:52.663214 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:52.664756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:52.676360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:52.677303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:52.678647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:52.682202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 522/1031] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:02:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:52.683413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:52.757692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:52.797135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485abf1f-12e0-4ad5-86c7-fd872f431ad9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:52.832502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8861cb4-7a8c-499d-8dd4-2359324fece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:52.833133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8861cb4-7a8c-499d-8dd4-2359324fece9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:52.833414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8861cb4-7a8c-499d-8dd4-2359324fece9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:52.833684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8861cb4-7a8c-499d-8dd4-2359324fece9 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:52.834528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8861cb4-7a8c-499d-8dd4-2359324fece9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:52.834836 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8861cb4-7a8c-499d-8dd4-2359324fece9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:52.836257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8861cb4-7a8c-499d-8dd4-2359324fece9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:52.858657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 510/1032] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:02:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:52.860123 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:52.892314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:52.969792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e652ba2-c60c-4e36-bd7d-7b962f61b025 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.004509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2263d31-6ee3-466a-bc43-2742ab260d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.005154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2263d31-6ee3-466a-bc43-2742ab260d79 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.005416 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2263d31-6ee3-466a-bc43-2742ab260d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.005687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2263d31-6ee3-466a-bc43-2742ab260d79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.049425 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6d01-54df-4533-ac52-9a12b810fc68 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.049714 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6d01-54df-4533-ac52-9a12b810fc68 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.049986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6d01-54df-4533-ac52-9a12b810fc68 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.051411 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 511/1033] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:53.056789 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0782502-99f6-463d-9cdc-c23410a9d46d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.057128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0782502-99f6-463d-9cdc-c23410a9d46d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.057349 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0782502-99f6-463d-9cdc-c23410a9d46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.075811 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d0782502-99f6-463d-9cdc-c23410a9d46d None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:53.077018 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d0782502-99f6-463d-9cdc-c23410a9d46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:53.084851 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2263d31-6ee3-466a-bc43-2742ab260d79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.086990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2263d31-6ee3-466a-bc43-2742ab260d79 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:53.117386 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:53.159418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 512/1034] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:02:53.171791 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0782502-99f6-463d-9cdc-c23410a9d46d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.201284 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 523/1035] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:52 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:02:53.207094 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2263d31-6ee3-466a-bc43-2742ab260d79 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.230809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:53.279219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0782502-99f6-463d-9cdc-c23410a9d46d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.311053 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2263d31-6ee3-466a-bc43-2742ab260d79 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.313406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55926f61-2bff-41bf-9dcb-742413c943e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.314091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926f61-2bff-41bf-9dcb-742413c943e6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.314399 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926f61-2bff-41bf-9dcb-742413c943e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.314668 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926f61-2bff-41bf-9dcb-742413c943e6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.315293 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926f61-2bff-41bf-9dcb-742413c943e6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.316658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926f61-2bff-41bf-9dcb-742413c943e6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-55926f61-2bff-41bf-9dcb-742413c943e6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:53.325716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:53.332152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:53.333027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 513/1036] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:53.333792 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e3e0-860d-40cf-9384-7f14d67fdd0e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.333792 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e3e0-860d-40cf-9384-7f14d67fdd0e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.333792 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e3e0-860d-40cf-9384-7f14d67fdd0e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.333792 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 514/1037] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:53.333792 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e3e0-860d-40cf-9384-7f14d67fdd0e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:53.354167 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16382fe0-6436-4f91-bc7b-ca3ec25ccabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.354167 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16382fe0-6436-4f91-bc7b-ca3ec25ccabc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.354167 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16382fe0-6436-4f91-bc7b-ca3ec25ccabc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.354167 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16382fe0-6436-4f91-bc7b-ca3ec25ccabc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.363787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16382fe0-6436-4f91-bc7b-ca3ec25ccabc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.365209 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16382fe0-6436-4f91-bc7b-ca3ec25ccabc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-16382fe0-6436-4f91-bc7b-ca3ec25ccabc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:53.380230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:53.381246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:53.382165 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:53.382165 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:53.382165 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:53.382165 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:53.382165 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 524/1038] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:53.382728 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3da3d-65eb-45e1-a5b8-77b023213615 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.382886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3da3d-65eb-45e1-a5b8-77b023213615 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.383155 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3da3d-65eb-45e1-a5b8-77b023213615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.404851 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-61c3da3d-65eb-45e1-a5b8-77b023213615 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:53.406234 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-61c3da3d-65eb-45e1-a5b8-77b023213615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:53.452663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e3e0-860d-40cf-9384-7f14d67fdd0e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.470038 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:53.483078 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 525/1039] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:53.487038 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3da3d-65eb-45e1-a5b8-77b023213615 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.496329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.496883 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.497165 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.497440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.498286 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] RBAC: Adding query filter params (name=image) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:53.498603 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.500120 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:53.505908 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 515/1040] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:53.506878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.607343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3da3d-65eb-45e1-a5b8-77b023213615 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.620360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:53.622441 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feaf3d-5679-4da8-ac8b-0c4ab0aad72b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.649696 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.650457 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.650739 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.651011 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.651717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:53.652027 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.653400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:53.657101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1d3e8-bbb6-4900-a274-ba3d03bab4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.657814 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d3e8-bbb6-4900-a274-ba3d03bab4de None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.658305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d3e8-bbb6-4900-a274-ba3d03bab4de None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.658428 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d3e8-bbb6-4900-a274-ba3d03bab4de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.671835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 526/1041] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:53.671835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.672231 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1d3e8-bbb6-4900-a274-ba3d03bab4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-48f1d3e8-bbb6-4900-a274-ba3d03bab4de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:53.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:53.677142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 516/1042] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:53.678602 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1d3e8-bbb6-4900-a274-ba3d03bab4de None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.722173 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:53.778130 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9159-fdcc-4508-9aaf-18bb0615b96f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.781709 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1d3e8-bbb6-4900-a274-ba3d03bab4de None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.812664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.813297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.813576 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.813842 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.814615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:53.814736 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.814919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.815323 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.815598 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.815864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.816292 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:53.821203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 527/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:53.821995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.823013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.824846 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:53.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:53.838546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:53.839438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 517/1044] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:53.840141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.865115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:53.940768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86500c-c52b-4eaa-98ba-3322d6f3a2f7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.952004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:53.955947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e2d54-ca6c-4096-abe2-1b41df18b336 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:53.980154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.980694 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.980982 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.981260 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.981985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:53.982327 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.983712 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:53.991394 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:53.992034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:53.992309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:53.992653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:53.993496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 528/1045] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:53.993651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] RBAC: Adding query filter params (interface=public, service_id=abe1e1c7f8314377bf49ce41a3db7bbf, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:53.993651 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:53.994403 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:53.995136 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:54.006411 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 518/1046] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/endpoints?service_id=abe1e1c7f8314377bf49ce41a3db7bbf&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:54.006411 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:54.092292 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:54.099712 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7daf00-62b2-4a8d-8863-36362eaf08a8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:54.104008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a3e70-3e17-4e56-a691-21df2d624210 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:54.136062 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:54.136623 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:54.136907 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:54.137187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:54.137951 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:54.138320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:54.138416 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0bd9a-038e-41d5-80de-b3c18879aa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:54.139010 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0bd9a-038e-41d5-80de-b3c18879aa30 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:54.139287 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0bd9a-038e-41d5-80de-b3c18879aa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:54.139556 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0bd9a-038e-41d5-80de-b3c18879aa30 None admin] PATH_INFO: `/v3/users/998480235ca644cbae7478924a4bf7f5` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:54.139773 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:54.153113 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 529/1047] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:54.153988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:54.154524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0bd9a-038e-41d5-80de-b3c18879aa30 None admin] RBAC: Authorizing `identity:get_user(user_id=998480235ca644cbae7478924a4bf7f5)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:54.156152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0bd9a-038e-41d5-80de-b3c18879aa30 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:54.158620 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 519/1048] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:02:54 2022] GET /identity/v3/users/998480235ca644cbae7478924a4bf7f5 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:54.163215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0bd9a-038e-41d5-80de-b3c18879aa30 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:54.189320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:54.266222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5e45c-066a-4282-b0f8-25647f9c3730 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:54.280258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0bd9a-038e-41d5-80de-b3c18879aa30 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:54.304753 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd55c3-ff3b-430a-9557-6ceafd0a22f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:54.305395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd55c3-ff3b-430a-9557-6ceafd0a22f9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:54.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd55c3-ff3b-430a-9557-6ceafd0a22f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:54.305949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd55c3-ff3b-430a-9557-6ceafd0a22f9 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:54.306646 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd55c3-ff3b-430a-9557-6ceafd0a22f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:54.308024 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd55c3-ff3b-430a-9557-6ceafd0a22f9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:54.314845 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 530/1049] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:02:54 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:02:54.315580 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd55c3-ff3b-430a-9557-6ceafd0a22f9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:54.320494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:54.321125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:54.321516 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:54.321797 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:54.322898 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] RBAC: Adding query filter params (role.id=d8710b1ea79f4ed78a1c4d54ee6e9148, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=4527e4500f984b1a81214c3656e7947f) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:54.323239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:54.325273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:54.350171 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 520/1050] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:54 2022] GET /identity/v3/role_assignments?user.id=4527e4500f984b1a81214c3656e7947f&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=d8710b1ea79f4ed78a1c4d54ee6e9148 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:54.355449 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:54.458444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd55c3-ff3b-430a-9557-6ceafd0a22f9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:54.503415 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b12b0-6b3b-4bd5-9f19-017d232616d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:54.504379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b12b0-6b3b-4bd5-9f19-017d232616d9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:54.504379 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b12b0-6b3b-4bd5-9f19-017d232616d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:54.504602 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b12b0-6b3b-4bd5-9f19-017d232616d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:54.579419 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b12b0-6b3b-4bd5-9f19-017d232616d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:54.580885 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b12b0-6b3b-4bd5-9f19-017d232616d9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:54.605436 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:54.699802 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 531/1051] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:02:54.705930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:54.805294 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:54.823336 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f613618-22ea-43e0-8ad7-1a37e3b99372 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:54.857890 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e34b2e-08b1-433c-82e6-fd5bcbfb534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:54.858477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e34b2e-08b1-433c-82e6-fd5bcbfb534b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:54.858751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e34b2e-08b1-433c-82e6-fd5bcbfb534b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:54.859025 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e34b2e-08b1-433c-82e6-fd5bcbfb534b None admin] PATH_INFO: `/v3/projects/ef09a1567b1f45ffbfce008c501d393e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:54.860219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e34b2e-08b1-433c-82e6-fd5bcbfb534b None admin] RBAC: Authorizing `identity:get_project(project_id=ef09a1567b1f45ffbfce008c501d393e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:54.861728 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e34b2e-08b1-433c-82e6-fd5bcbfb534b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:54.863668 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 521/1052] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:02:54 2022] GET /identity/v3/projects/ef09a1567b1f45ffbfce008c501d393e => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:54.869664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b12b0-6b3b-4bd5-9f19-017d232616d9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:54.976646 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b12b0-6b3b-4bd5-9f19-017d232616d9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:54.998496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:55.014941 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bf90f-1ab8-4291-9abf-542f7be96440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:55.015590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf90f-1ab8-4291-9abf-542f7be96440 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:55.015866 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf90f-1ab8-4291-9abf-542f7be96440 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:55.016138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf90f-1ab8-4291-9abf-542f7be96440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:55.016927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bf90f-1ab8-4291-9abf-542f7be96440 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=ef09a1567b1f45ffbfce008c501d393e, user.id=998480235ca644cbae7478924a4bf7f5) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:55.017298 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bf90f-1ab8-4291-9abf-542f7be96440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:55.018778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bf90f-1ab8-4291-9abf-542f7be96440 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:55.024792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 532/1053] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:02:54 2022] GET /identity/v3/role_assignments?user.id=998480235ca644cbae7478924a4bf7f5&scope.project.id=ef09a1567b1f45ffbfce008c501d393e&role.id=65cd37a73e4f498482fefabb8374006c => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:56.340854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a955c-45ab-443b-8500-0815def4abca None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:56.341391 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a955c-45ab-443b-8500-0815def4abca None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:56.341391 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a955c-45ab-443b-8500-0815def4abca None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:56.341848 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498feb6-aa7a-4a3e-aa7c-583a0cca96a6 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:56.342248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498feb6-aa7a-4a3e-aa7c-583a0cca96a6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:56.342320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498feb6-aa7a-4a3e-aa7c-583a0cca96a6 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:56.342956 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 522/1054] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:56.343800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 533/1055] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:56.347781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974487-f67f-4e6e-81c4-c4e21f3300f3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:56.348093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974487-f67f-4e6e-81c4-c4e21f3300f3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:56.348364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974487-f67f-4e6e-81c4-c4e21f3300f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:56.349478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da467b-41d4-49e3-aa19-e3f9a6def976 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:56.349746 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da467b-41d4-49e3-aa19-e3f9a6def976 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:56.350042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da467b-41d4-49e3-aa19-e3f9a6def976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:56.367393 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a1974487-f67f-4e6e-81c4-c4e21f3300f3 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:56.368588 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a1974487-f67f-4e6e-81c4-c4e21f3300f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:56.368717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a3da467b-41d4-49e3-aa19-e3f9a6def976 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:56.369787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a3da467b-41d4-49e3-aa19-e3f9a6def976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:56.443515 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 523/1056] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:56.446424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 534/1057] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:56.465890 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1974487-f67f-4e6e-81c4-c4e21f3300f3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:56.470872 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da467b-41d4-49e3-aa19-e3f9a6def976 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:56.559936 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:56.579066 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1974487-f67f-4e6e-81c4-c4e21f3300f3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:56.581551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da467b-41d4-49e3-aa19-e3f9a6def976 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:56.600007 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:56.617756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cf7ee-5c2a-4789-bb65-03a83ce2f044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:56.618445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cf7ee-5c2a-4789-bb65-03a83ce2f044 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:56.618722 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cf7ee-5c2a-4789-bb65-03a83ce2f044 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:56.618989 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cf7ee-5c2a-4789-bb65-03a83ce2f044 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:56.625053 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:56.626150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:56.626681 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:56.627151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] PATH_INFO: `/v3/services/image` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:56.627979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:56.629350 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:56.631049 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cf7ee-5c2a-4789-bb65-03a83ce2f044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-126cf7ee-5c2a-4789-bb65-03a83ce2f044 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:56.636097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:56.637021 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:56.643412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 524/1058] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:02:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:56.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:56.644986 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:56.645783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 535/1059] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:02:56 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:56.648527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cf7ee-5c2a-4789-bb65-03a83ce2f044 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:56.648750 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:56.766699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cf7ee-5c2a-4789-bb65-03a83ce2f044 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:56.769072 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269c419-afdd-481f-81c2-ad656c0d8485 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:56.806407 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:56.807041 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:56.807486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:56.807596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:56.808611 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:56.808611 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:56.809640 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:56.809979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:56.810685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:56.810995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:56.811243 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:56.812155 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:56.812622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:56.814047 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:56.820350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 525/1060] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:02:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:56.825947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 536/1061] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:56.826849 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:56.831661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:56.838563 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:56.885334 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:56.967334 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303ca6a-8ce0-4aaa-9862-2dbe9bc44e34 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:56.977460 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe5b18-8263-4fc0-9819-15d77f5a30d6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.008539 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f9312-4ed0-4816-84ad-40f3029e0376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.009198 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f9312-4ed0-4816-84ad-40f3029e0376 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.009479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f9312-4ed0-4816-84ad-40f3029e0376 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.009746 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f9312-4ed0-4816-84ad-40f3029e0376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.019924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f825a3b-4155-4cf8-b066-03580dd8269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.020566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f825a3b-4155-4cf8-b066-03580dd8269c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.020847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f825a3b-4155-4cf8-b066-03580dd8269c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.021117 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f825a3b-4155-4cf8-b066-03580dd8269c None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.021907 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f825a3b-4155-4cf8-b066-03580dd8269c None admin] RBAC: Adding query filter params (type=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:57.022238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f825a3b-4155-4cf8-b066-03580dd8269c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.023637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f825a3b-4155-4cf8-b066-03580dd8269c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.024550 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f9312-4ed0-4816-84ad-40f3029e0376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.026026 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f9312-4ed0-4816-84ad-40f3029e0376 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.033484 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 537/1062] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:57.034789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013ca4-58e6-4571-bf0b-914a14b571b7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.035114 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013ca4-58e6-4571-bf0b-914a14b571b7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.035390 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013ca4-58e6-4571-bf0b-914a14b571b7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.036795 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 538/1063] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:57.039633 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00013ca4-58e6-4571-bf0b-914a14b571b7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-268f9312-4ed0-4816-84ad-40f3029e0376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:57.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:57.041731 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:57.042672 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:57.042672 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:57.042672 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:57.042672 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:57.042672 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 526/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:57.043404 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c8810-71d9-4601-91cb-252f143e5a5e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.043723 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c8810-71d9-4601-91cb-252f143e5a5e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.043816 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c8810-71d9-4601-91cb-252f143e5a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.063032 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4b7c8810-71d9-4601-91cb-252f143e5a5e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:57.064266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4b7c8810-71d9-4601-91cb-252f143e5a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:57.118533 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:57.132363 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 527/1065] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:57.133296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c8810-71d9-4601-91cb-252f143e5a5e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.152366 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00013ca4-58e6-4571-bf0b-914a14b571b7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.165006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:57.196195 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499b1c5-9eaf-4e45-92e6-47e6cb686156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.196830 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499b1c5-9eaf-4e45-92e6-47e6cb686156 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.197108 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499b1c5-9eaf-4e45-92e6-47e6cb686156 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.197373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499b1c5-9eaf-4e45-92e6-47e6cb686156 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.198085 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499b1c5-9eaf-4e45-92e6-47e6cb686156 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.199850 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499b1c5-9eaf-4e45-92e6-47e6cb686156 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.230830 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 539/1066] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:02:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:02:57.230830 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499b1c5-9eaf-4e45-92e6-47e6cb686156 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.246502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c8810-71d9-4601-91cb-252f143e5a5e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.293246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.293922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.294236 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.294514 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.295239 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:57.295547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.296993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.314744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 528/1067] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:57.315971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.342520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499b1c5-9eaf-4e45-92e6-47e6cb686156 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.366041 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:57.388708 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48928f72-d3c2-44ca-9e28-39f3f75c5360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.390196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48928f72-d3c2-44ca-9e28-39f3f75c5360 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.390196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48928f72-d3c2-44ca-9e28-39f3f75c5360 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.390196 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48928f72-d3c2-44ca-9e28-39f3f75c5360 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.403090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48928f72-d3c2-44ca-9e28-39f3f75c5360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-48928f72-d3c2-44ca-9e28-39f3f75c5360 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:02:57.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:57.409801 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:02:57.410877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 540/1068] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:57.415469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48928f72-d3c2-44ca-9e28-39f3f75c5360 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.431624 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0943ce-d69f-4586-80b9-a83d9288f3e8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.443823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:57.468298 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.468955 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.469234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.469503 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.477637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.479184 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:02:57.491216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:57.492358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:57.498005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 529/1069] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:57.499951 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.532303 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48928f72-d3c2-44ca-9e28-39f3f75c5360 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.570751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.571699 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.572169 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.572606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.573510 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:57.573884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.575657 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.585381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 541/1070] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:57.591452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.607265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8405d-9815-442e-bb8a-1cdd8d2c150f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.611192 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:57.648208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.649234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.649817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.650382 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.651488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:57.651981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.653901 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.672421 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 530/1071] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:02:57.678976 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.679287 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:57.699902 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fc164-af76-4379-bac8-8fbe9e4e0f11 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.744688 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0d1a-3bea-4841-a7ae-3b3667d0eb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.745496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0d1a-3bea-4841-a7ae-3b3667d0eb2f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.745837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0d1a-3bea-4841-a7ae-3b3667d0eb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.746145 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0d1a-3bea-4841-a7ae-3b3667d0eb2f None admin] PATH_INFO: `/v3/users/998480235ca644cbae7478924a4bf7f5` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.747374 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c0d1a-3bea-4841-a7ae-3b3667d0eb2f None admin] RBAC: Authorizing `identity:get_user(user_id=998480235ca644cbae7478924a4bf7f5)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.748812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c0d1a-3bea-4841-a7ae-3b3667d0eb2f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.750494 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 542/1072] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/users/998480235ca644cbae7478924a4bf7f5 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:57.756725 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0d1a-3bea-4841-a7ae-3b3667d0eb2f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.794775 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586c83b-3434-420c-a21c-3e3a5a8bca7d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.831784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.832438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.832716 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.832990 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.844120 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.845603 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:57.861802 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:57.862781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:57.863630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:57.863630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:57.863630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:57.863630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:57.863630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 531/1073] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:57.868988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.869861 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0d1a-3bea-4841-a7ae-3b3667d0eb2f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:57.888520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:57.910410 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aa192-eef7-4b5a-8593-9e9db357bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:57.910970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aa192-eef7-4b5a-8593-9e9db357bad2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:57.911242 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aa192-eef7-4b5a-8593-9e9db357bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:57.911508 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aa192-eef7-4b5a-8593-9e9db357bad2 None admin] PATH_INFO: `/v3/projects/ef09a1567b1f45ffbfce008c501d393e` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:57.912659 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aa192-eef7-4b5a-8593-9e9db357bad2 None admin] RBAC: Authorizing `identity:get_project(project_id=ef09a1567b1f45ffbfce008c501d393e)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:57.914085 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aa192-eef7-4b5a-8593-9e9db357bad2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:57.918792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 543/1074] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/projects/ef09a1567b1f45ffbfce008c501d393e => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:57.922472 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aa192-eef7-4b5a-8593-9e9db357bad2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:57.948684 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:57.987089 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659db4c-c1eb-4ff6-a32e-438723ba5474 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:58.026347 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:58.027293 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:58.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:58.027838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:58.028528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:58.028830 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:58.030225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:58.041777 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aa192-eef7-4b5a-8593-9e9db357bad2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:58.050861 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 532/1075] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:58.052468 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:58.086592 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dda6a2-81e3-41be-ad1b-8fa9072d19ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:58.087273 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dda6a2-81e3-41be-ad1b-8fa9072d19ef None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:58.087555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dda6a2-81e3-41be-ad1b-8fa9072d19ef None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:58.087833 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dda6a2-81e3-41be-ad1b-8fa9072d19ef None admin] PATH_INFO: `/v3/projects/ef09a1567b1f45ffbfce008c501d393e/users/998480235ca644cbae7478924a4bf7f5/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:58.089799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dda6a2-81e3-41be-ad1b-8fa9072d19ef None admin] RBAC: Authorizing `identity:create_grant(project_id=ef09a1567b1f45ffbfce008c501d393e, user_id=998480235ca644cbae7478924a4bf7f5, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:58.091279 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dda6a2-81e3-41be-ad1b-8fa9072d19ef None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:58.106926 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 544/1076] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:02:57 2022] PUT /identity/v3/projects/ef09a1567b1f45ffbfce008c501d393e/users/998480235ca644cbae7478924a4bf7f5/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:02:58.138370 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:58.173334 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3605a5-5261-4d0b-bf2f-53ed3a4fad53 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:58.211932 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:58.212597 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:58.212877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:58.213151 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:58.221096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:58.222612 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:02:58.235266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:58.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:58.237071 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 533/1077] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:02:58.242687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dda6a2-81e3-41be-ad1b-8fa9072d19ef None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:58.337672 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:58.359613 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dda6a2-81e3-41be-ad1b-8fa9072d19ef None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:58.397185 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f8011-6339-4c47-99dd-a77a291c773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:58.397825 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f8011-6339-4c47-99dd-a77a291c773a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:58.398121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f8011-6339-4c47-99dd-a77a291c773a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:58.398400 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f8011-6339-4c47-99dd-a77a291c773a None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:58.399162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f8011-6339-4c47-99dd-a77a291c773a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:02:58.399469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f8011-6339-4c47-99dd-a77a291c773a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:58.400896 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f8011-6339-4c47-99dd-a77a291c773a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:58.418854 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 545/1078] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:02:58.420092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:58.524924 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13df8d-2dce-4705-9996-f132b728e9a3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:58.559992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3080a-1bf2-49e1-9ae4-ed194c39c6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:58.560671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3080a-1bf2-49e1-9ae4-ed194c39c6cb None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:58.560945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3080a-1bf2-49e1-9ae4-ed194c39c6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:58.561217 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3080a-1bf2-49e1-9ae4-ed194c39c6cb None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/4527e4500f984b1a81214c3656e7947f/roles/d8710b1ea79f4ed78a1c4d54ee6e9148` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:58.563437 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3080a-1bf2-49e1-9ae4-ed194c39c6cb None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=4527e4500f984b1a81214c3656e7947f, role_id=d8710b1ea79f4ed78a1c4d54ee6e9148)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:58.564930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3080a-1bf2-49e1-9ae4-ed194c39c6cb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:58.567212 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:58.586149 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 534/1079] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:02:58 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/4527e4500f984b1a81214c3656e7947f/roles/d8710b1ea79f4ed78a1c4d54ee6e9148 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:02:59.264981 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dd445-9afe-42f3-a795-0700841175bb None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:59.265360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dd445-9afe-42f3-a795-0700841175bb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:59.265521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dd445-9afe-42f3-a795-0700841175bb None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:59.267035 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 546/1080] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:02:59.273113 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9a4b-d502-46ae-bd1d-3e65f0734981 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:59.273405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9a4b-d502-46ae-bd1d-3e65f0734981 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:59.273677 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9a4b-d502-46ae-bd1d-3e65f0734981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:59.292601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bf8e9a4b-d502-46ae-bd1d-3e65f0734981 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:02:59.293810 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bf8e9a4b-d502-46ae-bd1d-3e65f0734981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:02:59.364729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 535/1081] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:02:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:02:59.386483 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dd445-9afe-42f3-a795-0700841175bb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:59.498063 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dd445-9afe-42f3-a795-0700841175bb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:59.534324 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43085608-1ae0-47bb-b1f7-37a162406c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:59.535113 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43085608-1ae0-47bb-b1f7-37a162406c3b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:59.535243 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43085608-1ae0-47bb-b1f7-37a162406c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:59.535410 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43085608-1ae0-47bb-b1f7-37a162406c3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:59.548193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:59.610083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43085608-1ae0-47bb-b1f7-37a162406c3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:59.611616 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43085608-1ae0-47bb-b1f7-37a162406c3b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:59.743500 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 547/1082] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:02:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:02:59.744854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e9a4b-d502-46ae-bd1d-3e65f0734981 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:59.859645 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e9a4b-d502-46ae-bd1d-3e65f0734981 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:02:59.885095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:02:59.898424 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95237ff7-b4f1-4d4b-a39f-3bf61ea37757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:02:59.899081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95237ff7-b4f1-4d4b-a39f-3bf61ea37757 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:02:59.899355 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95237ff7-b4f1-4d4b-a39f-3bf61ea37757 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:02:59.899624 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95237ff7-b4f1-4d4b-a39f-3bf61ea37757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:02:59.909935 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95237ff7-b4f1-4d4b-a39f-3bf61ea37757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:02:59.911421 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95237ff7-b4f1-4d4b-a39f-3bf61ea37757 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-95237ff7-b4f1-4d4b-a39f-3bf61ea37757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:02:59.927266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:02:59.928202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:02:59.929099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:02:59.929099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:02:59.929099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:02:59.929099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:02:59.929099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 536/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:02:59.936454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43085608-1ae0-47bb-b1f7-37a162406c3b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:02:59.980302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:00.040651 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43085608-1ae0-47bb-b1f7-37a162406c3b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:00.076070 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6e3c3-765e-487d-b27f-8ead9ea8a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:00.076700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e3c3-765e-487d-b27f-8ead9ea8a119 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.076972 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e3c3-765e-487d-b27f-8ead9ea8a119 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.077236 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e3c3-765e-487d-b27f-8ead9ea8a119 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.077926 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6e3c3-765e-487d-b27f-8ead9ea8a119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:00.078258 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6e3c3-765e-487d-b27f-8ead9ea8a119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:00.079741 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6e3c3-765e-487d-b27f-8ead9ea8a119 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:00.093993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 548/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:00.139567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc836-52a6-4b86-8b0d-5c03545288c4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.139911 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc836-52a6-4b86-8b0d-5c03545288c4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.140185 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc836-52a6-4b86-8b0d-5c03545288c4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.142243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 537/1085] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:00.148276 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc21c17-19a9-426e-b273-7744356090fa None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.148598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc21c17-19a9-426e-b273-7744356090fa None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.148840 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc21c17-19a9-426e-b273-7744356090fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.168555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4cc21c17-19a9-426e-b273-7744356090fa None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:00.169808 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4cc21c17-19a9-426e-b273-7744356090fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:00.248644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 549/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:00.272138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfc836-52a6-4b86-8b0d-5c03545288c4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:00.382487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfc836-52a6-4b86-8b0d-5c03545288c4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:00.416921 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc00ba-4775-446f-9df6-93ba8d679097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:00.417532 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc00ba-4775-446f-9df6-93ba8d679097 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.417813 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc00ba-4775-446f-9df6-93ba8d679097 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.418086 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc00ba-4775-446f-9df6-93ba8d679097 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.428605 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc00ba-4775-446f-9df6-93ba8d679097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a0fc00ba-4775-446f-9df6-93ba8d679097 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:03:00.433486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:00.434592 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:00.435623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 538/1087] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:00.441131 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc21c17-19a9-426e-b273-7744356090fa None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:00.472110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:00.546173 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc21c17-19a9-426e-b273-7744356090fa None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:00.577541 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ab74-0376-4ee3-aed3-32b6e24bfc74 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.577797 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ab74-0376-4ee3-aed3-32b6e24bfc74 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.578098 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ab74-0376-4ee3-aed3-32b6e24bfc74 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.579543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 539/1088] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:00.581866 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:00.582530 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.582813 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.583084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.583811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:00.584116 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:00.584625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6fc64-da2d-4feb-a429-5cc065fb5a2a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.584905 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6fc64-da2d-4feb-a429-5cc065fb5a2a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.585178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6fc64-da2d-4feb-a429-5cc065fb5a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.585489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:00.594462 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 550/1089] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:00.600253 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:00.603841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-53c6fc64-da2d-4feb-a429-5cc065fb5a2a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:00.605037 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-53c6fc64-da2d-4feb-a429-5cc065fb5a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:00.633821 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:00.674363 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 540/1090] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:03:00.697354 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6fc64-da2d-4feb-a429-5cc065fb5a2a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:00.702019 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:00.714873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3a9a7-7caa-4213-8ccc-913d3992181f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:00.753570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b665718-e1f5-4271-9161-9bffb9e1e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:00.754331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b665718-e1f5-4271-9161-9bffb9e1e359 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.754610 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b665718-e1f5-4271-9161-9bffb9e1e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.754871 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b665718-e1f5-4271-9161-9bffb9e1e359 None admin] PATH_INFO: `/v3/users/998480235ca644cbae7478924a4bf7f5` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.756035 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b665718-e1f5-4271-9161-9bffb9e1e359 None admin] RBAC: Authorizing `identity:get_user(user_id=998480235ca644cbae7478924a4bf7f5)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:00.757466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b665718-e1f5-4271-9161-9bffb9e1e359 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:00.759150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 551/1091] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:00 2022] GET /identity/v3/users/998480235ca644cbae7478924a4bf7f5 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:00.764956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b665718-e1f5-4271-9161-9bffb9e1e359 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:00.799154 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6fc64-da2d-4feb-a429-5cc065fb5a2a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:00.835926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bc839-a4cf-4847-91b8-cec79b6b1d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:00.836609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc839-a4cf-4847-91b8-cec79b6b1d11 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.836888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc839-a4cf-4847-91b8-cec79b6b1d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.837156 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc839-a4cf-4847-91b8-cec79b6b1d11 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.847325 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bc839-a4cf-4847-91b8-cec79b6b1d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-776bc839-a4cf-4847-91b8-cec79b6b1d11 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:03:00.852213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:00.853317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:00.854706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 541/1092] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:03:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:03:00.859477 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bc839-a4cf-4847-91b8-cec79b6b1d11 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:00.866882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b665718-e1f5-4271-9161-9bffb9e1e359 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:00.888969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:00.906087 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad810832-76ed-43f6-9a0a-cdca38bb7b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:00.906743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad810832-76ed-43f6-9a0a-cdca38bb7b20 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.907022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad810832-76ed-43f6-9a0a-cdca38bb7b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.907291 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad810832-76ed-43f6-9a0a-cdca38bb7b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.934802 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:00.961489 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bc839-a4cf-4847-91b8-cec79b6b1d11 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:00.975839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad810832-76ed-43f6-9a0a-cdca38bb7b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:00.977306 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad810832-76ed-43f6-9a0a-cdca38bb7b20 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:00.996625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:00.997245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:00.997518 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:00.997769 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:00.998529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:00.998831 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:01.000214 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:01.008210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 542/1093] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:03:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:03:01.013812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:01.078751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 552/1094] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:03:01.084924 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad810832-76ed-43f6-9a0a-cdca38bb7b20 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:01.101724 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:01.115153 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9abef-fe15-4fd4-b56a-f508f30bd0f2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:01.147590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:01.151951 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:01.152618 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:01.152895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:01.153163 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:01.164878 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:01.166418 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:03:01.180765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:01.181855 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:01.182864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:03:01.182864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:03:01.182864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:03:01.182864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:01.182864 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 543/1095] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:03:01.187790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:01.188899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad810832-76ed-43f6-9a0a-cdca38bb7b20 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:01.225466 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a8ef-0e53-45db-bdee-c031491dc4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:01.226136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a8ef-0e53-45db-bdee-c031491dc4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:01.226422 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a8ef-0e53-45db-bdee-c031491dc4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:01.226690 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a8ef-0e53-45db-bdee-c031491dc4f5 None admin] PATH_INFO: `/v3/projects/ef09a1567b1f45ffbfce008c501d393e` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:01.227859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822a8ef-0e53-45db-bdee-c031491dc4f5 None admin] RBAC: Authorizing `identity:get_project(project_id=ef09a1567b1f45ffbfce008c501d393e)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:01.229298 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822a8ef-0e53-45db-bdee-c031491dc4f5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:01.231001 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 553/1096] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:01 2022] GET /identity/v3/projects/ef09a1567b1f45ffbfce008c501d393e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:01.237030 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a8ef-0e53-45db-bdee-c031491dc4f5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:01.290742 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7d9c6-82f6-44e0-a182-3f080930026b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:01.324248 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:01.325235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:01.325512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:01.325776 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:01.326500 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:01.326801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:01.328173 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:01.337343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:01.340621 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 544/1097] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:03:01.341822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a8ef-0e53-45db-bdee-c031491dc4f5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:01.346433 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:01.376778 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:01.387100 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:01.387739 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:01.388023 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:01.388296 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:01.389107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=ef09a1567b1f45ffbfce008c501d393e, user.id=998480235ca644cbae7478924a4bf7f5) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:01.389421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:01.390899 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:01.397293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 554/1098] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:03:01 2022] GET /identity/v3/role_assignments?user.id=998480235ca644cbae7478924a4bf7f5&scope.project.id=ef09a1567b1f45ffbfce008c501d393e&role.id=65cd37a73e4f498482fefabb8374006c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:03:01.455468 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf18c-7705-4102-961d-5fc10fe11dfd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:01.494350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:01.495025 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:01.495293 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:01.495562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:01.503923 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:01.505387 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:03:01.518462 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:01.519595 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:01.520631 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 545/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:03:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:03:01.526142 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:01.634742 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04243d57-a071-463c-b48e-7075221297e4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:01.675596 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:01.676230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:01.676510 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:01.676784 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:01.677585 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:01.677897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:01.679351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:01.698116 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 555/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:03:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:01.704452 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:01.769971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:01.820755 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0359c-13b0-4dd9-89b6-3edb89614fed None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:01.857003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e592ef7-2945-4e3c-9823-f1208c4e81cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:01.857641 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e592ef7-2945-4e3c-9823-f1208c4e81cd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:01.857920 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e592ef7-2945-4e3c-9823-f1208c4e81cd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:01.858228 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e592ef7-2945-4e3c-9823-f1208c4e81cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:01.944272 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e592ef7-2945-4e3c-9823-f1208c4e81cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:01.945750 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e592ef7-2945-4e3c-9823-f1208c4e81cd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:02.008698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:02.052965 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 546/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:03:02.059038 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:02.110099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:02.162767 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd64715-7b0d-41ac-a486-d702ca48bf14 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:02.198166 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:02.198823 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:02.199097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:02.199737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:02.210629 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:02.212085 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:03:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:02.228826 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:02.229756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:03:02.229756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:03:02.229756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:03:02.229756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:03:02.229756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 556/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:02.235111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e592ef7-2945-4e3c-9823-f1208c4e81cd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:02.347663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e592ef7-2945-4e3c-9823-f1208c4e81cd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:02.385857 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:02.386538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:02.386819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:02.387094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:02.387804 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:02.388112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:02.389514 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:02.403171 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 547/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:02.409815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:02.518859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a61f8-e4cb-4c9a-9596-306c2ab4cdb8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:02.520478 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:02.559773 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:02.560459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:02.560689 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:02.560946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:02.568434 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:02.569910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:03:02.582244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:02.583316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:02.584882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:03:02.584882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:03:02.584882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:03:02.584882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:03:02.584882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:03:02.584882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:03:02.584882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:03:02.584882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:03:02.586497 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 557/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:03:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:02.589525 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:02.612848 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:02.701671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e5d03-49aa-4b15-9d67-1b8ab0c2cabd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:02.746528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f479e-00fe-4b0b-9ee3-792956450b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:02.749076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f479e-00fe-4b0b-9ee3-792956450b01 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:02.749076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f479e-00fe-4b0b-9ee3-792956450b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:02.749076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f479e-00fe-4b0b-9ee3-792956450b01 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:02.749076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f479e-00fe-4b0b-9ee3-792956450b01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:02.749076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f479e-00fe-4b0b-9ee3-792956450b01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:02.750169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f479e-00fe-4b0b-9ee3-792956450b01 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:02.765305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 548/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:03:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:02.771768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:02.892389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6575-04c1-4194-a0d1-9c172643fac0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:02.928262 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4866a7-9484-4c6d-9f17-cdec095a3d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:02.928897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4866a7-9484-4c6d-9f17-cdec095a3d49 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:02.929180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4866a7-9484-4c6d-9f17-cdec095a3d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:02.929450 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4866a7-9484-4c6d-9f17-cdec095a3d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:02.930270 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4866a7-9484-4c6d-9f17-cdec095a3d49 None admin] RBAC: Adding query filter params (role.id=d8710b1ea79f4ed78a1c4d54ee6e9148, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=4527e4500f984b1a81214c3656e7947f) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:02.930581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4866a7-9484-4c6d-9f17-cdec095a3d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:02.932042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4866a7-9484-4c6d-9f17-cdec095a3d49 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:02.938988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 558/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:03:02 2022] GET /identity/v3/role_assignments?user.id=4527e4500f984b1a81214c3656e7947f&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=d8710b1ea79f4ed78a1c4d54ee6e9148 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:03.467488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab425-cc6f-4aec-b4b8-81933f9d912f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:03.468236 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab425-cc6f-4aec-b4b8-81933f9d912f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:03.468236 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab425-cc6f-4aec-b4b8-81933f9d912f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:03.469438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 549/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:03.477099 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b46392-18f7-42be-9283-6544101aa4d5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:03.477548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b46392-18f7-42be-9283-6544101aa4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:03.478040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b46392-18f7-42be-9283-6544101aa4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:03.501481 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d7b46392-18f7-42be-9283-6544101aa4d5 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:03.502866 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d7b46392-18f7-42be-9283-6544101aa4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:03.530049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:03.586162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 559/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:03.615877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ab425-cc6f-4aec-b4b8-81933f9d912f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:03.713969 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:03.743044 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ab425-cc6f-4aec-b4b8-81933f9d912f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:03.782923 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab20240-c74a-4e72-b307-420cf7996a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:03.783980 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab20240-c74a-4e72-b307-420cf7996a61 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:03.784439 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab20240-c74a-4e72-b307-420cf7996a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:03.784983 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab20240-c74a-4e72-b307-420cf7996a61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:03.786700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab20240-c74a-4e72-b307-420cf7996a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:03.788273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab20240-c74a-4e72-b307-420cf7996a61 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:03.791191 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 550/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:03:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:03.797092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b46392-18f7-42be-9283-6544101aa4d5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:03.920171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b46392-18f7-42be-9283-6544101aa4d5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:03.958222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed759e-4772-45c8-b393-3e90b7bc2616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:03.958888 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed759e-4772-45c8-b393-3e90b7bc2616 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:03.959172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed759e-4772-45c8-b393-3e90b7bc2616 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:03.959448 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed759e-4772-45c8-b393-3e90b7bc2616 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:03.960160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed759e-4772-45c8-b393-3e90b7bc2616 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:03.961612 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed759e-4772-45c8-b393-3e90b7bc2616 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:03.984533 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 560/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:03:03 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:03:06.052042 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a87be6-ecfa-4922-886c-101a3fe71ac1 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:06.052583 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a87be6-ecfa-4922-886c-101a3fe71ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:06.052583 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a87be6-ecfa-4922-886c-101a3fe71ac1 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:06.053869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 551/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:06.059870 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035855f7-ee82-409a-acc3-0123bd8081ef None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:06.060397 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035855f7-ee82-409a-acc3-0123bd8081ef None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:06.060397 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035855f7-ee82-409a-acc3-0123bd8081ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:06.081570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-035855f7-ee82-409a-acc3-0123bd8081ef None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:06.082849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-035855f7-ee82-409a-acc3-0123bd8081ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:06.091802 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:06.158436 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 561/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:03:06.182859 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a87be6-ecfa-4922-886c-101a3fe71ac1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:06.296779 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a87be6-ecfa-4922-886c-101a3fe71ac1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:06.333674 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935294f-6db6-4b6a-a54d-09eb0c35fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:06.334262 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935294f-6db6-4b6a-a54d-09eb0c35fb2d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:06.334540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935294f-6db6-4b6a-a54d-09eb0c35fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:06.334909 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935294f-6db6-4b6a-a54d-09eb0c35fb2d None admin] PATH_INFO: `/v3/roles/94aa0ce403304ce8a21ef20db556d534` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:06.336302 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935294f-6db6-4b6a-a54d-09eb0c35fb2d None admin] RBAC: Authorizing `identity:get_role(role_id=94aa0ce403304ce8a21ef20db556d534)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:06.337705 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935294f-6db6-4b6a-a54d-09eb0c35fb2d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:06.339141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 552/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:06 2022] GET /identity/v3/roles/94aa0ce403304ce8a21ef20db556d534 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:06.345566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035855f7-ee82-409a-acc3-0123bd8081ef None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:06.453818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035855f7-ee82-409a-acc3-0123bd8081ef None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:06.490489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65831d99-7aed-4dc2-a27c-5ef8f3f1db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:06.491177 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65831d99-7aed-4dc2-a27c-5ef8f3f1db6e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:06.491439 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65831d99-7aed-4dc2-a27c-5ef8f3f1db6e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:06.491703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65831d99-7aed-4dc2-a27c-5ef8f3f1db6e None admin] PATH_INFO: `/v3/users/a7bb399daae6493b824d403437ffbfb9` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:06.504531 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:06.507905 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65831d99-7aed-4dc2-a27c-5ef8f3f1db6e None admin] RBAC: Authorizing `identity:get_user(user_id=a7bb399daae6493b824d403437ffbfb9)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:06.509752 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65831d99-7aed-4dc2-a27c-5ef8f3f1db6e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:06.511661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 562/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:06 2022] GET /identity/v3/users/a7bb399daae6493b824d403437ffbfb9 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:06.517752 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935294f-6db6-4b6a-a54d-09eb0c35fb2d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:06.543390 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:06.636190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935294f-6db6-4b6a-a54d-09eb0c35fb2d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:06.672213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9e368-c0ff-4995-89fe-f1d7b5040e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:06.672841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9e368-c0ff-4995-89fe-f1d7b5040e50 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:06.673191 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9e368-c0ff-4995-89fe-f1d7b5040e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:06.673464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9e368-c0ff-4995-89fe-f1d7b5040e50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:06.746577 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9e368-c0ff-4995-89fe-f1d7b5040e50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:06.748084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9e368-c0ff-4995-89fe-f1d7b5040e50 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:06.767384 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:06.858544 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 553/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:03:06.865354 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65831d99-7aed-4dc2-a27c-5ef8f3f1db6e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:06.986389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65831d99-7aed-4dc2-a27c-5ef8f3f1db6e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:07.034523 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4e8ed-82b9-46b8-86d8-913c9ab897f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:07.035222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4e8ed-82b9-46b8-86d8-913c9ab897f5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:07.035507 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4e8ed-82b9-46b8-86d8-913c9ab897f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:07.035780 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4e8ed-82b9-46b8-86d8-913c9ab897f5 None admin] PATH_INFO: `/v3/projects/ef09a1567b1f45ffbfce008c501d393e` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:07.037234 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4e8ed-82b9-46b8-86d8-913c9ab897f5 None admin] RBAC: Authorizing `identity:get_project(project_id=ef09a1567b1f45ffbfce008c501d393e)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:07.038764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4e8ed-82b9-46b8-86d8-913c9ab897f5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:07.040628 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 563/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:06 2022] GET /identity/v3/projects/ef09a1567b1f45ffbfce008c501d393e => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:07.047657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9e368-c0ff-4995-89fe-f1d7b5040e50 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:07.158810 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9e368-c0ff-4995-89fe-f1d7b5040e50 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:07.179529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:07.198970 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52e207-1a83-4a54-a341-a4230654687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:07.199955 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e207-1a83-4a54-a341-a4230654687c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:07.200229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e207-1a83-4a54-a341-a4230654687c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:07.200482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e207-1a83-4a54-a341-a4230654687c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:07.201274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52e207-1a83-4a54-a341-a4230654687c None admin] RBAC: Adding query filter params (role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=ef09a1567b1f45ffbfce008c501d393e, user.id=a7bb399daae6493b824d403437ffbfb9) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:07.201594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52e207-1a83-4a54-a341-a4230654687c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:07.203046 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52e207-1a83-4a54-a341-a4230654687c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:07.208819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 554/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:03:07 2022] GET /identity/v3/role_assignments?user.id=a7bb399daae6493b824d403437ffbfb9&scope.project.id=ef09a1567b1f45ffbfce008c501d393e&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:09.427184 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7d9cf-a588-4444-8481-2a59554a75d4 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:09.427733 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7d9cf-a588-4444-8481-2a59554a75d4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:09.427859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7d9cf-a588-4444-8481-2a59554a75d4 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:09.429306 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 564/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:09.436008 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb62f3-c3b7-47d0-9b14-90f6b4201609 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:09.436507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb62f3-c3b7-47d0-9b14-90f6b4201609 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:09.436656 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb62f3-c3b7-47d0-9b14-90f6b4201609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:09.458815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3fcb62f3-c3b7-47d0-9b14-90f6b4201609 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:09.460129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3fcb62f3-c3b7-47d0-9b14-90f6b4201609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:09.540603 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 555/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:09.566840 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7d9cf-a588-4444-8481-2a59554a75d4 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:09.641794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:09.687610 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7d9cf-a588-4444-8481-2a59554a75d4 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:09.730855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6243d-a06e-4396-acae-ed917d6c625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:09.731710 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6243d-a06e-4396-acae-ed917d6c625c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:09.732043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6243d-a06e-4396-acae-ed917d6c625c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:09.732320 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6243d-a06e-4396-acae-ed917d6c625c None admin] PATH_INFO: `/v3/roles/94aa0ce403304ce8a21ef20db556d534` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:09.734319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6243d-a06e-4396-acae-ed917d6c625c None admin] RBAC: Authorizing `identity:get_role(role_id=94aa0ce403304ce8a21ef20db556d534)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:09.735971 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6243d-a06e-4396-acae-ed917d6c625c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:09.737428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 565/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:09 2022] GET /identity/v3/roles/94aa0ce403304ce8a21ef20db556d534 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:09.744842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb62f3-c3b7-47d0-9b14-90f6b4201609 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:09.862734 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb62f3-c3b7-47d0-9b14-90f6b4201609 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:09.875791 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:09.910815 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5455f21-4f6c-49a4-8167-e3b2a4a2f0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:09.911472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5455f21-4f6c-49a4-8167-e3b2a4a2f0cf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:09.911741 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5455f21-4f6c-49a4-8167-e3b2a4a2f0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:09.912013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5455f21-4f6c-49a4-8167-e3b2a4a2f0cf None admin] PATH_INFO: `/v3/users/a7bb399daae6493b824d403437ffbfb9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:09.913266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5455f21-4f6c-49a4-8167-e3b2a4a2f0cf None admin] RBAC: Authorizing `identity:get_user(user_id=a7bb399daae6493b824d403437ffbfb9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:09.914744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5455f21-4f6c-49a4-8167-e3b2a4a2f0cf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:09.916374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 556/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:09 2022] GET /identity/v3/users/a7bb399daae6493b824d403437ffbfb9 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:09.922731 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6243d-a06e-4396-acae-ed917d6c625c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:10.046935 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6243d-a06e-4396-acae-ed917d6c625c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:10.086255 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec60c5-a8af-4bf0-9705-45d13eed54ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:10.086993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec60c5-a8af-4bf0-9705-45d13eed54ea None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:10.087317 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec60c5-a8af-4bf0-9705-45d13eed54ea None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:10.087648 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec60c5-a8af-4bf0-9705-45d13eed54ea None admin] PATH_INFO: `/v3/projects/ef09a1567b1f45ffbfce008c501d393e` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:10.089066 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec60c5-a8af-4bf0-9705-45d13eed54ea None admin] RBAC: Authorizing `identity:get_project(project_id=ef09a1567b1f45ffbfce008c501d393e)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:10.090735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec60c5-a8af-4bf0-9705-45d13eed54ea None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:10.092520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 566/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:09 2022] GET /identity/v3/projects/ef09a1567b1f45ffbfce008c501d393e => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:10.099690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5455f21-4f6c-49a4-8167-e3b2a4a2f0cf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:10.212531 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5455f21-4f6c-49a4-8167-e3b2a4a2f0cf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:10.257836 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152340e-e635-4352-9294-c26920b6a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:10.258717 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152340e-e635-4352-9294-c26920b6a710 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:10.258999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152340e-e635-4352-9294-c26920b6a710 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:10.259297 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152340e-e635-4352-9294-c26920b6a710 None admin] PATH_INFO: `/v3/projects/ef09a1567b1f45ffbfce008c501d393e/users/a7bb399daae6493b824d403437ffbfb9/roles/94aa0ce403304ce8a21ef20db556d534` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:10.262123 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152340e-e635-4352-9294-c26920b6a710 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef09a1567b1f45ffbfce008c501d393e, user_id=a7bb399daae6493b824d403437ffbfb9, role_id=94aa0ce403304ce8a21ef20db556d534)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:10.264147 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152340e-e635-4352-9294-c26920b6a710 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:10.279479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 557/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:03:10 2022] PUT /identity/v3/projects/ef09a1567b1f45ffbfce008c501d393e/users/a7bb399daae6493b824d403437ffbfb9/roles/94aa0ce403304ce8a21ef20db556d534 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:03:12.518950 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b368-952d-4e9c-b621-c88813ccb9b2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:12.519642 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b368-952d-4e9c-b621-c88813ccb9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:12.519642 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b368-952d-4e9c-b621-c88813ccb9b2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:12.520978 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 567/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:12.527487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326c23-a9eb-4877-8ea0-69f3bf0416d8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:12.527990 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326c23-a9eb-4877-8ea0-69f3bf0416d8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:12.528454 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326c23-a9eb-4877-8ea0-69f3bf0416d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:12.555296 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c9326c23-a9eb-4877-8ea0-69f3bf0416d8 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:12.556676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c9326c23-a9eb-4877-8ea0-69f3bf0416d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:12.594367 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:12.646041 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 558/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:12.673096 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88b368-952d-4e9c-b621-c88813ccb9b2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:12.677586 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:12.800263 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88b368-952d-4e9c-b621-c88813ccb9b2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:12.843794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5af93e-d7f4-4a6a-b46c-19c45ca470f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:12.844467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5af93e-d7f4-4a6a-b46c-19c45ca470f1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:12.844743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5af93e-d7f4-4a6a-b46c-19c45ca470f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:12.845050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5af93e-d7f4-4a6a-b46c-19c45ca470f1 None admin] PATH_INFO: `/v3/roles/94aa0ce403304ce8a21ef20db556d534` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:12.846711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5af93e-d7f4-4a6a-b46c-19c45ca470f1 None admin] RBAC: Authorizing `identity:get_role(role_id=94aa0ce403304ce8a21ef20db556d534)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:12.848219 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5af93e-d7f4-4a6a-b46c-19c45ca470f1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:12.849660 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 568/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:12 2022] GET /identity/v3/roles/94aa0ce403304ce8a21ef20db556d534 => generated 223 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:12.855853 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9326c23-a9eb-4877-8ea0-69f3bf0416d8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:12.970918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9326c23-a9eb-4877-8ea0-69f3bf0416d8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:13.009918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bf34c-2945-4dbe-ad75-a4b7e18559bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:13.010648 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bf34c-2945-4dbe-ad75-a4b7e18559bb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:13.010928 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bf34c-2945-4dbe-ad75-a4b7e18559bb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:13.011207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bf34c-2945-4dbe-ad75-a4b7e18559bb None admin] PATH_INFO: `/v3/users/a7bb399daae6493b824d403437ffbfb9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:13.012575 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bf34c-2945-4dbe-ad75-a4b7e18559bb None admin] RBAC: Authorizing `identity:get_user(user_id=a7bb399daae6493b824d403437ffbfb9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:13.014226 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bf34c-2945-4dbe-ad75-a4b7e18559bb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:13.016002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 559/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:12 2022] GET /identity/v3/users/a7bb399daae6493b824d403437ffbfb9 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:13.022446 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5af93e-d7f4-4a6a-b46c-19c45ca470f1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:13.102455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:13.140936 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5af93e-d7f4-4a6a-b46c-19c45ca470f1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:13.179351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016795-c3f8-4d5f-a002-494ffe6d711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:13.179963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016795-c3f8-4d5f-a002-494ffe6d711f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:13.180248 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016795-c3f8-4d5f-a002-494ffe6d711f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:13.180520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016795-c3f8-4d5f-a002-494ffe6d711f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:13.256748 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63016795-c3f8-4d5f-a002-494ffe6d711f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:13.258238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63016795-c3f8-4d5f-a002-494ffe6d711f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:13.331834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:13.365471 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 569/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:03:13.371719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bf34c-2945-4dbe-ad75-a4b7e18559bb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:13.410123 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:13.487609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bf34c-2945-4dbe-ad75-a4b7e18559bb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:13.525461 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a9851-6696-4ad7-8604-cd3f87f4fdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:13.526170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9851-6696-4ad7-8604-cd3f87f4fdaa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:13.526455 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9851-6696-4ad7-8604-cd3f87f4fdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:13.526729 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9851-6696-4ad7-8604-cd3f87f4fdaa None admin] PATH_INFO: `/v3/projects/ef09a1567b1f45ffbfce008c501d393e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:13.527999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a9851-6696-4ad7-8604-cd3f87f4fdaa None admin] RBAC: Authorizing `identity:get_project(project_id=ef09a1567b1f45ffbfce008c501d393e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:13.529482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a9851-6696-4ad7-8604-cd3f87f4fdaa None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:13.531331 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 560/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:13 2022] GET /identity/v3/projects/ef09a1567b1f45ffbfce008c501d393e => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:13.537989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016795-c3f8-4d5f-a002-494ffe6d711f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:13.652900 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016795-c3f8-4d5f-a002-494ffe6d711f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:13.696469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb77038-0f1e-4e49-928b-6f08aa7f901b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:13.697168 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb77038-0f1e-4e49-928b-6f08aa7f901b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:13.697463 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb77038-0f1e-4e49-928b-6f08aa7f901b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:13.697743 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb77038-0f1e-4e49-928b-6f08aa7f901b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:13.698663 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb77038-0f1e-4e49-928b-6f08aa7f901b None admin] RBAC: Adding query filter params (role.id=94aa0ce403304ce8a21ef20db556d534, scope.project.id=ef09a1567b1f45ffbfce008c501d393e, user.id=a7bb399daae6493b824d403437ffbfb9) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:13.699054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb77038-0f1e-4e49-928b-6f08aa7f901b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:13.700993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb77038-0f1e-4e49-928b-6f08aa7f901b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:13.707903 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 570/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:03:13 2022] GET /identity/v3/role_assignments?user.id=a7bb399daae6493b824d403437ffbfb9&scope.project.id=ef09a1567b1f45ffbfce008c501d393e&role.id=94aa0ce403304ce8a21ef20db556d534 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:15.841827 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933fdd8-0da6-49c9-bba5-e004039719a8 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:15.842455 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933fdd8-0da6-49c9-bba5-e004039719a8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:15.842455 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933fdd8-0da6-49c9-bba5-e004039719a8 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:15.843801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 561/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:15.851444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67300fff-eeb8-4358-80d4-1e05f41e3d69 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:15.852015 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67300fff-eeb8-4358-80d4-1e05f41e3d69 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:15.852344 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67300fff-eeb8-4358-80d4-1e05f41e3d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:15.874796 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-67300fff-eeb8-4358-80d4-1e05f41e3d69 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:15.876214 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-67300fff-eeb8-4358-80d4-1e05f41e3d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:15.919679 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:15.964488 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 571/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:15.987692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933fdd8-0da6-49c9-bba5-e004039719a8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:16.101695 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933fdd8-0da6-49c9-bba5-e004039719a8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:16.104202 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:16.142343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188ce4e-8b86-4f84-b946-3926e06afb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:16.142992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188ce4e-8b86-4f84-b946-3926e06afb17 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:16.143271 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188ce4e-8b86-4f84-b946-3926e06afb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:16.143540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188ce4e-8b86-4f84-b946-3926e06afb17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:16.144772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188ce4e-8b86-4f84-b946-3926e06afb17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:16.146324 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188ce4e-8b86-4f84-b946-3926e06afb17 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:16.147990 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 562/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:03:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:16.154344 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67300fff-eeb8-4358-80d4-1e05f41e3d69 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:16.259089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67300fff-eeb8-4358-80d4-1e05f41e3d69 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:16.294927 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87902ee2-0a43-482e-8a59-d949068d612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:16.295558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87902ee2-0a43-482e-8a59-d949068d612f None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:16.295834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87902ee2-0a43-482e-8a59-d949068d612f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:16.296107 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87902ee2-0a43-482e-8a59-d949068d612f None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:16.296794 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87902ee2-0a43-482e-8a59-d949068d612f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:16.298304 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87902ee2-0a43-482e-8a59-d949068d612f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:16.328544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 572/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:03:16 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:03:18.363366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53befd7c-44b9-404d-9e7b-4a80cb108828 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:18.363902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53befd7c-44b9-404d-9e7b-4a80cb108828 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:18.363902 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53befd7c-44b9-404d-9e7b-4a80cb108828 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:18.365275 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 563/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:18.370831 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264690-c8b1-468b-be3f-65c1ed55c546 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:18.371221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264690-c8b1-468b-be3f-65c1ed55c546 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:18.371378 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264690-c8b1-468b-be3f-65c1ed55c546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:18.390659 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-71264690-c8b1-468b-be3f-65c1ed55c546 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:18.391868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-71264690-c8b1-468b-be3f-65c1ed55c546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:18.406225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:18.464428 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 573/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:18.488899 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53befd7c-44b9-404d-9e7b-4a80cb108828 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:18.599025 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53befd7c-44b9-404d-9e7b-4a80cb108828 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:18.637270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9ceeb-d587-41a4-baa6-06532ce9d8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:18.637923 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ceeb-d587-41a4-baa6-06532ce9d8df None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:18.638232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ceeb-d587-41a4-baa6-06532ce9d8df None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:18.638507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ceeb-d587-41a4-baa6-06532ce9d8df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:18.639659 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9ceeb-d587-41a4-baa6-06532ce9d8df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:18.641111 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9ceeb-d587-41a4-baa6-06532ce9d8df None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:18.642833 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 564/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:03:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:18.648980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71264690-c8b1-468b-be3f-65c1ed55c546 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:18.760054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71264690-c8b1-468b-be3f-65c1ed55c546 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:18.796420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cea08a-182b-41bc-b1d9-226e9da151f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:18.797046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cea08a-182b-41bc-b1d9-226e9da151f9 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:18.797332 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cea08a-182b-41bc-b1d9-226e9da151f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:18.797603 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cea08a-182b-41bc-b1d9-226e9da151f9 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:18.798392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cea08a-182b-41bc-b1d9-226e9da151f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:18.799836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cea08a-182b-41bc-b1d9-226e9da151f9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:18.823388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 574/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:03:18 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:03:20.885095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff80b0-4657-4bbb-840b-0b1daee70b32 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:20.885616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff80b0-4657-4bbb-840b-0b1daee70b32 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:20.885616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff80b0-4657-4bbb-840b-0b1daee70b32 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:20.887561 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 565/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:20.893502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeed3e7-0232-418e-b4e2-dfbabb5c18b8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:20.893987 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeed3e7-0232-418e-b4e2-dfbabb5c18b8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:20.894078 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeed3e7-0232-418e-b4e2-dfbabb5c18b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:20.902066 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:20.914373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-eaeed3e7-0232-418e-b4e2-dfbabb5c18b8 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:20.915645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-eaeed3e7-0232-418e-b4e2-dfbabb5c18b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:20.983989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 575/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:21.007143 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff80b0-4657-4bbb-840b-0b1daee70b32 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:21.053392 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:21.114924 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff80b0-4657-4bbb-840b-0b1daee70b32 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:21.155376 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348dd2-f6f2-404c-ba40-e183618ebbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:21.156042 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348dd2-f6f2-404c-ba40-e183618ebbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:21.156319 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348dd2-f6f2-404c-ba40-e183618ebbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:21.156594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348dd2-f6f2-404c-ba40-e183618ebbc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:21.171583 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348dd2-f6f2-404c-ba40-e183618ebbc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-87348dd2-f6f2-404c-ba40-e183618ebbc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:03:21.176751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:21.177847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:21.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 566/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:03:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:21.184430 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeed3e7-0232-418e-b4e2-dfbabb5c18b8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:21.289264 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeed3e7-0232-418e-b4e2-dfbabb5c18b8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:21.325219 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:21.325852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:21.326149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:21.326429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:21.327150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:21.327457 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:21.328839 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:21.330318 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:21.338611 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 576/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:03:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:21.344765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348dd2-f6f2-404c-ba40-e183618ebbc1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:21.447049 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:21.450543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348dd2-f6f2-404c-ba40-e183618ebbc1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:21.485448 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062ea8-67a0-45b4-8a4b-63fb220e92f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:21.486085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062ea8-67a0-45b4-8a4b-63fb220e92f8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:21.486415 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062ea8-67a0-45b4-8a4b-63fb220e92f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:21.486681 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062ea8-67a0-45b4-8a4b-63fb220e92f8 None admin] PATH_INFO: `/v3/users/fb649356b495468c9d04b0d31ca2cf79` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:21.500305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1062ea8-67a0-45b4-8a4b-63fb220e92f8 None admin] RBAC: Authorizing `identity:get_user(user_id=fb649356b495468c9d04b0d31ca2cf79)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:21.501884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1062ea8-67a0-45b4-8a4b-63fb220e92f8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:21.503566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 567/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:21 2022] GET /identity/v3/users/fb649356b495468c9d04b0d31ca2cf79 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:21.509657 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:21.615673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce30859-c508-430d-8a2f-394f4563870e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:21.650643 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8893d-180b-4352-922c-3307c65fac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:21.651264 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8893d-180b-4352-922c-3307c65fac30 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:21.651543 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8893d-180b-4352-922c-3307c65fac30 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:21.651814 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8893d-180b-4352-922c-3307c65fac30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:21.715652 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:21.723164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8893d-180b-4352-922c-3307c65fac30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:21.724647 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8893d-180b-4352-922c-3307c65fac30 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:21.832667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 577/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:03:21.839010 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062ea8-67a0-45b4-8a4b-63fb220e92f8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:21.955966 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062ea8-67a0-45b4-8a4b-63fb220e92f8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:21.991570 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7e003-aa7b-4b32-bf42-fe09840c2e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:21.992215 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7e003-aa7b-4b32-bf42-fe09840c2e8c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:21.992501 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7e003-aa7b-4b32-bf42-fe09840c2e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:21.992773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7e003-aa7b-4b32-bf42-fe09840c2e8c None admin] PATH_INFO: `/v3/projects/1b182e5eb1d3407eb38edb7ac0687484` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:21.994022 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7e003-aa7b-4b32-bf42-fe09840c2e8c None admin] RBAC: Authorizing `identity:get_project(project_id=1b182e5eb1d3407eb38edb7ac0687484)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:21.995515 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7e003-aa7b-4b32-bf42-fe09840c2e8c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:21.997205 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 568/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:21 2022] GET /identity/v3/projects/1b182e5eb1d3407eb38edb7ac0687484 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:22.003703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8893d-180b-4352-922c-3307c65fac30 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:22.106884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:22.113381 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8893d-180b-4352-922c-3307c65fac30 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:22.148587 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc160b4d-28e9-4a32-b001-59da404e7733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:22.149199 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc160b4d-28e9-4a32-b001-59da404e7733 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:22.149429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc160b4d-28e9-4a32-b001-59da404e7733 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:22.149751 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc160b4d-28e9-4a32-b001-59da404e7733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:22.150566 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc160b4d-28e9-4a32-b001-59da404e7733 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=1b182e5eb1d3407eb38edb7ac0687484, user.id=fb649356b495468c9d04b0d31ca2cf79) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:22.150874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc160b4d-28e9-4a32-b001-59da404e7733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:22.152316 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc160b4d-28e9-4a32-b001-59da404e7733 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:22.158419 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 578/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:03:22 2022] GET /identity/v3/role_assignments?user.id=fb649356b495468c9d04b0d31ca2cf79&scope.project.id=1b182e5eb1d3407eb38edb7ac0687484&role.id=65cd37a73e4f498482fefabb8374006c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:24.151139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f52a1-64a8-4aad-9a15-fe10e3e03293 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:24.151754 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f52a1-64a8-4aad-9a15-fe10e3e03293 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:24.151754 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f52a1-64a8-4aad-9a15-fe10e3e03293 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:24.153064 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 569/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:24.159251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66029-6eca-4850-a593-5e4e83110e7d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:24.159827 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66029-6eca-4850-a593-5e4e83110e7d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:24.159827 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66029-6eca-4850-a593-5e4e83110e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:24.179228 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-2ba66029-6eca-4850-a593-5e4e83110e7d None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:24.180449 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2ba66029-6eca-4850-a593-5e4e83110e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:24.259537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 579/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:03:24.285102 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f52a1-64a8-4aad-9a15-fe10e3e03293 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:24.314586 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:24.392977 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f52a1-64a8-4aad-9a15-fe10e3e03293 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:24.428818 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155dcc8-e72b-4757-82cc-8b90007e94f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:24.429395 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155dcc8-e72b-4757-82cc-8b90007e94f4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:24.429671 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155dcc8-e72b-4757-82cc-8b90007e94f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:24.429946 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155dcc8-e72b-4757-82cc-8b90007e94f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:24.441220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155dcc8-e72b-4757-82cc-8b90007e94f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9155dcc8-e72b-4757-82cc-8b90007e94f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:03:24.446282 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:24.447412 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:24.448340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 570/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:03:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:24.456205 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba66029-6eca-4850-a593-5e4e83110e7d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:24.550526 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:24.561174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba66029-6eca-4850-a593-5e4e83110e7d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:24.598389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:24.599022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:24.599299 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:24.599567 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:24.600296 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:24.600581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:24.601960 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:24.611138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 580/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:03:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:24.617402 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155dcc8-e72b-4757-82cc-8b90007e94f4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:24.714784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:24.738918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155dcc8-e72b-4757-82cc-8b90007e94f4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:24.774685 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160038-9bbf-4641-bd01-8d9d5309f574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:24.775322 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160038-9bbf-4641-bd01-8d9d5309f574 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:24.775596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160038-9bbf-4641-bd01-8d9d5309f574 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:24.775867 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160038-9bbf-4641-bd01-8d9d5309f574 None admin] PATH_INFO: `/v3/users/fb649356b495468c9d04b0d31ca2cf79` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:24.777061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09160038-9bbf-4641-bd01-8d9d5309f574 None admin] RBAC: Authorizing `identity:get_user(user_id=fb649356b495468c9d04b0d31ca2cf79)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:24.778522 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09160038-9bbf-4641-bd01-8d9d5309f574 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:24.780179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 571/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:24 2022] GET /identity/v3/users/fb649356b495468c9d04b0d31ca2cf79 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:24.788916 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:24.902037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d90bc6-aeac-479a-be3d-7ab4fe61f2a9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:24.946593 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01739d0a-7c78-449d-85e4-e3a4a6609d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:24.947259 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01739d0a-7c78-449d-85e4-e3a4a6609d68 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:24.947542 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01739d0a-7c78-449d-85e4-e3a4a6609d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:24.947822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01739d0a-7c78-449d-85e4-e3a4a6609d68 None admin] PATH_INFO: `/v3/projects/1b182e5eb1d3407eb38edb7ac0687484` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:24.949009 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01739d0a-7c78-449d-85e4-e3a4a6609d68 None admin] RBAC: Authorizing `identity:get_project(project_id=1b182e5eb1d3407eb38edb7ac0687484)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:24.950861 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01739d0a-7c78-449d-85e4-e3a4a6609d68 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:24.952570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 581/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:24 2022] GET /identity/v3/projects/1b182e5eb1d3407eb38edb7ac0687484 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:24.959033 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160038-9bbf-4641-bd01-8d9d5309f574 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:25.065092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160038-9bbf-4641-bd01-8d9d5309f574 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:25.102895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46a62e-83b5-46a1-a6a9-6de61fc1a78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:25.103578 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46a62e-83b5-46a1-a6a9-6de61fc1a78e None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:25.103858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46a62e-83b5-46a1-a6a9-6de61fc1a78e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:25.104126 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46a62e-83b5-46a1-a6a9-6de61fc1a78e None admin] PATH_INFO: `/v3/projects/1b182e5eb1d3407eb38edb7ac0687484/users/fb649356b495468c9d04b0d31ca2cf79/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:25.106136 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46a62e-83b5-46a1-a6a9-6de61fc1a78e None admin] RBAC: Authorizing `identity:create_grant(project_id=1b182e5eb1d3407eb38edb7ac0687484, user_id=fb649356b495468c9d04b0d31ca2cf79, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:25.107593 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46a62e-83b5-46a1-a6a9-6de61fc1a78e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:25.122315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 572/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:03:24 2022] PUT /identity/v3/projects/1b182e5eb1d3407eb38edb7ac0687484/users/fb649356b495468c9d04b0d31ca2cf79/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:03:27.102883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeb58c-0bc2-4330-8f0a-0ce9fe631d8e None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:27.103405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeb58c-0bc2-4330-8f0a-0ce9fe631d8e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:27.103405 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeb58c-0bc2-4330-8f0a-0ce9fe631d8e None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:27.104800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 582/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:27.110362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb839d-1a29-43c0-9b5c-f6ecf53ec432 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:27.111088 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb839d-1a29-43c0-9b5c-f6ecf53ec432 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:27.111088 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb839d-1a29-43c0-9b5c-f6ecf53ec432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:27.130121 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-40eb839d-1a29-43c0-9b5c-f6ecf53ec432 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:27.131338 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-40eb839d-1a29-43c0-9b5c-f6ecf53ec432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:27.141788 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:27.203405 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 573/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:27.226544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfeb58c-0bc2-4330-8f0a-0ce9fe631d8e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:27.250955 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:27.335321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfeb58c-0bc2-4330-8f0a-0ce9fe631d8e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:27.376889 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca095232-20c3-45f4-a614-53a02a98105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:27.377551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca095232-20c3-45f4-a614-53a02a98105c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:27.377834 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca095232-20c3-45f4-a614-53a02a98105c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:27.378130 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca095232-20c3-45f4-a614-53a02a98105c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:27.389154 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca095232-20c3-45f4-a614-53a02a98105c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ca095232-20c3-45f4-a614-53a02a98105c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:03:27.394093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:27.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:03:27.396134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 583/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:27.401668 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb839d-1a29-43c0-9b5c-f6ecf53ec432 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:27.518502 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb839d-1a29-43c0-9b5c-f6ecf53ec432 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:27.553971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:27.554630 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:27.554906 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:27.555170 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:27.555891 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:27.556194 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:27.557560 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:27.566524 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 574/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:03:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:27.572735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca095232-20c3-45f4-a614-53a02a98105c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:27.658447 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:27.679546 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca095232-20c3-45f4-a614-53a02a98105c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:27.716461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf272f-9a28-4f26-9f7f-835c00ee7739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:27.717119 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf272f-9a28-4f26-9f7f-835c00ee7739 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:27.717392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf272f-9a28-4f26-9f7f-835c00ee7739 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:27.717660 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf272f-9a28-4f26-9f7f-835c00ee7739 None admin] PATH_INFO: `/v3/users/fb649356b495468c9d04b0d31ca2cf79` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:27.718861 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf272f-9a28-4f26-9f7f-835c00ee7739 None admin] RBAC: Authorizing `identity:get_user(user_id=fb649356b495468c9d04b0d31ca2cf79)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:27.720326 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf272f-9a28-4f26-9f7f-835c00ee7739 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:27.721973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 584/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:27 2022] GET /identity/v3/users/fb649356b495468c9d04b0d31ca2cf79 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:27.728457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:27.738951 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:27.836932 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d91cdc-a45b-4883-8d71-22fe9dff6305 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:27.871450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af8f71-5af8-4e2a-b4e5-9dc56239c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:27.872121 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af8f71-5af8-4e2a-b4e5-9dc56239c9fc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:27.872427 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af8f71-5af8-4e2a-b4e5-9dc56239c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:27.872678 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af8f71-5af8-4e2a-b4e5-9dc56239c9fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:27.952992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af8f71-5af8-4e2a-b4e5-9dc56239c9fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:27.954701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af8f71-5af8-4e2a-b4e5-9dc56239c9fc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:27.963796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:28.064116 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 575/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:03:28.070440 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf272f-9a28-4f26-9f7f-835c00ee7739 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:28.183480 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf272f-9a28-4f26-9f7f-835c00ee7739 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:28.220177 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f0e32-bee3-4ab5-8ac7-13cd76fcf2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:28.220730 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f0e32-bee3-4ab5-8ac7-13cd76fcf2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:28.221008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f0e32-bee3-4ab5-8ac7-13cd76fcf2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:28.221280 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f0e32-bee3-4ab5-8ac7-13cd76fcf2f1 None admin] PATH_INFO: `/v3/projects/1b182e5eb1d3407eb38edb7ac0687484` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:28.222489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f0e32-bee3-4ab5-8ac7-13cd76fcf2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=1b182e5eb1d3407eb38edb7ac0687484)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:28.223967 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f0e32-bee3-4ab5-8ac7-13cd76fcf2f1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:28.225649 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 585/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:28 2022] GET /identity/v3/projects/1b182e5eb1d3407eb38edb7ac0687484 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:28.231944 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af8f71-5af8-4e2a-b4e5-9dc56239c9fc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:28.335879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af8f71-5af8-4e2a-b4e5-9dc56239c9fc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:28.343292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:28.371819 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54d14e-6774-45f4-a496-859068918371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:28.372445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54d14e-6774-45f4-a496-859068918371 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:28.372724 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54d14e-6774-45f4-a496-859068918371 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:28.372997 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54d14e-6774-45f4-a496-859068918371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:28.373787 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54d14e-6774-45f4-a496-859068918371 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=1b182e5eb1d3407eb38edb7ac0687484, user.id=fb649356b495468c9d04b0d31ca2cf79) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:28.374066 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54d14e-6774-45f4-a496-859068918371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:28.375643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54d14e-6774-45f4-a496-859068918371 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:28.383983 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 576/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:03:28 2022] GET /identity/v3/role_assignments?user.id=fb649356b495468c9d04b0d31ca2cf79&scope.project.id=1b182e5eb1d3407eb38edb7ac0687484&role.id=65cd37a73e4f498482fefabb8374006c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:30.437965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ddc42-83b0-41fb-b900-9b878c9aa8eb None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:30.438755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ddc42-83b0-41fb-b900-9b878c9aa8eb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:30.438755 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ddc42-83b0-41fb-b900-9b878c9aa8eb None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:30.440029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 586/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:30.445429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149516b-abec-4d42-8aff-220ab335f89a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:30.445884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149516b-abec-4d42-8aff-220ab335f89a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:30.445998 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149516b-abec-4d42-8aff-220ab335f89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:30.465686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8149516b-abec-4d42-8aff-220ab335f89a None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:30.466916 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8149516b-abec-4d42-8aff-220ab335f89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:30.535793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 577/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:30.557882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ddc42-83b0-41fb-b900-9b878c9aa8eb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:30.577938 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:30.664650 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ddc42-83b0-41fb-b900-9b878c9aa8eb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:30.701769 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830696d-afcb-42fb-bf63-cfad954a6eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:30.702329 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830696d-afcb-42fb-bf63-cfad954a6eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:30.702613 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830696d-afcb-42fb-bf63-cfad954a6eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:30.702892 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830696d-afcb-42fb-bf63-cfad954a6eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:30.776491 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830696d-afcb-42fb-bf63-cfad954a6eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:30.777954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830696d-afcb-42fb-bf63-cfad954a6eb2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:30.796029 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:30.882988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 587/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:03:30.890481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149516b-abec-4d42-8aff-220ab335f89a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:31.002266 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:31.005758 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149516b-abec-4d42-8aff-220ab335f89a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:31.047803 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649133e3-f50c-4983-9122-8d97720afa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:31.048454 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649133e3-f50c-4983-9122-8d97720afa99 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:31.048725 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649133e3-f50c-4983-9122-8d97720afa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:31.048991 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649133e3-f50c-4983-9122-8d97720afa99 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:31.056190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649133e3-f50c-4983-9122-8d97720afa99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:31.057629 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649133e3-f50c-4983-9122-8d97720afa99 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-649133e3-f50c-4983-9122-8d97720afa99 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:31.073256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:31.074934 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:31.075772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 578/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:03:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:31.081422 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830696d-afcb-42fb-bf63-cfad954a6eb2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:31.187369 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830696d-afcb-42fb-bf63-cfad954a6eb2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:31.204910 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:31.225508 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ed12-4bf1-436a-ac7b-13f436307432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:31.226148 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ed12-4bf1-436a-ac7b-13f436307432 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:31.226438 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ed12-4bf1-436a-ac7b-13f436307432 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:31.226712 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ed12-4bf1-436a-ac7b-13f436307432 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:31.227415 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27ed12-4bf1-436a-ac7b-13f436307432 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:31.227719 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27ed12-4bf1-436a-ac7b-13f436307432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:31.229127 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27ed12-4bf1-436a-ac7b-13f436307432 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:31.237364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 588/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:03:31 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:33.229568 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a956-c1c2-4d08-9dc8-207610491426 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:33.230176 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a956-c1c2-4d08-9dc8-207610491426 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:33.230176 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a956-c1c2-4d08-9dc8-207610491426 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:33.231421 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 579/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:33.236812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baccb5-ec1c-400d-ad40-b70ade7691b3 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:33.237272 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baccb5-ec1c-400d-ad40-b70ade7691b3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:33.237366 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baccb5-ec1c-400d-ad40-b70ade7691b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:33.257357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-51baccb5-ec1c-400d-ad40-b70ade7691b3 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:33.259440 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-51baccb5-ec1c-400d-ad40-b70ade7691b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:33.335111 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 589/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:33.357745 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97a956-c1c2-4d08-9dc8-207610491426 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:33.463698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97a956-c1c2-4d08-9dc8-207610491426 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:33.499138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef41e82-f5f9-48ab-b6ec-d10772b381f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:33.499678 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef41e82-f5f9-48ab-b6ec-d10772b381f4 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:33.499956 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef41e82-f5f9-48ab-b6ec-d10772b381f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:33.500225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef41e82-f5f9-48ab-b6ec-d10772b381f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:33.501029 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef41e82-f5f9-48ab-b6ec-d10772b381f4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:33.502419 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef41e82-f5f9-48ab-b6ec-d10772b381f4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:33.526211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 580/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jul 28 08:03:33 2022] POST /identity/v3/domains => generated 267 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:03:35.554499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7b9d8-5e30-40b1-aa42-81f6f784d93a None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:35.555056 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7b9d8-5e30-40b1-aa42-81f6f784d93a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:35.555056 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7b9d8-5e30-40b1-aa42-81f6f784d93a None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:35.556423 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 590/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:35.561761 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8582a5e-9435-4e22-9100-1e2f0f40ae09 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:35.562240 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8582a5e-9435-4e22-9100-1e2f0f40ae09 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:35.562340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8582a5e-9435-4e22-9100-1e2f0f40ae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:35.581423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c8582a5e-9435-4e22-9100-1e2f0f40ae09 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:35.582657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c8582a5e-9435-4e22-9100-1e2f0f40ae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:35.597452 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:35.650999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 581/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:35.674213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7b9d8-5e30-40b1-aa42-81f6f784d93a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:35.784213 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7b9d8-5e30-40b1-aa42-81f6f784d93a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:35.794917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:35.822889 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704665a-3aa6-4015-8d88-a86da20761f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:35.823477 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704665a-3aa6-4015-8d88-a86da20761f4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:35.823761 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704665a-3aa6-4015-8d88-a86da20761f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:35.824033 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704665a-3aa6-4015-8d88-a86da20761f4 None admin] PATH_INFO: `/v3/domains/303ff1990de3467d9558297f0e0e9082` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:35.835262 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704665a-3aa6-4015-8d88-a86da20761f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=303ff1990de3467d9558297f0e0e9082)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:35.836713 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704665a-3aa6-4015-8d88-a86da20761f4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:35.838371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 591/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:03:35 2022] GET /identity/v3/domains/303ff1990de3467d9558297f0e0e9082 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:35.844950 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8582a5e-9435-4e22-9100-1e2f0f40ae09 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:35.957046 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8582a5e-9435-4e22-9100-1e2f0f40ae09 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:35.996196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1710c4-8c8c-41eb-83b4-1f13a536bf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:35.996805 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1710c4-8c8c-41eb-83b4-1f13a536bf86 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:35.997082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1710c4-8c8c-41eb-83b4-1f13a536bf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:35.997352 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1710c4-8c8c-41eb-83b4-1f13a536bf86 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:35.998029 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1710c4-8c8c-41eb-83b4-1f13a536bf86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:35.999515 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1710c4-8c8c-41eb-83b4-1f13a536bf86 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:36.031002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 582/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:03:35 2022] POST /identity/v3/projects => generated 361 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:03:38.059620 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e2e9d-da40-42e5-b725-9a3910141a63 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:38.060159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e2e9d-da40-42e5-b725-9a3910141a63 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:38.060159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e2e9d-da40-42e5-b725-9a3910141a63 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:38.061470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 592/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:38.066988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85532c4-c7e0-4647-8380-6f3a8af109e6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:38.067423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85532c4-c7e0-4647-8380-6f3a8af109e6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:38.067520 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85532c4-c7e0-4647-8380-6f3a8af109e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:38.074706 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:38.088992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f85532c4-c7e0-4647-8380-6f3a8af109e6 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:38.090210 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f85532c4-c7e0-4647-8380-6f3a8af109e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:38.161229 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 583/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:38.185882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e2e9d-da40-42e5-b725-9a3910141a63 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:38.300083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e2e9d-da40-42e5-b725-9a3910141a63 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:38.337343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ef1d1-263a-4128-9995-b10bf6091463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:38.337900 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef1d1-263a-4128-9995-b10bf6091463 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:38.338201 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef1d1-263a-4128-9995-b10bf6091463 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:38.338473 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef1d1-263a-4128-9995-b10bf6091463 None admin] PATH_INFO: `/v3/domains/303ff1990de3467d9558297f0e0e9082` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:38.339664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ef1d1-263a-4128-9995-b10bf6091463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=303ff1990de3467d9558297f0e0e9082)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:38.341116 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ef1d1-263a-4128-9995-b10bf6091463 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:38.342747 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 593/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:03:38 2022] GET /identity/v3/domains/303ff1990de3467d9558297f0e0e9082 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:38.350480 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85532c4-c7e0-4647-8380-6f3a8af109e6 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:38.453294 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85532c4-c7e0-4647-8380-6f3a8af109e6 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:38.486844 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:38.492809 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bee639-23ae-42b0-9918-d98f7d9ce1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:38.493437 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bee639-23ae-42b0-9918-d98f7d9ce1b7 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:38.493708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bee639-23ae-42b0-9918-d98f7d9ce1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:38.493975 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bee639-23ae-42b0-9918-d98f7d9ce1b7 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:38.494696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bee639-23ae-42b0-9918-d98f7d9ce1b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:38.496131 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bee639-23ae-42b0-9918-d98f7d9ce1b7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:38.518118 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 584/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:03:38 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:03:40.583244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf398748-6c42-4b03-a97a-0385183979b6 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:40.583800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf398748-6c42-4b03-a97a-0385183979b6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:40.583800 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf398748-6c42-4b03-a97a-0385183979b6 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:40.585178 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 594/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:40.590733 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5667d8-4826-4412-b7c9-54b46498deab None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:40.591193 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5667d8-4826-4412-b7c9-54b46498deab None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:40.591293 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5667d8-4826-4412-b7c9-54b46498deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:40.610128 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3e5667d8-4826-4412-b7c9-54b46498deab None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:40.611388 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3e5667d8-4826-4412-b7c9-54b46498deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:40.682001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 585/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:40.705764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf398748-6c42-4b03-a97a-0385183979b6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:40.751995 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:40.817752 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf398748-6c42-4b03-a97a-0385183979b6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:40.855401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf464f-1020-4f0c-8577-9ea1c9f96bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:40.855969 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf464f-1020-4f0c-8577-9ea1c9f96bbb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:40.856251 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf464f-1020-4f0c-8577-9ea1c9f96bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:40.856521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf464f-1020-4f0c-8577-9ea1c9f96bbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:40.868182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf464f-1020-4f0c-8577-9ea1c9f96bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-fbcf464f-1020-4f0c-8577-9ea1c9f96bbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:03:40.873126 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:40.874302 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:03:40.875522 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 595/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:40.880964 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5667d8-4826-4412-b7c9-54b46498deab None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:40.983734 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5667d8-4826-4412-b7c9-54b46498deab None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:41.001692 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:41.020084 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:41.020713 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:41.020993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:41.021269 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:41.021997 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:41.022339 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:41.023730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:41.032132 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 586/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:41.038241 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf464f-1020-4f0c-8577-9ea1c9f96bbb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:41.153108 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:41.157837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf464f-1020-4f0c-8577-9ea1c9f96bbb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:41.196437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd22c9-230c-4539-a061-121bf9a8627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:41.197096 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd22c9-230c-4539-a061-121bf9a8627b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:41.197371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd22c9-230c-4539-a061-121bf9a8627b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:41.197644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd22c9-230c-4539-a061-121bf9a8627b None admin] PATH_INFO: `/v3/users/b50b0b4db3de4c50a769d96a19134078` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:41.213392 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd22c9-230c-4539-a061-121bf9a8627b None admin] RBAC: Authorizing `identity:get_user(user_id=b50b0b4db3de4c50a769d96a19134078)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:41.214883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd22c9-230c-4539-a061-121bf9a8627b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:41.216591 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 596/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:41 2022] GET /identity/v3/users/b50b0b4db3de4c50a769d96a19134078 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:41.222918 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:41.329232 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9310-00f1-444b-bcbe-e43de3dd770c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:41.363712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fc619-dce6-4c8c-aeae-4e49861f290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:41.364329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fc619-dce6-4c8c-aeae-4e49861f290e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:41.364609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fc619-dce6-4c8c-aeae-4e49861f290e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:41.364876 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fc619-dce6-4c8c-aeae-4e49861f290e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:41.413662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:41.434696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fc619-dce6-4c8c-aeae-4e49861f290e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:41.436173 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fc619-dce6-4c8c-aeae-4e49861f290e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:41.535241 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 587/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:03:41.541463 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd22c9-230c-4539-a061-121bf9a8627b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:41.646758 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd22c9-230c-4539-a061-121bf9a8627b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:41.681849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07193b7f-21cf-48e6-a656-83a6723e44c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:41.682459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07193b7f-21cf-48e6-a656-83a6723e44c0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:41.682735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07193b7f-21cf-48e6-a656-83a6723e44c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:41.683006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07193b7f-21cf-48e6-a656-83a6723e44c0 None admin] PATH_INFO: `/v3/projects/165b2bc2c2c44a9eb19fa3478449d618` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:41.684207 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07193b7f-21cf-48e6-a656-83a6723e44c0 None admin] RBAC: Authorizing `identity:get_project(project_id=165b2bc2c2c44a9eb19fa3478449d618)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:41.685645 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07193b7f-21cf-48e6-a656-83a6723e44c0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:41.687377 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 597/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:41 2022] GET /identity/v3/projects/165b2bc2c2c44a9eb19fa3478449d618 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:03:41.693917 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fc619-dce6-4c8c-aeae-4e49861f290e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:41.778029 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:41.799743 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fc619-dce6-4c8c-aeae-4e49861f290e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:41.836698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017ba7d-dc6d-428b-afec-5a234250aba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:41.837321 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ba7d-dc6d-428b-afec-5a234250aba7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:41.837601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ba7d-dc6d-428b-afec-5a234250aba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:41.837869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ba7d-dc6d-428b-afec-5a234250aba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:41.838700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017ba7d-dc6d-428b-afec-5a234250aba7 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=165b2bc2c2c44a9eb19fa3478449d618, user.id=b50b0b4db3de4c50a769d96a19134078) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:41.839014 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017ba7d-dc6d-428b-afec-5a234250aba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:41.840541 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017ba7d-dc6d-428b-afec-5a234250aba7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:41.846284 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 588/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:03:41 2022] GET /identity/v3/role_assignments?user.id=b50b0b4db3de4c50a769d96a19134078&scope.project.id=165b2bc2c2c44a9eb19fa3478449d618&role.id=65cd37a73e4f498482fefabb8374006c => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:43.843564 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c134f-b9a0-462a-9112-4d54745c6234 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:43.844240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c134f-b9a0-462a-9112-4d54745c6234 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:43.844240 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c134f-b9a0-462a-9112-4d54745c6234 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:43.845572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 598/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:43.851691 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4363260-6f2a-4fe1-ae8b-4e3d8697c9e9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:43.852211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4363260-6f2a-4fe1-ae8b-4e3d8697c9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:43.852479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4363260-6f2a-4fe1-ae8b-4e3d8697c9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:43.876949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b4363260-6f2a-4fe1-ae8b-4e3d8697c9e9 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:43.878188 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b4363260-6f2a-4fe1-ae8b-4e3d8697c9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:43.948024 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 589/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:43.974048 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c134f-b9a0-462a-9112-4d54745c6234 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:44.000601 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:44.085424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c134f-b9a0-462a-9112-4d54745c6234 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:44.122364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e1e8-1093-4478-b86d-34a17197fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:44.123007 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e1e8-1093-4478-b86d-34a17197fe03 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:44.123288 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e1e8-1093-4478-b86d-34a17197fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:44.123559 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e1e8-1093-4478-b86d-34a17197fe03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:44.133991 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957e1e8-1093-4478-b86d-34a17197fe03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0957e1e8-1093-4478-b86d-34a17197fe03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:03:44.138886 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:44.139979 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:03:44.141343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 599/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:03:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:44.146523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4363260-6f2a-4fe1-ae8b-4e3d8697c9e9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:44.231903 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:44.259525 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4363260-6f2a-4fe1-ae8b-4e3d8697c9e9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:44.297220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:44.297837 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:44.298140 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:44.298407 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:44.299121 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:44.299423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:44.300826 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:44.309127 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 590/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:44.315255 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e1e8-1093-4478-b86d-34a17197fe03 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:44.410394 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:44.428246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e1e8-1093-4478-b86d-34a17197fe03 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:44.465053 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31f58a-a368-4a31-b2e6-0b3aba852148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:44.465706 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31f58a-a368-4a31-b2e6-0b3aba852148 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:44.465989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31f58a-a368-4a31-b2e6-0b3aba852148 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:44.466295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31f58a-a368-4a31-b2e6-0b3aba852148 None admin] PATH_INFO: `/v3/users/b50b0b4db3de4c50a769d96a19134078` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:44.467474 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31f58a-a368-4a31-b2e6-0b3aba852148 None admin] RBAC: Authorizing `identity:get_user(user_id=b50b0b4db3de4c50a769d96a19134078)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:44.468985 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31f58a-a368-4a31-b2e6-0b3aba852148 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:44.470670 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 600/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:44 2022] GET /identity/v3/users/b50b0b4db3de4c50a769d96a19134078 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:44.476823 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:44.583981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086e36c-bcb7-43e2-af25-3b16be2f34f1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:44.619044 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cd335-f1d6-4115-8072-caf9bd9d6dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:44.619699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd335-f1d6-4115-8072-caf9bd9d6dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:44.619976 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd335-f1d6-4115-8072-caf9bd9d6dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:44.620245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd335-f1d6-4115-8072-caf9bd9d6dd0 None admin] PATH_INFO: `/v3/projects/165b2bc2c2c44a9eb19fa3478449d618` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:44.621457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cd335-f1d6-4115-8072-caf9bd9d6dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=165b2bc2c2c44a9eb19fa3478449d618)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:44.622926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cd335-f1d6-4115-8072-caf9bd9d6dd0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:44.624614 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 591/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:44 2022] GET /identity/v3/projects/165b2bc2c2c44a9eb19fa3478449d618 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:44.631486 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31f58a-a368-4a31-b2e6-0b3aba852148 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:44.746203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31f58a-a368-4a31-b2e6-0b3aba852148 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:44.781829 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc927a-aa1b-4814-be98-a5a82ee821f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:44.782527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc927a-aa1b-4814-be98-a5a82ee821f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:44.782815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc927a-aa1b-4814-be98-a5a82ee821f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:44.783084 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc927a-aa1b-4814-be98-a5a82ee821f5 None admin] PATH_INFO: `/v3/projects/165b2bc2c2c44a9eb19fa3478449d618/users/b50b0b4db3de4c50a769d96a19134078/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:44.785054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc927a-aa1b-4814-be98-a5a82ee821f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=165b2bc2c2c44a9eb19fa3478449d618, user_id=b50b0b4db3de4c50a769d96a19134078, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:44.786532 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc927a-aa1b-4814-be98-a5a82ee821f5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:44.801504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 601/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:03:44 2022] PUT /identity/v3/projects/165b2bc2c2c44a9eb19fa3478449d618/users/b50b0b4db3de4c50a769d96a19134078/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:03:46.788429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11805a-eb84-4201-81dd-7be453e39df7 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:46.788997 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11805a-eb84-4201-81dd-7be453e39df7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:46.788997 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11805a-eb84-4201-81dd-7be453e39df7 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:46.790415 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 592/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:03:46.796526 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aede34-0ae6-4b90-b7e8-4ba91f4519b8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:46.797146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aede34-0ae6-4b90-b7e8-4ba91f4519b8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:46.797146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aede34-0ae6-4b90-b7e8-4ba91f4519b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:46.817716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d6aede34-0ae6-4b90-b7e8-4ba91f4519b8 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:03:46.818998 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d6aede34-0ae6-4b90-b7e8-4ba91f4519b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:03:46.828293 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:46.895210 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 602/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:03:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:03:46.920744 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11805a-eb84-4201-81dd-7be453e39df7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:46.925280 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:47.028657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11805a-eb84-4201-81dd-7be453e39df7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:47.065504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f79f30-ac22-41bf-aa0f-460189648122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:47.066153 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79f30-ac22-41bf-aa0f-460189648122 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:47.066457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79f30-ac22-41bf-aa0f-460189648122 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:47.066720 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79f30-ac22-41bf-aa0f-460189648122 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:47.078243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f79f30-ac22-41bf-aa0f-460189648122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-52f79f30-ac22-41bf-aa0f-460189648122 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:03:47.083017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:03:47.084034 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:03:47.085097 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 593/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:03:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:03:47.090551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aede34-0ae6-4b90-b7e8-4ba91f4519b8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:47.196479 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aede34-0ae6-4b90-b7e8-4ba91f4519b8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:47.231807 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:47.232429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:47.232705 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:47.232975 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:47.233690 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:47.233992 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:47.235455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:47.243676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 603/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:47.249792 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f79f30-ac22-41bf-aa0f-460189648122 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:47.311364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:47.354133 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f79f30-ac22-41bf-aa0f-460189648122 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:47.389615 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe464ba-7c07-45ca-8fb1-beba6643fe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:47.390293 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe464ba-7c07-45ca-8fb1-beba6643fe0d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:47.390575 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe464ba-7c07-45ca-8fb1-beba6643fe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:47.390911 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe464ba-7c07-45ca-8fb1-beba6643fe0d None admin] PATH_INFO: `/v3/users/b50b0b4db3de4c50a769d96a19134078` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:47.392115 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe464ba-7c07-45ca-8fb1-beba6643fe0d None admin] RBAC: Authorizing `identity:get_user(user_id=b50b0b4db3de4c50a769d96a19134078)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:47.393573 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe464ba-7c07-45ca-8fb1-beba6643fe0d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:47.395227 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 594/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:03:47 2022] GET /identity/v3/users/b50b0b4db3de4c50a769d96a19134078 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:47.401350 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:47.405346 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:47.511780 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e0dc8-d70a-43be-bae3-7df9f5487026 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:47.546416 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3e34b-4fab-4dbb-8f5e-728d82d1b57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:47.547065 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e34b-4fab-4dbb-8f5e-728d82d1b57a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:47.547342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e34b-4fab-4dbb-8f5e-728d82d1b57a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:47.547612 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e34b-4fab-4dbb-8f5e-728d82d1b57a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:47.615704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3e34b-4fab-4dbb-8f5e-728d82d1b57a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:47.617077 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3e34b-4fab-4dbb-8f5e-728d82d1b57a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:47.619740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:47.719093 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 604/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:03:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:03:47.725212 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe464ba-7c07-45ca-8fb1-beba6643fe0d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:47.827861 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe464ba-7c07-45ca-8fb1-beba6643fe0d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:47.857699 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:47.863649 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dac4ef-38da-4b8b-ae56-d6c3f27c62ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:47.864291 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dac4ef-38da-4b8b-ae56-d6c3f27c62ad None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:47.864572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dac4ef-38da-4b8b-ae56-d6c3f27c62ad None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:47.864842 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dac4ef-38da-4b8b-ae56-d6c3f27c62ad None admin] PATH_INFO: `/v3/projects/165b2bc2c2c44a9eb19fa3478449d618` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:47.866124 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dac4ef-38da-4b8b-ae56-d6c3f27c62ad None admin] RBAC: Authorizing `identity:get_project(project_id=165b2bc2c2c44a9eb19fa3478449d618)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:47.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dac4ef-38da-4b8b-ae56-d6c3f27c62ad None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:47.869218 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 595/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:03:47 2022] GET /identity/v3/projects/165b2bc2c2c44a9eb19fa3478449d618 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:47.875917 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3e34b-4fab-4dbb-8f5e-728d82d1b57a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:03:47.981208 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3e34b-4fab-4dbb-8f5e-728d82d1b57a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:03:47.983624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:03:48.018804 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155807e-efd7-4467-9a31-bc8273fc0c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:03:48.019429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155807e-efd7-4467-9a31-bc8273fc0c75 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:03:48.019706 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155807e-efd7-4467-9a31-bc8273fc0c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:03:48.019977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155807e-efd7-4467-9a31-bc8273fc0c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:03:48.020837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155807e-efd7-4467-9a31-bc8273fc0c75 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=165b2bc2c2c44a9eb19fa3478449d618, user.id=b50b0b4db3de4c50a769d96a19134078) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:03:48.021149 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155807e-efd7-4467-9a31-bc8273fc0c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:03:48.022711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155807e-efd7-4467-9a31-bc8273fc0c75 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:03:48.028873 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 605/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:03:47 2022] GET /identity/v3/role_assignments?user.id=b50b0b4db3de4c50a769d96a19134078&scope.project.id=165b2bc2c2c44a9eb19fa3478449d618&role.id=65cd37a73e4f498482fefabb8374006c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:03:58.026465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=16 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:44.541657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160cce6-2d11-4605-acef-a820453290d8 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:44.542413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160cce6-2d11-4605-acef-a820453290d8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:44.542413 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160cce6-2d11-4605-acef-a820453290d8 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:44.544102 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 596/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:04:44.549393 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa155009-afa0-48c2-aa32-52f3807d61ea None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:44.550136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa155009-afa0-48c2-aa32-52f3807d61ea None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:44.550136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa155009-afa0-48c2-aa32-52f3807d61ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:44.571549 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-fa155009-afa0-48c2-aa32-52f3807d61ea None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:04:44.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fa155009-afa0-48c2-aa32-52f3807d61ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:04:44.662244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 606/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:04:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:04:44.685925 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160cce6-2d11-4605-acef-a820453290d8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:44.861436 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160cce6-2d11-4605-acef-a820453290d8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:44.929624 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:44.930270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:44.930528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:44.930770 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:44.949648 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:44.951993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:04:44.970225 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:44.971471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:44.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:04:44.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:04:44.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:44.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:04:44.972937 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 597/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:44.978877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa155009-afa0-48c2-aa32-52f3807d61ea None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:45.117376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa155009-afa0-48c2-aa32-52f3807d61ea None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:45.163922 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:45.171940 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0fb32-8552-4ca0-b03f-f195ce18a073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:45.172586 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0fb32-8552-4ca0-b03f-f195ce18a073 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:45.172866 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0fb32-8552-4ca0-b03f-f195ce18a073 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:45.173137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0fb32-8552-4ca0-b03f-f195ce18a073 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:45.173900 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0fb32-8552-4ca0-b03f-f195ce18a073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:45.174257 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0fb32-8552-4ca0-b03f-f195ce18a073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:45.175716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0fb32-8552-4ca0-b03f-f195ce18a073 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:45.194158 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 607/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:45.197604 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:45.290972 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:45.364843 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c404944-2735-451f-8aca-2c76c3a55abe None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:45.408219 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c11a3-89c1-4547-acba-787c06b6ce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:45.408887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c11a3-89c1-4547-acba-787c06b6ce6b None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:45.409175 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c11a3-89c1-4547-acba-787c06b6ce6b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:45.409446 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c11a3-89c1-4547-acba-787c06b6ce6b None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:45.410173 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c11a3-89c1-4547-acba-787c06b6ce6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:45.411636 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c11a3-89c1-4547-acba-787c06b6ce6b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:45.442617 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 598/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:04:45 2022] POST /identity/v3/users => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:04:47.709225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba335e-59c3-4eaa-b2ef-630f29c03300 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:47.709225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba335e-59c3-4eaa-b2ef-630f29c03300 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:47.709225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba335e-59c3-4eaa-b2ef-630f29c03300 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:47.709225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 608/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:04:47.715712 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fa734-d10c-4785-a4c9-723ae8c0788b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:47.716358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fa734-d10c-4785-a4c9-723ae8c0788b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:47.716358 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fa734-d10c-4785-a4c9-723ae8c0788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:47.741015 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bb2fa734-d10c-4785-a4c9-723ae8c0788b None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:04:47.742328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bb2fa734-d10c-4785-a4c9-723ae8c0788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:04:47.817986 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 599/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:04:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:04:47.841568 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba335e-59c3-4eaa-b2ef-630f29c03300 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:48.009174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba335e-59c3-4eaa-b2ef-630f29c03300 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:48.061424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe7d08-4dfb-4e7c-9064-9cbb735d789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:48.062509 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe7d08-4dfb-4e7c-9064-9cbb735d789b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:48.062978 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe7d08-4dfb-4e7c-9064-9cbb735d789b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:48.063548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe7d08-4dfb-4e7c-9064-9cbb735d789b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:48.089221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe7d08-4dfb-4e7c-9064-9cbb735d789b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-61fe7d08-4dfb-4e7c-9064-9cbb735d789b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:04:48.097946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:48.099757 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:04:48.100789 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 609/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:48.107335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fa734-d10c-4785-a4c9-723ae8c0788b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:48.160607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:48.230511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fa734-d10c-4785-a4c9-723ae8c0788b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:48.270893 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:48.271648 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:48.271930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:48.272201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:48.272974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:48.273276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:48.274884 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:48.285041 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 600/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:04:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:48.290999 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe7d08-4dfb-4e7c-9064-9cbb735d789b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:48.356363 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:48.425160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe7d08-4dfb-4e7c-9064-9cbb735d789b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:48.483382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:48.484912 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:48.485430 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:48.485901 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:48.504775 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:48.507070 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:04:48.532271 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:48.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:48.535783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:04:48.535783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:04:48.535783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:48.535783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:04:48.535783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 610/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:48.539686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:48.656127 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:48.659698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946546b-cf8e-4576-8269-863417bb1859 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:48.714070 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:48.715112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:48.715348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:48.715632 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:48.716342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:48.716910 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:48.718096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:48.737566 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 601/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:48.738997 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:48.880797 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be73066-95b6-4f87-bb5d-bc0bcf0199e3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:48.883894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:48.942424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:48.943669 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:48.943960 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:48.944235 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:48.956148 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:48.957609 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:04:48.970783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:48.973080 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:04:48.974815 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 611/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:48.978445 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:49.170075 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0265347-3301-4c3b-bcb4-f13a0032061b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:49.236028 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:49.236945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:49.237988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:49.237988 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:49.239287 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:49.239743 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:49.241943 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:49.270656 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 602/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:04:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:04:49.277189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:49.444965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dacbec-1f94-43cd-b228-a65c2bbea11e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:49.498144 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760da92-dad9-4cc3-8c2f-5be07be03c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:49.498965 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760da92-dad9-4cc3-8c2f-5be07be03c81 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:49.499275 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760da92-dad9-4cc3-8c2f-5be07be03c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:49.499560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760da92-dad9-4cc3-8c2f-5be07be03c81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:49.559799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:49.635836 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760da92-dad9-4cc3-8c2f-5be07be03c81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:49.637452 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760da92-dad9-4cc3-8c2f-5be07be03c81 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:49.802675 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 612/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:04:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:04:49.804858 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:49.832286 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:49.985807 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c301620-50c8-47fd-92c1-3d78d81b3fb7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:50.052285 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:50.053615 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:50.054340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:50.054738 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:50.070302 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:50.072317 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:04:50.103930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:50.105879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:50.107702 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:04:50.107702 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:04:50.107702 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:50.107702 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:04:50.107702 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 603/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:50.112690 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760da92-dad9-4cc3-8c2f-5be07be03c81 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:50.196418 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:50.256506 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760da92-dad9-4cc3-8c2f-5be07be03c81 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:50.317524 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:50.318706 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:50.319334 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:50.319885 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:50.321220 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:50.321848 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:50.325115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:50.347410 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 613/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:04:50.353292 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:50.450556 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:50.488882 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd26b0d-1a98-4982-a095-dddeb34ad199 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:50.537065 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:50.537718 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:50.538099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:50.538348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:50.548440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:50.550037 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:04:50.565167 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:50.566368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:50.567690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:04:50.567690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:04:50.567690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:04:50.567690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:04:50.567690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:04:50.567690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:04:50.567690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:04:50.567690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:04:50.572480 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 604/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:50.573849 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:50.772494 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab5127-66d5-46a1-808d-bb2bc0317df5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:50.818517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:50.850136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef6bc7-ab36-4d6a-8cd2-23bad36465cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:50.850136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef6bc7-ab36-4d6a-8cd2-23bad36465cc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:50.850136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef6bc7-ab36-4d6a-8cd2-23bad36465cc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:50.850136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef6bc7-ab36-4d6a-8cd2-23bad36465cc None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:50.850622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef6bc7-ab36-4d6a-8cd2-23bad36465cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:50.850622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef6bc7-ab36-4d6a-8cd2-23bad36465cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:50.850622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef6bc7-ab36-4d6a-8cd2-23bad36465cc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:50.860816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 614/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:04:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:50.868243 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:51.081966 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4700e-1d7b-4bcd-b4bf-30d594864393 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:51.120129 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:51.130346 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf00a3-226f-4dc3-bf21-ed1013e44676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:51.131089 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf00a3-226f-4dc3-bf21-ed1013e44676 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:51.131518 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf00a3-226f-4dc3-bf21-ed1013e44676 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:51.131885 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf00a3-226f-4dc3-bf21-ed1013e44676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:51.132857 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf00a3-226f-4dc3-bf21-ed1013e44676 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=21d16f9ab0ae45b69b495ed579f66521) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:51.133273 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf00a3-226f-4dc3-bf21-ed1013e44676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:51.135112 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf00a3-226f-4dc3-bf21-ed1013e44676 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:51.142594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 605/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:04:50 2022] GET /identity/v3/role_assignments?user.id=21d16f9ab0ae45b69b495ed579f66521&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:04:53.511818 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2c07f-15fb-4aa4-8eca-8b0261b792b0 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:53.512572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2c07f-15fb-4aa4-8eca-8b0261b792b0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:53.512572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2c07f-15fb-4aa4-8eca-8b0261b792b0 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:53.514157 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 615/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:04:53.522422 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f15e4-c99f-46a8-ab50-42278853f57f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:53.523247 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f15e4-c99f-46a8-ab50-42278853f57f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:53.523431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f15e4-c99f-46a8-ab50-42278853f57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:53.545051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ad7f15e4-c99f-46a8-ab50-42278853f57f None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:04:53.546450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ad7f15e4-c99f-46a8-ab50-42278853f57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:04:53.641974 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 606/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:04:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:04:53.664812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2c07f-15fb-4aa4-8eca-8b0261b792b0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:53.819955 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2c07f-15fb-4aa4-8eca-8b0261b792b0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:53.866598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329c487-7464-4471-9e96-57bbe1cf46ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:53.867465 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329c487-7464-4471-9e96-57bbe1cf46ac None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:53.867847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329c487-7464-4471-9e96-57bbe1cf46ac None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:53.868222 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329c487-7464-4471-9e96-57bbe1cf46ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:53.880868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329c487-7464-4471-9e96-57bbe1cf46ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3329c487-7464-4471-9e96-57bbe1cf46ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:04:53.886551 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:53.889092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:04:53.890443 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 616/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:04:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:04:53.898310 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f15e4-c99f-46a8-ab50-42278853f57f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:54.041316 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f15e4-c99f-46a8-ab50-42278853f57f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:54.073562 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:54.089476 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:54.090138 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:54.090663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:54.090957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:54.091831 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:54.092220 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:54.094141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:54.114701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 607/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:04:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:04:54.116077 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329c487-7464-4471-9e96-57bbe1cf46ac None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:54.183983 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:54.271904 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329c487-7464-4471-9e96-57bbe1cf46ac None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:54.318812 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:54.320075 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:54.320740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:54.321237 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:54.343159 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:54.344772 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:04:54.363606 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:54.365717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:54.367537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:04:54.367537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:04:54.367537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:54.367537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:04:54.367537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 617/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:54.374523 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:54.554586 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99495072-2bb2-452e-aecc-0f0ba9585dca None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:54.605940 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:54.605940 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:54.605940 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:54.605940 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:54.606697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:54.606886 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:54.608939 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:54.624606 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 608/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:54.633544 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:54.778290 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:54.786043 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89ad7f-3d51-4525-86e5-e2839e50aaee None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:54.831525 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:54.832607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:54.833090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:54.833565 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:54.844980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:54.846951 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:04:54.862521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:54.864552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:04:54.865687 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 618/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:04:54.876265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:54.954554 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:55.037332 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34746c41-152f-4d92-b27f-776f0805a30d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:55.083509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:55.084099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:55.084419 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:55.084648 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:55.085460 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:55.085772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:55.087268 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:55.106854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 609/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:04:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:55.115416 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:55.287598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b24071-bed6-421f-a793-2f4448308c77 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:55.343572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:55.344695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:55.345177 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:55.345622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:55.363458 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:55.366177 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:04:55.391847 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:55.393342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:55.401171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:04:55.401171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:04:55.401171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:55.401171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:04:55.401171 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 619/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:04:55.402094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:55.573092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1199f-5e54-4b5f-949c-e9595361cd25 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:55.578237 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:55.622982 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17503151-39bf-4390-adec-050c177986c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:55.623414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17503151-39bf-4390-adec-050c177986c4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:55.623780 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17503151-39bf-4390-adec-050c177986c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:55.623979 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17503151-39bf-4390-adec-050c177986c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:55.624772 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17503151-39bf-4390-adec-050c177986c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:55.625152 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17503151-39bf-4390-adec-050c177986c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:55.626619 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17503151-39bf-4390-adec-050c177986c4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:55.640436 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 610/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:55.649760 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:55.661940 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:55.803493 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59f687-3b36-4ed1-83fb-0ad90a92e1ff None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:55.862401 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:55.863236 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:55.863590 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:55.864152 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:55.876110 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:55.878444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:04:55.900197 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:55.902061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:04:55.903852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 620/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:04:55.911089 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17503151-39bf-4390-adec-050c177986c4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:56.056420 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17503151-39bf-4390-adec-050c177986c4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:56.101973 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88bdc2-6c32-458b-830f-2941a237a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:56.102877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88bdc2-6c32-458b-830f-2941a237a9bf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:56.103368 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88bdc2-6c32-458b-830f-2941a237a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:56.103753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88bdc2-6c32-458b-830f-2941a237a9bf None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:56.104625 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88bdc2-6c32-458b-830f-2941a237a9bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:56.105015 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88bdc2-6c32-458b-830f-2941a237a9bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:56.106684 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88bdc2-6c32-458b-830f-2941a237a9bf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:56.123686 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 611/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:56.133161 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:56.243903 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:56.296626 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecac76-8e65-4281-9c41-b0cee279be09 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:56.355483 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c64e3-d8ab-4be1-aa01-f07243d7100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:56.356239 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c64e3-d8ab-4be1-aa01-f07243d7100a None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:56.356538 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c64e3-d8ab-4be1-aa01-f07243d7100a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:56.356813 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c64e3-d8ab-4be1-aa01-f07243d7100a None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/21d16f9ab0ae45b69b495ed579f66521/roles/b5c148877d4a489f85a6d554d44dba3f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:56.377875 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c64e3-d8ab-4be1-aa01-f07243d7100a None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=21d16f9ab0ae45b69b495ed579f66521, role_id=b5c148877d4a489f85a6d554d44dba3f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:56.379501 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c64e3-d8ab-4be1-aa01-f07243d7100a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:56.396942 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 621/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:04:56 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/21d16f9ab0ae45b69b495ed579f66521/roles/b5c148877d4a489f85a6d554d44dba3f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:04:58.495976 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7cc80-a6cd-4081-afaf-f4df13e62aec None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:58.496549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7cc80-a6cd-4081-afaf-f4df13e62aec None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:58.496549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7cc80-a6cd-4081-afaf-f4df13e62aec None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:58.497895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 612/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:04:58.503824 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72902d-0102-45ff-a67b-597c28c7f057 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:58.504360 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72902d-0102-45ff-a67b-597c28c7f057 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:58.504464 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72902d-0102-45ff-a67b-597c28c7f057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:58.525990 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3d72902d-0102-45ff-a67b-597c28c7f057 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:04:58.527414 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3d72902d-0102-45ff-a67b-597c28c7f057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:04:58.604298 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 622/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:04:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:04:58.630718 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7cc80-a6cd-4081-afaf-f4df13e62aec None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:58.666298 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:58.779690 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7cc80-a6cd-4081-afaf-f4df13e62aec None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:58.816913 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3c65-da05-4831-b9d6-cfa6236efbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:58.817575 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3c65-da05-4831-b9d6-cfa6236efbdd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:58.817849 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3c65-da05-4831-b9d6-cfa6236efbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:58.818142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3c65-da05-4831-b9d6-cfa6236efbdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:58.831896 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf3c65-da05-4831-b9d6-cfa6236efbdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1dcf3c65-da05-4831-b9d6-cfa6236efbdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:04:58.838496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:58.840307 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:04:58.842328 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 613/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:04:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:58.848284 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72902d-0102-45ff-a67b-597c28c7f057 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:58.908791 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:58.974177 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72902d-0102-45ff-a67b-597c28c7f057 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:59.022583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:59.023661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:59.024141 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:59.024615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:59.025884 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:59.026431 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:59.029042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:59.043285 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 623/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:04:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:04:59.050311 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3c65-da05-4831-b9d6-cfa6236efbdd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:59.153727 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:59.173492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3c65-da05-4831-b9d6-cfa6236efbdd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:59.214533 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:59.215213 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:59.215490 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:59.215759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:59.227790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:59.229340 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:04:59.249003 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:59.250051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:59.251110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:04:59.251110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:04:59.251110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:04:59.251110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:04:59.251110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 614/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:59.257628 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:59.402756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf140905-1788-4b4f-b164-1ce77244a468 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:59.416068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:59.448768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:59.449781 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:59.450060 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:59.450425 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:59.451138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:59.451581 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:59.452973 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:59.466906 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 624/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:04:59.473702 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:59.605909 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61629bb-b9a5-4c5e-8668-760196628d2b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:59.653344 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:59.654394 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:59.654981 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:59.655467 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:59.659354 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:04:59.667324 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:59.669388 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:04:59.686201 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:04:59.687567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:04:59.688479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 615/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:04:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:04:59.693453 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:04:59.808579 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675caf2-fc93-4b98-9bb0-0b7eb89dc54b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:04:59.847321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:04:59.848003 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:04:59.848312 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:04:59.848588 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:04:59.849504 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:04:59.849811 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:04:59.851416 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:04:59.873922 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 625/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:04:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:04:59.882052 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:00.033005 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfb56a-0305-4d12-8f27-ffdd76ff09e6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:00.077513 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944f347-46b2-4d5d-b790-c393a178a5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:00.078199 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944f347-46b2-4d5d-b790-c393a178a5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:00.078489 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944f347-46b2-4d5d-b790-c393a178a5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:00.078769 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944f347-46b2-4d5d-b790-c393a178a5a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:00.168584 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:00.179059 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944f347-46b2-4d5d-b790-c393a178a5a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:00.180516 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944f347-46b2-4d5d-b790-c393a178a5a5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:00.305785 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 616/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:04:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:05:00.313468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:00.352260 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:00.451470 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e8a53-c287-4261-b218-56470150e72f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:00.493358 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:00.494076 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:00.494413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:00.494756 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:00.507259 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:00.508854 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:05:00.529005 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:00.530869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:00.532455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:05:00.532455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:05:00.532455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:00.532455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:00.532455 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 626/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:00.537727 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944f347-46b2-4d5d-b790-c393a178a5a5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:00.669274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:00.677945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944f347-46b2-4d5d-b790-c393a178a5a5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:00.723032 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:00.723796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:00.724142 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:00.724756 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:00.725483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:00.725798 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:00.727234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:00.743403 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 617/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:05:00.750946 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:00.852252 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:00.881031 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d349b-9198-4636-bcf4-b02a7eba0adf None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:00.930217 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:00.931221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:00.931779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:00.932245 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:00.943349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:00.944835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:05:00.966541 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:00.967852 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:00.968810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:05:00.968810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:05:00.968810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:05:00.968810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:05:00.968810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:05:00.968810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:05:00.968810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:05:00.968810 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:00.969227 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 627/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:00.977492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:01.118748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2a1a-2111-41fd-8683-872a8c972de8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:01.160540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d790ccb-8116-439b-a40e-ceab1ca34e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:01.161223 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790ccb-8116-439b-a40e-ceab1ca34e0b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:01.161573 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790ccb-8116-439b-a40e-ceab1ca34e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:01.162013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790ccb-8116-439b-a40e-ceab1ca34e0b None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:01.162887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d790ccb-8116-439b-a40e-ceab1ca34e0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:01.163258 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d790ccb-8116-439b-a40e-ceab1ca34e0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:01.165079 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d790ccb-8116-439b-a40e-ceab1ca34e0b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:01.182001 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 618/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:01.189115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:01.319051 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693e6c7-9cf0-4cb7-bfcd-c41890e650a6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:01.362750 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aa8b0-c381-4611-8240-91dc49d42be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:01.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa8b0-c381-4611-8240-91dc49d42be1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:01.363963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa8b0-c381-4611-8240-91dc49d42be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:01.364384 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa8b0-c381-4611-8240-91dc49d42be1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:01.365343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aa8b0-c381-4611-8240-91dc49d42be1 None admin] RBAC: Adding query filter params (role.id=b5c148877d4a489f85a6d554d44dba3f, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=21d16f9ab0ae45b69b495ed579f66521) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:01.365769 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aa8b0-c381-4611-8240-91dc49d42be1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:01.367637 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aa8b0-c381-4611-8240-91dc49d42be1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:01.378451 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 628/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:05:01 2022] GET /identity/v3/role_assignments?user.id=21d16f9ab0ae45b69b495ed579f66521&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=b5c148877d4a489f85a6d554d44dba3f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:03.644511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24366576-44ce-48e8-8416-e059d0257c0b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:03.645057 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24366576-44ce-48e8-8416-e059d0257c0b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:03.645259 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24366576-44ce-48e8-8416-e059d0257c0b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:03.646841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 619/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:05:03.653339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f3133-5ac0-4ee7-bf30-451100f89adb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:03.654026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f3133-5ac0-4ee7-bf30-451100f89adb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:03.654026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f3133-5ac0-4ee7-bf30-451100f89adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:03.661636 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:03.677249 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6b0f3133-5ac0-4ee7-bf30-451100f89adb None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:05:03.678723 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6b0f3133-5ac0-4ee7-bf30-451100f89adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:05:03.776597 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 629/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:05:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:05:03.801946 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24366576-44ce-48e8-8416-e059d0257c0b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:03.835131 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:03.956115 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24366576-44ce-48e8-8416-e059d0257c0b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:04.006558 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af16c2-55b7-4b6a-ac15-e23b72359a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:04.007265 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af16c2-55b7-4b6a-ac15-e23b72359a6b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:04.007598 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af16c2-55b7-4b6a-ac15-e23b72359a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:04.007907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af16c2-55b7-4b6a-ac15-e23b72359a6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:04.023074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af16c2-55b7-4b6a-ac15-e23b72359a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-52af16c2-55b7-4b6a-ac15-e23b72359a6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:05:04.028216 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:04.029487 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:04.030423 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 620/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:04.039916 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f3133-5ac0-4ee7-bf30-451100f89adb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:04.186809 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f3133-5ac0-4ee7-bf30-451100f89adb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:04.223651 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:04.229947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:04.230653 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:04.230983 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:04.231306 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:04.232136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:04.232441 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:04.234595 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:04.245923 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 630/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:04.251915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af16c2-55b7-4b6a-ac15-e23b72359a6b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:04.359916 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:04.394601 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af16c2-55b7-4b6a-ac15-e23b72359a6b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:04.443922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:04.444715 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:04.445049 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:04.445300 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:04.458932 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:04.460581 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:05:04.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:04.479245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:04.480174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:05:04.480174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:05:04.480174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:04.480174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:04.480174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 621/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:05:04.490219 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:04.639572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcee7af-5516-4917-a7e1-e921c9b2f981 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:04.680779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:04.682195 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:04.682654 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:04.682919 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:04.683631 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:04.683938 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:04.685464 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:04.700578 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 631/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:04.707004 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:04.840342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c98411-9a69-42f3-b66b-33ae45cdd27d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:04.878304 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:04.891144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:04.892135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:04.892546 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:04.892943 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:04.905319 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:04.907119 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:05:04.923829 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:04.925234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:04.926342 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 622/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:04.932138 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:05.019963 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:05.087711 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4981c-3ef1-488e-beef-831a65bedfaa None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:05.130921 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:05.131816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:05.132189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:05.132584 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:05.133624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:05.134163 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:05.136420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:05.171822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 632/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:05:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:05:05.173384 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:05.312901 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3baceb-670f-4474-b3b2-8aabd19f3c65 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:05.355942 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540cefd-3a25-4367-9a4e-683191725b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:05.356596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540cefd-3a25-4367-9a4e-683191725b20 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:05.356881 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540cefd-3a25-4367-9a4e-683191725b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:05.357158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540cefd-3a25-4367-9a4e-683191725b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:05.427044 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:05.443046 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540cefd-3a25-4367-9a4e-683191725b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:05.444639 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540cefd-3a25-4367-9a4e-683191725b20 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:05.596341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 623/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:05:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:05:05.602571 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:05.754547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554cf0f-ad21-48dd-9ed8-0e0281aac773 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:05.771922 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:05.802983 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:05.804078 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:05.804417 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:05.804790 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:05.820957 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:05.822485 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:05:05.847046 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:05.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:05.849943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:05:05.849943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:05:05.849943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:05.849943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:05.849943 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 633/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:05.855676 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540cefd-3a25-4367-9a4e-683191725b20 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:05.970659 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:05.987548 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540cefd-3a25-4367-9a4e-683191725b20 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:06.038662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:06.039416 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:06.039799 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:06.040147 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:06.040952 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:06.041504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:06.043310 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:06.056310 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 624/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:05:06.064164 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:06.231695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87780e30-c589-46ca-8de0-a5e742706f7b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:06.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:06.277746 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:06.278064 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:06.278368 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:06.287993 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:06.289808 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:05:06.303977 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:06.305988 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:06.307883 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 634/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:06.311586 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:06.464147 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e9092-919a-4e7b-ab1d-1c7ca1b8e021 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:06.517545 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2e47e-25ca-474e-bb93-4509f4a9b273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:06.518350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2e47e-25ca-474e-bb93-4509f4a9b273 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:06.518822 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2e47e-25ca-474e-bb93-4509f4a9b273 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:06.519309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2e47e-25ca-474e-bb93-4509f4a9b273 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:06.520798 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2e47e-25ca-474e-bb93-4509f4a9b273 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:06.521279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2e47e-25ca-474e-bb93-4509f4a9b273 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:06.523876 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2e47e-25ca-474e-bb93-4509f4a9b273 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:06.546661 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 625/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:06.553403 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:06.592052 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:06.702898 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f006206-de31-4b88-a1e2-6b7c44d4c0e2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:06.766716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d97b6-07e9-4068-af92-b836c48ba5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:06.767469 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d97b6-07e9-4068-af92-b836c48ba5dd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:06.767674 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d97b6-07e9-4068-af92-b836c48ba5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:06.767897 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d97b6-07e9-4068-af92-b836c48ba5dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:06.768708 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d97b6-07e9-4068-af92-b836c48ba5dd None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=21d16f9ab0ae45b69b495ed579f66521) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:06.769018 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d97b6-07e9-4068-af92-b836c48ba5dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:06.770570 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d97b6-07e9-4068-af92-b836c48ba5dd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:06.777526 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 635/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:05:06 2022] GET /identity/v3/role_assignments?user.id=21d16f9ab0ae45b69b495ed579f66521&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=65cd37a73e4f498482fefabb8374006c => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:08.877892 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b96066-bf63-49b7-be71-8c4d8e133f68 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:08.878451 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b96066-bf63-49b7-be71-8c4d8e133f68 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:08.878540 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b96066-bf63-49b7-be71-8c4d8e133f68 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:08.879968 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 626/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:05:08.886203 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a62195-1230-4cf5-a7be-4b9c56cd8a7b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:08.886920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a62195-1230-4cf5-a7be-4b9c56cd8a7b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:08.886920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a62195-1230-4cf5-a7be-4b9c56cd8a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:08.905700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-73a62195-1230-4cf5-a7be-4b9c56cd8a7b None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:05:08.906956 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-73a62195-1230-4cf5-a7be-4b9c56cd8a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:05:08.993792 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 636/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:05:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:05:09.019092 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b96066-bf63-49b7-be71-8c4d8e133f68 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:09.033940 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:09.132035 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b96066-bf63-49b7-be71-8c4d8e133f68 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:09.171781 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93864476-590f-4374-bad6-653d31690d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:09.172572 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93864476-590f-4374-bad6-653d31690d0f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:09.172947 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93864476-590f-4374-bad6-653d31690d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:09.173312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93864476-590f-4374-bad6-653d31690d0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:09.184718 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93864476-590f-4374-bad6-653d31690d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-93864476-590f-4374-bad6-653d31690d0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:05:09.189773 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:09.191543 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:09.193458 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 627/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:05:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:09.198783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a62195-1230-4cf5-a7be-4b9c56cd8a7b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:09.219459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:09.315984 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a62195-1230-4cf5-a7be-4b9c56cd8a7b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:09.361026 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:09.361644 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:09.361949 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:09.362259 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:09.363189 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:09.363536 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:09.364932 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:09.375803 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 637/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:05:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:09.381926 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93864476-590f-4374-bad6-653d31690d0f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:09.454913 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:09.490567 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93864476-590f-4374-bad6-653d31690d0f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:09.526295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:09.526941 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:09.527218 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:09.527479 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:09.538509 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:09.540014 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:05:09.556378 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:09.557607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:09.559208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:05:09.559208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:05:09.559208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:09.559208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:09.559208 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 628/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:09.564118 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:09.651388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:09.671939 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09534865-b58e-482f-b83f-ed1a021e5282 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:09.710537 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:09.711180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:09.711454 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:09.711722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:09.712437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:09.712732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:09.714162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:09.728006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 638/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:09.736261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:09.841896 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fcd780-34ba-4844-9ce1-1c369be98455 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:09.862749 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:09.886488 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:09.887525 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:09.887812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:09.888295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:09.902211 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:09.904945 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:05:09.925840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:09.926971 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:05:09.928087 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:09.928841 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 629/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:09.934143 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:10.070376 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863402af-7846-42fb-a48a-f09b7e8f9c21 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:10.107157 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:10.107819 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:10.108095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:10.108365 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:10.109162 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:10.109461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:10.110935 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:10.120206 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:10.130190 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 639/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:05:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:10.136355 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:10.249752 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af247156-20db-44ad-8783-7449afd1aaad None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:10.289235 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:10.289907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:10.290221 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:10.290496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:10.302863 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:10.304366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:05:10.320320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:10.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:10.322438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:05:10.322438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:05:10.322438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:10.322438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:10.322438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 630/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:10.328805 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:10.437127 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade64268-a93c-4db2-8634-939c6ab6d37a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:10.473356 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:10.473962 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:10.474249 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:10.474550 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:10.475235 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:10.475536 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:10.476913 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:10.490151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 640/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:10.496263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:10.531174 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:10.607832 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eda6e-8a87-45ca-8bc6-03278974883f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:10.644542 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:10.645300 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:10.645608 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:10.645879 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:10.653695 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:10.655251 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:05:10.668466 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:10.669431 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:10.670511 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 631/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:05:10.676704 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:10.754632 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:10.801664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884f945-0ccc-423f-bac3-7af2b2c9a89c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:10.839421 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec5ebb-17cb-4ab0-b591-ddc9eeff68d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:10.840045 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec5ebb-17cb-4ab0-b591-ddc9eeff68d6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:10.840319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec5ebb-17cb-4ab0-b591-ddc9eeff68d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:10.840586 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec5ebb-17cb-4ab0-b591-ddc9eeff68d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:10.841375 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec5ebb-17cb-4ab0-b591-ddc9eeff68d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:10.841740 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec5ebb-17cb-4ab0-b591-ddc9eeff68d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:10.843205 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec5ebb-17cb-4ab0-b591-ddc9eeff68d6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:10.856341 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 641/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:05:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:10.863057 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:10.979683 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:10.990852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b01ceb-ff8b-4127-9f25-bc91bda3fd37 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:11.029397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5096aa-bc15-4b0b-8455-825624850800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:11.030077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5096aa-bc15-4b0b-8455-825624850800 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:11.030388 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5096aa-bc15-4b0b-8455-825624850800 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:11.030662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5096aa-bc15-4b0b-8455-825624850800 None admin] PATH_INFO: `/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/21d16f9ab0ae45b69b495ed579f66521/roles/65cd37a73e4f498482fefabb8374006c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:11.032649 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5096aa-bc15-4b0b-8455-825624850800 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f0819449c034a08b5a366324ca3a3f4, user_id=21d16f9ab0ae45b69b495ed579f66521, role_id=65cd37a73e4f498482fefabb8374006c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:11.034099 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5096aa-bc15-4b0b-8455-825624850800 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:11.048214 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 632/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:05:10 2022] PUT /identity/v3/projects/6f0819449c034a08b5a366324ca3a3f4/users/21d16f9ab0ae45b69b495ed579f66521/roles/65cd37a73e4f498482fefabb8374006c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:05:13.103068 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54ce43-c246-49a2-8198-701f8be906e1 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:13.103870 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54ce43-c246-49a2-8198-701f8be906e1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:13.103870 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54ce43-c246-49a2-8198-701f8be906e1 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:13.105398 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 642/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:05:13.111144 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6bfd8-8ed4-4a21-944d-3ff0a36e6c32 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:13.111589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6bfd8-8ed4-4a21-944d-3ff0a36e6c32 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:13.111737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6bfd8-8ed4-4a21-944d-3ff0a36e6c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:13.131419 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-19d6bfd8-8ed4-4a21-944d-3ff0a36e6c32 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:05:13.132638 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-19d6bfd8-8ed4-4a21-944d-3ff0a36e6c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:05:13.202849 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 633/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:05:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:05:13.226661 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54ce43-c246-49a2-8198-701f8be906e1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:13.341404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54ce43-c246-49a2-8198-701f8be906e1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:13.359077 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:13.376430 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dfe69-95a5-40e0-b8a5-2a3ff64ea369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:13.377076 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dfe69-95a5-40e0-b8a5-2a3ff64ea369 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:13.377357 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dfe69-95a5-40e0-b8a5-2a3ff64ea369 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:13.377624 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dfe69-95a5-40e0-b8a5-2a3ff64ea369 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:13.392730 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6dfe69-95a5-40e0-b8a5-2a3ff64ea369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-cf6dfe69-95a5-40e0-b8a5-2a3ff64ea369 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:05:13.397662 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:13.399313 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:13.400289 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 643/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:13.406406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6bfd8-8ed4-4a21-944d-3ff0a36e6c32 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:13.492637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:13.513357 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6bfd8-8ed4-4a21-944d-3ff0a36e6c32 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:13.547822 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:13.548440 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:13.548722 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:13.548993 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:13.549717 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:13.550025 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:13.551457 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:13.560518 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 634/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:05:13.569586 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dfe69-95a5-40e0-b8a5-2a3ff64ea369 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:13.683511 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dfe69-95a5-40e0-b8a5-2a3ff64ea369 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:13.721276 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:13.721930 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:13.722238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:13.722513 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:13.734489 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:13.735832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:05:13.752089 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:13.754006 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:13.755695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:05:13.755695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:05:13.755695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:13.755695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:13.755695 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 644/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:13.760331 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:13.882793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fb8ac-7be5-4ec2-8a4e-b1c705195942 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:13.919683 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:13.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:13.920589 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:13.920859 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:13.921557 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:13.921860 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:13.923276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:13.935132 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 635/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:13.944404 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:13.986685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:14.076467 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164db4e-2115-499c-98f0-abbeb35c05f3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:14.112488 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:14.113244 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:14.113340 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:14.113639 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:14.123334 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:14.126085 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:05:14.137095 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:14.138146 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:14.139370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 645/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:14.146915 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:14.151507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:14.259463 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d7572-92fd-4e78-b165-cf3652bef676 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:14.295379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:14.295999 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:14.296279 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:14.296551 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:14.297365 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:14.297670 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:14.299135 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:14.316796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 636/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:05:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:14.325685 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:14.436444 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:14.448351 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e291cd3-8de0-4949-b637-caa906e691e1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:14.496133 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e5a4e-1a81-4b92-a41e-60ae5bdbbe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:14.496762 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e5a4e-1a81-4b92-a41e-60ae5bdbbe0d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:14.497045 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e5a4e-1a81-4b92-a41e-60ae5bdbbe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:14.497321 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e5a4e-1a81-4b92-a41e-60ae5bdbbe0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:14.572339 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e5a4e-1a81-4b92-a41e-60ae5bdbbe0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:14.573722 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e5a4e-1a81-4b92-a41e-60ae5bdbbe0d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:14.665891 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:14.677654 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 646/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:05:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:05:14.684473 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:14.752652 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:14.794547 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4a025-01bc-486f-99e4-7d9d7eecbecc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:14.834594 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:14.835379 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:14.835749 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:14.836103 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:14.849504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:14.851023 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:05:14.867494 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:14.868496 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:14.869753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:05:14.869753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:05:14.869753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:05:14.869753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:14.869753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 637/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:14.876869 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e5a4e-1a81-4b92-a41e-60ae5bdbbe0d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:15.006474 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e5a4e-1a81-4b92-a41e-60ae5bdbbe0d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:15.044548 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:15.045182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:15.045451 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:15.045720 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:15.046558 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:15.046859 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:15.048225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:15.059616 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 647/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:15.066018 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:15.179711 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e7a7-fd07-4932-af37-85a9877f3612 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:15.193840 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:15.216101 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:15.216751 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:15.217027 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:15.217297 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:15.224204 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:15.225662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:05:15.237081 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:15.238166 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 28 08:05:15.239385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 638/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:15.244461 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:15.297301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:15.350373 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd581ff0-bf75-422d-a599-8d6bcd3fad70 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:15.387951 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafd5f8-825d-47b2-b693-0b2c7d91ec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:15.388562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafd5f8-825d-47b2-b693-0b2c7d91ec8b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:15.388841 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafd5f8-825d-47b2-b693-0b2c7d91ec8b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:15.389115 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafd5f8-825d-47b2-b693-0b2c7d91ec8b None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:15.389890 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafd5f8-825d-47b2-b693-0b2c7d91ec8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:15.390227 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafd5f8-825d-47b2-b693-0b2c7d91ec8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:15.391678 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafd5f8-825d-47b2-b693-0b2c7d91ec8b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:15.404737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 648/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:05:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:15.411764 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:15.523309 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8463b-f88f-446c-8bb1-9b027199b2e6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:15.559455 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050494b-6875-4294-b99b-b1fbefc70190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:15.560059 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050494b-6875-4294-b99b-b1fbefc70190 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:15.560334 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050494b-6875-4294-b99b-b1fbefc70190 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:15.560599 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050494b-6875-4294-b99b-b1fbefc70190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:15.561385 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050494b-6875-4294-b99b-b1fbefc70190 None admin] RBAC: Adding query filter params (role.id=65cd37a73e4f498482fefabb8374006c, scope.project.id=6f0819449c034a08b5a366324ca3a3f4, user.id=21d16f9ab0ae45b69b495ed579f66521) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:15.561768 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050494b-6875-4294-b99b-b1fbefc70190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:15.563236 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050494b-6875-4294-b99b-b1fbefc70190 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:15.569319 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 639/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:05:15 2022] GET /identity/v3/role_assignments?user.id=21d16f9ab0ae45b69b495ed579f66521&scope.project.id=6f0819449c034a08b5a366324ca3a3f4&role.id=65cd37a73e4f498482fefabb8374006c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:17.602605 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed2525-eed5-40df-a2c6-ae0a2819d5b4 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:17.603202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed2525-eed5-40df-a2c6-ae0a2819d5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:17.603202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed2525-eed5-40df-a2c6-ae0a2819d5b4 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:17.604555 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 649/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:05:17.610784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d878bbc-2290-4b6c-9a6c-4a2b1967a39e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:17.611499 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d878bbc-2290-4b6c-9a6c-4a2b1967a39e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:17.611697 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d878bbc-2290-4b6c-9a6c-4a2b1967a39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:17.634366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3d878bbc-2290-4b6c-9a6c-4a2b1967a39e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:05:17.635554 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3d878bbc-2290-4b6c-9a6c-4a2b1967a39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:05:17.670541 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:17.711482 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 640/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:05:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:05:17.733737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed2525-eed5-40df-a2c6-ae0a2819d5b4 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:17.855334 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed2525-eed5-40df-a2c6-ae0a2819d5b4 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:17.866499 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:17.893008 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:17.893656 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:17.893934 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:17.894238 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:17.894866 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:17.896241 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:17.905297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:17.906556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:17.907607 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 650/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:05:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:17.912961 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d878bbc-2290-4b6c-9a6c-4a2b1967a39e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:18.045672 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d878bbc-2290-4b6c-9a6c-4a2b1967a39e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:18.082301 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003699-c94d-43db-a36c-cdc3f02d3089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:18.082930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003699-c94d-43db-a36c-cdc3f02d3089 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:18.083207 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003699-c94d-43db-a36c-cdc3f02d3089 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:18.083478 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003699-c94d-43db-a36c-cdc3f02d3089 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:18.084169 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003699-c94d-43db-a36c-cdc3f02d3089 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:18.084471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003699-c94d-43db-a36c-cdc3f02d3089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:18.085853 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003699-c94d-43db-a36c-cdc3f02d3089 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:18.091077 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 641/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:05:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:18.097352 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:18.213027 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a70bbe-cc8a-44e2-9e75-377f8225c678 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:18.250983 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a3c54-5ac5-4f36-b9ef-8ebbc97a3f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:18.251611 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3c54-5ac5-4f36-b9ef-8ebbc97a3f6c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:18.251891 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3c54-5ac5-4f36-b9ef-8ebbc97a3f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:18.252160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3c54-5ac5-4f36-b9ef-8ebbc97a3f6c None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:18.252844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a3c54-5ac5-4f36-b9ef-8ebbc97a3f6c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:18.253134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a3c54-5ac5-4f36-b9ef-8ebbc97a3f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:18.254549 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a3c54-5ac5-4f36-b9ef-8ebbc97a3f6c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:18.259396 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 651/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:05:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:20.217179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31aa2-f0c3-4485-a8ce-f374048d82b4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:20.217746 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31aa2-f0c3-4485-a8ce-f374048d82b4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:20.217746 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31aa2-f0c3-4485-a8ce-f374048d82b4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:20.219155 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 642/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:05:20.224874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5068-d3c0-4cf2-a0bf-f103c6b5fca3 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:20.225323 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5068-d3c0-4cf2-a0bf-f103c6b5fca3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:20.225413 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5068-d3c0-4cf2-a0bf-f103c6b5fca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:20.247174 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-be9a5068-d3c0-4cf2-a0bf-f103c6b5fca3 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:05:20.248717 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-be9a5068-d3c0-4cf2-a0bf-f103c6b5fca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:05:20.284474 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:20.326670 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 652/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:05:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:05:20.352862 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d31aa2-f0c3-4485-a8ce-f374048d82b4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:20.389158 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:20.466196 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d31aa2-f0c3-4485-a8ce-f374048d82b4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:20.502140 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6fa9-8369-430f-ba7e-9658e0db6c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:20.502700 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6fa9-8369-430f-ba7e-9658e0db6c03 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:20.502983 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6fa9-8369-430f-ba7e-9658e0db6c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:20.503257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6fa9-8369-430f-ba7e-9658e0db6c03 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:20.503957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b6fa9-8369-430f-ba7e-9658e0db6c03 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:20.505350 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b6fa9-8369-430f-ba7e-9658e0db6c03 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:20.521679 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 643/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:05:20 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:05:22.563835 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf9f2-220d-4f25-aab5-006cbdddd28b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:22.564429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf9f2-220d-4f25-aab5-006cbdddd28b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:22.564429 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf9f2-220d-4f25-aab5-006cbdddd28b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:22.565769 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 653/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:05:22.571557 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a687365-c955-4440-b6dd-ab3404c78640 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:22.572002 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a687365-c955-4440-b6dd-ab3404c78640 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:22.572091 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a687365-c955-4440-b6dd-ab3404c78640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:22.593464 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8a687365-c955-4440-b6dd-ab3404c78640 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:05:22.594726 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8a687365-c955-4440-b6dd-ab3404c78640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:05:22.671528 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 644/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:05:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:05:22.694954 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daf9f2-220d-4f25-aab5-006cbdddd28b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:22.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daf9f2-220d-4f25-aab5-006cbdddd28b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:22.844183 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:22.844736 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:22.845010 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:22.845277 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:22.845916 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:22.847314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:22.856764 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:22.858295 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:22.859837 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 654/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:05:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:22.864270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a687365-c955-4440-b6dd-ab3404c78640 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:22.883256 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:22.980341 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a687365-c955-4440-b6dd-ab3404c78640 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:23.035885 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:23.036549 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:23.036826 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:23.037095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:23.037824 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:23.038154 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:23.039637 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:23.046727 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 645/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:05:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:23.055369 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:23.096854 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:23.182657 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20119efb-edf7-46f1-b26d-c72525dcb73f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:23.227450 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882899e9-a95d-49b3-a975-478f5e6a1554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:23.228098 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882899e9-a95d-49b3-a975-478f5e6a1554 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:23.228378 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882899e9-a95d-49b3-a975-478f5e6a1554 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:23.228648 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882899e9-a95d-49b3-a975-478f5e6a1554 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:23.229446 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882899e9-a95d-49b3-a975-478f5e6a1554 None admin] RBAC: Adding query filter params (interface=public, service_id=ac3ef96d001b412e99d38dec4c42da7c, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:23.229767 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882899e9-a95d-49b3-a975-478f5e6a1554 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:23.231277 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882899e9-a95d-49b3-a975-478f5e6a1554 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:23.237889 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 655/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:05:23 2022] GET /identity/v3/endpoints?service_id=ac3ef96d001b412e99d38dec4c42da7c&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:23.244343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:23.345085 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:23.369854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe685d42-fa4c-46f1-b090-ca83532a7be0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:23.416799 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff8b58-d555-45f7-9127-60c8a5425a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:23.417449 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8b58-d555-45f7-9127-60c8a5425a80 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:23.417732 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8b58-d555-45f7-9127-60c8a5425a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:23.418006 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8b58-d555-45f7-9127-60c8a5425a80 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:23.418730 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff8b58-d555-45f7-9127-60c8a5425a80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:23.420177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff8b58-d555-45f7-9127-60c8a5425a80 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:23.428406 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 646/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:05:23 2022] GET /identity/v3/services => generated 2191 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:05:25.427363 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc30b0-eb32-4c24-966c-b78818c3c4a7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:25.428156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc30b0-eb32-4c24-966c-b78818c3c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:25.428156 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc30b0-eb32-4c24-966c-b78818c3c4a7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:25.430323 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 656/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:05:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:05:25.439032 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea0b28-2ee9-45b6-9b7f-5c03878e39b2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:25.439434 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea0b28-2ee9-45b6-9b7f-5c03878e39b2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:25.439660 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea0b28-2ee9-45b6-9b7f-5c03878e39b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:25.465294 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0bea0b28-2ee9-45b6-9b7f-5c03878e39b2 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:05:25.466573 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0bea0b28-2ee9-45b6-9b7f-5c03878e39b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:05:25.549489 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 647/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:05:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:05:25.571816 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc30b0-eb32-4c24-966c-b78818c3c4a7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:25.672814 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:25.680179 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc30b0-eb32-4c24-966c-b78818c3c4a7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:25.717042 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:25.717783 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:25.718066 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:25.718370 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:25.718994 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:25.720389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:05:25.729527 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:05:25.730622 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 28 08:05:25.731615 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 657/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:05:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:05:25.737320 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea0b28-2ee9-45b6-9b7f-5c03878e39b2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:25.816529 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:25.842082 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea0b28-2ee9-45b6-9b7f-5c03878e39b2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:25.875888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986b3bd-c78d-41e4-9ce6-f56446978acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:25.876513 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b3bd-c78d-41e4-9ce6-f56446978acd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:25.876797 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b3bd-c78d-41e4-9ce6-f56446978acd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:25.877074 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b3bd-c78d-41e4-9ce6-f56446978acd None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:25.877753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b3bd-c78d-41e4-9ce6-f56446978acd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:05:25.878061 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b3bd-c78d-41e4-9ce6-f56446978acd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:25.879483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b3bd-c78d-41e4-9ce6-f56446978acd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:25.884839 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 648/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:05:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:05:25.891040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:05:26.011633 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a432-d9be-4f1f-bcca-ac706eb148f2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:05:26.044225 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd3ea6-bcc5-40d7-bbdf-8acd1ab77c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf187c6216740528137e4e8f0aad7b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:05:26.044844 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd3ea6-bcc5-40d7-bbdf-8acd1ab77c53 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:05:26.045120 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd3ea6-bcc5-40d7-bbdf-8acd1ab77c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:05:26.045395 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd3ea6-bcc5-40d7-bbdf-8acd1ab77c53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:05:26.046150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd3ea6-bcc5-40d7-bbdf-8acd1ab77c53 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:05:26.047517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd3ea6-bcc5-40d7-bbdf-8acd1ab77c53 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:05:26.068680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 658/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:05:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:05:35.881707 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=30 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:05:36.064894 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:30.332863 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb3fdc-142a-42ae-aad0-ab3eed59cfd5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:30.333343 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb3fdc-142a-42ae-aad0-ab3eed59cfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:30.333444 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb3fdc-142a-42ae-aad0-ab3eed59cfd5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:30.338696 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 649/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:30.340090 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661ce29-5052-4952-8a8d-f80f27a66baf None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:30.340508 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661ce29-5052-4952-8a8d-f80f27a66baf None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:30.340630 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661ce29-5052-4952-8a8d-f80f27a66baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:30.456855 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6661ce29-5052-4952-8a8d-f80f27a66baf None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:30.459003 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6661ce29-5052-4952-8a8d-f80f27a66baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:30.596749 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 659/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:30.646970 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0db3fc-2b2e-4c9b-861d-8ed317e505e1 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:30.647311 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0db3fc-2b2e-4c9b-861d-8ed317e505e1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:30.647508 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0db3fc-2b2e-4c9b-861d-8ed317e505e1 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:30.649246 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 650/1309] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 08:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:30.656131 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afa927-eebe-4616-9c69-c61fad0b74e3 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:30.656424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afa927-eebe-4616-9c69-c61fad0b74e3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:30.656700 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afa927-eebe-4616-9c69-c61fad0b74e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:30.753349 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-90afa927-eebe-4616-9c69-c61fad0b74e3 None None] MFA Rules not processed for user `2ae2e70a28824ac58182627a8c7bf702`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:30.754771 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-90afa927-eebe-4616-9c69-c61fad0b74e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:30.780918 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:30.855642 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 660/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 08:06:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:30.864801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0db3fc-2b2e-4c9b-861d-8ed317e505e1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:30.867397 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1f0db3fc-2b2e-4c9b-861d-8ed317e505e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:30.997157 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0db3fc-2b2e-4c9b-861d-8ed317e505e1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:31.078940 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e1c9b-b4bd-4895-b045-1d08ed83b62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:31.079480 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1c9b-b4bd-4895-b045-1d08ed83b62b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:31.079757 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1c9b-b4bd-4895-b045-1d08ed83b62b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:31.080025 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1c9b-b4bd-4895-b045-1d08ed83b62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:31.081701 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2c4e1c9b-b4bd-4895-b045-1d08ed83b62b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:31.137975 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:31.206557 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e1c9b-b4bd-4895-b045-1d08ed83b62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:31.208056 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e1c9b-b4bd-4895-b045-1d08ed83b62b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:31.231312 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 651/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:33.513472 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d798-c00f-4e2c-bbc5-9f2115dce89a None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:33.514035 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d798-c00f-4e2c-bbc5-9f2115dce89a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:33.514035 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d798-c00f-4e2c-bbc5-9f2115dce89a None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:33.515598 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 661/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:33.520968 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f4ce-979b-4a60-ba9b-9312045cdc93 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:33.521401 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f4ce-979b-4a60-ba9b-9312045cdc93 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:33.521491 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f4ce-979b-4a60-ba9b-9312045cdc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:33.615345 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4077f4ce-979b-4a60-ba9b-9312045cdc93 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:33.616569 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4077f4ce-979b-4a60-ba9b-9312045cdc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:33.730389 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 652/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:33.781211 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a025e-f2f0-4b45-bcec-344e2ac96b32 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:33.781947 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a025e-f2f0-4b45-bcec-344e2ac96b32 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:33.782269 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a025e-f2f0-4b45-bcec-344e2ac96b32 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:33.783716 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 662/1314] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 08:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:33.790895 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4588ac3-69e6-4b0f-8b89-5ef49b6f24c9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:33.791252 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4588ac3-69e6-4b0f-8b89-5ef49b6f24c9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:33.791471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4588ac3-69e6-4b0f-8b89-5ef49b6f24c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:33.804877 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:33.885106 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e4588ac3-69e6-4b0f-8b89-5ef49b6f24c9 None None] MFA Rules not processed for user `2ae2e70a28824ac58182627a8c7bf702`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:33.886429 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e4588ac3-69e6-4b0f-8b89-5ef49b6f24c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:33.977516 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 653/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 08:06:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:33.987735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a025e-f2f0-4b45-bcec-344e2ac96b32 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:33.989361 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-524a025e-f2f0-4b45-bcec-344e2ac96b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:34.109989 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a025e-f2f0-4b45-bcec-344e2ac96b32 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:34.190539 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:34.193481 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1bb19-3367-4ac0-9e74-646958bbbab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:34.194437 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1bb19-3367-4ac0-9e74-646958bbbab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:34.194729 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1bb19-3367-4ac0-9e74-646958bbbab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:34.195236 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1bb19-3367-4ac0-9e74-646958bbbab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:34.197135 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-50a1bb19-3367-4ac0-9e74-646958bbbab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:34.310703 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1bb19-3367-4ac0-9e74-646958bbbab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:34.312061 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1bb19-3367-4ac0-9e74-646958bbbab4 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:34.331142 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 663/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:36.836414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250e99-9006-4e8e-a345-d9f11c5addbe None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:36.837030 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250e99-9006-4e8e-a345-d9f11c5addbe None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:36.837030 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250e99-9006-4e8e-a345-d9f11c5addbe None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:36.838351 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 654/1317] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:36.844180 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5d485-c89d-41bb-a3ef-170fad2d4bcf None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:36.844664 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5d485-c89d-41bb-a3ef-170fad2d4bcf None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:36.844765 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5d485-c89d-41bb-a3ef-170fad2d4bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:36.913057 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1ef5d485-c89d-41bb-a3ef-170fad2d4bcf None None] MFA Rules not processed for user `0ed09815b5214d75993e263bb3fd05c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:36.914333 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1ef5d485-c89d-41bb-a3ef-170fad2d4bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:36.972935 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:37.020639 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 664/1318] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:37.060017 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08250e99-9006-4e8e-a345-d9f11c5addbe None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:37.061596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-08250e99-9006-4e8e-a345-d9f11c5addbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:37.172157 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08250e99-9006-4e8e-a345-d9f11c5addbe None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:37.175090 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:37.250123 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541e72f-228c-49e6-81ba-689894416c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:37.250758 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541e72f-228c-49e6-81ba-689894416c5d service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:37.251039 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541e72f-228c-49e6-81ba-689894416c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:37.251310 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541e72f-228c-49e6-81ba-689894416c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:37.252960 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a541e72f-228c-49e6-81ba-689894416c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:37.371179 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541e72f-228c-49e6-81ba-689894416c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:37.372600 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541e72f-228c-49e6-81ba-689894416c5d service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:37.392095 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 655/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:39.980182 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d8e2-33a5-4eeb-b323-77a85edeaf68 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:39.980737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d8e2-33a5-4eeb-b323-77a85edeaf68 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:39.980737 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d8e2-33a5-4eeb-b323-77a85edeaf68 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:39.982049 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 665/1320] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:39.987414 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1df592-9530-45cd-9b8b-36f59414d22f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:39.987854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1df592-9530-45cd-9b8b-36f59414d22f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:39.987957 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1df592-9530-45cd-9b8b-36f59414d22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:40.029990 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:40.057636 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8f1df592-9530-45cd-9b8b-36f59414d22f None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:40.058900 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8f1df592-9530-45cd-9b8b-36f59414d22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:40.159181 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 656/1321] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:40.197680 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d8e2-33a5-4eeb-b323-77a85edeaf68 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:40.199658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a866d8e2-33a5-4eeb-b323-77a85edeaf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:40.311030 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d8e2-33a5-4eeb-b323-77a85edeaf68 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:40.386868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff6954-347f-41a2-a2d8-70054f6dcdea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:40.387547 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff6954-347f-41a2-a2d8-70054f6dcdea service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:40.387779 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff6954-347f-41a2-a2d8-70054f6dcdea service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:40.388058 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff6954-347f-41a2-a2d8-70054f6dcdea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:40.389724 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-08ff6954-347f-41a2-a2d8-70054f6dcdea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:40.397961 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:40.506088 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff6954-347f-41a2-a2d8-70054f6dcdea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:40.507592 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff6954-347f-41a2-a2d8-70054f6dcdea service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:40.526643 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 666/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:43.272830 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f4dd-a227-4d12-a150-3b6f1a39749d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:43.273416 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f4dd-a227-4d12-a150-3b6f1a39749d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:43.273416 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f4dd-a227-4d12-a150-3b6f1a39749d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:43.274801 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 657/1323] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:43.280260 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194898c8-2d0e-4edb-b1a1-8833ab117a22 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:43.280739 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194898c8-2d0e-4edb-b1a1-8833ab117a22 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:43.280838 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194898c8-2d0e-4edb-b1a1-8833ab117a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:43.355185 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-194898c8-2d0e-4edb-b1a1-8833ab117a22 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:43.356577 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-194898c8-2d0e-4edb-b1a1-8833ab117a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:43.422125 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:43.467642 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 667/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:43.505015 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8f4dd-a227-4d12-a150-3b6f1a39749d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:43.506604 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-03f8f4dd-a227-4d12-a150-3b6f1a39749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:43.625587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8f4dd-a227-4d12-a150-3b6f1a39749d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:43.639444 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:43.709623 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eca2c8-ed3f-4898-8c23-0fb20dd49d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:43.710374 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eca2c8-ed3f-4898-8c23-0fb20dd49d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:43.710711 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eca2c8-ed3f-4898-8c23-0fb20dd49d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:43.711048 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eca2c8-ed3f-4898-8c23-0fb20dd49d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:43.712737 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-98eca2c8-ed3f-4898-8c23-0fb20dd49d82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:43.828769 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eca2c8-ed3f-4898-8c23-0fb20dd49d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:43.830270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eca2c8-ed3f-4898-8c23-0fb20dd49d82 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:43.856419 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 658/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:45.947502 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a787cc-a4cf-451a-a073-5c669e182ea7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:45.948193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a787cc-a4cf-451a-a073-5c669e182ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:45.948193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a787cc-a4cf-451a-a073-5c669e182ea7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:45.950151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 668/1326] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:45.956735 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359feb-f819-42b5-85de-da7a411f11ad None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:45.957223 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359feb-f819-42b5-85de-da7a411f11ad None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:45.957483 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359feb-f819-42b5-85de-da7a411f11ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:46.018094 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:46.037359 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-26359feb-f819-42b5-85de-da7a411f11ad None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:46.038643 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-26359feb-f819-42b5-85de-da7a411f11ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:46.144607 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 659/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:46.180766 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a787cc-a4cf-451a-a073-5c669e182ea7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:46.182348 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-08a787cc-a4cf-451a-a073-5c669e182ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:46.308520 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a787cc-a4cf-451a-a073-5c669e182ea7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:46.394121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf12df-838c-4a30-be9c-17d060a0ed58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:46.394768 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf12df-838c-4a30-be9c-17d060a0ed58 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:46.395054 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf12df-838c-4a30-be9c-17d060a0ed58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:46.395325 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf12df-838c-4a30-be9c-17d060a0ed58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:46.396980 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8cbf12df-838c-4a30-be9c-17d060a0ed58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:46.415848 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:46.514468 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf12df-838c-4a30-be9c-17d060a0ed58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:46.515922 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf12df-838c-4a30-be9c-17d060a0ed58 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:46.535709 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 669/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:48.599251 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e6588-29b5-43fa-8531-4ca20f2954b5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:48.599838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e6588-29b5-43fa-8531-4ca20f2954b5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:48.599838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e6588-29b5-43fa-8531-4ca20f2954b5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:48.601178 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 660/1329] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:48.606676 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82072-01fe-44c7-8d51-f4551b3defd5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:48.607136 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82072-01fe-44c7-8d51-f4551b3defd5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:48.607246 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82072-01fe-44c7-8d51-f4551b3defd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:48.686355 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-89b82072-01fe-44c7-8d51-f4551b3defd5 None None] MFA Rules not processed for user `0ed09815b5214d75993e263bb3fd05c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:48.687632 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-89b82072-01fe-44c7-8d51-f4551b3defd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:48.763456 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:48.801338 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 670/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:48 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:48.839274 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e6588-29b5-43fa-8531-4ca20f2954b5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:48.840852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5f1e6588-29b5-43fa-8531-4ca20f2954b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:48.953013 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e6588-29b5-43fa-8531-4ca20f2954b5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:48.976922 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:49.038450 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233aed-cd48-4715-832d-88283de9234c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:49.039079 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233aed-cd48-4715-832d-88283de9234c service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:49.039355 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233aed-cd48-4715-832d-88283de9234c service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:49.039609 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233aed-cd48-4715-832d-88283de9234c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:49.041244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-03233aed-cd48-4715-832d-88283de9234c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:49.153932 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233aed-cd48-4715-832d-88283de9234c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:49.155435 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233aed-cd48-4715-832d-88283de9234c service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:49.176796 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 661/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:51.607178 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b312e-8dbd-46ba-9dc8-2bd0afac1888 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:51.607748 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b312e-8dbd-46ba-9dc8-2bd0afac1888 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:51.607748 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b312e-8dbd-46ba-9dc8-2bd0afac1888 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:51.609020 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 671/1332] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:51.614698 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a87b10-ea66-4e2d-aff3-5e88ca4a31eb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:51.615113 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a87b10-ea66-4e2d-aff3-5e88ca4a31eb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:51.615244 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a87b10-ea66-4e2d-aff3-5e88ca4a31eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:51.681276 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:51.684053 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d6a87b10-ea66-4e2d-aff3-5e88ca4a31eb None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:51.685384 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d6a87b10-ea66-4e2d-aff3-5e88ca4a31eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:51.792747 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 662/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:51.829882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b312e-8dbd-46ba-9dc8-2bd0afac1888 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:51.831496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-416b312e-8dbd-46ba-9dc8-2bd0afac1888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:51.944193 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b312e-8dbd-46ba-9dc8-2bd0afac1888 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:52.021735 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a3df6-9974-4798-a20a-60ae0e4ba62e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:52.022389 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3df6-9974-4798-a20a-60ae0e4ba62e service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:52.022665 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3df6-9974-4798-a20a-60ae0e4ba62e service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:52.022930 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3df6-9974-4798-a20a-60ae0e4ba62e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:52.024562 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-494a3df6-9974-4798-a20a-60ae0e4ba62e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:52.052252 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:52.140868 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a3df6-9974-4798-a20a-60ae0e4ba62e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:52.142345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a3df6-9974-4798-a20a-60ae0e4ba62e service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:52.161677 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 672/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:54.198127 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29632f-1518-4b8e-bf25-5d1dbfa5bc8a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:54.199045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29632f-1518-4b8e-bf25-5d1dbfa5bc8a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:54.199045 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29632f-1518-4b8e-bf25-5d1dbfa5bc8a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:54.200587 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 663/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:54.206388 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0798c-faab-46e2-a095-6d23d771dc01 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:54.206882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0798c-faab-46e2-a095-6d23d771dc01 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:54.206882 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0798c-faab-46e2-a095-6d23d771dc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:54.286166 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-06b0798c-faab-46e2-a095-6d23d771dc01 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:54.287494 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-06b0798c-faab-46e2-a095-6d23d771dc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:54.376560 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:54.400247 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 673/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:06:54.441410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29632f-1518-4b8e-bf25-5d1dbfa5bc8a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:54.443176 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1f29632f-1518-4b8e-bf25-5d1dbfa5bc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:54.557329 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29632f-1518-4b8e-bf25-5d1dbfa5bc8a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:54.589409 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:54.640093 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4e899-2f80-4494-81cd-a094666d6145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:54.640628 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4e899-2f80-4494-81cd-a094666d6145 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:54.640904 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4e899-2f80-4494-81cd-a094666d6145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:54.641177 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4e899-2f80-4494-81cd-a094666d6145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:54.642830 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e1a4e899-2f80-4494-81cd-a094666d6145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:54.760076 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4e899-2f80-4494-81cd-a094666d6145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:54.761535 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4e899-2f80-4494-81cd-a094666d6145 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:54.780366 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 664/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:06:57.181371 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f7df6-644d-4f31-99d7-2bfd53650aef None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:57.181970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f7df6-644d-4f31-99d7-2bfd53650aef None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:57.181970 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f7df6-644d-4f31-99d7-2bfd53650aef None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:57.183345 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 674/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:06:57.189025 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb97eb-9e13-47dd-bcad-caca90ef2ef0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:57.189538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb97eb-9e13-47dd-bcad-caca90ef2ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:57.189538 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb97eb-9e13-47dd-bcad-caca90ef2ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:57.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-dedb97eb-9e13-47dd-bcad-caca90ef2ef0 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:06:57.265930 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dedb97eb-9e13-47dd-bcad-caca90ef2ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:57.275380 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:57.372364 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 665/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:06:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:06:57.408596 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f7df6-644d-4f31-99d7-2bfd53650aef None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:06:57.410221 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0d9f7df6-644d-4f31-99d7-2bfd53650aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:57.520895 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f7df6-644d-4f31-99d7-2bfd53650aef None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:06:57.603134 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277de28-9d2d-4b10-b14d-cedbf6ec1e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:06:57.603770 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277de28-9d2d-4b10-b14d-cedbf6ec1e8e service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:06:57.604051 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277de28-9d2d-4b10-b14d-cedbf6ec1e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:06:57.604297 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277de28-9d2d-4b10-b14d-cedbf6ec1e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:06:57.606083 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6277de28-9d2d-4b10-b14d-cedbf6ec1e8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:06:57.643337 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:06:57.721673 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277de28-9d2d-4b10-b14d-cedbf6ec1e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:06:57.723172 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277de28-9d2d-4b10-b14d-cedbf6ec1e8e service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:06:57.743050 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 675/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:06:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:07:01.745438 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d740b0-5255-4454-a81a-c5186ecc0644 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:01.745949 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d740b0-5255-4454-a81a-c5186ecc0644 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:01.746038 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d740b0-5255-4454-a81a-c5186ecc0644 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:01.747527 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 666/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:01.752808 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5fb6-7868-44a7-b50c-e3a997a8baec None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:01.753230 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5fb6-7868-44a7-b50c-e3a997a8baec None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:01.753342 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5fb6-7868-44a7-b50c-e3a997a8baec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:01.822041 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-bd3f5fb6-7868-44a7-b50c-e3a997a8baec None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:01.823202 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bd3f5fb6-7868-44a7-b50c-e3a997a8baec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:01.916583 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:01.929832 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 676/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:01.970709 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d740b0-5255-4454-a81a-c5186ecc0644 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:07:01.972261 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-07d740b0-5255-4454-a81a-c5186ecc0644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:02.089055 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d740b0-5255-4454-a81a-c5186ecc0644 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:07:02.130190 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:02.171237 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f63123-0a8d-4e39-9914-44e48cffd0cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:07:02.172039 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f63123-0a8d-4e39-9914-44e48cffd0cd service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:02.172410 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f63123-0a8d-4e39-9914-44e48cffd0cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:02.172768 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f63123-0a8d-4e39-9914-44e48cffd0cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:02.174653 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f5f63123-0a8d-4e39-9914-44e48cffd0cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:02.293038 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f63123-0a8d-4e39-9914-44e48cffd0cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:07:02.295100 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f63123-0a8d-4e39-9914-44e48cffd0cd service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:07:02.314449 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 667/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:07:05.018893 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819d9d7-9c2b-4d5b-b7e5-aa3091984ace None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:05.019440 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819d9d7-9c2b-4d5b-b7e5-aa3091984ace None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:05.019440 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819d9d7-9c2b-4d5b-b7e5-aa3091984ace None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:05.020923 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 677/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:05.026793 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c6c70-2195-4af4-8962-b4b7d13254da None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:05.027471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c6c70-2195-4af4-8962-b4b7d13254da None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:05.027471 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c6c70-2195-4af4-8962-b4b7d13254da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:05.105236 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a54c6c70-2195-4af4-8962-b4b7d13254da None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:05.106662 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a54c6c70-2195-4af4-8962-b4b7d13254da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:05.126869 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:05.210616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 668/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:07:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:05.247572 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819d9d7-9c2b-4d5b-b7e5-aa3091984ace None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:07:05.249150 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d819d9d7-9c2b-4d5b-b7e5-aa3091984ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:05.363579 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819d9d7-9c2b-4d5b-b7e5-aa3091984ace None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:07:05.445614 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e74d33-bc25-414d-b134-b72728597118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:07:05.446474 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74d33-bc25-414d-b134-b72728597118 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:05.446762 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74d33-bc25-414d-b134-b72728597118 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:05.447040 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74d33-bc25-414d-b134-b72728597118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:05.449109 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-23e74d33-bc25-414d-b134-b72728597118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:05.496975 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:05.566683 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e74d33-bc25-414d-b134-b72728597118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:07:05.568137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e74d33-bc25-414d-b134-b72728597118 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:07:05.587459 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 678/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:07:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:07:09.978519 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8dc5c-7dd0-41a0-8c54-98e7f8f59a14 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:09.979139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8dc5c-7dd0-41a0-8c54-98e7f8f59a14 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:09.979139 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8dc5c-7dd0-41a0-8c54-98e7f8f59a14 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:09.980740 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 669/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:09.987039 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e6beb-7cc7-49a4-8f68-a2429b9c2f0e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:09.987600 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e6beb-7cc7-49a4-8f68-a2429b9c2f0e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:09.987600 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e6beb-7cc7-49a4-8f68-a2429b9c2f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:10.063898 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-814e6beb-7cc7-49a4-8f68-a2429b9c2f0e None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:10.065137 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-814e6beb-7cc7-49a4-8f68-a2429b9c2f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:10.181911 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:10.190364 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 679/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:10.224596 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8dc5c-7dd0-41a0-8c54-98e7f8f59a14 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:07:10.226400 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-eef8dc5c-7dd0-41a0-8c54-98e7f8f59a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:10.348621 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8dc5c-7dd0-41a0-8c54-98e7f8f59a14 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:07:10.401184 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:10.431838 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1de70-8811-43a1-8853-4c51bc84992b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae2e70a28824ac58182627a8c7bf702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:07:10.432486 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1de70-8811-43a1-8853-4c51bc84992b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:10.432765 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1de70-8811-43a1-8853-4c51bc84992b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:10.433038 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1de70-8811-43a1-8853-4c51bc84992b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:10.434812 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c5e1de70-8811-43a1-8853-4c51bc84992b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:10.557096 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e1de70-8811-43a1-8853-4c51bc84992b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:07:10.558657 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e1de70-8811-43a1-8853-4c51bc84992b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:07:10.578270 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 670/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:07:15.425354 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b9910-80b8-4bdb-b5bb-38b91ae13737 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:15.425874 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b9910-80b8-4bdb-b5bb-38b91ae13737 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:15.425955 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b9910-80b8-4bdb-b5bb-38b91ae13737 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:15.427732 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 680/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:07:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:15.433263 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db119-7eb6-4bfa-a63c-ea7ba77c6a40 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:15.433776 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db119-7eb6-4bfa-a63c-ea7ba77c6a40 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:15.433888 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db119-7eb6-4bfa-a63c-ea7ba77c6a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:15.575790 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-033db119-7eb6-4bfa-a63c-ea7ba77c6a40 None None] MFA Rules not processed for user `21d16f9ab0ae45b69b495ed579f66521`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:15.576960 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-033db119-7eb6-4bfa-a63c-ea7ba77c6a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:15.616500 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:15.757535 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 671/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:07:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:19.964055 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3eae-d9be-4b94-a068-e03a8b255eeb None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:19.965409 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3eae-d9be-4b94-a068-e03a8b255eeb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:19.966097 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3eae-d9be-4b94-a068-e03a8b255eeb None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:19.968347 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 681/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:07:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:19.973534 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8eb3b1-0d2b-46b8-81c3-d7dcc5ce3c06 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:19.974389 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8eb3b1-0d2b-46b8-81c3-d7dcc5ce3c06 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:19.974748 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8eb3b1-0d2b-46b8-81c3-d7dcc5ce3c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:20.078156 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fe8eb3b1-0d2b-46b8-81c3-d7dcc5ce3c06 None None] MFA Rules not processed for user `21d16f9ab0ae45b69b495ed579f66521`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:20.080069 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fe8eb3b1-0d2b-46b8-81c3-d7dcc5ce3c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:20.210526 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:20.281504 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 672/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:07:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:21.523267 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6feff-8150-424b-a753-23211abebd61 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:21.523804 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6feff-8150-424b-a753-23211abebd61 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:21.523804 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6feff-8150-424b-a753-23211abebd61 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:21.525185 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 682/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:21.532371 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66b6b4-9149-4a79-8836-add18a1efbd7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:21.532847 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66b6b4-9149-4a79-8836-add18a1efbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:21.533058 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66b6b4-9149-4a79-8836-add18a1efbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:21.628719 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6f66b6b4-9149-4a79-8836-add18a1efbd7 None None] MFA Rules not processed for user `21d16f9ab0ae45b69b495ed579f66521`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:21.629955 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6f66b6b4-9149-4a79-8836-add18a1efbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:21.739854 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 673/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:07:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:25.086773 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6d3dc-e964-423b-b06d-08a7175c97b4 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:25.087314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6d3dc-e964-423b-b06d-08a7175c97b4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:25.087314 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6d3dc-e964-423b-b06d-08a7175c97b4 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:25.089307 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 683/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:25.099365 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22330b-3b56-43a6-8cba-322a4669e18d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:25.099890 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22330b-3b56-43a6-8cba-322a4669e18d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:25.099890 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22330b-3b56-43a6-8cba-322a4669e18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:25.152417 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:25.192764 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6c22330b-3b56-43a6-8cba-322a4669e18d None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:25.193992 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6c22330b-3b56-43a6-8cba-322a4669e18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:25.311300 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 674/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:25.369301 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9a4a-2f06-4a28-85a3-b0e4cbaa86b4 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:25.369975 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9a4a-2f06-4a28-85a3-b0e4cbaa86b4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:25.370343 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9a4a-2f06-4a28-85a3-b0e4cbaa86b4 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:25.371822 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 684/1358] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 08:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:25.380497 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0757f9-c637-41b9-8184-6426c02bf58d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:25.380784 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0757f9-c637-41b9-8184-6426c02bf58d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:25.381060 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0757f9-c637-41b9-8184-6426c02bf58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:25.475039 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8d0757f9-c637-41b9-8184-6426c02bf58d None None] MFA Rules not processed for user `4527e4500f984b1a81214c3656e7947f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:25.476295 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8d0757f9-c637-41b9-8184-6426c02bf58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:25.531665 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:25.617534 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 675/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 08:07:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:25.625877 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9a4a-2f06-4a28-85a3-b0e4cbaa86b4 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:07:25.627799 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-61ea9a4a-2f06-4a28-85a3-b0e4cbaa86b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:25.798496 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9a4a-2f06-4a28-85a3-b0e4cbaa86b4 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:07:25.879787 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:25.923151 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb346-74f0-4d3a-b650-eab9f9e363e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4527e4500f984b1a81214c3656e7947f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:07:25.923788 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb346-74f0-4d3a-b650-eab9f9e363e0 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:25.924187 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb346-74f0-4d3a-b650-eab9f9e363e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:25.924359 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb346-74f0-4d3a-b650-eab9f9e363e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:25.926067 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-100cb346-74f0-4d3a-b650-eab9f9e363e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:26.028260 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cb346-74f0-4d3a-b650-eab9f9e363e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:07:26.030022 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cb346-74f0-4d3a-b650-eab9f9e363e0 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:07:26.049319 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 685/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:07:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:07:28.271907 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09714a6e-ba52-4966-8999-3df0f9dac8b3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:28.272512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09714a6e-ba52-4966-8999-3df0f9dac8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:28.272512 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09714a6e-ba52-4966-8999-3df0f9dac8b3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:28.277689 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 676/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:07:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:28.279667 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817c2a-d07b-4ddd-bcae-dfea47570803 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:28.280160 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817c2a-d07b-4ddd-bcae-dfea47570803 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:28.280420 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817c2a-d07b-4ddd-bcae-dfea47570803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:28.356746 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0b817c2a-d07b-4ddd-bcae-dfea47570803 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:28.358027 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0b817c2a-d07b-4ddd-bcae-dfea47570803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:28.399556 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:28.473738 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 686/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:28.512927 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09714a6e-ba52-4966-8999-3df0f9dac8b3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:07:28.514348 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-09714a6e-ba52-4966-8999-3df0f9dac8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:28.678616 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09714a6e-ba52-4966-8999-3df0f9dac8b3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:07:28.688817 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:28.808759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553b53c-6cc3-440b-a2ca-9ca0746ccfa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4527e4500f984b1a81214c3656e7947f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:07:28.808759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553b53c-6cc3-440b-a2ca-9ca0746ccfa0 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:28.808759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553b53c-6cc3-440b-a2ca-9ca0746ccfa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:28.808759 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553b53c-6cc3-440b-a2ca-9ca0746ccfa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:28.811852 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6553b53c-6cc3-440b-a2ca-9ca0746ccfa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:28.911633 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553b53c-6cc3-440b-a2ca-9ca0746ccfa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:07:28.913420 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553b53c-6cc3-440b-a2ca-9ca0746ccfa0 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:07:28.933398 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 677/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:07:31.272102 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101c965-3eab-4b70-81f4-b73942e3eebb None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:31.272658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101c965-3eab-4b70-81f4-b73942e3eebb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:31.272658 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101c965-3eab-4b70-81f4-b73942e3eebb None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:31.274198 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 687/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:31.279663 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bfb9c-ffa5-402a-9964-955dbea0bbfd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:31.280125 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bfb9c-ffa5-402a-9964-955dbea0bbfd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:31.280481 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bfb9c-ffa5-402a-9964-955dbea0bbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:31.297102 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:31.351234 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ca5bfb9c-ffa5-402a-9964-955dbea0bbfd None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:31.352581 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ca5bfb9c-ffa5-402a-9964-955dbea0bbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:31.458290 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 678/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:31.504510 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee2ac-228c-42fd-86eb-c62e09ddb8bd None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:31.504936 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee2ac-228c-42fd-86eb-c62e09ddb8bd None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:31.505451 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee2ac-228c-42fd-86eb-c62e09ddb8bd None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:31.507178 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 688/1366] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 08:07:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:31.512943 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2334a2-9070-4c38-9c6b-49c7c395abfd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:31.513224 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2334a2-9070-4c38-9c6b-49c7c395abfd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:31.513507 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2334a2-9070-4c38-9c6b-49c7c395abfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:31.592536 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1d2334a2-9070-4c38-9c6b-49c7c395abfd None None] MFA Rules not processed for user `4527e4500f984b1a81214c3656e7947f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:31.593682 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1d2334a2-9070-4c38-9c6b-49c7c395abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:31.630305 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:31.719530 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 679/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 08:07:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:31.727692 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ee2ac-228c-42fd-86eb-c62e09ddb8bd None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:07:31.729550 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-282ee2ac-228c-42fd-86eb-c62e09ddb8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:31.894915 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ee2ac-228c-42fd-86eb-c62e09ddb8bd None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:07:31.921479 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:32.035382 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5eb3a-2be6-4cb7-9225-22549933d965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4527e4500f984b1a81214c3656e7947f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:07:32.036116 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5eb3a-2be6-4cb7-9225-22549933d965 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:32.036460 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5eb3a-2be6-4cb7-9225-22549933d965 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:32.036804 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5eb3a-2be6-4cb7-9225-22549933d965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:32.038553 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8dd5eb3a-2be6-4cb7-9225-22549933d965 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:32.182211 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5eb3a-2be6-4cb7-9225-22549933d965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:07:32.184092 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5eb3a-2be6-4cb7-9225-22549933d965 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:07:32.218937 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 689/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:07:32.816136 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2f176-4200-41c8-9f75-34ebdfa7e58d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:32.816708 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2f176-4200-41c8-9f75-34ebdfa7e58d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:32.817110 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2f176-4200-41c8-9f75-34ebdfa7e58d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:32.819051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 680/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:32.827037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815c1d1-d32b-420e-b6d1-e3e6202cbf27 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:32.827037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815c1d1-d32b-420e-b6d1-e3e6202cbf27 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:32.827037 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815c1d1-d32b-420e-b6d1-e3e6202cbf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:32.863331 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:32.904920 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e815c1d1-d32b-420e-b6d1-e3e6202cbf27 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:32.906191 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e815c1d1-d32b-420e-b6d1-e3e6202cbf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:33.027082 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 690/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:34.377257 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d405b1-577c-458b-9fdc-63f04c41a06b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:34.377887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d405b1-577c-458b-9fdc-63f04c41a06b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:34.377887 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d405b1-577c-458b-9fdc-63f04c41a06b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:34.379436 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 681/1371] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:07:34.385691 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa699c27-61a4-4b6a-9af2-2421139f3fe6 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:34.386311 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa699c27-61a4-4b6a-9af2-2421139f3fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:34.386424 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa699c27-61a4-4b6a-9af2-2421139f3fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:34.453517 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-fa699c27-61a4-4b6a-9af2-2421139f3fe6 None None] MFA Rules not processed for user `acf187c6216740528137e4e8f0aad7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:07:34.454521 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fa699c27-61a4-4b6a-9af2-2421139f3fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:34.510824 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:34.555930 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 691/1372] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:07:34.601051 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d405b1-577c-458b-9fdc-63f04c41a06b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:07:34.602658 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a6d405b1-577c-458b-9fdc-63f04c41a06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:34.758417 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:34.767300 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d405b1-577c-458b-9fdc-63f04c41a06b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:07:34.983388 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402416f-6974-4e11-a1b9-efdb5016cdd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4527e4500f984b1a81214c3656e7947f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f0819449c034a08b5a366324ca3a3f4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:07:34.984434 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402416f-6974-4e11-a1b9-efdb5016cdd5 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:07:34.986245 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402416f-6974-4e11-a1b9-efdb5016cdd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:07:34.986753 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402416f-6974-4e11-a1b9-efdb5016cdd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:07:35.014890 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a402416f-6974-4e11-a1b9-efdb5016cdd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:07:35.183492 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402416f-6974-4e11-a1b9-efdb5016cdd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:07:35.185249 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402416f-6974-4e11-a1b9-efdb5016cdd5 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:07:35.203362 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:35.207452 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 682/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:07:44.553829 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=19 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:07:55.720225 dv-cinder-851351-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Jul 28 08:07:55.720627 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: Thu Jul 28 08:07:55 2022 - graceful shutdown triggered... Jul 28 08:07:55.721121 dv-cinder-851351-1-iscsi devstack@keystone.service[80369]: Gracefully killing worker 2 (pid: 80369)... Jul 28 08:07:55.721472 dv-cinder-851351-1-iscsi devstack@keystone.service[80368]: Gracefully killing worker 1 (pid: 80368)... Jul 28 08:07:56.721848 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: worker 1 buried after 1 seconds Jul 28 08:07:56.721848 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: worker 2 buried after 1 seconds Jul 28 08:07:56.721848 dv-cinder-851351-1-iscsi devstack@keystone.service[80356]: goodbye to uWSGI. Jul 28 08:07:56.723567 dv-cinder-851351-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Jul 28 08:07:56.724125 dv-cinder-851351-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. Jul 28 08:12:44.615281 dv-cinder-851351-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 08:12:44.627928 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 08:12:44 2022] *** Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: nodename: dv-cinder-851351-1-iscsi Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: machine: x86_64 Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: clock source: unix Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: pcre jit disabled Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: detected number of CPU cores: 4 Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: current working directory: / Jul 28 08:12:44.631190 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: detected binary path: /usr/bin/uwsgi-core Jul 28 08:12:44.632031 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: your processes number limit is 47680 Jul 28 08:12:44.632031 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: your memory page size is 4096 bytes Jul 28 08:12:44.632031 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: detected max file descriptor number: 2048 Jul 28 08:12:44.632031 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: lock engine: pthread robust mutexes Jul 28 08:12:44.632031 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: thunder lock: enabled Jul 28 08:12:44.632031 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 08:12:44.632031 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 08:12:44.672998 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: Python main interpreter initialized at 0x563b1ea369c0 Jul 28 08:12:44.672998 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: python threads support enabled Jul 28 08:12:44.673248 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: your server socket listen backlog is limited to 100 connections Jul 28 08:12:44.673248 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: your mercy for graceful operations on workers is 90 seconds Jul 28 08:12:44.673399 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 08:12:44.673478 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: *** Operational MODE: preforking *** Jul 28 08:12:44.673754 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: *** uWSGI is running in multiple interpreter mode *** Jul 28 08:12:44.673754 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: spawned uWSGI master process (pid: 144028) Jul 28 08:12:44.674038 dv-cinder-851351-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 08:12:44.674637 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: spawned uWSGI worker 1 (pid: 144042, cores: 1) Jul 28 08:12:44.674982 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: spawned uWSGI worker 2 (pid: 144043, cores: 1) Jul 28 08:12:44.675149 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 08:12:50.212824 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:12:50.212824 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: from cryptography.utils import int_from_bytes Jul 28 08:12:50.213994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:12:50.213994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: from cryptography.utils import int_from_bytes Jul 28 08:12:50.214319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:12:50.214319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: from cryptography.utils import int_from_bytes Jul 28 08:12:50.214319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:12:50.214319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: from cryptography.utils import int_from_bytes Jul 28 08:12:50.263427 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:12:50.263780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:12:50.263780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] command line args: [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:12:50.264099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:12:50.264208 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ================================================================================ {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:12:50.264469 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] admin_token = **** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.264685 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:12:50.264802 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] config_dir = [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.264904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.265075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:12:50.265184 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] config_source = [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.265283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] command line args: [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:12:50.265367 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] debug = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.265457 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:12:50.265794 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.267265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ================================================================================ {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:12:50.267265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] admin_token = **** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.267504 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.267504 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] insecure_debug = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.267754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] config_dir = [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.267861 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.267991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.267991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] config_source = [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.268370 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] debug = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.268460 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] list_limit = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.268604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.268727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] log_config_append = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.268863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.268983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.268983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] log_dir = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.269340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] insecure_debug = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.269516 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] log_file = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.269622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.269756 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.269923 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] list_limit = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] log_config_append = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270594 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270714 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] log_dir = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270887 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.270984 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] log_file = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271516 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271516 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271676 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.271850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272116 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272211 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272394 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272483 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] notification_format = cadf {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.272997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] publish_errors = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] notification_format = cadf {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273668 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273668 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273915 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.273915 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] standard_threads = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] publish_errors = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274390 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] strict_password_check = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274486 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274791 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] use_eventlog = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.274948 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.275093 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] use_journal = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.275261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.275429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] use_json = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.275429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] use_stderr = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.275626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.275718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] use_syslog = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.275869 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] standard_threads = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.275976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] watch_log_file = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.276074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] strict_password_check = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.276166 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.276262 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.276349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.276435 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] use_eventlog = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.276435 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] use_journal = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.276767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.276930 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] use_json = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.277049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.277150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] use_stderr = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.277236 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.277406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] use_syslog = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.277511 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.277597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] watch_log_file = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:12:50.277680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.277780 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.277875 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.277979 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.278128 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.278237 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.278324 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.278418 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.278509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.278634 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.278793 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.278938 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279027 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279505 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279691 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.279883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.280000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.280159 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.280316 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.280477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.280600 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.280697 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.280835 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.280975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281086 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281281 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281376 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281469 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281647 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281824 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.281919 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282257 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282642 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.282954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283057 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283319 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283489 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283691 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283787 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283872 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.283955 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284157 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284248 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284731 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284831 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.284973 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.285138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.285316 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.285480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.285636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.285740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.285827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.285933 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.286057 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.286205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.286315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.286412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.286512 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.286597 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.286718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.286873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287290 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287653 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287834 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.287931 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288014 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288606 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288781 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288874 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.288978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289595 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289773 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289869 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.289955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.290065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.290338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.290494 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.290656 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.290837 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.290963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291062 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291252 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291359 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291464 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291646 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.291991 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292652 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.enabled = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292827 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.292958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293124 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293368 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293461 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293618 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293830 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.293984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294148 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.enabled = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294502 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294600 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294702 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294789 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294868 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.294952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.295056 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.295218 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.295399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.295557 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.295674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.295770 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.295857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.295952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296048 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296122 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296122 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296369 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.296963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297260 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297362 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297462 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297553 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297632 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297801 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.297991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298087 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298325 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298593 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] auth.external = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298750 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] auth.mapped = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.298957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299137 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] auth.password = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] auth.token = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299668 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.299979 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.300383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] catalog.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.300516 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.300607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.300680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.300751 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.300895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] auth.external = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.301034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.301227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] auth.mapped = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.301431 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.301563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.301693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.301843 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.302015 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] credential.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.302180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] auth.password = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.302338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] credential.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.302482 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] auth.token = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.302626 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.302807 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.302950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.303121 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] catalog.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.303264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.303407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.303550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.303712 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.303845 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.303984 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304257 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304512 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] credential.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304594 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] credential.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.304936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305028 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305108 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305474 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305558 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] federation.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305832 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.305926 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] federation.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306379 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306539 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306610 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.306886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307081 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307805 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] federation.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307805 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.307984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.308146 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] federation.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.308225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.308311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.308512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.308655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.308748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.308823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.308956 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.309065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.309427 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.309583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.309731 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.309826 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.309935 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310028 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310163 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310249 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310347 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310539 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310624 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310725 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.310987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.311122 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.311275 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.311435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.311579 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.311725 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.311870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312361 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312455 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312778 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.312928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313171 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313733 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.313979 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.314212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.314374 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.314541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.314685 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.314835 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.314971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.315115 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.315250 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.315395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.315553 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.315703 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.315854 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.315990 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.316147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.316293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.316430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.316581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.316731 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.316867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.317012 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.317147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.password = **** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.317283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.317526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.317664 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.317811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.317959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.318195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.318419 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.318652 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.318866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.319022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.319159 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.319295 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.319432 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.319588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.319737 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.319885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.320020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.320175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.320350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.320537 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.320669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.password = **** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.320821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.321004 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.321231 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.321446 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.322000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.322000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.322381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.322519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.322676 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.322830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.322976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.323119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.323271 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.323484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.323699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.323900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.324050 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.324209 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.324345 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.324488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.324691 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.324867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.325067 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.325217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.325362 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.325511 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.325647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.325782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.325926 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.326094 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.326254 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.326391 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.326547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.326754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.326996 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.327154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.327309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.327460 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.327612 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.327612 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.328018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.328235 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.328398 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.328535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.328671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.328807 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.328963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.329100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.329262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.329447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.329657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.329805 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.329952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.330121 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.330275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.330412 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.330555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.330700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.330836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.330836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.331195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.331358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] policy.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.331583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.331744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.331896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.332038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.332255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.332583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.332737 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.332906 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] receipt.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.333043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.333196 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.333335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.333478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.333622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.333780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.333916 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.334080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.334080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.334358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.334493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] resource.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.334650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.334804 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.334946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.335098 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] resource.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.335246 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] policy.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.335381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.335525 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.335661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.335797 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.335994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.336548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.336704 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] revoke.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.336865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] receipt.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.337003 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.337139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.337275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.337410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.337552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] role.cache_time = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.337711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.337901 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] role.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.338213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.338356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] role.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.338509 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.338645 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] role.list_limit = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.338808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] resource.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.338962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.339097 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.339240 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.339422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] resource.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.339648 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.339798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.339935 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.340098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.340242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.340445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.340669 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.340890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] revoke.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.341046 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.341184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.341333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.341471 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.341684 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.341883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] role.cache_time = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.342048 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.342166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] role.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.342363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.342500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] role.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.342637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.342793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] role.list_limit = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.342936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.343085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.343085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.343466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.343606 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.343758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.343902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.344051 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.344051 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.344436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.344670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.344827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.344963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.345100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.345258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.345439 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.345643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.345836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.345989 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.346245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.346412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.346549 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.346704 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.346841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.346988 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.347199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.347393 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.347393 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.347698 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.347851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.348069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.348285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.348513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.348659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.348807 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.348958 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.349139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.349328 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] token.cache_time = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.349542 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.349680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] token.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.349910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.350145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.350299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.350450 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] token.provider = fernet {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.350642 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.350781 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.350938 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.351087 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.351274 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.351420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.351563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.351699 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.351887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.352038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.352194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.352405 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.352548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.352709 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] trust.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.352909 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.353057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.353193 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.353328 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.353483 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.353686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.353823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] token.cache_time = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.353960 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.354135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] token.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.354272 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.354442 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.354653 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.354855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] token.provider = fernet {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.355038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.355178 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.355384 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.355614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.355829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.connection = **** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.356035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.356220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.356389 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.356546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.356690 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.356896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.357087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.357227 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.357444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] trust.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.357602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.357739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.357875 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.357875 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.358287 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.358427 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.358593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.358744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.358893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.359051 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.359280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.359458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.359616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.359760 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.359904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.360039 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.360181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.360372 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.360529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.connection = **** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.360673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.360862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361086 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361434 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361596 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=144042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.361965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.363407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.363407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.363407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.363407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.363407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.363407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:12:50.363407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=144043) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:12:50.377698 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.378042 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.378269 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.380710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.380947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.381178 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.388298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.389963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.418802 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.419131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.422339 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.422563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.472487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.472819 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.524557 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.525137 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.525364 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.592207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.592545 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.592755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.592755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.593179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.593381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.593556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.593737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.593941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.594164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.594415 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.594415 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.618283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.618618 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.618853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.619109 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.619342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.619555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.619698 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.619853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.619990 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.620137 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.620271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.620486 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.620713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.620934 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.631503 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.631821 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.644935 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.645305 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.645535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.645812 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.646042 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.646300 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.646541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.646790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.647011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.647246 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=144043) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.647464 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.647464 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.647464 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.647464 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=144042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:12:50.664358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.664687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:12:50.664880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.665099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.665301 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:12:50.665532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:12:50.665719 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.665906 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.666145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:12:50.666360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.666360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.666738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.666933 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.667149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.667149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.667149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.676057 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.676380 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.676661 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.676943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.679338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.679338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.679650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.679883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.684080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.686930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.689068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.689369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.689644 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.689927 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.690238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.690526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.690793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.691053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.691314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.691593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.691917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.692102 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.692269 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.692477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.692754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.693025 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.693285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.693558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.693822 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.694087 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.694383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.694637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.715609 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.717125 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.717397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.718018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.719552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.719828 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.722074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.722386 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.722747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.724550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.724821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.725100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.729521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.729880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.731755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.732037 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.734422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.734783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.735118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.735472 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.735803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.736137 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.736250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.736507 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.736630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.736785 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.736940 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.737038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.737307 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.737954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.737954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.738264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.754347 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.754645 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.758615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.761754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.762052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.766480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.768654 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.769152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.769642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.772869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.773167 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.773445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.773724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.774012 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.774318 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.776960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.783300 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.783796 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.790456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.790757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.791018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.791280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.792480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.798137 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.798742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.799726 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.799999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.802933 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.803211 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.803512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.803790 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.804070 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.805037 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.805537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.806016 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.806499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.806974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.807504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.815533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.815811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.816085 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.816358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.816634 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.816942 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.817210 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.833819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.837424 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.840412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.840556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.840883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.841335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.841616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.843104 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.844697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.844977 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.845255 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.851806 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.852935 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.853412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.855367 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.855754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.856355 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.856644 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.856923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.857193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.860854 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.861346 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.861830 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.862347 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.862831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.865356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.865649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.865923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.872804 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=144042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.873082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.873362 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.873634 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.873917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.874243 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.874513 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.874787 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.875157 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.875435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.875714 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.875973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.876247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.876503 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=144042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.885521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.886010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.886528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.887015 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.887499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.887968 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.888456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.910025 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563b1ea369c0 pid: 144042 (default app) Jul 28 08:12:50.913641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755af244-db00-4c9e-ab45-cca7980b31a2 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:12:50.913928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755af244-db00-4c9e-ab45-cca7980b31a2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:12:50.914234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755af244-db00-4c9e-ab45-cca7980b31a2 None None] PATH_INFO: `/v3` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:12:50.918972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jul 28 08:12:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:12:50.919786 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.925702 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.931471 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.936142 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.939065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.939570 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.940042 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.953250 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.956366 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.962281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.962792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.963277 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.963789 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.980165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.980599 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.981076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.995042 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=144043) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:12:50.995546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.996008 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.996478 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.996981 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.997467 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.997953 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.998453 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.999112 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:50.999607 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:51.000118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:51.000601 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:51.001073 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:51.001539 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=144043) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:12:51.066924 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563b1ea369c0 pid: 144043 (default app) Jul 28 08:12:58.725588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d3b0b-45f3-4fe7-a673-92dff9ba77fb None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:12:58.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d3b0b-45f3-4fe7-a673-92dff9ba77fb None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:12:58.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d3b0b-45f3-4fe7-a673-92dff9ba77fb None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:12:58.727628 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 2/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:12:58.735465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:12:58.736095 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:12:58.736095 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:12:58.807035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: INFO dbcounter [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] Registered counter for database keystone Jul 28 08:12:58.829695 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG oslo_db.sqlalchemy.engines [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=144043) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:12:58.837523 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writer thread running {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:12:59.029426 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG passlib.handlers.bcrypt [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=144043) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:12:59.029748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG passlib.handlers.bcrypt [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=144043) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:12:59.086328 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:12:59.087949 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:12:59.219364 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 1/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:12:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 28 08:12:59.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d3b0b-45f3-4fe7-a673-92dff9ba77fb None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:12:59.323630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: INFO dbcounter [None req-740d3b0b-45f3-4fe7-a673-92dff9ba77fb None None] Registered counter for database keystone Jul 28 08:12:59.344688 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG oslo_db.sqlalchemy.engines [None req-740d3b0b-45f3-4fe7-a673-92dff9ba77fb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=144042) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:12:59.353884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writer thread running {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:12:59.473998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d3b0b-45f3-4fe7-a673-92dff9ba77fb None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:12:59.479012 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afd09d-4d42-412e-86eb-0e014a331261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d6cb8b74c64466888b7108b567aa82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:12:59.479662 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afd09d-4d42-412e-86eb-0e014a331261 admin admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:12:59.479945 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afd09d-4d42-412e-86eb-0e014a331261 admin admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:12:59.480228 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afd09d-4d42-412e-86eb-0e014a331261 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:12:59.488979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afd09d-4d42-412e-86eb-0e014a331261 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:12:59.539682 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b1afd09d-4d42-412e-86eb-0e014a331261 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:12:59.540026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afd09d-4d42-412e-86eb-0e014a331261 admin admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:12:59.550604 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 3/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:12:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:12:59.557131 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:12:59.575270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c5885-4480-43b5-a8f7-8efcfd09c613 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:12:59.583831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca5782-7e62-4391-8ca0-83b78f8a8534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d6cb8b74c64466888b7108b567aa82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:12:59.585494 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca5782-7e62-4391-8ca0-83b78f8a8534 admin admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:12:59.585902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca5782-7e62-4391-8ca0-83b78f8a8534 admin admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:12:59.586316 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca5782-7e62-4391-8ca0-83b78f8a8534 admin admin] PATH_INFO: `/v3/projects/c6d6cb8b74c64466888b7108b567aa82` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:12:59.589112 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca5782-7e62-4391-8ca0-83b78f8a8534 admin admin] RBAC: Authorizing `identity:get_project(project_id=c6d6cb8b74c64466888b7108b567aa82)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:12:59.647659 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: WARNING keystone.common.rbac_enforcer.enforcer [None req-03ca5782-7e62-4391-8ca0-83b78f8a8534 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:12:59.647988 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca5782-7e62-4391-8ca0-83b78f8a8534 admin admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:12:59.652904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 2/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jul 28 08:12:59 2022] GET /identity/v3/projects/c6d6cb8b74c64466888b7108b567aa82 => generated 344 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:01.620752 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66fcf-0920-44eb-a1be-1daa855fee7f None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:01.621375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66fcf-0920-44eb-a1be-1daa855fee7f None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:01.621375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66fcf-0920-44eb-a1be-1daa855fee7f None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:01.625492 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 4/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:01.631207 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119084e-40f8-4ee7-9c30-0ce6d23cb396 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:01.631668 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119084e-40f8-4ee7-9c30-0ce6d23cb396 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:01.631782 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119084e-40f8-4ee7-9c30-0ce6d23cb396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:01.653802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-1119084e-40f8-4ee7-9c30-0ce6d23cb396 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:01.655550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-1119084e-40f8-4ee7-9c30-0ce6d23cb396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:01.665473 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 3/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:13:01.688105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b66fcf-0920-44eb-a1be-1daa855fee7f None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:01.702914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b66fcf-0920-44eb-a1be-1daa855fee7f None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:01.707919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5274e-e194-4934-8771-09fb088a03ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d6cb8b74c64466888b7108b567aa82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:01.708581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5274e-e194-4934-8771-09fb088a03ac admin admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:01.708840 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5274e-e194-4934-8771-09fb088a03ac admin admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:01.709132 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5274e-e194-4934-8771-09fb088a03ac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:01.717404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5274e-e194-4934-8771-09fb088a03ac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:01.719266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5274e-e194-4934-8771-09fb088a03ac admin admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:01.728605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 5/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:13:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:13:01.734678 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119084e-40f8-4ee7-9c30-0ce6d23cb396 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:01.744107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119084e-40f8-4ee7-9c30-0ce6d23cb396 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:01.750074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f637bd-e05c-4c5a-82a8-f7d22a031262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d6cb8b74c64466888b7108b567aa82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:01.750769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f637bd-e05c-4c5a-82a8-f7d22a031262 admin admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:01.751060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f637bd-e05c-4c5a-82a8-f7d22a031262 admin admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:01.751344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f637bd-e05c-4c5a-82a8-f7d22a031262 admin admin] PATH_INFO: `/v3/users/ab47d79a05944b2eb59466f830650026` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:01.752910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f637bd-e05c-4c5a-82a8-f7d22a031262 admin admin] RBAC: Authorizing `identity:get_user(user_id=ab47d79a05944b2eb59466f830650026)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:01.754997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f637bd-e05c-4c5a-82a8-f7d22a031262 admin admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:01.756741 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 4/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jul 28 08:13:01 2022] GET /identity/v3/users/ab47d79a05944b2eb59466f830650026 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:03.851760 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e66c77-e578-4fbd-ae7c-0c31904321a8 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:03.852329 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e66c77-e578-4fbd-ae7c-0c31904321a8 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:03.852329 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e66c77-e578-4fbd-ae7c-0c31904321a8 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:03.853810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 6/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:13:03.859530 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac57c54-a1d8-4400-91bf-f0e0c53605cc None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:03.860043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac57c54-a1d8-4400-91bf-f0e0c53605cc None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:03.860159 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac57c54-a1d8-4400-91bf-f0e0c53605cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:03.866888 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837844b-3816-4785-a026-f6ce0b83df88 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:03.867176 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837844b-3816-4785-a026-f6ce0b83df88 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:03.867452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837844b-3816-4785-a026-f6ce0b83df88 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:03.868854 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 7/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:13:03.874150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:03.874451 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:03.874730 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:03.880966 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-1ac57c54-a1d8-4400-91bf-f0e0c53605cc None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:03.882234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-1ac57c54-a1d8-4400-91bf-f0e0c53605cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:03.917192 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG passlib.handlers.bcrypt [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=144042) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:13:03.917626 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG passlib.handlers.bcrypt [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=144042) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:13:03.963252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:03.964782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:03.972601 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 5/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:03.996000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac57c54-a1d8-4400-91bf-f0e0c53605cc None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:04.021561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:04.068423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 8/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:04.091003 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:04.105617 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac57c54-a1d8-4400-91bf-f0e0c53605cc None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:04.140747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c294-bf64-4422-8f19-1810e2785a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:04.141431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c294-bf64-4422-8f19-1810e2785a64 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:04.141710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c294-bf64-4422-8f19-1810e2785a64 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:04.141980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c294-bf64-4422-8f19-1810e2785a64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:04.156043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640c294-bf64-4422-8f19-1810e2785a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-8640c294-bf64-4422-8f19-1810e2785a64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:04.165188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:04.166424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:04.167417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 6/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:04.172621 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c294-bf64-4422-8f19-1810e2785a64 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:04.199159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3ad4f-8c81-433f-9f3b-0a3822b2a0e7 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:04.201614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:04.238885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2376f28-d59e-4cc0-a004-3575186cb809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:04.239696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2376f28-d59e-4cc0-a004-3575186cb809 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:04.239984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2376f28-d59e-4cc0-a004-3575186cb809 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:04.240258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2376f28-d59e-4cc0-a004-3575186cb809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:04.256401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:04.275710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c294-bf64-4422-8f19-1810e2785a64 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:04.310828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:04.311468 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:04.311755 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:04.311866 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2376f28-d59e-4cc0-a004-3575186cb809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:04.312036 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:04.312803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:04.313107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:04.313590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2376f28-d59e-4cc0-a004-3575186cb809 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:04.314550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:04.327611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 7/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:04.333751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:04.425074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 9/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:13:04.431335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2376f28-d59e-4cc0-a004-3575186cb809 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:04.434775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:04.549129 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2376f28-d59e-4cc0-a004-3575186cb809 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:04.552908 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfb0e0-a41d-4516-a02b-41d1393eda7f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:04.588532 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:04.589189 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27b58-ab94-4427-89be-0c79c0680731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:04.589454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:04.589741 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:04.589910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27b58-ab94-4427-89be-0c79c0680731 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:04.590015 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:04.590128 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27b58-ab94-4427-89be-0c79c0680731 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:04.590427 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27b58-ab94-4427-89be-0c79c0680731 None admin] PATH_INFO: `/v3/users/ab47d79a05944b2eb59466f830650026` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:04.591636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e27b58-ab94-4427-89be-0c79c0680731 None admin] RBAC: Authorizing `identity:get_user(user_id=ab47d79a05944b2eb59466f830650026)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:04.593434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e27b58-ab94-4427-89be-0c79c0680731 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:04.594785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 8/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:04 2022] GET /identity/v3/users/ab47d79a05944b2eb59466f830650026 => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:04.600747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27b58-ab94-4427-89be-0c79c0680731 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:04.610098 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:04.611641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:04.616827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:13:04.630950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:04.632159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:04.633158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:13:04.633158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:13:04.633158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:13:04.633158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:04.633158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:04.639167 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:04.702685 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:04.710025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27b58-ab94-4427-89be-0c79c0680731 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:04.747271 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab8ed2-f231-42c3-b588-949eb49e60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:04.747875 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8ed2-f231-42c3-b588-949eb49e60bf None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:04.748160 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8ed2-f231-42c3-b588-949eb49e60bf None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:04.748438 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8ed2-f231-42c3-b588-949eb49e60bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:04.749565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d39f2-eebf-48aa-8c85-e800eeab7ef9 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:04.749673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab8ed2-f231-42c3-b588-949eb49e60bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:04.751153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab8ed2-f231-42c3-b588-949eb49e60bf None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:04.752950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:04.760293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab8ed2-f231-42c3-b588-949eb49e60bf None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:04.787907 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e0459-2849-46c0-ad8f-ba0b9ec45696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:04.788587 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0459-2849-46c0-ad8f-ba0b9ec45696 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:04.788828 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0459-2849-46c0-ad8f-ba0b9ec45696 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:04.789108 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0459-2849-46c0-ad8f-ba0b9ec45696 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:04.789823 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e0459-2849-46c0-ad8f-ba0b9ec45696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:04.790133 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e0459-2849-46c0-ad8f-ba0b9ec45696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:04.791939 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e0459-2849-46c0-ad8f-ba0b9ec45696 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:04.810456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:04.852792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:04.876633 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab8ed2-f231-42c3-b588-949eb49e60bf None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:04.917760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162faa-314f-451e-915e-9e3d21510966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:04.918459 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162faa-314f-451e-915e-9e3d21510966 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:04.918742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162faa-314f-451e-915e-9e3d21510966 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:04.919019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162faa-314f-451e-915e-9e3d21510966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:04.919832 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f162faa-314f-451e-915e-9e3d21510966 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.domain.id=default, user.id=ab47d79a05944b2eb59466f830650026) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:04.920143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f162faa-314f-451e-915e-9e3d21510966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:04.921662 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f162faa-314f-451e-915e-9e3d21510966 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:04.930040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:13:04 2022] GET /identity/v3/role_assignments?user.id=ab47d79a05944b2eb59466f830650026&scope.domain.id=default&role.id=40504e80dbb845aeb07ca258977ecacb => generated 233 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:07.314369 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d26dd7-5a0d-453e-bfe4-973431788c6c None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:07.315210 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d26dd7-5a0d-453e-bfe4-973431788c6c None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:07.315395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d26dd7-5a0d-453e-bfe4-973431788c6c None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:07.317953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:07.324833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500ca17-14ce-482d-8744-ba2f56be7e34 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:07.326261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500ca17-14ce-482d-8744-ba2f56be7e34 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:07.327073 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500ca17-14ce-482d-8744-ba2f56be7e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:07.359648 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-1500ca17-14ce-482d-8744-ba2f56be7e34 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:07.360873 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-1500ca17-14ce-482d-8744-ba2f56be7e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:07.370629 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574e08-4d93-4b2d-80d1-affe5c50b36a None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:07.371102 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574e08-4d93-4b2d-80d1-affe5c50b36a None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:07.371587 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574e08-4d93-4b2d-80d1-affe5c50b36a None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:07.374082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 13/23] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:07.381200 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217815b-d305-4246-bb6c-1af2144e18ea None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:07.381672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217815b-d305-4246-bb6c-1af2144e18ea None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:07.382165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217815b-d305-4246-bb6c-1af2144e18ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:07.439880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-7217815b-d305-4246-bb6c-1af2144e18ea None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:07.441999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-7217815b-d305-4246-bb6c-1af2144e18ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:07.480522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 11/24] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:07.519610 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500ca17-14ce-482d-8744-ba2f56be7e34 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:07.620732 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 14/25] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:07.623079 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922e892-e27e-4b72-a1f4-fcfb6f9c16ce None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:07.623423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922e892-e27e-4b72-a1f4-fcfb6f9c16ce None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:07.623891 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922e892-e27e-4b72-a1f4-fcfb6f9c16ce None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:07.626344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 15/26] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:07.635026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38edb76-799d-4969-b731-4b0a00418686 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:07.635637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38edb76-799d-4969-b731-4b0a00418686 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:07.636126 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38edb76-799d-4969-b731-4b0a00418686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:07.670150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-a38edb76-799d-4969-b731-4b0a00418686 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:07.670150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-a38edb76-799d-4969-b731-4b0a00418686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:07.699211 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:07.743186 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500ca17-14ce-482d-8744-ba2f56be7e34 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:07.764183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:07.806603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 16/27] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:07.812700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38edb76-799d-4969-b731-4b0a00418686 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:07.842997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cc6c5-f7ef-4320-8c1a-3f1e66b7fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:07.844050 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cc6c5-f7ef-4320-8c1a-3f1e66b7fb1b None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:07.844541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cc6c5-f7ef-4320-8c1a-3f1e66b7fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:07.845056 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cc6c5-f7ef-4320-8c1a-3f1e66b7fb1b None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:07.846916 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cc6c5-f7ef-4320-8c1a-3f1e66b7fb1b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:07.849922 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cc6c5-f7ef-4320-8c1a-3f1e66b7fb1b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:08.098601 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38edb76-799d-4969-b731-4b0a00418686 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:08.147518 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 12/28] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:13:07 2022] POST /identity/v3/roles => generated 219 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:13:08.148598 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cc6c5-f7ef-4320-8c1a-3f1e66b7fb1b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:08.158731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:08.160186 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.160749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.161295 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.175262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:08.177373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:13:08.215650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:08.216743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:08.217728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:13:08.217728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:13:08.217728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:13:08.217728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:08.219953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 17/29] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:08.227132 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:08.414543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:08.456737 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cc6c5-f7ef-4320-8c1a-3f1e66b7fb1b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:08.495975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ff09d-280e-42e6-a0d1-b0a2d6fe9b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:08.496654 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ff09d-280e-42e6-a0d1-b0a2d6fe9b05 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.497045 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ff09d-280e-42e6-a0d1-b0a2d6fe9b05 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.497322 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ff09d-280e-42e6-a0d1-b0a2d6fe9b05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.504554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:08.514260 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ff09d-280e-42e6-a0d1-b0a2d6fe9b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:08.517146 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d75a4-1ffa-4f64-999e-8bfc43486a05 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-ac0ff09d-280e-42e6-a0d1-b0a2d6fe9b05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:08.520249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:08.521303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:08.522291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 13/30] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:08.523083 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ca557-d638-4c22-9ecb-92e74d0a2947 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.523724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ca557-d638-4c22-9ecb-92e74d0a2947 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.523724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ca557-d638-4c22-9ecb-92e74d0a2947 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.525022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 14/31] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:08.531061 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ca557-d638-4c22-9ecb-92e74d0a2947 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:08.560476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94090cf5-4616-443e-ae83-0c4944f5c302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:08.561126 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090cf5-4616-443e-ae83-0c4944f5c302 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.561569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090cf5-4616-443e-ae83-0c4944f5c302 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.561849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090cf5-4616-443e-ae83-0c4944f5c302 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.563105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94090cf5-4616-443e-ae83-0c4944f5c302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:08.563431 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94090cf5-4616-443e-ae83-0c4944f5c302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:08.564866 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94090cf5-4616-443e-ae83-0c4944f5c302 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:08.584969 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 18/32] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:08.586327 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763f7c1-086c-4493-899c-b58b36c36c42 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.586621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763f7c1-086c-4493-899c-b58b36c36c42 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.586901 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763f7c1-086c-4493-899c-b58b36c36c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-0763f7c1-086c-4493-899c-b58b36c36c42 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:08.610780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-0763f7c1-086c-4493-899c-b58b36c36c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:08.667892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ca557-d638-4c22-9ecb-92e74d0a2947 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:08.705283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 19/33] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:08.707137 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763f7c1-086c-4493-899c-b58b36c36c42 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:08.707386 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8124b-8f3a-40fa-927a-33706dd40d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:08.708268 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8124b-8f3a-40fa-927a-33706dd40d37 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.708768 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8124b-8f3a-40fa-927a-33706dd40d37 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.709239 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8124b-8f3a-40fa-927a-33706dd40d37 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8124b-8f3a-40fa-927a-33706dd40d37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:08.710751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8124b-8f3a-40fa-927a-33706dd40d37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:08.712550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8124b-8f3a-40fa-927a-33706dd40d37 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:08.722388 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 15/34] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:08.724172 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0f59c-e1ea-4d6e-b37e-e59a503bf74c None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.724678 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0f59c-e1ea-4d6e-b37e-e59a503bf74c None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.725176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0f59c-e1ea-4d6e-b37e-e59a503bf74c None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.729607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:08.730268 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 16/35] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:08.731229 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0f59c-e1ea-4d6e-b37e-e59a503bf74c None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:08.798070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:08.826348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763f7c1-086c-4493-899c-b58b36c36c42 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:08.850189 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0f59c-e1ea-4d6e-b37e-e59a503bf74c None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:08.861416 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352420d-00ff-4a56-b76d-2f39450bed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:08.862058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352420d-00ff-4a56-b76d-2f39450bed03 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.862377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352420d-00ff-4a56-b76d-2f39450bed03 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.862657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352420d-00ff-4a56-b76d-2f39450bed03 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.863667 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352420d-00ff-4a56-b76d-2f39450bed03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:08.865144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352420d-00ff-4a56-b76d-2f39450bed03 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:08.891201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaec3d-0918-48fb-84fc-a1f1ce0c5001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:08.891921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaec3d-0918-48fb-84fc-a1f1ce0c5001 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:08.892357 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaec3d-0918-48fb-84fc-a1f1ce0c5001 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:08.892675 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaec3d-0918-48fb-84fc-a1f1ce0c5001 None admin] PATH_INFO: `/v3/users/ab47d79a05944b2eb59466f830650026` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:08.893999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eaec3d-0918-48fb-84fc-a1f1ce0c5001 None admin] RBAC: Authorizing `identity:get_user(user_id=ab47d79a05944b2eb59466f830650026)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:08.895587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eaec3d-0918-48fb-84fc-a1f1ce0c5001 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:08.897530 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 17/36] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:08 2022] GET /identity/v3/users/ab47d79a05944b2eb59466f830650026 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:08.898028 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaec3d-0918-48fb-84fc-a1f1ce0c5001 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:09.012949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 20/37] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:13:08 2022] POST /identity/v3/projects => generated 301 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:09.012949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119bc5b-1645-4b8a-92fa-a1f7e77815af None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.012949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119bc5b-1645-4b8a-92fa-a1f7e77815af None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.012949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119bc5b-1645-4b8a-92fa-a1f7e77815af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.016334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaec3d-0918-48fb-84fc-a1f1ce0c5001 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:09.026584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-4119bc5b-1645-4b8a-92fa-a1f7e77815af None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:09.028211 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-4119bc5b-1645-4b8a-92fa-a1f7e77815af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:09.045987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:09.064942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019815b-94c6-40bf-ab8b-b8a622cf2ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:09.066089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019815b-94c6-40bf-ab8b-b8a622cf2ea2 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.066723 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019815b-94c6-40bf-ab8b-b8a622cf2ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.067191 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019815b-94c6-40bf-ab8b-b8a622cf2ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.068180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019815b-94c6-40bf-ab8b-b8a622cf2ea2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:09.071063 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019815b-94c6-40bf-ab8b-b8a622cf2ea2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:09.096706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 18/38] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:13:08 2022] POST /identity/v3/roles => generated 225 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:09.098854 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019815b-94c6-40bf-ab8b-b8a622cf2ea2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:09.109995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:09.119316 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 21/39] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:09.143073 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119bc5b-1645-4b8a-92fa-a1f7e77815af None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:09.260613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019815b-94c6-40bf-ab8b-b8a622cf2ea2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:09.273220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119bc5b-1645-4b8a-92fa-a1f7e77815af None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:09.309503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f4f79-5b2b-4075-8ddd-c24db9fcc0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:09.310454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f4f79-5b2b-4075-8ddd-c24db9fcc0b4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.310917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f4f79-5b2b-4075-8ddd-c24db9fcc0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.311284 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f4f79-5b2b-4075-8ddd-c24db9fcc0b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.312727 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f4f79-5b2b-4075-8ddd-c24db9fcc0b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:09.314451 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f4f79-5b2b-4075-8ddd-c24db9fcc0b4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:09.320997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 19/40] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:09.320997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe77a-317e-49cb-9f67-3867b0eb889f None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.320997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe77a-317e-49cb-9f67-3867b0eb889f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.320997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe77a-317e-49cb-9f67-3867b0eb889f None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.322316 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 20/41] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:09.323196 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fe77a-317e-49cb-9f67-3867b0eb889f None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:09.325587 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693287dd-8d6d-4ba2-bd42-47f44e784ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:09.326416 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693287dd-8d6d-4ba2-bd42-47f44e784ba7 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.326708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693287dd-8d6d-4ba2-bd42-47f44e784ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.326993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693287dd-8d6d-4ba2-bd42-47f44e784ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.327870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693287dd-8d6d-4ba2-bd42-47f44e784ba7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:09.329538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693287dd-8d6d-4ba2-bd42-47f44e784ba7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:09.351171 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 22/42] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:13:09 2022] POST /identity/v3/roles => generated 223 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:13:09.353599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2f9ae-606f-40a5-bd3f-6cb0f2b6a8d8 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.354317 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2f9ae-606f-40a5-bd3f-6cb0f2b6a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.354744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2f9ae-606f-40a5-bd3f-6cb0f2b6a8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.378806 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-0de2f9ae-606f-40a5-bd3f-6cb0f2b6a8d8 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:09.380152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-0de2f9ae-606f-40a5-bd3f-6cb0f2b6a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:09.409992 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:09.466849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:09.480278 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fe77a-317e-49cb-9f67-3867b0eb889f None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:09.481820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 23/43] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:09.506890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2f9ae-606f-40a5-bd3f-6cb0f2b6a8d8 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:09.529203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4cc7-93a9-45f2-8773-060c047c368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:09.529823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4cc7-93a9-45f2-8773-060c047c368c None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.530122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4cc7-93a9-45f2-8773-060c047c368c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.530425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4cc7-93a9-45f2-8773-060c047c368c None admin] PATH_INFO: `/v3/domains/default/users/ab47d79a05944b2eb59466f830650026/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.533231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d4cc7-93a9-45f2-8773-060c047c368c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ab47d79a05944b2eb59466f830650026, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:09.534861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d4cc7-93a9-45f2-8773-060c047c368c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:09.553459 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 21/44] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jul 28 08:13:09 2022] PUT /identity/v3/domains/default/users/ab47d79a05944b2eb59466f830650026/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:09.627372 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2f9ae-606f-40a5-bd3f-6cb0f2b6a8d8 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:09.664842 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004104fe-d0b8-4c97-a681-940dcd1250a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:09.665533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004104fe-d0b8-4c97-a681-940dcd1250a7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.665795 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004104fe-d0b8-4c97-a681-940dcd1250a7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.666128 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004104fe-d0b8-4c97-a681-940dcd1250a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.667465 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004104fe-d0b8-4c97-a681-940dcd1250a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:09.669000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004104fe-d0b8-4c97-a681-940dcd1250a7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:09.670764 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 24/45] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:09.676923 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4cc7-93a9-45f2-8773-060c047c368c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:09.784944 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4cc7-93a9-45f2-8773-060c047c368c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:09.798312 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:09.822189 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d2d22-8f63-4093-9bfc-7d8a0e0a7c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:09.822845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d2d22-8f63-4093-9bfc-7d8a0e0a7c34 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:09.823135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d2d22-8f63-4093-9bfc-7d8a0e0a7c34 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:09.823417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d2d22-8f63-4093-9bfc-7d8a0e0a7c34 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:09.824647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d2d22-8f63-4093-9bfc-7d8a0e0a7c34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:09.826605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d2d22-8f63-4093-9bfc-7d8a0e0a7c34 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:09.864677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 22/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:13:09 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:11.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0c84f-2cca-46e6-9533-575dd3c19c42 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:11.599583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0c84f-2cca-46e6-9533-575dd3c19c42 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:11.599583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0c84f-2cca-46e6-9533-575dd3c19c42 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:11.600967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 25/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:11.606477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a568-87c9-41e2-afd3-1973594c342f None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:11.606915 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a568-87c9-41e2-afd3-1973594c342f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:11.607028 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a568-87c9-41e2-afd3-1973594c342f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:11.626757 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-0147a568-87c9-41e2-afd3-1973594c342f None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:11.628315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-0147a568-87c9-41e2-afd3-1973594c342f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:11.703356 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 23/48] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:11.726406 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0c84f-2cca-46e6-9533-575dd3c19c42 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:11.791869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:11.837338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0c84f-2cca-46e6-9533-575dd3c19c42 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:11.873471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3dd68-8c60-46df-aa00-12021ea96450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:11.874165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3dd68-8c60-46df-aa00-12021ea96450 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:11.874456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3dd68-8c60-46df-aa00-12021ea96450 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:11.874732 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3dd68-8c60-46df-aa00-12021ea96450 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:11.886572 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3dd68-8c60-46df-aa00-12021ea96450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:11.887694 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a77f-a083-43d2-bad3-8754c6ca8c35 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:11.887998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a77f-a083-43d2-bad3-8754c6ca8c35 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:11.888283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a77f-a083-43d2-bad3-8754c6ca8c35 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:11.889684 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 24/49] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-72c3dd68-8c60-46df-aa00-12021ea96450 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:11.894125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:11.895195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:11.896361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:11.897223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c20cc-6871-4b52-a562-9e1cf73f5373 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:11.897223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c20cc-6871-4b52-a562-9e1cf73f5373 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:11.897223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c20cc-6871-4b52-a562-9e1cf73f5373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:11.897427 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 26/50] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:11.901263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3dd68-8c60-46df-aa00-12021ea96450 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:11.914546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-1e8c20cc-6871-4b52-a562-9e1cf73f5373 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:11.915774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-1e8c20cc-6871-4b52-a562-9e1cf73f5373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:11.986003 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 25/51] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:12.002528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3dd68-8c60-46df-aa00-12021ea96450 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:12.008898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c20cc-6871-4b52-a562-9e1cf73f5373 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:12.019908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:12.034250 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:12.037234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:12.037905 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:12.038211 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:12.038501 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:12.039315 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:12.039616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:12.041036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:12.058084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 27/52] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:12.059328 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:12.114866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c20cc-6871-4b52-a562-9e1cf73f5373 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:12.149086 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6536-3c16-4dd6-a7c2-364288ebe167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:12.149659 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6536-3c16-4dd6-a7c2-364288ebe167 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:12.149941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6536-3c16-4dd6-a7c2-364288ebe167 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:12.150244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6536-3c16-4dd6-a7c2-364288ebe167 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:12.151444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e6536-3c16-4dd6-a7c2-364288ebe167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:12.152929 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e6536-3c16-4dd6-a7c2-364288ebe167 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:12.154601 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 26/53] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:12.159691 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2b77c-e623-4790-a899-f4baf058a309 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:12.160775 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6536-3c16-4dd6-a7c2-364288ebe167 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:12.193882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95018362-6825-46e4-9a44-843de7950fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:12.194571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95018362-6825-46e4-9a44-843de7950fe8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:12.194860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95018362-6825-46e4-9a44-843de7950fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:12.195143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95018362-6825-46e4-9a44-843de7950fe8 None admin] PATH_INFO: `/v3/users/ab47d79a05944b2eb59466f830650026` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:12.196960 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95018362-6825-46e4-9a44-843de7950fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=ab47d79a05944b2eb59466f830650026)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:12.198502 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95018362-6825-46e4-9a44-843de7950fe8 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:12.200164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 28/54] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:12 2022] GET /identity/v3/users/ab47d79a05944b2eb59466f830650026 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:12.205702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95018362-6825-46e4-9a44-843de7950fe8 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:12.256033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:12.258242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:12.266032 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6536-3c16-4dd6-a7c2-364288ebe167 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:12.302888 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c64cb5-f178-4023-ad56-0ed38e012793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:12.303528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c64cb5-f178-4023-ad56-0ed38e012793 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:12.303816 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c64cb5-f178-4023-ad56-0ed38e012793 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:12.304137 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c64cb5-f178-4023-ad56-0ed38e012793 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:12.304836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c64cb5-f178-4023-ad56-0ed38e012793 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:12.306336 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c64cb5-f178-4023-ad56-0ed38e012793 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:12.312737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95018362-6825-46e4-9a44-843de7950fe8 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:12.333410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 27/55] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:13:12 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:12.348404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f164a4-ab8f-4532-a872-1c1074605812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:12.349100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f164a4-ab8f-4532-a872-1c1074605812 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:12.349380 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f164a4-ab8f-4532-a872-1c1074605812 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:12.349656 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f164a4-ab8f-4532-a872-1c1074605812 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:12.350986 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f164a4-ab8f-4532-a872-1c1074605812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:12.352518 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f164a4-ab8f-4532-a872-1c1074605812 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:12.354300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 29/56] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:12.360777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c64cb5-f178-4023-ad56-0ed38e012793 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:12.470709 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c64cb5-f178-4023-ad56-0ed38e012793 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:12.508534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc9b6e-d59b-4fdb-97b6-1cf79480a9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:12.509202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9b6e-d59b-4fdb-97b6-1cf79480a9b1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:12.509486 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9b6e-d59b-4fdb-97b6-1cf79480a9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:12.509760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9b6e-d59b-4fdb-97b6-1cf79480a9b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:12.510628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc9b6e-d59b-4fdb-97b6-1cf79480a9b1 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.domain.id=default, user.id=ab47d79a05944b2eb59466f830650026) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:12.510966 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc9b6e-d59b-4fdb-97b6-1cf79480a9b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:12.512472 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc9b6e-d59b-4fdb-97b6-1cf79480a9b1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:12.519722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 28/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:13:12 2022] GET /identity/v3/role_assignments?user.id=ab47d79a05944b2eb59466f830650026&scope.domain.id=default&role.id=40504e80dbb845aeb07ca258977ecacb => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:14.378657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50cb3a-a8b4-4dae-b341-6175faf365b8 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:14.379274 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50cb3a-a8b4-4dae-b341-6175faf365b8 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:14.379274 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50cb3a-a8b4-4dae-b341-6175faf365b8 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:14.380598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 30/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:14.386157 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5263c66-3e14-4b4f-a1cd-447ff9c8312f None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:14.386605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5263c66-3e14-4b4f-a1cd-447ff9c8312f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:14.386699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5263c66-3e14-4b4f-a1cd-447ff9c8312f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:14.405467 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-a5263c66-3e14-4b4f-a1cd-447ff9c8312f None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:14.406740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-a5263c66-3e14-4b4f-a1cd-447ff9c8312f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:14.408792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:14.478003 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 29/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:14.501929 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50cb3a-a8b4-4dae-b341-6175faf365b8 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:14.611907 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50cb3a-a8b4-4dae-b341-6175faf365b8 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:14.628505 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:14.648375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd02cb-541b-4c0c-9e02-27358da01c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:14.648932 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd02cb-541b-4c0c-9e02-27358da01c17 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:14.649211 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd02cb-541b-4c0c-9e02-27358da01c17 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:14.649454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd02cb-541b-4c0c-9e02-27358da01c17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:14.650727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd02cb-541b-4c0c-9e02-27358da01c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:14.652190 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd02cb-541b-4c0c-9e02-27358da01c17 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:14.653804 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 31/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:14.659762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5263c66-3e14-4b4f-a1cd-447ff9c8312f None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:14.765999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5263c66-3e14-4b4f-a1cd-447ff9c8312f None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:14.802153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059d96b-5b98-4b00-a98d-23cdd966b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:14.802790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059d96b-5b98-4b00-a98d-23cdd966b066 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:14.803080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059d96b-5b98-4b00-a98d-23cdd966b066 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:14.803389 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059d96b-5b98-4b00-a98d-23cdd966b066 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:14.804060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059d96b-5b98-4b00-a98d-23cdd966b066 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:14.806072 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059d96b-5b98-4b00-a98d-23cdd966b066 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:14.820438 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:14.836151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 30/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:13:14 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:17.847203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed822b5-51c2-4d1b-a1e0-9463c7dc077d None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:17.848245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed822b5-51c2-4d1b-a1e0-9463c7dc077d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:17.848786 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed822b5-51c2-4d1b-a1e0-9463c7dc077d None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:17.850423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 32/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:17.856038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61391b-1f9f-4dc3-a2ed-85ccc1ef09a0 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:17.856921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61391b-1f9f-4dc3-a2ed-85ccc1ef09a0 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:17.857416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61391b-1f9f-4dc3-a2ed-85ccc1ef09a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:17.877706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-be61391b-1f9f-4dc3-a2ed-85ccc1ef09a0 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:17.879364 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-be61391b-1f9f-4dc3-a2ed-85ccc1ef09a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:17.950674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 31/63] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:17.975700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed822b5-51c2-4d1b-a1e0-9463c7dc077d None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:18.110840 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed822b5-51c2-4d1b-a1e0-9463c7dc077d None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:18.153295 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd34723-5d33-46a8-9cfa-a9cc5d8e3dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:18.153986 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd34723-5d33-46a8-9cfa-a9cc5d8e3dd8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.154312 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd34723-5d33-46a8-9cfa-a9cc5d8e3dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.154597 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd34723-5d33-46a8-9cfa-a9cc5d8e3dd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.156900 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0ffa3-bdf8-461c-852a-30d272a31ae9 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.157425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0ffa3-bdf8-461c-852a-30d272a31ae9 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.157912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0ffa3-bdf8-461c-852a-30d272a31ae9 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.159565 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 32/64] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:18.164589 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab35e-495a-4c7e-9f23-7a39518cc006 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.165190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab35e-495a-4c7e-9f23-7a39518cc006 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.165671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab35e-495a-4c7e-9f23-7a39518cc006 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.167270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 33/65] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:13:18.175810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5d33-cd22-4ad6-a89e-92c168d5b164 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.176099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5d33-cd22-4ad6-a89e-92c168d5b164 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.176645 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5d33-cd22-4ad6-a89e-92c168d5b164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.176779 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd34723-5d33-46a8-9cfa-a9cc5d8e3dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-afd34723-5d33-46a8-9cfa-a9cc5d8e3dd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:18.181876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:18.182962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:18.183914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 33/66] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:18.184703 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eed0e-3925-4b04-b0d3-7ff3ef351a6c None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.184703 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eed0e-3925-4b04-b0d3-7ff3ef351a6c None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.185334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eed0e-3925-4b04-b0d3-7ff3ef351a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.199806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-c48b5d33-cd22-4ad6-a89e-92c168d5b164 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:18.201448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-c48b5d33-cd22-4ad6-a89e-92c168d5b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:18.208940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-f64eed0e-3925-4b04-b0d3-7ff3ef351a6c None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:18.210291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-f64eed0e-3925-4b04-b0d3-7ff3ef351a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:18.261286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:18.291404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 34/67] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:18.292632 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b5d33-cd22-4ad6-a89e-92c168d5b164 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:18.309205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 34/68] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:18.317977 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64eed0e-3925-4b04-b0d3-7ff3ef351a6c None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:18.390473 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:18.454521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b5d33-cd22-4ad6-a89e-92c168d5b164 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:18.456149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64eed0e-3925-4b04-b0d3-7ff3ef351a6c None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:18.496640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a373c-4fc0-4d55-9a29-88572e5b20fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:18.497690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a373c-4fc0-4d55-9a29-88572e5b20fa None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.498245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a373c-4fc0-4d55-9a29-88572e5b20fa None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.498919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a373c-4fc0-4d55-9a29-88572e5b20fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.499958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:18.500613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.500904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.501188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.502268 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:18.502595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:18.504560 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:18.514719 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 35/69] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:18.516151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:18.521815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a373c-4fc0-4d55-9a29-88572e5b20fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-9c4a373c-4fc0-4d55-9a29-88572e5b20fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:18.532341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:18.533423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:18.534376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 35/70] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:18.540057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a373c-4fc0-4d55-9a29-88572e5b20fa None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:18.586836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:18.641439 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b3028-3f00-404a-9d4d-f3a41ffc9991 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:18.661663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:18.685242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a373c-4fc0-4d55-9a29-88572e5b20fa None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:18.692051 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f02f86-a6ae-46a1-a85d-4323cfa53dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:18.692722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f02f86-a6ae-46a1-a85d-4323cfa53dda None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.693014 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f02f86-a6ae-46a1-a85d-4323cfa53dda None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.693293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f02f86-a6ae-46a1-a85d-4323cfa53dda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.704455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f02f86-a6ae-46a1-a85d-4323cfa53dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-40f02f86-a6ae-46a1-a85d-4323cfa53dda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:18.710227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:18.711667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 36/71] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:18.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f02f86-a6ae-46a1-a85d-4323cfa53dda None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:18.727060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a3e9c-314d-461f-bce9-aafc6577eee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:18.727760 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3e9c-314d-461f-bce9-aafc6577eee9 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.728046 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3e9c-314d-461f-bce9-aafc6577eee9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.728348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3e9c-314d-461f-bce9-aafc6577eee9 None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.744741 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a3e9c-314d-461f-bce9-aafc6577eee9 None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:18.746286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a3e9c-314d-461f-bce9-aafc6577eee9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:18.748063 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 36/72] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:18 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:18.749289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77442a9-86d2-49c9-b976-7ea5b5ad6c18 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.749578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77442a9-86d2-49c9-b976-7ea5b5ad6c18 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.749855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77442a9-86d2-49c9-b976-7ea5b5ad6c18 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.751287 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 37/73] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:18.752214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9fc25-58f4-4375-bcbf-a1974aba552b None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.752476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9fc25-58f4-4375-bcbf-a1974aba552b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.752758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9fc25-58f4-4375-bcbf-a1974aba552b None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.754262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 38/74] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:18.754929 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9fc25-58f4-4375-bcbf-a1974aba552b None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:18.830876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f02f86-a6ae-46a1-a85d-4323cfa53dda None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:18.854893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:18.865743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9fc25-58f4-4375-bcbf-a1974aba552b None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:18.869577 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:18.870256 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.870565 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.870836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.871582 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:18.871892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:18.873338 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:18.882392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 37/75] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:18.882392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:18.902817 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5267ddb-e711-434c-b1f9-1e47692a2afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:18.903462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267ddb-e711-434c-b1f9-1e47692a2afc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.903748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267ddb-e711-434c-b1f9-1e47692a2afc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.904026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267ddb-e711-434c-b1f9-1e47692a2afc None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.904768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5267ddb-e711-434c-b1f9-1e47692a2afc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:18.905090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5267ddb-e711-434c-b1f9-1e47692a2afc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:18.906545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5267ddb-e711-434c-b1f9-1e47692a2afc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:18.916036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 39/76] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:18.917080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a94b69-1dad-4c2f-a935-69454c5a9c8d None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:18.917373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a94b69-1dad-4c2f-a935-69454c5a9c8d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:18.917653 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a94b69-1dad-4c2f-a935-69454c5a9c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:18.931284 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:18.937618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-04a94b69-1dad-4c2f-a935-69454c5a9c8d None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:18.938920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-04a94b69-1dad-4c2f-a935-69454c5a9c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:19.003024 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3467d9-fce6-4baa-a3c8-12c07bd8ae0a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:19.013962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 40/77] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:19.019210 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6681e96-82de-4277-a78c-6051920473b4 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.019493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6681e96-82de-4277-a78c-6051920473b4 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.019774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6681e96-82de-4277-a78c-6051920473b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.040237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-f6681e96-82de-4277-a78c-6051920473b4 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:19.041460 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-f6681e96-82de-4277-a78c-6051920473b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:19.043077 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606f60e-b073-49f2-81d9-9fd4b12f97c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:19.043730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f60e-b073-49f2-81d9-9fd4b12f97c5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.044020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f60e-b073-49f2-81d9-9fd4b12f97c5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.044301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f60e-b073-49f2-81d9-9fd4b12f97c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.111054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 41/78] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:19.112019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6681e96-82de-4277-a78c-6051920473b4 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:19.127196 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:19.127960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606f60e-b073-49f2-81d9-9fd4b12f97c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:19.130385 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606f60e-b073-49f2-81d9-9fd4b12f97c5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:19.172088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:19.216060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6681e96-82de-4277-a78c-6051920473b4 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:19.240392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 38/79] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:19.240392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606f60e-b073-49f2-81d9-9fd4b12f97c5 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:19.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2de68b-747c-4574-aee3-c5b560b8ebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:19.256282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de68b-747c-4574-aee3-c5b560b8ebcf None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.256572 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de68b-747c-4574-aee3-c5b560b8ebcf None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.256849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de68b-747c-4574-aee3-c5b560b8ebcf None admin] PATH_INFO: `/v3/users/ab47d79a05944b2eb59466f830650026` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.258096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2de68b-747c-4574-aee3-c5b560b8ebcf None admin] RBAC: Authorizing `identity:get_user(user_id=ab47d79a05944b2eb59466f830650026)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:19.259570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2de68b-747c-4574-aee3-c5b560b8ebcf None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:19.261191 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 42/80] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:19 2022] GET /identity/v3/users/ab47d79a05944b2eb59466f830650026 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:19.262060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2de68b-747c-4574-aee3-c5b560b8ebcf None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:19.343779 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606f60e-b073-49f2-81d9-9fd4b12f97c5 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:19.348839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:19.368819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2de68b-747c-4574-aee3-c5b560b8ebcf None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:19.379635 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b66a29-4b64-485e-8423-bc900aec9375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:19.380321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b66a29-4b64-485e-8423-bc900aec9375 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.380612 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b66a29-4b64-485e-8423-bc900aec9375 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.380892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b66a29-4b64-485e-8423-bc900aec9375 None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.382141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b66a29-4b64-485e-8423-bc900aec9375 None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:19.383640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b66a29-4b64-485e-8423-bc900aec9375 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:19.385303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 39/81] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:19 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:19.386087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b66a29-4b64-485e-8423-bc900aec9375 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:19.396913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:19.409739 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931794-e4b9-487a-8ced-1bd878be280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:19.410510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931794-e4b9-487a-8ced-1bd878be280f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.410794 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931794-e4b9-487a-8ced-1bd878be280f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.411074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931794-e4b9-487a-8ced-1bd878be280f None admin] PATH_INFO: `/v3/domains/default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.412275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b931794-e4b9-487a-8ced-1bd878be280f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:19.413746 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b931794-e4b9-487a-8ced-1bd878be280f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:19.415683 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 43/82] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:19.416196 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931794-e4b9-487a-8ced-1bd878be280f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:19.494095 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b66a29-4b64-485e-8423-bc900aec9375 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:19.520552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931794-e4b9-487a-8ced-1bd878be280f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:19.528310 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2f567-0077-45f9-9f13-f6a52dcd7b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:19.528977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2f567-0077-45f9-9f13-f6a52dcd7b49 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.529269 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2f567-0077-45f9-9f13-f6a52dcd7b49 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.529556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2f567-0077-45f9-9f13-f6a52dcd7b49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.540480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2f567-0077-45f9-9f13-f6a52dcd7b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-b1f2f567-0077-45f9-9f13-f6a52dcd7b49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:19.546040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:19.547206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 40/83] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:19.548130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2f567-0077-45f9-9f13-f6a52dcd7b49 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:19.563672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e252b-cbe0-4bd3-bbc0-98506f0eb71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:19.564345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e252b-cbe0-4bd3-bbc0-98506f0eb71d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.564630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e252b-cbe0-4bd3-bbc0-98506f0eb71d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.564911 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e252b-cbe0-4bd3-bbc0-98506f0eb71d None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.566396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e252b-cbe0-4bd3-bbc0-98506f0eb71d None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:19.568060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e252b-cbe0-4bd3-bbc0-98506f0eb71d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:19.569970 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 44/84] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:19 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:19.570873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e252b-cbe0-4bd3-bbc0-98506f0eb71d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:19.591337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:19.622949 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:19.664395 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2f567-0077-45f9-9f13-f6a52dcd7b49 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:19.675657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e252b-cbe0-4bd3-bbc0-98506f0eb71d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:19.702723 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e7dd3-3375-4373-8a43-ec0f3c50fdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:19.703380 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e7dd3-3375-4373-8a43-ec0f3c50fdb2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.703667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e7dd3-3375-4373-8a43-ec0f3c50fdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.703946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e7dd3-3375-4373-8a43-ec0f3c50fdb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.711925 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e353f06-201c-4a44-b84a-8c1469e572e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:19.711925 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e353f06-201c-4a44-b84a-8c1469e572e3 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:19.712283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e353f06-201c-4a44-b84a-8c1469e572e3 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:19.712503 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e353f06-201c-4a44-b84a-8c1469e572e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:19.780462 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e7dd3-3375-4373-8a43-ec0f3c50fdb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:19.782022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e7dd3-3375-4373-8a43-ec0f3c50fdb2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:19.788225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e353f06-201c-4a44-b84a-8c1469e572e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:19.789729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e353f06-201c-4a44-b84a-8c1469e572e3 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:19.808574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:19.851499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:19.891161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 41/85] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:19.892156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e7dd3-3375-4373-8a43-ec0f3c50fdb2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:19.900083 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 45/86] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:19.901016 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e353f06-201c-4a44-b84a-8c1469e572e3 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:20.004346 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e7dd3-3375-4373-8a43-ec0f3c50fdb2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.009615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e353f06-201c-4a44-b84a-8c1469e572e3 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.027104 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:20.038946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554f363-303b-4cb2-a36f-de5d08af3ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.039416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554f363-303b-4cb2-a36f-de5d08af3ae0 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.039703 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554f363-303b-4cb2-a36f-de5d08af3ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.039991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554f363-303b-4cb2-a36f-de5d08af3ae0 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.040688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554f363-303b-4cb2-a36f-de5d08af3ae0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.042222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554f363-303b-4cb2-a36f-de5d08af3ae0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:20.044685 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.045326 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.045614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.045896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.046670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:20.046980 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.048367 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:20.057473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 46/87] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:20.058418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:20.069316 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 42/88] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:13:19 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:20.070205 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554f363-303b-4cb2-a36f-de5d08af3ae0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:20.099189 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:20.179505 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1932629-222a-4f43-b4ff-2620b336744a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.195769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554f363-303b-4cb2-a36f-de5d08af3ae0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.217923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.218625 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.218906 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.219182 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.220619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] RBAC: Adding query filter params (role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb, user.id=cee2a726271640868cff7cbf256bac2d) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:20.221096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.222515 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:20.231546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 47/89] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:20 2022] GET /identity/v3/role_assignments?user.id=cee2a726271640868cff7cbf256bac2d&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:20.234150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:20.243681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2702c-64d6-40bf-885d-16320308802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.244372 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2702c-64d6-40bf-885d-16320308802d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.244656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2702c-64d6-40bf-885d-16320308802d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.244935 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2702c-64d6-40bf-885d-16320308802d None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.246300 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2702c-64d6-40bf-885d-16320308802d None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.247889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2702c-64d6-40bf-885d-16320308802d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:20.249713 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 43/90] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:20 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:20.250564 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2702c-64d6-40bf-885d-16320308802d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:20.296791 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:20.383033 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a809110-7a5e-45cb-9642-4d237a928cfd None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.389728 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:20.415751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2702c-64d6-40bf-885d-16320308802d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.430594 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156ed77-c108-44a2-9126-1b5f4fec4161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.431280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156ed77-c108-44a2-9126-1b5f4fec4161 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.431767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156ed77-c108-44a2-9126-1b5f4fec4161 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.432053 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156ed77-c108-44a2-9126-1b5f4fec4161 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.434358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156ed77-c108-44a2-9126-1b5f4fec4161 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.435864 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156ed77-c108-44a2-9126-1b5f4fec4161 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:20.437828 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 48/91] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:20 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:20.440193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156ed77-c108-44a2-9126-1b5f4fec4161 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:20.467153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c82dd-7e8d-4fa2-8a6a-4f29b5a13d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.467967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c82dd-7e8d-4fa2-8a6a-4f29b5a13d1f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.468262 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c82dd-7e8d-4fa2-8a6a-4f29b5a13d1f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.468550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c82dd-7e8d-4fa2-8a6a-4f29b5a13d1f None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.470200 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c82dd-7e8d-4fa2-8a6a-4f29b5a13d1f None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.472170 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c82dd-7e8d-4fa2-8a6a-4f29b5a13d1f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:20.475051 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 44/92] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:20 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:20.475892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c82dd-7e8d-4fa2-8a6a-4f29b5a13d1f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:20.558880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156ed77-c108-44a2-9126-1b5f4fec4161 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.586719 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:20.626095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.626821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.627114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.627396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.628213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb, user.id=ab47d79a05944b2eb59466f830650026) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:20.628603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.630093 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:20.636869 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c82dd-7e8d-4fa2-8a6a-4f29b5a13d1f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.643806 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 49/93] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:20 2022] GET /identity/v3/role_assignments?user.id=ab47d79a05944b2eb59466f830650026&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=40504e80dbb845aeb07ca258977ecacb => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:20.644735 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:20.677803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.678567 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.678857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.679134 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.679970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] RBAC: Adding query filter params (role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb, user.id=cee2a726271640868cff7cbf256bac2d) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:20.680286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.682172 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:20.689045 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 45/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:20 2022] GET /identity/v3/role_assignments?user.id=cee2a726271640868cff7cbf256bac2d&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:20.858938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285cab0-d5b9-41e5-8316-638d477669d5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:20.908145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d71950-7e21-4eab-97b7-b0aac8628116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:20.908817 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d71950-7e21-4eab-97b7-b0aac8628116 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:20.909089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d71950-7e21-4eab-97b7-b0aac8628116 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:20.909507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d71950-7e21-4eab-97b7-b0aac8628116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:20.970080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:20.996723 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d71950-7e21-4eab-97b7-b0aac8628116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:20.998284 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d71950-7e21-4eab-97b7-b0aac8628116 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:21.118578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 50/95] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:21.124475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:21.169541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:21.252867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7171b-cd39-4944-9503-2c32e9c53604 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:21.304334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28696b-891b-467e-acdb-f4cb50bab704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:21.305418 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28696b-891b-467e-acdb-f4cb50bab704 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:21.305940 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28696b-891b-467e-acdb-f4cb50bab704 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:21.306441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28696b-891b-467e-acdb-f4cb50bab704 None admin] PATH_INFO: `/v3/projects/7d381fee65074e4380907d65ce60d876` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:21.307971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28696b-891b-467e-acdb-f4cb50bab704 None admin] RBAC: Authorizing `identity:get_project(project_id=7d381fee65074e4380907d65ce60d876)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:21.309769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28696b-891b-467e-acdb-f4cb50bab704 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:21.311849 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 46/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:21 2022] GET /identity/v3/projects/7d381fee65074e4380907d65ce60d876 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:21.318843 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d71950-7e21-4eab-97b7-b0aac8628116 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:21.450938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:21.482512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d71950-7e21-4eab-97b7-b0aac8628116 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:21.531389 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a14e5a-f5d6-4f07-a2d6-047b5251cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:21.532035 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a14e5a-f5d6-4f07-a2d6-047b5251cca2 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:21.532325 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a14e5a-f5d6-4f07-a2d6-047b5251cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:21.532607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a14e5a-f5d6-4f07-a2d6-047b5251cca2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:21.533409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a14e5a-f5d6-4f07-a2d6-047b5251cca2 None admin] RBAC: Adding query filter params (role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=7d381fee65074e4380907d65ce60d876, user.id=cee2a726271640868cff7cbf256bac2d) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:21.533717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a14e5a-f5d6-4f07-a2d6-047b5251cca2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:21.535265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a14e5a-f5d6-4f07-a2d6-047b5251cca2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:21.546594 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 51/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:21 2022] GET /identity/v3/role_assignments?user.id=cee2a726271640868cff7cbf256bac2d&scope.project.id=7d381fee65074e4380907d65ce60d876&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:23.260566 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44398b20-5bdb-4afd-b834-11cf37288c12 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.261237 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44398b20-5bdb-4afd-b834-11cf37288c12 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.261237 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44398b20-5bdb-4afd-b834-11cf37288c12 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.262543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 47/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:23.275862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e97a-dc32-4f42-ad46-565ddc66f61c None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.276429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e97a-dc32-4f42-ad46-565ddc66f61c None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.276429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e97a-dc32-4f42-ad46-565ddc66f61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.299954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-2812e97a-dc32-4f42-ad46-565ddc66f61c None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:23.301266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-2812e97a-dc32-4f42-ad46-565ddc66f61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:23.379110 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 52/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:23.385876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41730518-a000-43d0-8495-9a0836d95437 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.386201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41730518-a000-43d0-8495-9a0836d95437 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.386489 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41730518-a000-43d0-8495-9a0836d95437 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.388164 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 48/100] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:23.395150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4abb3e-21c8-4a89-9e46-27ce31575471 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.395442 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4abb3e-21c8-4a89-9e46-27ce31575471 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.395726 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4abb3e-21c8-4a89-9e46-27ce31575471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.413369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41730518-a000-43d0-8495-9a0836d95437 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:23.437391 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-6c4abb3e-21c8-4a89-9e46-27ce31575471 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:23.438652 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-6c4abb3e-21c8-4a89-9e46-27ce31575471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:23.548535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:23.553868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:23.561310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 53/101] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:23.562764 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f0ba-8fb9-4517-b6f3-3ba235913b80 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.563051 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f0ba-8fb9-4517-b6f3-3ba235913b80 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.563245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41730518-a000-43d0-8495-9a0836d95437 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:23.563402 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f0ba-8fb9-4517-b6f3-3ba235913b80 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.564857 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 54/102] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:23.566129 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c9752-3532-43d5-b327-fed2448160c4 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.566444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c9752-3532-43d5-b327-fed2448160c4 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.566720 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c9752-3532-43d5-b327-fed2448160c4 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.572472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 55/103] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:23.573949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8ac6b-f222-471b-a470-da139a015553 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.574265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8ac6b-f222-471b-a470-da139a015553 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.574545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8ac6b-f222-471b-a470-da139a015553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.597356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-86d8ac6b-f222-471b-a470-da139a015553 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:23.599052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-86d8ac6b-f222-471b-a470-da139a015553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:23.603310 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af4a9a-45c0-4828-b8b8-cd02c0b2acc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:23.603977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4a9a-45c0-4828-b8b8-cd02c0b2acc1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.604260 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4a9a-45c0-4828-b8b8-cd02c0b2acc1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.604541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4a9a-45c0-4828-b8b8-cd02c0b2acc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.605713 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af4a9a-45c0-4828-b8b8-cd02c0b2acc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:23.607227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af4a9a-45c0-4828-b8b8-cd02c0b2acc1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:23.609218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 49/104] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:23.610585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a5b0f-c628-4a58-9d3f-ee3976e145af None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.610871 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a5b0f-c628-4a58-9d3f-ee3976e145af None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.611145 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a5b0f-c628-4a58-9d3f-ee3976e145af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.630275 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d56a5b0f-c628-4a58-9d3f-ee3976e145af None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:23.631499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d56a5b0f-c628-4a58-9d3f-ee3976e145af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:23.695565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 56/105] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:23.696899 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8ac6b-f222-471b-a470-da139a015553 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:23.717201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 50/106] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:23.718692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a5b0f-c628-4a58-9d3f-ee3976e145af None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:23.818410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:23.834083 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8ac6b-f222-471b-a470-da139a015553 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:23.834534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a5b0f-c628-4a58-9d3f-ee3976e145af None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:23.865064 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:23.872544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01470f7-fb3d-444b-b672-dc9ed9a54e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:23.873445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01470f7-fb3d-444b-b672-dc9ed9a54e2d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.873912 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01470f7-fb3d-444b-b672-dc9ed9a54e2d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.874225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01470f7-fb3d-444b-b672-dc9ed9a54e2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.878627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74898321-6748-4fc5-a126-677c45b6c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:23.879276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74898321-6748-4fc5-a126-677c45b6c74d None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.879618 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74898321-6748-4fc5-a126-677c45b6c74d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.879924 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74898321-6748-4fc5-a126-677c45b6c74d None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.880616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74898321-6748-4fc5-a126-677c45b6c74d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:23.882298 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74898321-6748-4fc5-a126-677c45b6c74d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:23.885519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01470f7-fb3d-444b-b672-dc9ed9a54e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-d01470f7-fb3d-444b-b672-dc9ed9a54e2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:23.890461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:23.891869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 57/107] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:23.893036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01470f7-fb3d-444b-b672-dc9ed9a54e2d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:23.917748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 51/108] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:13:23 2022] POST /identity/v3/projects => generated 302 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:23.919378 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e07ac-f8bc-40b6-9d41-c6119379ce98 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:23.919827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e07ac-f8bc-40b6-9d41-c6119379ce98 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:23.920279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e07ac-f8bc-40b6-9d41-c6119379ce98 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:23.926002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 52/109] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:23.926002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e07ac-f8bc-40b6-9d41-c6119379ce98 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.046584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01470f7-fb3d-444b-b672-dc9ed9a54e2d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.048550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e07ac-f8bc-40b6-9d41-c6119379ce98 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.087275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17265db2-50f6-4d0f-9279-abe067840f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.087936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17265db2-50f6-4d0f-9279-abe067840f01 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.088203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17265db2-50f6-4d0f-9279-abe067840f01 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.088494 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17265db2-50f6-4d0f-9279-abe067840f01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.090708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d8ef6-bfad-4296-936f-de57134b4ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.091845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d8ef6-bfad-4296-936f-de57134b4ee9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.092321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d8ef6-bfad-4296-936f-de57134b4ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.092809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d8ef6-bfad-4296-936f-de57134b4ee9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.101615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17265db2-50f6-4d0f-9279-abe067840f01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:24.104563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-17265db2-50f6-4d0f-9279-abe067840f01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:24.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:24.108999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:24.109920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:24.112749 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d8ef6-bfad-4296-936f-de57134b4ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:24.114538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 58/110] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:24.115484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17265db2-50f6-4d0f-9279-abe067840f01 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-f89d8ef6-bfad-4296-936f-de57134b4ee9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:24.120158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:24.121236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:24.122425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:24.130497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 53/111] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:24.132334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cc730-c484-4a0e-b3f3-638e24e520ae None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.132855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cc730-c484-4a0e-b3f3-638e24e520ae None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.133363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cc730-c484-4a0e-b3f3-638e24e520ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.153401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-447cc730-c484-4a0e-b3f3-638e24e520ae None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:24.154652 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-447cc730-c484-4a0e-b3f3-638e24e520ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:24.190885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:24.250129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 54/112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:24.250129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cc730-c484-4a0e-b3f3-638e24e520ae None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.273680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17265db2-50f6-4d0f-9279-abe067840f01 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.309316 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.310955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.311446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.311935 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.312888 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:24.313430 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:24.315127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:24.325379 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 59/113] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:24.326641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.361180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cc730-c484-4a0e-b3f3-638e24e520ae None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.400400 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.400792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.400962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.401236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.401970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:24.402311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:24.403730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:24.406533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:24.413012 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 55/114] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:24.422896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ac140-bb6c-43f9-8809-5a615f903f4e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.453738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:24.477365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.478467 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.479038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.479536 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.480534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:24.481053 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:24.482708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:24.492206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 60/115] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:24.493461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.540637 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c487d0-eeb6-4f98-b70f-ee15ee6d5b04 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.577782 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecb4ca-b1ce-4e34-bd9d-6790a630606a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.578481 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecb4ca-b1ce-4e34-bd9d-6790a630606a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.578765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecb4ca-b1ce-4e34-bd9d-6790a630606a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.579044 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecb4ca-b1ce-4e34-bd9d-6790a630606a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.590751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecb4ca-b1ce-4e34-bd9d-6790a630606a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-b0ecb4ca-b1ce-4e34-bd9d-6790a630606a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:24.596669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:24.597793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:24.598760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 56/116] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:24.599541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecb4ca-b1ce-4e34-bd9d-6790a630606a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.615437 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021915f-a91b-4c63-8371-0268f6bc6eda None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.652229 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625751a-cb12-4a15-b7b6-f2a7125491db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.652889 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625751a-cb12-4a15-b7b6-f2a7125491db None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.653174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625751a-cb12-4a15-b7b6-f2a7125491db None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.653532 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625751a-cb12-4a15-b7b6-f2a7125491db None admin] PATH_INFO: `/v3/users/ab47d79a05944b2eb59466f830650026` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.654850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625751a-cb12-4a15-b7b6-f2a7125491db None admin] RBAC: Authorizing `identity:get_user(user_id=ab47d79a05944b2eb59466f830650026)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:24.656119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625751a-cb12-4a15-b7b6-f2a7125491db None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:24.662392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 61/117] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/users/ab47d79a05944b2eb59466f830650026 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:24.662392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625751a-cb12-4a15-b7b6-f2a7125491db None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.667020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:24.706401 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:24.729381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecb4ca-b1ce-4e34-bd9d-6790a630606a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.767313 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb37c1-3821-4e50-8852-61ea20b6d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.767981 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb37c1-3821-4e50-8852-61ea20b6d281 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.768265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb37c1-3821-4e50-8852-61ea20b6d281 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.768546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb37c1-3821-4e50-8852-61ea20b6d281 None admin] PATH_INFO: `/v3/users/6d07581a4e294fb09b80c406fc0264c9` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.770984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625751a-cb12-4a15-b7b6-f2a7125491db None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.783876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb37c1-3821-4e50-8852-61ea20b6d281 None admin] RBAC: Authorizing `identity:get_user(user_id=6d07581a4e294fb09b80c406fc0264c9)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:24.785410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb37c1-3821-4e50-8852-61ea20b6d281 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:24.787204 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 57/118] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/users/6d07581a4e294fb09b80c406fc0264c9 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:24.791207 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb37c1-3821-4e50-8852-61ea20b6d281 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.807113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e218e9-38eb-4359-a7f6-06161b3de62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.808135 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e218e9-38eb-4359-a7f6-06161b3de62d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.808626 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e218e9-38eb-4359-a7f6-06161b3de62d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.809273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e218e9-38eb-4359-a7f6-06161b3de62d None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.810892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e218e9-38eb-4359-a7f6-06161b3de62d None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:24.812568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e218e9-38eb-4359-a7f6-06161b3de62d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:24.814687 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 62/119] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:24.816151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59facd6-b05d-44f9-a69b-f8a53a7923bf None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.816436 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59facd6-b05d-44f9-a69b-f8a53a7923bf None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.816710 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59facd6-b05d-44f9-a69b-f8a53a7923bf None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.818061 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 63/120] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:24.818655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59facd6-b05d-44f9-a69b-f8a53a7923bf None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.906487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb37c1-3821-4e50-8852-61ea20b6d281 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.920227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:24.928924 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59facd6-b05d-44f9-a69b-f8a53a7923bf None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:24.941266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:24.948951 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.949602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.949891 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.950213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.950955 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:24.951271 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:24.952748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:24.961793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 58/121] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:24.962696 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:24.974717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9ba63-6a6e-4cca-b021-bd4274f8f977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:24.975407 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9ba63-6a6e-4cca-b021-bd4274f8f977 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:24.975679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9ba63-6a6e-4cca-b021-bd4274f8f977 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:24.975962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9ba63-6a6e-4cca-b021-bd4274f8f977 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:24.977231 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9ba63-6a6e-4cca-b021-bd4274f8f977 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:24.978862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9ba63-6a6e-4cca-b021-bd4274f8f977 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:24.980682 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 64/122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:24.980682 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9ba63-6a6e-4cca-b021-bd4274f8f977 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:25.077700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6af37e-f313-4f26-a577-7e5d103fea20 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:25.102972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9ba63-6a6e-4cca-b021-bd4274f8f977 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:25.119218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9bf51-e611-4b9b-b82c-10f8e5f07865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:25.119866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9bf51-e611-4b9b-b82c-10f8e5f07865 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.120119 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9bf51-e611-4b9b-b82c-10f8e5f07865 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.120392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9bf51-e611-4b9b-b82c-10f8e5f07865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.142422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f94dc-0d1e-4b7a-9597-2d80eb4886de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:25.143459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f94dc-0d1e-4b7a-9597-2d80eb4886de None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.144508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f94dc-0d1e-4b7a-9597-2d80eb4886de None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.145013 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f94dc-0d1e-4b7a-9597-2d80eb4886de None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.147040 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f94dc-0d1e-4b7a-9597-2d80eb4886de None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:25.149489 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f94dc-0d1e-4b7a-9597-2d80eb4886de None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:25.152368 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 65/123] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:25.154633 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe319e87-369d-4e30-91ce-a76e7f2c8a1c None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.155138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe319e87-369d-4e30-91ce-a76e7f2c8a1c None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.155610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe319e87-369d-4e30-91ce-a76e7f2c8a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.165429 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:25.176592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-fe319e87-369d-4e30-91ce-a76e7f2c8a1c None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:25.177993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-fe319e87-369d-4e30-91ce-a76e7f2c8a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:25.202454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9bf51-e611-4b9b-b82c-10f8e5f07865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:25.203997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9bf51-e611-4b9b-b82c-10f8e5f07865 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:25.228525 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:25.257658 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 66/124] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:25.266342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe319e87-369d-4e30-91ce-a76e7f2c8a1c None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:25.336588 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 59/125] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:25.337459 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9bf51-e611-4b9b-b82c-10f8e5f07865 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:25.383018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe319e87-369d-4e30-91ce-a76e7f2c8a1c None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:25.410717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:25.423467 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a5e8b-c986-4368-b178-26df9eebe145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:25.424152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a5e8b-c986-4368-b178-26df9eebe145 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.424437 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a5e8b-c986-4368-b178-26df9eebe145 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.424740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a5e8b-c986-4368-b178-26df9eebe145 None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.426252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a5e8b-c986-4368-b178-26df9eebe145 None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:25.427931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a5e8b-c986-4368-b178-26df9eebe145 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:25.434658 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 67/126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:25 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:25.435083 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a5e8b-c986-4368-b178-26df9eebe145 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:25.447356 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9bf51-e611-4b9b-b82c-10f8e5f07865 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:25.482613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b681c-773e-477f-8485-486935d9b79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:25.483319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b681c-773e-477f-8485-486935d9b79c None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.483608 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b681c-773e-477f-8485-486935d9b79c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.483892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b681c-773e-477f-8485-486935d9b79c None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/users/ab47d79a05944b2eb59466f830650026/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.485920 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b681c-773e-477f-8485-486935d9b79c None admin] RBAC: Authorizing `identity:create_grant(project_id=8ac986a4d83f4f07af22c95818f1b5bb, user_id=ab47d79a05944b2eb59466f830650026, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:25.486752 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:25.487444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b681c-773e-477f-8485-486935d9b79c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:25.500821 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 60/127] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:25 2022] PUT /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/users/ab47d79a05944b2eb59466f830650026/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:25.501415 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b681c-773e-477f-8485-486935d9b79c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:25.581535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a5e8b-c986-4368-b178-26df9eebe145 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:25.621208 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba0fbc-6c86-46bc-96ec-af0b79e4b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:25.621929 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba0fbc-6c86-46bc-96ec-af0b79e4b69a None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.622246 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba0fbc-6c86-46bc-96ec-af0b79e4b69a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.622537 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba0fbc-6c86-46bc-96ec-af0b79e4b69a None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/users/cee2a726271640868cff7cbf256bac2d/roles/0aacd40c75f64f668308391b44dc4e85` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.625386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba0fbc-6c86-46bc-96ec-af0b79e4b69a None admin] RBAC: Authorizing `identity:create_grant(project_id=8ac986a4d83f4f07af22c95818f1b5bb, user_id=cee2a726271640868cff7cbf256bac2d, role_id=0aacd40c75f64f668308391b44dc4e85)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:25.626960 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba0fbc-6c86-46bc-96ec-af0b79e4b69a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:25.646543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 68/128] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:25 2022] PUT /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/users/cee2a726271640868cff7cbf256bac2d/roles/0aacd40c75f64f668308391b44dc4e85 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:25.647713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba0fbc-6c86-46bc-96ec-af0b79e4b69a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:25.705313 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b681c-773e-477f-8485-486935d9b79c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:25.752681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:25.761795 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb4ebe-ed32-4b80-90bb-612f063da7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:25.764676 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4ebe-ed32-4b80-90bb-612f063da7d9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.765226 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4ebe-ed32-4b80-90bb-612f063da7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.765709 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4ebe-ed32-4b80-90bb-612f063da7d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.783465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb4ebe-ed32-4b80-90bb-612f063da7d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-eacb4ebe-ed32-4b80-90bb-612f063da7d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:25.790147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:25.791477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:25.792611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:25.795748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 61/129] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:25.797446 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb4ebe-ed32-4b80-90bb-612f063da7d9 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:25.829920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba0fbc-6c86-46bc-96ec-af0b79e4b69a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:25.834416 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:25.882253 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11563193-14c5-4f6b-a5e5-cf0e0672b860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:25.882982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563193-14c5-4f6b-a5e5-cf0e0672b860 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.883267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563193-14c5-4f6b-a5e5-cf0e0672b860 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.883545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563193-14c5-4f6b-a5e5-cf0e0672b860 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.885158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11563193-14c5-4f6b-a5e5-cf0e0672b860 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:25.886647 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11563193-14c5-4f6b-a5e5-cf0e0672b860 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:25.888704 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 69/130] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:25 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:25.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11563193-14c5-4f6b-a5e5-cf0e0672b860 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:25.939423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb4ebe-ed32-4b80-90bb-612f063da7d9 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:25.993118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aac31c-5501-4238-84db-fc2883469a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:25.994203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac31c-5501-4238-84db-fc2883469a38 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:25.994698 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac31c-5501-4238-84db-fc2883469a38 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:25.995216 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac31c-5501-4238-84db-fc2883469a38 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:25.996883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aac31c-5501-4238-84db-fc2883469a38 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:25.998738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aac31c-5501-4238-84db-fc2883469a38 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:26.000833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 62/131] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:25 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:26.001861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aac31c-5501-4238-84db-fc2883469a38 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:26.003674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11563193-14c5-4f6b-a5e5-cf0e0672b860 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:26.038599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:26.039239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.039525 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.039823 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.040545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:26.040851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:26.046143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:26.051635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 70/132] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:26.052478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:26.102503 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:26.113163 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:26.171022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbd75e-57f1-410b-8850-4f93f990010a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:26.204397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aac31c-5501-4238-84db-fc2883469a38 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:26.206829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c8d11-79e6-482c-a5c3-a013e67ec503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:26.207598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c8d11-79e6-482c-a5c3-a013e67ec503 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.207944 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c8d11-79e6-482c-a5c3-a013e67ec503 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.208293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c8d11-79e6-482c-a5c3-a013e67ec503 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/users/cee2a726271640868cff7cbf256bac2d/roles/0f002225dde34ef1a2adda2f583b8520` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.210732 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c8d11-79e6-482c-a5c3-a013e67ec503 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ac986a4d83f4f07af22c95818f1b5bb, user_id=cee2a726271640868cff7cbf256bac2d, role_id=0f002225dde34ef1a2adda2f583b8520)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:26.212282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c8d11-79e6-482c-a5c3-a013e67ec503 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:26.232028 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 71/133] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:26 2022] PUT /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/users/cee2a726271640868cff7cbf256bac2d/roles/0f002225dde34ef1a2adda2f583b8520 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:26.233918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c8d11-79e6-482c-a5c3-a013e67ec503 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:26.272884 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d29e8-9ccd-452b-9ae8-854e5afbf7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:26.273448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d29e8-9ccd-452b-9ae8-854e5afbf7e5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.273965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d29e8-9ccd-452b-9ae8-854e5afbf7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.274283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d29e8-9ccd-452b-9ae8-854e5afbf7e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.275811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d29e8-9ccd-452b-9ae8-854e5afbf7e5 None admin] RBAC: Adding query filter params (role.id=a0a0e158ed4f4f9496ff04139752f839, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb, user.id=6d07581a4e294fb09b80c406fc0264c9) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:26.276134 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d29e8-9ccd-452b-9ae8-854e5afbf7e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:26.278411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d29e8-9ccd-452b-9ae8-854e5afbf7e5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:26.286694 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 63/134] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:26 2022] GET /identity/v3/role_assignments?user.id=6d07581a4e294fb09b80c406fc0264c9&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=a0a0e158ed4f4f9496ff04139752f839 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:26.291730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313f44a-3b0e-48b7-8e9f-d0c54786726f None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.292106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313f44a-3b0e-48b7-8e9f-d0c54786726f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.292309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313f44a-3b0e-48b7-8e9f-d0c54786726f None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.294014 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 64/135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:26.302313 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fba46-9a31-4bfb-b939-4e9a48190285 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.302676 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fba46-9a31-4bfb-b939-4e9a48190285 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.302877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fba46-9a31-4bfb-b939-4e9a48190285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.343034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-5f1fba46-9a31-4bfb-b939-4e9a48190285 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:26.344285 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-5f1fba46-9a31-4bfb-b939-4e9a48190285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:26.404148 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:26.414460 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c8d11-79e6-482c-a5c3-a013e67ec503 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:26.457920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17be7ac-c654-4901-b953-66f1e886cd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:26.459156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be7ac-c654-4901-b953-66f1e886cd8a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.459695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be7ac-c654-4901-b953-66f1e886cd8a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.460227 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be7ac-c654-4901-b953-66f1e886cd8a None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.461825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17be7ac-c654-4901-b953-66f1e886cd8a None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:26.463698 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17be7ac-c654-4901-b953-66f1e886cd8a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:26.466320 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 72/136] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:26 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:26.475626 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17be7ac-c654-4901-b953-66f1e886cd8a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:26.480729 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 65/137] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:26.506169 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fba46-9a31-4bfb-b939-4e9a48190285 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:26.600536 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17be7ac-c654-4901-b953-66f1e886cd8a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:26.630555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:26.643134 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2616f-00bc-4ab1-b133-674d8cecc23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:26.643872 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2616f-00bc-4ab1-b133-674d8cecc23a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.644165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2616f-00bc-4ab1-b133-674d8cecc23a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.644446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2616f-00bc-4ab1-b133-674d8cecc23a None admin] PATH_INFO: `/v3/projects/7d381fee65074e4380907d65ce60d876` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.645742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2616f-00bc-4ab1-b133-674d8cecc23a None admin] RBAC: Authorizing `identity:get_project(project_id=7d381fee65074e4380907d65ce60d876)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:26.647289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2616f-00bc-4ab1-b133-674d8cecc23a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:26.649296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 73/138] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:26 2022] GET /identity/v3/projects/7d381fee65074e4380907d65ce60d876 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:26.655349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2616f-00bc-4ab1-b133-674d8cecc23a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:26.692388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:26.714133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fba46-9a31-4bfb-b939-4e9a48190285 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:26.774925 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960650f2-3795-47b3-9dbc-d4f62d8db89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:26.775610 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960650f2-3795-47b3-9dbc-d4f62d8db89c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.775896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960650f2-3795-47b3-9dbc-d4f62d8db89c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.776176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960650f2-3795-47b3-9dbc-d4f62d8db89c None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.782956 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960650f2-3795-47b3-9dbc-d4f62d8db89c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:26.784604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960650f2-3795-47b3-9dbc-d4f62d8db89c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:26.786545 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 66/139] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:26.789435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2616f-00bc-4ab1-b133-674d8cecc23a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:26.795143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960650f2-3795-47b3-9dbc-d4f62d8db89c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:26.826611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c563d-0d2c-4bd5-b620-37ae682aff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:26.827316 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c563d-0d2c-4bd5-b620-37ae682aff97 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:26.827615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c563d-0d2c-4bd5-b620-37ae682aff97 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:26.827902 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c563d-0d2c-4bd5-b620-37ae682aff97 None admin] PATH_INFO: `/v3/projects/7d381fee65074e4380907d65ce60d876/users/cee2a726271640868cff7cbf256bac2d/roles/0aacd40c75f64f668308391b44dc4e85` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:26.829897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c563d-0d2c-4bd5-b620-37ae682aff97 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d381fee65074e4380907d65ce60d876, user_id=cee2a726271640868cff7cbf256bac2d, role_id=0aacd40c75f64f668308391b44dc4e85)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:26.831498 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c563d-0d2c-4bd5-b620-37ae682aff97 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:26.850346 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 74/140] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:26 2022] PUT /identity/v3/projects/7d381fee65074e4380907d65ce60d876/users/cee2a726271640868cff7cbf256bac2d/roles/0aacd40c75f64f668308391b44dc4e85 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:26.971469 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960650f2-3795-47b3-9dbc-d4f62d8db89c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:27.006803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:27.022646 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0739eb-cb8c-42f0-8c2b-d53ed1181a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:27.023301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0739eb-cb8c-42f0-8c2b-d53ed1181a86 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:27.023587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0739eb-cb8c-42f0-8c2b-d53ed1181a86 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:27.023865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0739eb-cb8c-42f0-8c2b-d53ed1181a86 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:27.024572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0739eb-cb8c-42f0-8c2b-d53ed1181a86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:27.026590 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0739eb-cb8c-42f0-8c2b-d53ed1181a86 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:27.058351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 67/141] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:13:26 2022] POST /identity/v3/users => generated 283 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:28.540282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5f07-f180-48a1-8d66-2c391681a90a None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:28.540879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5f07-f180-48a1-8d66-2c391681a90a None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:28.541147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5f07-f180-48a1-8d66-2c391681a90a None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:28.542676 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 75/142] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:28.548602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13519-6aba-448d-9568-b5f91afc2597 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:28.549094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13519-6aba-448d-9568-b5f91afc2597 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:28.549180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13519-6aba-448d-9568-b5f91afc2597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:28.574734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-c8c13519-6aba-448d-9568-b5f91afc2597 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:28.575961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-c8c13519-6aba-448d-9568-b5f91afc2597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:28.659823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 68/143] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:28.707692 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b5f07-f180-48a1-8d66-2c391681a90a None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:28.729772 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6ade-905b-4309-95b8-62458c1980eb None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:28.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6ade-905b-4309-95b8-62458c1980eb None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:28.730353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6ade-905b-4309-95b8-62458c1980eb None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:28.736657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 69/144] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:28.740581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d681d-62ea-4877-a2eb-cdc99b025c3a None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:28.740581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d681d-62ea-4877-a2eb-cdc99b025c3a None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:28.740581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d681d-62ea-4877-a2eb-cdc99b025c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:28.767139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-5d1d681d-62ea-4877-a2eb-cdc99b025c3a None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:28.768408 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-5d1d681d-62ea-4877-a2eb-cdc99b025c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:28.837036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:28.851821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b5f07-f180-48a1-8d66-2c391681a90a None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:28.892192 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e0b7d-342b-4a3f-abb5-dd11efe87344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:28.893299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e0b7d-342b-4a3f-abb5-dd11efe87344 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:28.893849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e0b7d-342b-4a3f-abb5-dd11efe87344 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:28.894381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e0b7d-342b-4a3f-abb5-dd11efe87344 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:28.903227 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 70/145] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:28.909965 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e0b7d-342b-4a3f-abb5-dd11efe87344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-650e0b7d-342b-4a3f-abb5-dd11efe87344 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:28.916164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:28.917213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:28.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:28.921027 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 76/146] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:28.927127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d681d-62ea-4877-a2eb-cdc99b025c3a None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:28.949804 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e0b7d-342b-4a3f-abb5-dd11efe87344 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:29.003867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:29.073218 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e0b7d-342b-4a3f-abb5-dd11efe87344 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:29.115204 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8d6b1-fc4e-4e19-9045-abd431c9cb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:29.116233 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8d6b1-fc4e-4e19-9045-abd431c9cb98 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:29.116743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8d6b1-fc4e-4e19-9045-abd431c9cb98 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:29.117246 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8d6b1-fc4e-4e19-9045-abd431c9cb98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:29.125501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d681d-62ea-4877-a2eb-cdc99b025c3a None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:29.132134 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8d6b1-fc4e-4e19-9045-abd431c9cb98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6cd8d6b1-fc4e-4e19-9045-abd431c9cb98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:29.138036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:29.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:29.140049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:29.142601 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 77/147] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:29.148768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8d6b1-fc4e-4e19-9045-abd431c9cb98 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:29.169747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:29.194051 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:29.195249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:29.195769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:29.196257 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:29.197623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:29.199786 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:29.202211 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:29.212168 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 71/148] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:29.218437 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:29.406642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8d6b1-fc4e-4e19-9045-abd431c9cb98 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:29.425222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884cc11-e753-4de1-8c22-9e47dd63342d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:29.428217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:29.485491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:29.486494 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:29.486932 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:29.487380 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:29.488464 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:29.489145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:29.490841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:29.492316 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238f58a-de4d-402f-aa94-20a5d72cc2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:29.498133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f58a-de4d-402f-aa94-20a5d72cc2ff None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:29.498133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f58a-de4d-402f-aa94-20a5d72cc2ff None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:29.498133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f58a-de4d-402f-aa94-20a5d72cc2ff None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:29.499493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238f58a-de4d-402f-aa94-20a5d72cc2ff None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:29.501766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238f58a-de4d-402f-aa94-20a5d72cc2ff None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:29.504703 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 72/149] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:29 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:29.511854 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238f58a-de4d-402f-aa94-20a5d72cc2ff None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:29.513351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 78/150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:29.519659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:29.633826 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:29.707204 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a5998-b6dc-49cf-bf9e-1ed156044ad6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:29.727535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238f58a-de4d-402f-aa94-20a5d72cc2ff None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:29.772183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136077b-5dd1-4644-bc0a-85d179ec8540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:29.772853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136077b-5dd1-4644-bc0a-85d179ec8540 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:29.773138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136077b-5dd1-4644-bc0a-85d179ec8540 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:29.773414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136077b-5dd1-4644-bc0a-85d179ec8540 None admin] PATH_INFO: `/v3/users/ab47d79a05944b2eb59466f830650026` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:29.775332 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136077b-5dd1-4644-bc0a-85d179ec8540 None admin] RBAC: Authorizing `identity:get_user(user_id=ab47d79a05944b2eb59466f830650026)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:29.778529 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136077b-5dd1-4644-bc0a-85d179ec8540 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:29.780394 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 79/151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:29 2022] GET /identity/v3/users/ab47d79a05944b2eb59466f830650026 => generated 247 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:29.783381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b627e6-6ed7-4d84-a00c-d311d1cfde81 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:29.783910 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b627e6-6ed7-4d84-a00c-d311d1cfde81 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:29.784481 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b627e6-6ed7-4d84-a00c-d311d1cfde81 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:29.785990 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 80/152] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:29.787008 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b627e6-6ed7-4d84-a00c-d311d1cfde81 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:29.802091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916def0-cdc1-485e-ac8a-93e8e8c08a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:29.802792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916def0-cdc1-485e-ac8a-93e8e8c08a16 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:29.803113 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916def0-cdc1-485e-ac8a-93e8e8c08a16 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:29.803395 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916def0-cdc1-485e-ac8a-93e8e8c08a16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:29.870808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:29.901373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b627e6-6ed7-4d84-a00c-d311d1cfde81 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:29.907960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916def0-cdc1-485e-ac8a-93e8e8c08a16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:29.909882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916def0-cdc1-485e-ac8a-93e8e8c08a16 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:29.945312 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f8124-8e21-4477-b347-5ad0e6f9949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:29.947471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f8124-8e21-4477-b347-5ad0e6f9949e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:29.947985 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f8124-8e21-4477-b347-5ad0e6f9949e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:29.948491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f8124-8e21-4477-b347-5ad0e6f9949e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:29.974846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:30.088113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f8124-8e21-4477-b347-5ad0e6f9949e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:30.091131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f8124-8e21-4477-b347-5ad0e6f9949e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:30.108569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 73/153] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:30.110294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5896d4-9b1f-47d5-849c-c21518ee468d None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:30.110828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5896d4-9b1f-47d5-849c-c21518ee468d None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:30.111325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5896d4-9b1f-47d5-849c-c21518ee468d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:30.131386 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-9c5896d4-9b1f-47d5-849c-c21518ee468d None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:30.132967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-9c5896d4-9b1f-47d5-849c-c21518ee468d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:30.201756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:30.208498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 74/154] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:30.210148 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c7ce-294d-4f6b-b3ad-03a04143a01e None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:30.210683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c7ce-294d-4f6b-b3ad-03a04143a01e None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:30.211291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c7ce-294d-4f6b-b3ad-03a04143a01e None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:30.213067 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 75/155] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:30.214375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3c7ce-294d-4f6b-b3ad-03a04143a01e None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:30.287080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 81/156] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:30.288403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461dfd9-32b5-450f-a75b-ad0ef7fb7666 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:30.288694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461dfd9-32b5-450f-a75b-ad0ef7fb7666 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:30.288972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461dfd9-32b5-450f-a75b-ad0ef7fb7666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:30.325351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-3461dfd9-32b5-450f-a75b-ad0ef7fb7666 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:30.326593 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-3461dfd9-32b5-450f-a75b-ad0ef7fb7666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:30.392155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3c7ce-294d-4f6b-b3ad-03a04143a01e None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:30.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:30.480578 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606702b1-694f-42ed-b1a0-01dd4bd9244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:30.481159 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606702b1-694f-42ed-b1a0-01dd4bd9244f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:30.481438 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606702b1-694f-42ed-b1a0-01dd4bd9244f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:30.481714 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606702b1-694f-42ed-b1a0-01dd4bd9244f None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:30.483531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 82/157] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:30.484403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461dfd9-32b5-450f-a75b-ad0ef7fb7666 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:30.485472 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606702b1-694f-42ed-b1a0-01dd4bd9244f None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:30.493160 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606702b1-694f-42ed-b1a0-01dd4bd9244f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:30.495005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 76/158] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:30 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:30.495789 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606702b1-694f-42ed-b1a0-01dd4bd9244f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:30.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:30.662591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461dfd9-32b5-450f-a75b-ad0ef7fb7666 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:30.732333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606702b1-694f-42ed-b1a0-01dd4bd9244f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:30.751509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc17c-2f88-493f-b51a-99a2df286e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:30.752188 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc17c-2f88-493f-b51a-99a2df286e39 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:30.752466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc17c-2f88-493f-b51a-99a2df286e39 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:30.752734 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc17c-2f88-493f-b51a-99a2df286e39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:30.768262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affc17c-2f88-493f-b51a-99a2df286e39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-0affc17c-2f88-493f-b51a-99a2df286e39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:30.775497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:30.776611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 83/159] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:30.777550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc17c-2f88-493f-b51a-99a2df286e39 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:30.785807 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075302b-f4c1-4ee0-9f79-c0b7a5b16939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:30.786534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075302b-f4c1-4ee0-9f79-c0b7a5b16939 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:30.786818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075302b-f4c1-4ee0-9f79-c0b7a5b16939 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:30.787094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075302b-f4c1-4ee0-9f79-c0b7a5b16939 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:30.788398 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075302b-f4c1-4ee0-9f79-c0b7a5b16939 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:30.789979 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075302b-f4c1-4ee0-9f79-c0b7a5b16939 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:30.797294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 77/160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:30 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:30.798486 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075302b-f4c1-4ee0-9f79-c0b7a5b16939 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:30.848719 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:30.976422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc17c-2f88-493f-b51a-99a2df286e39 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:30.982070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075302b-f4c1-4ee0-9f79-c0b7a5b16939 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:31.044994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe14003-a214-452a-8033-8a0ba1028697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:31.045562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe14003-a214-452a-8033-8a0ba1028697 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.045845 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe14003-a214-452a-8033-8a0ba1028697 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.046140 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe14003-a214-452a-8033-8a0ba1028697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.046971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe14003-a214-452a-8033-8a0ba1028697 None admin] RBAC: Adding query filter params (role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb, user.id=cee2a726271640868cff7cbf256bac2d) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:31.048100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df3c48-bf09-4f95-afc4-4e5f0297aaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:31.048686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df3c48-bf09-4f95-afc4-4e5f0297aaa5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.048970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df3c48-bf09-4f95-afc4-4e5f0297aaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.049323 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df3c48-bf09-4f95-afc4-4e5f0297aaa5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.053600 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe14003-a214-452a-8033-8a0ba1028697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:31.053600 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe14003-a214-452a-8033-8a0ba1028697 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:31.055685 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:31.078298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 84/161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:30 2022] GET /identity/v3/role_assignments?user.id=cee2a726271640868cff7cbf256bac2d&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:31.079920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e7d8-1396-45b3-a53d-f56a066a65cd None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.080216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e7d8-1396-45b3-a53d-f56a066a65cd None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.080492 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e7d8-1396-45b3-a53d-f56a066a65cd None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.083116 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df3c48-bf09-4f95-afc4-4e5f0297aaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-18df3c48-bf09-4f95-afc4-4e5f0297aaa5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:31.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:31.091039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:31.093377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:31.095228 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 85/162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:31.095228 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771e7d8-1396-45b3-a53d-f56a066a65cd None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:31.101053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 78/163] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:31.102161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df3c48-bf09-4f95-afc4-4e5f0297aaa5 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:31.302803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:31.314872 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df3c48-bf09-4f95-afc4-4e5f0297aaa5 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:31.323420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771e7d8-1396-45b3-a53d-f56a066a65cd None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:31.385861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:31.386604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.386881 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.387151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.387959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb, user.id=ab47d79a05944b2eb59466f830650026) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:31.388198 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b170c4-580f-4662-9fe7-aa8821687ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:31.388417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:31.388840 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b170c4-580f-4662-9fe7-aa8821687ef5 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.389174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b170c4-580f-4662-9fe7-aa8821687ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.389457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b170c4-580f-4662-9fe7-aa8821687ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.389725 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:31.390230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b170c4-580f-4662-9fe7-aa8821687ef5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:31.390547 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b170c4-580f-4662-9fe7-aa8821687ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:31.392106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b170c4-580f-4662-9fe7-aa8821687ef5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:31.396494 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 79/164] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:31 2022] GET /identity/v3/role_assignments?user.id=ab47d79a05944b2eb59466f830650026&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=40504e80dbb845aeb07ca258977ecacb => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:31.397355 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:31.412672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 86/165] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:31.417997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef81a7c-6dd7-4d9f-b99b-1d4165b8132e None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.417997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef81a7c-6dd7-4d9f-b99b-1d4165b8132e None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.417997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef81a7c-6dd7-4d9f-b99b-1d4165b8132e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.462389 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-aef81a7c-6dd7-4d9f-b99b-1d4165b8132e None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:31.463641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-aef81a7c-6dd7-4d9f-b99b-1d4165b8132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:31.492500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:31.573863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a082e-110d-4590-9386-de4e2a8bd8c3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:31.622581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 87/166] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:31.624961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137e03-2dec-4204-aa90-7d9fe520902a None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.625247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137e03-2dec-4204-aa90-7d9fe520902a None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.625529 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137e03-2dec-4204-aa90-7d9fe520902a None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.626994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 88/167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:31.628635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d137e03-2dec-4204-aa90-7d9fe520902a None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:31.655353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0269a0b-d002-4034-ad2c-d834a3eded03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:31.655353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0269a0b-d002-4034-ad2c-d834a3eded03 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.658581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0269a0b-d002-4034-ad2c-d834a3eded03 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.658581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0269a0b-d002-4034-ad2c-d834a3eded03 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.658581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0269a0b-d002-4034-ad2c-d834a3eded03 None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:31.658581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0269a0b-d002-4034-ad2c-d834a3eded03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:31.658581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0269a0b-d002-4034-ad2c-d834a3eded03 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:31.674081 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 80/168] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:31.676413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50f790-fc7a-4d68-b6db-78c5822a86f1 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.676890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50f790-fc7a-4d68-b6db-78c5822a86f1 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.677336 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50f790-fc7a-4d68-b6db-78c5822a86f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.700832 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-4f50f790-fc7a-4d68-b6db-78c5822a86f1 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:31.702375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-4f50f790-fc7a-4d68-b6db-78c5822a86f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:31.778581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:31.789841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 81/169] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:31.793765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50f790-fc7a-4d68-b6db-78c5822a86f1 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:31.807782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d137e03-2dec-4204-aa90-7d9fe520902a None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:31.852313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315920-71b7-4fdd-8e5e-55b83d4c463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:31.853311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315920-71b7-4fdd-8e5e-55b83d4c463d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.853747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315920-71b7-4fdd-8e5e-55b83d4c463d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315920-71b7-4fdd-8e5e-55b83d4c463d None admin] PATH_INFO: `/v3/users/6d07581a4e294fb09b80c406fc0264c9` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:31.855738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29315920-71b7-4fdd-8e5e-55b83d4c463d None admin] RBAC: Authorizing `identity:get_user(user_id=6d07581a4e294fb09b80c406fc0264c9)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:31.857521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29315920-71b7-4fdd-8e5e-55b83d4c463d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:31.859545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 89/170] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:31 2022] GET /identity/v3/users/6d07581a4e294fb09b80c406fc0264c9 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:31.860915 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315920-71b7-4fdd-8e5e-55b83d4c463d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:31.874033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:31.938274 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50f790-fc7a-4d68-b6db-78c5822a86f1 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:31.984090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315920-71b7-4fdd-8e5e-55b83d4c463d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:31.986986 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb3065-67e1-4e31-846c-f12933691985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:31.987794 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb3065-67e1-4e31-846c-f12933691985 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:31.988161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb3065-67e1-4e31-846c-f12933691985 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:31.988506 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb3065-67e1-4e31-846c-f12933691985 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.001088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb3065-67e1-4e31-846c-f12933691985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-0afb3065-67e1-4e31-846c-f12933691985 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:32.007950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:32.009181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:32.010501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 82/171] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:32.011390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb3065-67e1-4e31-846c-f12933691985 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.028277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00654d77-91c5-4315-889e-521cb776eae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.028974 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00654d77-91c5-4315-889e-521cb776eae7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.029252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00654d77-91c5-4315-889e-521cb776eae7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.029527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00654d77-91c5-4315-889e-521cb776eae7 None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.030892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00654d77-91c5-4315-889e-521cb776eae7 None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:32.032375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00654d77-91c5-4315-889e-521cb776eae7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:32.034142 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 90/172] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:31 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:32.035636 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e1ba-ba33-4392-a250-09b0803ad935 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.035912 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e1ba-ba33-4392-a250-09b0803ad935 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.036186 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e1ba-ba33-4392-a250-09b0803ad935 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.038199 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 91/173] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:32.038323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6e1ba-ba33-4392-a250-09b0803ad935 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.066303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:32.133617 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb3065-67e1-4e31-846c-f12933691985 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.145546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:32.153340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6e1ba-ba33-4392-a250-09b0803ad935 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.172051 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c083de7-2764-4c17-bf36-ebc6ad846a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.172813 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c083de7-2764-4c17-bf36-ebc6ad846a27 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.173123 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c083de7-2764-4c17-bf36-ebc6ad846a27 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.173350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c083de7-2764-4c17-bf36-ebc6ad846a27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.187431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c083de7-2764-4c17-bf36-ebc6ad846a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:32.190732 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ef700-42d5-4544-ad30-92d0ed3a655e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.191392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ef700-42d5-4544-ad30-92d0ed3a655e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.191677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ef700-42d5-4544-ad30-92d0ed3a655e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.191955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ef700-42d5-4544-ad30-92d0ed3a655e None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-2c083de7-2764-4c17-bf36-ebc6ad846a27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:32.192640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:32.193831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:32.195026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:32.195823 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ef700-42d5-4544-ad30-92d0ed3a655e None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:32.195900 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 83/174] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:32.195982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ef700-42d5-4544-ad30-92d0ed3a655e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:32.196064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e113-0667-4ab2-b911-7dc85c68e48f None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.196064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e113-0667-4ab2-b911-7dc85c68e48f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.196064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e113-0667-4ab2-b911-7dc85c68e48f None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.201826 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 84/176] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:32.201968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 92/176] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:32.202053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694e113-0667-4ab2-b911-7dc85c68e48f None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.202152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ef700-42d5-4544-ad30-92d0ed3a655e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.292768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:32.303844 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ef700-42d5-4544-ad30-92d0ed3a655e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.304161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694e113-0667-4ab2-b911-7dc85c68e48f None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.338858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42e0cb-5aba-4bd6-85a3-501773e021f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.339138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e9dcc-cb41-44e9-8126-6f3a7cade0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.339501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42e0cb-5aba-4bd6-85a3-501773e021f1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.339769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e9dcc-cb41-44e9-8126-6f3a7cade0db None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.339867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42e0cb-5aba-4bd6-85a3-501773e021f1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.340045 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42e0cb-5aba-4bd6-85a3-501773e021f1 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.340139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e9dcc-cb41-44e9-8126-6f3a7cade0db None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.340315 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e9dcc-cb41-44e9-8126-6f3a7cade0db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.340781 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42e0cb-5aba-4bd6-85a3-501773e021f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:32.341088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42e0cb-5aba-4bd6-85a3-501773e021f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:32.342550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42e0cb-5aba-4bd6-85a3-501773e021f1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:32.351698 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 85/177] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:32.353892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143dc9e-8d3b-4a32-a339-0e788af8a04b None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.354289 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143dc9e-8d3b-4a32-a339-0e788af8a04b None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.354571 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143dc9e-8d3b-4a32-a339-0e788af8a04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.373011 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-6143dc9e-8d3b-4a32-a339-0e788af8a04b None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:32.374338 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-6143dc9e-8d3b-4a32-a339-0e788af8a04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:32.402028 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:32.412014 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e9dcc-cb41-44e9-8126-6f3a7cade0db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:32.413505 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e9dcc-cb41-44e9-8126-6f3a7cade0db None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:32.444222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 86/178] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:32.445170 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143dc9e-8d3b-4a32-a339-0e788af8a04b None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.506168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:32.516911 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 93/179] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:32.517814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e9dcc-cb41-44e9-8126-6f3a7cade0db None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.556647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143dc9e-8d3b-4a32-a339-0e788af8a04b None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.592429 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58d002-9508-4c2a-a996-c42a5c281f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.593062 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d002-9508-4c2a-a996-c42a5c281f71 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.593341 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d002-9508-4c2a-a996-c42a5c281f71 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.593614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d002-9508-4c2a-a996-c42a5c281f71 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.594456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58d002-9508-4c2a-a996-c42a5c281f71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:32.594765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58d002-9508-4c2a-a996-c42a5c281f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:32.596188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58d002-9508-4c2a-a996-c42a5c281f71 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:32.605354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 87/180] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:32.606624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de479121-b3b9-4352-b567-3d62ab54a551 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.606915 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de479121-b3b9-4352-b567-3d62ab54a551 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.607191 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de479121-b3b9-4352-b567-3d62ab54a551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.620735 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e9dcc-cb41-44e9-8126-6f3a7cade0db None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.625604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-de479121-b3b9-4352-b567-3d62ab54a551 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:32.626855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-de479121-b3b9-4352-b567-3d62ab54a551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:32.644513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:32.655850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c894-368c-4c13-af81-c2dd3e0c8cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.656484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c894-368c-4c13-af81-c2dd3e0c8cc4 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.656773 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c894-368c-4c13-af81-c2dd3e0c8cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.657051 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c894-368c-4c13-af81-c2dd3e0c8cc4 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/users/6d07581a4e294fb09b80c406fc0264c9/roles/a0a0e158ed4f4f9496ff04139752f839` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.659193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c894-368c-4c13-af81-c2dd3e0c8cc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ac986a4d83f4f07af22c95818f1b5bb, user_id=6d07581a4e294fb09b80c406fc0264c9, role_id=a0a0e158ed4f4f9496ff04139752f839)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:32.660780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c894-368c-4c13-af81-c2dd3e0c8cc4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:32.677535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 94/181] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:32 2022] PUT /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/users/6d07581a4e294fb09b80c406fc0264c9/roles/a0a0e158ed4f4f9496ff04139752f839 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:32.678711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c894-368c-4c13-af81-c2dd3e0c8cc4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.708535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 88/182] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:32.709843 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de479121-b3b9-4352-b567-3d62ab54a551 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.753439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:32.813551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c894-368c-4c13-af81-c2dd3e0c8cc4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.843000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de479121-b3b9-4352-b567-3d62ab54a551 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.858809 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3483f9f-5533-402e-9442-2ae0b3a10672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.866161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3483f9f-5533-402e-9442-2ae0b3a10672 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.866161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3483f9f-5533-402e-9442-2ae0b3a10672 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.866161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3483f9f-5533-402e-9442-2ae0b3a10672 None admin] PATH_INFO: `/v3/users/cee2a726271640868cff7cbf256bac2d` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.866161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3483f9f-5533-402e-9442-2ae0b3a10672 None admin] RBAC: Authorizing `identity:get_user(user_id=cee2a726271640868cff7cbf256bac2d)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:32.866161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3483f9f-5533-402e-9442-2ae0b3a10672 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:32.866161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 95/183] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/users/cee2a726271640868cff7cbf256bac2d => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:32.866769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3483f9f-5533-402e-9442-2ae0b3a10672 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.887444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929713f-4bbf-44f5-a62e-b93be0eddbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:32.888413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929713f-4bbf-44f5-a62e-b93be0eddbae None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:32.888845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929713f-4bbf-44f5-a62e-b93be0eddbae None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:32.889264 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929713f-4bbf-44f5-a62e-b93be0eddbae None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:32.891317 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929713f-4bbf-44f5-a62e-b93be0eddbae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:32.893428 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929713f-4bbf-44f5-a62e-b93be0eddbae None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:32.895703 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 89/184] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:32.897078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929713f-4bbf-44f5-a62e-b93be0eddbae None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:32.927281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:32.976508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3483f9f-5533-402e-9442-2ae0b3a10672 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:32.996851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:33.016401 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c7ccd-f223-442a-a972-90675b91563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:33.017190 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c7ccd-f223-442a-a972-90675b91563e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:33.017451 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c7ccd-f223-442a-a972-90675b91563e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:33.017728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c7ccd-f223-442a-a972-90675b91563e None admin] PATH_INFO: `/v3/projects/7d381fee65074e4380907d65ce60d876` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:33.019112 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c7ccd-f223-442a-a972-90675b91563e None admin] RBAC: Authorizing `identity:get_project(project_id=7d381fee65074e4380907d65ce60d876)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:33.020571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c7ccd-f223-442a-a972-90675b91563e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:33.020918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929713f-4bbf-44f5-a62e-b93be0eddbae None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:33.027228 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 96/185] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/projects/7d381fee65074e4380907d65ce60d876 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:33.027228 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c7ccd-f223-442a-a972-90675b91563e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:33.065715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d137a4-b146-4068-b8aa-20700862dc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:33.066423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d137a4-b146-4068-b8aa-20700862dc1e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:33.066704 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d137a4-b146-4068-b8aa-20700862dc1e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:33.066973 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d137a4-b146-4068-b8aa-20700862dc1e None admin] PATH_INFO: `/v3/users/a2683672ffd64d99823348e2781589fc` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:33.083093 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d137a4-b146-4068-b8aa-20700862dc1e None admin] RBAC: Authorizing `identity:get_user(user_id=a2683672ffd64d99823348e2781589fc)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:33.084629 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d137a4-b146-4068-b8aa-20700862dc1e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:33.086762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 90/186] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:32 2022] GET /identity/v3/users/a2683672ffd64d99823348e2781589fc => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:33.087429 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d137a4-b146-4068-b8aa-20700862dc1e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:33.149343 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c7ccd-f223-442a-a972-90675b91563e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:33.176522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:33.199281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d137a4-b146-4068-b8aa-20700862dc1e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:33.199482 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63ef06-5b96-4734-bf50-eb5b4df104e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:33.200019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63ef06-5b96-4734-bf50-eb5b4df104e1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:33.200302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63ef06-5b96-4734-bf50-eb5b4df104e1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:33.200604 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63ef06-5b96-4734-bf50-eb5b4df104e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:33.211318 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63ef06-5b96-4734-bf50-eb5b4df104e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-2e63ef06-5b96-4734-bf50-eb5b4df104e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:33.216344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:33.217635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 97/187] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:33.219291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63ef06-5b96-4734-bf50-eb5b4df104e1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:33.242158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf541e-8044-4b97-bd26-19594c1bdfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:33.242806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf541e-8044-4b97-bd26-19594c1bdfc4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:33.243115 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf541e-8044-4b97-bd26-19594c1bdfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:33.243353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf541e-8044-4b97-bd26-19594c1bdfc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:33.268382 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:33.315210 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf541e-8044-4b97-bd26-19594c1bdfc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:33.317190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf541e-8044-4b97-bd26-19594c1bdfc4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:33.324639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63ef06-5b96-4734-bf50-eb5b4df104e1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:33.363243 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364d5f7-308f-442e-a48f-b599ee4a76bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:33.364289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364d5f7-308f-442e-a48f-b599ee4a76bd None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:33.364790 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364d5f7-308f-442e-a48f-b599ee4a76bd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:33.365160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364d5f7-308f-442e-a48f-b599ee4a76bd None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:33.366025 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364d5f7-308f-442e-a48f-b599ee4a76bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:33.367573 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364d5f7-308f-442e-a48f-b599ee4a76bd None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:33.410163 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 98/188] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:13:33 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:33.410163 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364d5f7-308f-442e-a48f-b599ee4a76bd None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:33.411808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:33.428025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 91/189] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:33.434650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf541e-8044-4b97-bd26-19594c1bdfc4 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:33.557715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf541e-8044-4b97-bd26-19594c1bdfc4 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:33.557939 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364d5f7-308f-442e-a48f-b599ee4a76bd None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:33.560151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:33.606953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:33.608430 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:33.608711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:33.609679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:33.609812 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] RBAC: Adding query filter params (role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=7d381fee65074e4380907d65ce60d876, user.id=cee2a726271640868cff7cbf256bac2d) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:33.610119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:33.611604 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:33.616753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68328275-a547-4048-a024-8c7081a283a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:33.617490 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68328275-a547-4048-a024-8c7081a283a5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:33.617690 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68328275-a547-4048-a024-8c7081a283a5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:33.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68328275-a547-4048-a024-8c7081a283a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:33.619519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 99/190] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:33 2022] GET /identity/v3/role_assignments?user.id=cee2a726271640868cff7cbf256bac2d&scope.project.id=7d381fee65074e4380907d65ce60d876&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:33.622248 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:33.699871 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:33.729182 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68328275-a547-4048-a024-8c7081a283a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:33.731242 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68328275-a547-4048-a024-8c7081a283a5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:33.780815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31751cfb-6348-422d-8bcb-fa86e484e7c4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:33.830971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:33.833156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:33.833678 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:33.834176 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:33.835153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:33.835678 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:33.837342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:33.847801 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 100/191] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:33.853503 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:33.871429 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 92/192] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:33.871967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68328275-a547-4048-a024-8c7081a283a5 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:33.903695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:33.981339 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:34.003519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766e235-7e87-4830-b30d-94d39a2defeb None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:34.009212 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68328275-a547-4048-a024-8c7081a283a5 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:34.059973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6faf06-dfef-4512-a97f-6fdb7b34a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:34.060890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6faf06-dfef-4512-a97f-6fdb7b34a139 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:34.061179 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6faf06-dfef-4512-a97f-6fdb7b34a139 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:34.061458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6faf06-dfef-4512-a97f-6fdb7b34a139 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:34.062902 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6faf06-dfef-4512-a97f-6fdb7b34a139 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:34.064394 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6faf06-dfef-4512-a97f-6fdb7b34a139 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:34.066022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da095d-c502-4c8d-aa15-8898a0290c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:34.066830 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da095d-c502-4c8d-aa15-8898a0290c39 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:34.067173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da095d-c502-4c8d-aa15-8898a0290c39 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:34.067600 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da095d-c502-4c8d-aa15-8898a0290c39 None admin] PATH_INFO: `/v3/users/a2683672ffd64d99823348e2781589fc` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:34.069031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da095d-c502-4c8d-aa15-8898a0290c39 None admin] RBAC: Authorizing `identity:get_user(user_id=a2683672ffd64d99823348e2781589fc)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:34.070220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 101/193] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:33 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:34.070888 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da095d-c502-4c8d-aa15-8898a0290c39 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:34.071145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6faf06-dfef-4512-a97f-6fdb7b34a139 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:34.073448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 93/194] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:33 2022] GET /identity/v3/users/a2683672ffd64d99823348e2781589fc => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:34.078958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da095d-c502-4c8d-aa15-8898a0290c39 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:34.208701 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:34.222162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6faf06-dfef-4512-a97f-6fdb7b34a139 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:34.243904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da095d-c502-4c8d-aa15-8898a0290c39 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:34.252829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff08d-a6af-49be-a887-be46b55436f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:34.253294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff08d-a6af-49be-a887-be46b55436f9 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:34.253582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff08d-a6af-49be-a887-be46b55436f9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:34.253862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff08d-a6af-49be-a887-be46b55436f9 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:34.263164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fff08d-a6af-49be-a887-be46b55436f9 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:34.264980 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fff08d-a6af-49be-a887-be46b55436f9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:34.271650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 102/195] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:34 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:34.271650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff08d-a6af-49be-a887-be46b55436f9 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:34.289098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:34.290042 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:34.290502 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:34.290931 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:34.291893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] RBAC: Adding query filter params (role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb, user.id=cee2a726271640868cff7cbf256bac2d) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:34.292420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:34.294213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:34.301265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 94/196] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:34 2022] GET /identity/v3/role_assignments?user.id=cee2a726271640868cff7cbf256bac2d&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:34.307537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:34.333547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:34.450717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff08d-a6af-49be-a887-be46b55436f9 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:34.508884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32720e97-bb5b-48bb-908f-3983acfc2aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:34.509510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720e97-bb5b-48bb-908f-3983acfc2aed None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:34.509799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720e97-bb5b-48bb-908f-3983acfc2aed None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:34.510076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720e97-bb5b-48bb-908f-3983acfc2aed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:34.579138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:34.596538 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32b919-28c9-4b6f-9f79-b678fd0ce9c1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:34.650702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32720e97-bb5b-48bb-908f-3983acfc2aed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:34.652252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32720e97-bb5b-48bb-908f-3983acfc2aed None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:34.678036 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:34.678721 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:34.679010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:34.679289 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:34.680109 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee, user.id=a2683672ffd64d99823348e2781589fc) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:34.680421 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:34.682474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:34.698554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 95/197] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:34 2022] GET /identity/v3/role_assignments?user.id=a2683672ffd64d99823348e2781589fc&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=40504e80dbb845aeb07ca258977ecacb => generated 259 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:34.851334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 103/198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:34.857377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:34.966711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:35.002441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb5c6a-2e3b-402b-a334-80e5b108c4e2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:35.009768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5087d-77a7-4211-9e46-89723b36af65 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:35.010522 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5087d-77a7-4211-9e46-89723b36af65 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:35.011036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5087d-77a7-4211-9e46-89723b36af65 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:35.012967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 104/199] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:35.019123 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db468f-2711-4049-9ee9-7200d5af73af None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:35.019690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db468f-2711-4049-9ee9-7200d5af73af None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:35.020180 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db468f-2711-4049-9ee9-7200d5af73af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:35.044588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-21db468f-2711-4049-9ee9-7200d5af73af None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:35.045241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-21db468f-2711-4049-9ee9-7200d5af73af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:35.065186 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bf587-c682-42ef-acf4-b430330cdbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:35.066334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf587-c682-42ef-acf4-b430330cdbc2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:35.066836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf587-c682-42ef-acf4-b430330cdbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:35.067339 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf587-c682-42ef-acf4-b430330cdbc2 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:35.068567 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bf587-c682-42ef-acf4-b430330cdbc2 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:35.070059 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bf587-c682-42ef-acf4-b430330cdbc2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:35.072628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 96/200] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:34 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:35.085191 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bf587-c682-42ef-acf4-b430330cdbc2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:35.156384 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:35.170558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 105/201] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:35.222680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db468f-2711-4049-9ee9-7200d5af73af None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:35.275889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bf587-c682-42ef-acf4-b430330cdbc2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:35.335878 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:35.336811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:35.337105 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:35.337390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:35.338240 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] RBAC: Adding query filter params (role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee, user.id=a2683672ffd64d99823348e2781589fc) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:35.338560 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:35.340182 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:35.347275 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 97/202] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:35 2022] GET /identity/v3/role_assignments?user.id=a2683672ffd64d99823348e2781589fc&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:35.429674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db468f-2711-4049-9ee9-7200d5af73af None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:35.477268 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd5287-9fa0-44c0-8552-c0b941ab2d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:35.477959 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5287-9fa0-44c0-8552-c0b941ab2d57 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:35.478268 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5287-9fa0-44c0-8552-c0b941ab2d57 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:35.478546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5287-9fa0-44c0-8552-c0b941ab2d57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:35.495491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd5287-9fa0-44c0-8552-c0b941ab2d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-cfdd5287-9fa0-44c0-8552-c0b941ab2d57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:35.500951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:35.502020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:35.503562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 106/203] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:35.508538 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:35.524506 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:35.661741 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e487ebb-708b-4056-848f-882d9b8100b8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:35.709935 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:35.710525 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:35.710811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:35.711084 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:35.711818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:35.712126 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:35.713546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:35.722626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 98/204] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:35.731236 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd5287-9fa0-44c0-8552-c0b941ab2d57 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:35.807541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:35.867277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd5287-9fa0-44c0-8552-c0b941ab2d57 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:35.917711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae24a77-1def-4acd-9ce5-c6b7608b935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:35.918414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae24a77-1def-4acd-9ce5-c6b7608b935a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:35.918701 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae24a77-1def-4acd-9ce5-c6b7608b935a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:35.918982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae24a77-1def-4acd-9ce5-c6b7608b935a None admin] PATH_INFO: `/v3/users/6d07581a4e294fb09b80c406fc0264c9` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:35.920202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae24a77-1def-4acd-9ce5-c6b7608b935a None admin] RBAC: Authorizing `identity:get_user(user_id=6d07581a4e294fb09b80c406fc0264c9)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:35.921683 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae24a77-1def-4acd-9ce5-c6b7608b935a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:35.923381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 107/205] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:35 2022] GET /identity/v3/users/6d07581a4e294fb09b80c406fc0264c9 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:35.931262 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:36.011635 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:36.046235 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceb606-f05a-464d-960d-5ee5874bbf99 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:36.081705 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1e801-8eab-49ed-af5f-b8732edd9dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:36.082455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1e801-8eab-49ed-af5f-b8732edd9dac None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:36.082740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1e801-8eab-49ed-af5f-b8732edd9dac None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:36.083013 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1e801-8eab-49ed-af5f-b8732edd9dac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:36.152522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1e801-8eab-49ed-af5f-b8732edd9dac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:36.154014 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1e801-8eab-49ed-af5f-b8732edd9dac None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:36.226290 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:36.259888 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 99/206] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:36.267361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae24a77-1def-4acd-9ce5-c6b7608b935a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:36.351421 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35506e9-b12e-4123-9425-60488684dd2a None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:36.351762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35506e9-b12e-4123-9425-60488684dd2a None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:36.351949 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35506e9-b12e-4123-9425-60488684dd2a None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:36.353620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 100/207] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:36.359100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6145654-7f9c-42cd-bb77-020fd99b9984 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:36.359377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6145654-7f9c-42cd-bb77-020fd99b9984 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:36.359649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6145654-7f9c-42cd-bb77-020fd99b9984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:36.408221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-a6145654-7f9c-42cd-bb77-020fd99b9984 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:36.409527 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-a6145654-7f9c-42cd-bb77-020fd99b9984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:36.425686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae24a77-1def-4acd-9ce5-c6b7608b935a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:36.468591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:36.492658 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da72b-3a73-4ebc-90a1-cc82a5c0c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:36.493336 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da72b-3a73-4ebc-90a1-cc82a5c0c2be None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:36.493621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da72b-3a73-4ebc-90a1-cc82a5c0c2be None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:36.493894 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da72b-3a73-4ebc-90a1-cc82a5c0c2be None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:36.497201 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7da72b-3a73-4ebc-90a1-cc82a5c0c2be None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:36.498981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7da72b-3a73-4ebc-90a1-cc82a5c0c2be None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:36.503355 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 108/208] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:36 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:36.509288 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da72b-3a73-4ebc-90a1-cc82a5c0c2be None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:36.544245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 101/209] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:36.545913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2e6e-4a2c-42a8-8c6d-587a7ddac9f4 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:36.546444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2e6e-4a2c-42a8-8c6d-587a7ddac9f4 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:36.546959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2e6e-4a2c-42a8-8c6d-587a7ddac9f4 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:36.548501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 102/210] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:36.555553 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80899a48-5be1-464c-ba11-8a4ac1cc66b1 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:36.555831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80899a48-5be1-464c-ba11-8a4ac1cc66b1 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:36.557107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80899a48-5be1-464c-ba11-8a4ac1cc66b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:36.577037 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-80899a48-5be1-464c-ba11-8a4ac1cc66b1 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:36.578296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-80899a48-5be1-464c-ba11-8a4ac1cc66b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:36.642784 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da72b-3a73-4ebc-90a1-cc82a5c0c2be None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:36.674201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 103/211] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:36.675493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80899a48-5be1-464c-ba11-8a4ac1cc66b1 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:36.706968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:36.707885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:36.708459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:36.708732 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:36.709540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] RBAC: Adding query filter params (role.id=a0a0e158ed4f4f9496ff04139752f839, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb, user.id=6d07581a4e294fb09b80c406fc0264c9) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:36.709846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:36.711356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:36.718156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 109/212] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:36 2022] GET /identity/v3/role_assignments?user.id=6d07581a4e294fb09b80c406fc0264c9&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=a0a0e158ed4f4f9496ff04139752f839 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:36.719026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:36.729107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:36.823984 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80899a48-5be1-464c-ba11-8a4ac1cc66b1 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:36.870902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a2bb-5a0d-4a89-be11-75cbc1a665cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:36.872439 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a2bb-5a0d-4a89-be11-75cbc1a665cd None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:36.872943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a2bb-5a0d-4a89-be11-75cbc1a665cd None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:36.873421 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a2bb-5a0d-4a89-be11-75cbc1a665cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:36.883876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:36.889819 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981a2bb-5a0d-4a89-be11-75cbc1a665cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-3981a2bb-5a0d-4a89-be11-75cbc1a665cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:36.896631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:36.897762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:36.898767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:36.901668 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 104/213] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:36.909350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a2bb-5a0d-4a89-be11-75cbc1a665cd None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:37.023647 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddacaba-ef7d-4435-8beb-174a8f0efc1e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:37.025458 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:37.050546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a2bb-5a0d-4a89-be11-75cbc1a665cd None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:37.076668 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527b7bc-edb2-4429-855d-ee063e1be065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:37.077483 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b7bc-edb2-4429-855d-ee063e1be065 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.077599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b7bc-edb2-4429-855d-ee063e1be065 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.077777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b7bc-edb2-4429-855d-ee063e1be065 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.079029 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527b7bc-edb2-4429-855d-ee063e1be065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:37.080501 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527b7bc-edb2-4429-855d-ee063e1be065 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:37.086878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 110/214] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:37.089702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527b7bc-edb2-4429-855d-ee063e1be065 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:37.105434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:37.106451 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.107410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.107903 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.109267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:37.110055 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:37.112015 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:37.125199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 105/215] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:37.131395 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:37.241998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527b7bc-edb2-4429-855d-ee063e1be065 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:37.274871 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000aab96-8a73-49d1-875c-dbdb1996930e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:37.276903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:37.290680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb2359-4ce5-4162-bccb-32bdffa674d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:37.291338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2359-4ce5-4162-bccb-32bdffa674d1 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.291620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2359-4ce5-4162-bccb-32bdffa674d1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.291899 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2359-4ce5-4162-bccb-32bdffa674d1 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.292615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb2359-4ce5-4162-bccb-32bdffa674d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:37.294130 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb2359-4ce5-4162-bccb-32bdffa674d1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:37.317353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d200f1d-c0cb-49bc-bf74-d0463746c19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:37.322351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d200f1d-c0cb-49bc-bf74-d0463746c19c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.322592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 111/216] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:13:37 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:37.322698 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d200f1d-c0cb-49bc-bf74-d0463746c19c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.322698 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d200f1d-c0cb-49bc-bf74-d0463746c19c None admin] PATH_INFO: `/v3/users/8ed76d53fe964bf790be7083bc868622` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.322876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13fa6-20a5-4d05-aca4-a987954e5959 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.322876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13fa6-20a5-4d05-aca4-a987954e5959 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.322876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13fa6-20a5-4d05-aca4-a987954e5959 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.326352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 112/217] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:37.327958 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819cd6-ef41-4f1a-8332-fd5ca0b0ae15 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.328186 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819cd6-ef41-4f1a-8332-fd5ca0b0ae15 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.328466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819cd6-ef41-4f1a-8332-fd5ca0b0ae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.347779 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d200f1d-c0cb-49bc-bf74-d0463746c19c None admin] RBAC: Authorizing `identity:get_user(user_id=8ed76d53fe964bf790be7083bc868622)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:37.349531 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d200f1d-c0cb-49bc-bf74-d0463746c19c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:37.351189 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-fe819cd6-ef41-4f1a-8332-fd5ca0b0ae15 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:37.351586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 106/218] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:37 2022] GET /identity/v3/users/8ed76d53fe964bf790be7083bc868622 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:37.353128 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-fe819cd6-ef41-4f1a-8332-fd5ca0b0ae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:37.358861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d200f1d-c0cb-49bc-bf74-d0463746c19c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:37.374214 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:37.456384 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 113/219] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:37.485720 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819cd6-ef41-4f1a-8332-fd5ca0b0ae15 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:37.504789 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d200f1d-c0cb-49bc-bf74-d0463746c19c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:37.553681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da503-fd7b-4602-ac96-565c8e529427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:37.554445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da503-fd7b-4602-ac96-565c8e529427 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.554730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da503-fd7b-4602-ac96-565c8e529427 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.555006 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da503-fd7b-4602-ac96-565c8e529427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.617030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819cd6-ef41-4f1a-8332-fd5ca0b0ae15 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:37.628535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:37.658482 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da503-fd7b-4602-ac96-565c8e529427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:37.660031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da503-fd7b-4602-ac96-565c8e529427 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:37.667994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a3f02-1e70-4272-933c-e86a72d48e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:37.670074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a3f02-1e70-4272-933c-e86a72d48e0e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.670074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a3f02-1e70-4272-933c-e86a72d48e0e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.670074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a3f02-1e70-4272-933c-e86a72d48e0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.678697 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:37.684848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a3f02-1e70-4272-933c-e86a72d48e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6c1a3f02-1e70-4272-933c-e86a72d48e0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:37.690952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:37.692193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:37.693223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 114/220] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:37.702079 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a3f02-1e70-4272-933c-e86a72d48e0e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:37.807208 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 107/221] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:13:37.808555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45650c-d175-4fc0-829c-32308b97a0fe None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.808847 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45650c-d175-4fc0-829c-32308b97a0fe None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.809224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45650c-d175-4fc0-829c-32308b97a0fe None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.815090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 108/222] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:37.816088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45650c-d175-4fc0-829c-32308b97a0fe None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:37.835584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a3f02-1e70-4272-933c-e86a72d48e0e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:37.877078 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2066a-bf84-42dc-96bb-7d2cc2efc639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:37.877797 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2066a-bf84-42dc-96bb-7d2cc2efc639 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.878165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2066a-bf84-42dc-96bb-7d2cc2efc639 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2066a-bf84-42dc-96bb-7d2cc2efc639 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.882333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2066a-bf84-42dc-96bb-7d2cc2efc639 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:37.882333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2066a-bf84-42dc-96bb-7d2cc2efc639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:37.882333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2066a-bf84-42dc-96bb-7d2cc2efc639 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:37.894230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 115/223] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:37.895648 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e33ba9-ff3f-41aa-ad3d-53ed1d0a3034 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.895941 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e33ba9-ff3f-41aa-ad3d-53ed1d0a3034 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.896220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e33ba9-ff3f-41aa-ad3d-53ed1d0a3034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.915773 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-67e33ba9-ff3f-41aa-ad3d-53ed1d0a3034 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:37.916976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-67e33ba9-ff3f-41aa-ad3d-53ed1d0a3034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:37.939553 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45650c-d175-4fc0-829c-32308b97a0fe None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:37.945705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:37.952499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:37.981350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fff843-f054-47a0-9acc-d13febadc5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:37.982604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fff843-f054-47a0-9acc-d13febadc5df None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:37.983063 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fff843-f054-47a0-9acc-d13febadc5df None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:37.983638 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fff843-f054-47a0-9acc-d13febadc5df None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:37.985697 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fff843-f054-47a0-9acc-d13febadc5df None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:37.987897 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fff843-f054-47a0-9acc-d13febadc5df None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:37.990140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 109/224] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:37 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:37.995826 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fff843-f054-47a0-9acc-d13febadc5df None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:37.995991 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 116/225] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:37.997030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e33ba9-ff3f-41aa-ad3d-53ed1d0a3034 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:38.118999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e33ba9-ff3f-41aa-ad3d-53ed1d0a3034 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:38.131836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fff843-f054-47a0-9acc-d13febadc5df None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:38.155259 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:38.155897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:38.156179 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:38.156458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:38.157565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] RBAC: Adding query filter params (role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee, user.id=8ed76d53fe964bf790be7083bc868622) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:38.158136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:38.160760 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:38.174149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 117/226] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:37 2022] GET /identity/v3/role_assignments?user.id=8ed76d53fe964bf790be7083bc868622&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:38.174149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:38.186578 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7e4f9-2677-411c-9b4e-04a106872337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:38.187691 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7e4f9-2677-411c-9b4e-04a106872337 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:38.188268 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7e4f9-2677-411c-9b4e-04a106872337 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:38.188798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7e4f9-2677-411c-9b4e-04a106872337 None admin] PATH_INFO: `/v3/users/a2683672ffd64d99823348e2781589fc` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:38.190604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7e4f9-2677-411c-9b4e-04a106872337 None admin] RBAC: Authorizing `identity:get_user(user_id=a2683672ffd64d99823348e2781589fc)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:38.192380 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7e4f9-2677-411c-9b4e-04a106872337 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:38.195447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 110/227] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:37 2022] GET /identity/v3/users/a2683672ffd64d99823348e2781589fc => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:38.201998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7e4f9-2677-411c-9b4e-04a106872337 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:38.251820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:38.262285 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:38.332163 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7e4f9-2677-411c-9b4e-04a106872337 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:38.413876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425e541-ab45-4e4d-89cd-05b1efab76bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:38.414540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425e541-ab45-4e4d-89cd-05b1efab76bc None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:38.414827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425e541-ab45-4e4d-89cd-05b1efab76bc None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:38.415108 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425e541-ab45-4e4d-89cd-05b1efab76bc None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:38.416913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425e541-ab45-4e4d-89cd-05b1efab76bc None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:38.418875 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425e541-ab45-4e4d-89cd-05b1efab76bc None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:38.420161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c4a1-e230-4cab-8afd-de6935219e54 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:38.420984 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 111/228] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:38 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:38.426740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425e541-ab45-4e4d-89cd-05b1efab76bc None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:38.478255 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc3d5c-b4eb-450e-b7e2-8b1d3c40d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:38.479233 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc3d5c-b4eb-450e-b7e2-8b1d3c40d316 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:38.479684 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc3d5c-b4eb-450e-b7e2-8b1d3c40d316 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:38.480179 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc3d5c-b4eb-450e-b7e2-8b1d3c40d316 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:38.507093 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc3d5c-b4eb-450e-b7e2-8b1d3c40d316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-20bc3d5c-b4eb-450e-b7e2-8b1d3c40d316 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:38.512623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:38.513718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:38.515050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 118/229] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:38.523297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc3d5c-b4eb-450e-b7e2-8b1d3c40d316 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:38.592307 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425e541-ab45-4e4d-89cd-05b1efab76bc None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:38.621428 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:38.627311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:38.640566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc3d5c-b4eb-450e-b7e2-8b1d3c40d316 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:38.681358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:38.682469 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:38.683018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:38.683506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:38.684465 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:38.684994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:38.686693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:38.695449 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de749dcd-59c8-4df0-b4ad-5dd0b3a54efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:38.696167 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de749dcd-59c8-4df0-b4ad-5dd0b3a54efe None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:38.696815 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de749dcd-59c8-4df0-b4ad-5dd0b3a54efe None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:38.697092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de749dcd-59c8-4df0-b4ad-5dd0b3a54efe None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/users/a2683672ffd64d99823348e2781589fc/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:38.697847 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 119/230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:38.699209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de749dcd-59c8-4df0-b4ad-5dd0b3a54efe None admin] RBAC: Authorizing `identity:create_grant(project_id=e8083f126cd24c4b93c7be83ca71f0ee, user_id=a2683672ffd64d99823348e2781589fc, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:38.700715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de749dcd-59c8-4df0-b4ad-5dd0b3a54efe None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:38.711039 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:38.754021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 112/231] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:38 2022] PUT /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/users/a2683672ffd64d99823348e2781589fc/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:38.836840 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb1bab-8b67-4c2a-95b4-2cc48e33f5ae None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:38.891852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d40af5-90d9-4f99-acda-c1cdb8ef5bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:38.892972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40af5-90d9-4f99-acda-c1cdb8ef5bd8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:38.893519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40af5-90d9-4f99-acda-c1cdb8ef5bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:38.894064 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40af5-90d9-4f99-acda-c1cdb8ef5bd8 None admin] PATH_INFO: `/v3/users/a2683672ffd64d99823348e2781589fc` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:38.895674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d40af5-90d9-4f99-acda-c1cdb8ef5bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=a2683672ffd64d99823348e2781589fc)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:38.897908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d40af5-90d9-4f99-acda-c1cdb8ef5bd8 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:38.904268 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 120/232] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:38 2022] GET /identity/v3/users/a2683672ffd64d99823348e2781589fc => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:38.910537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de749dcd-59c8-4df0-b4ad-5dd0b3a54efe None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:39.040180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de749dcd-59c8-4df0-b4ad-5dd0b3a54efe None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:39.076203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f8dd4-74c9-4955-ac9c-2dd34c3b2361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:39.076934 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8dd4-74c9-4955-ac9c-2dd34c3b2361 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:39.077135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8dd4-74c9-4955-ac9c-2dd34c3b2361 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:39.077408 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8dd4-74c9-4955-ac9c-2dd34c3b2361 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:39.078606 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8dd4-74c9-4955-ac9c-2dd34c3b2361 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:39.080074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8dd4-74c9-4955-ac9c-2dd34c3b2361 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:39.082243 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 113/233] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:38 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:39.087953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d40af5-90d9-4f99-acda-c1cdb8ef5bd8 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:39.111426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:39.226613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d40af5-90d9-4f99-acda-c1cdb8ef5bd8 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:39.275085 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e4d99-1076-4ca0-b455-6b2ddfd23d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:39.275782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e4d99-1076-4ca0-b455-6b2ddfd23d3f None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:39.276074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e4d99-1076-4ca0-b455-6b2ddfd23d3f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:39.276356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e4d99-1076-4ca0-b455-6b2ddfd23d3f None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/users/a2683672ffd64d99823348e2781589fc/roles/0f002225dde34ef1a2adda2f583b8520` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:39.284253 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e4d99-1076-4ca0-b455-6b2ddfd23d3f None admin] RBAC: Authorizing `identity:create_grant(project_id=e8083f126cd24c4b93c7be83ca71f0ee, user_id=a2683672ffd64d99823348e2781589fc, role_id=0f002225dde34ef1a2adda2f583b8520)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:39.285826 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e4d99-1076-4ca0-b455-6b2ddfd23d3f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:39.310679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 121/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:39 2022] PUT /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/users/a2683672ffd64d99823348e2781589fc/roles/0f002225dde34ef1a2adda2f583b8520 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:39.986425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d6a1-b745-4ff8-b6b5-bd5c59cb2e5f None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:39.986970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d6a1-b745-4ff8-b6b5-bd5c59cb2e5f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:39.986970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d6a1-b745-4ff8-b6b5-bd5c59cb2e5f None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:39.988306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 114/235] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:39.993995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabb194-dd0d-462f-91f8-da20dc5d113b None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:39.994505 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabb194-dd0d-462f-91f8-da20dc5d113b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:39.994613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabb194-dd0d-462f-91f8-da20dc5d113b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.019679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-6fabb194-dd0d-462f-91f8-da20dc5d113b None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:40.020899 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-6fabb194-dd0d-462f-91f8-da20dc5d113b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:40.086288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb6e76-f86e-43d1-b8ac-408bfb224720 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:40.086650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb6e76-f86e-43d1-b8ac-408bfb224720 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:40.086873 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb6e76-f86e-43d1-b8ac-408bfb224720 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.088335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 115/236] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:13:40.094065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f473bc-c9a7-4953-85b7-425cc2d3fe80 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:40.094392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f473bc-c9a7-4953-85b7-425cc2d3fe80 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:40.094649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f473bc-c9a7-4953-85b7-425cc2d3fe80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.111396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 122/237] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:40.124774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-07f473bc-c9a7-4953-85b7-425cc2d3fe80 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:40.125990 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-07f473bc-c9a7-4953-85b7-425cc2d3fe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:40.146885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabb194-dd0d-462f-91f8-da20dc5d113b None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:40.158809 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:40.210321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:40.255176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 116/238] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:40.285847 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f473bc-c9a7-4953-85b7-425cc2d3fe80 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:40.397706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f473bc-c9a7-4953-85b7-425cc2d3fe80 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:40.432138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabb194-dd0d-462f-91f8-da20dc5d113b None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:40.437321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b22aa-da12-442c-a423-4ab581203aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:40.438402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b22aa-da12-442c-a423-4ab581203aa0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:40.438921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b22aa-da12-442c-a423-4ab581203aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:40.439395 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b22aa-da12-442c-a423-4ab581203aa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.440849 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b22aa-da12-442c-a423-4ab581203aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:40.443513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b22aa-da12-442c-a423-4ab581203aa0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:40.445782 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 117/239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:40.452612 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b22aa-da12-442c-a423-4ab581203aa0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:40.514847 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:40.562904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000314c2-80fe-46f6-a029-87d5770ebc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:40.564923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000314c2-80fe-46f6-a029-87d5770ebc56 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:40.564923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000314c2-80fe-46f6-a029-87d5770ebc56 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:40.564923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000314c2-80fe-46f6-a029-87d5770ebc56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.616005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b22aa-da12-442c-a423-4ab581203aa0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:40.622849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000314c2-80fe-46f6-a029-87d5770ebc56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-000314c2-80fe-46f6-a029-87d5770ebc56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:40.627957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:40.629136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:40.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:40.633855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 123/240] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:40.635120 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000314c2-80fe-46f6-a029-87d5770ebc56 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:40.671262 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cf128-5c65-4db2-8a47-15ae78d24a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:40.671619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf128-5c65-4db2-8a47-15ae78d24a18 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:40.677901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf128-5c65-4db2-8a47-15ae78d24a18 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:40.677901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf128-5c65-4db2-8a47-15ae78d24a18 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.677901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cf128-5c65-4db2-8a47-15ae78d24a18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:40.677901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cf128-5c65-4db2-8a47-15ae78d24a18 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:40.733406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 118/241] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:13:40 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:13:40.740700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d2aae-3c38-4bd8-a7ab-03c53fb1d714 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:40.740700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d2aae-3c38-4bd8-a7ab-03c53fb1d714 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:40.740700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d2aae-3c38-4bd8-a7ab-03c53fb1d714 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.741870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 119/242] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:40.752315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91a36-8808-4608-9acd-9e36f2b6898b None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:40.752691 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91a36-8808-4608-9acd-9e36f2b6898b None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:40.753067 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91a36-8808-4608-9acd-9e36f2b6898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.790664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:40.795671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-6de91a36-8808-4608-9acd-9e36f2b6898b None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:40.802198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-6de91a36-8808-4608-9acd-9e36f2b6898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:40.867612 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000314c2-80fe-46f6-a029-87d5770ebc56 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:40.942387 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:40.943490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:40.943973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:40.944821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:40.945762 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] RBAC: Adding query filter params (name=reader) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:40.947333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:40.950485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:40.951001 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 120/243] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:40.961535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 124/244] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:40.967029 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de91a36-8808-4608-9acd-9e36f2b6898b None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:40.985642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:40.997711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:41.099332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de91a36-8808-4608-9acd-9e36f2b6898b None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:41.128607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e57dc-b1f5-4976-be8b-8478a1a6139e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:41.138273 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8b0bd-13b7-42f0-964b-ce9e75191881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:41.139017 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b0bd-13b7-42f0-964b-ce9e75191881 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.139379 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b0bd-13b7-42f0-964b-ce9e75191881 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.139972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b0bd-13b7-42f0-964b-ce9e75191881 None admin] PATH_INFO: `/v3/users/260a9f255f384247b2c1b56c56257a67` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.162247 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8b0bd-13b7-42f0-964b-ce9e75191881 None admin] RBAC: Authorizing `identity:get_user(user_id=260a9f255f384247b2c1b56c56257a67)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:41.165068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8b0bd-13b7-42f0-964b-ce9e75191881 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:41.166222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:41.168452 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 121/245] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:40 2022] GET /identity/v3/users/260a9f255f384247b2c1b56c56257a67 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:41.173748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8b0bd-13b7-42f0-964b-ce9e75191881 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:41.180035 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391691e-8a98-4260-bd88-481a1bf39c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:41.180035 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391691e-8a98-4260-bd88-481a1bf39c1a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.180402 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391691e-8a98-4260-bd88-481a1bf39c1a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.180811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391691e-8a98-4260-bd88-481a1bf39c1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.193689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391691e-8a98-4260-bd88-481a1bf39c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-7391691e-8a98-4260-bd88-481a1bf39c1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:41.201220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:41.202377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:41.206937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 125/246] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:41.214049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391691e-8a98-4260-bd88-481a1bf39c1a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:41.306501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:41.327744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8b0bd-13b7-42f0-964b-ce9e75191881 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:41.350212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391691e-8a98-4260-bd88-481a1bf39c1a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:41.387439 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7633f-e8c0-43f6-be30-6c8024f8a2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:41.387711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:41.388189 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7633f-e8c0-43f6-be30-6c8024f8a2ff None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.388307 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.388468 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7633f-e8c0-43f6-be30-6c8024f8a2ff None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.388567 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.388730 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.388823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7633f-e8c0-43f6-be30-6c8024f8a2ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.389575 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:41.389886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:41.391369 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:41.402849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 126/247] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:41.410180 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:41.470743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:41.475378 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7633f-e8c0-43f6-be30-6c8024f8a2ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:41.477776 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7633f-e8c0-43f6-be30-6c8024f8a2ff None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:41.533780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb40c0-8e0c-463e-804e-f2e8c2652c45 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:41.564197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:41.582967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 122/248] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:13:41.583916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a115ad8-db11-419f-9776-8f33b6038c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:41.584516 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fc7da-427d-4ef0-84fd-8d8be02fbed1 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.584965 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a115ad8-db11-419f-9776-8f33b6038c60 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.585403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a115ad8-db11-419f-9776-8f33b6038c60 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.585895 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a115ad8-db11-419f-9776-8f33b6038c60 None admin] PATH_INFO: `/v3/users/8ed76d53fe964bf790be7083bc868622` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.589518 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fc7da-427d-4ef0-84fd-8d8be02fbed1 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.589811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a115ad8-db11-419f-9776-8f33b6038c60 None admin] RBAC: Authorizing `identity:get_user(user_id=8ed76d53fe964bf790be7083bc868622)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:41.589993 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fc7da-427d-4ef0-84fd-8d8be02fbed1 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.591649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a115ad8-db11-419f-9776-8f33b6038c60 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:41.591823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 123/249] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:41.592651 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fc7da-427d-4ef0-84fd-8d8be02fbed1 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:41.594067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 127/250] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:41 2022] GET /identity/v3/users/8ed76d53fe964bf790be7083bc868622 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:41.598480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea508697-a95e-4a1e-a7b7-e72d515097c5 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.599054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea508697-a95e-4a1e-a7b7-e72d515097c5 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.599485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea508697-a95e-4a1e-a7b7-e72d515097c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.636457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-ea508697-a95e-4a1e-a7b7-e72d515097c5 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:41.637624 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-ea508697-a95e-4a1e-a7b7-e72d515097c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:41.761455 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 128/251] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:41.762687 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea508697-a95e-4a1e-a7b7-e72d515097c5 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:41.790832 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fc7da-427d-4ef0-84fd-8d8be02fbed1 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:41.813844 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:41.832463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb055af-bcf8-4a58-829d-855e6b3999c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:41.833156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb055af-bcf8-4a58-829d-855e6b3999c0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.833443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb055af-bcf8-4a58-829d-855e6b3999c0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.833723 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb055af-bcf8-4a58-829d-855e6b3999c0 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.835077 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb055af-bcf8-4a58-829d-855e6b3999c0 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:41.836585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb055af-bcf8-4a58-829d-855e6b3999c0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:41.838416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 124/252] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:41 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:41.839651 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd838e26-5571-4409-ab70-5e9c238b24cd None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.839936 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd838e26-5571-4409-ab70-5e9c238b24cd None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.840213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd838e26-5571-4409-ab70-5e9c238b24cd None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.846657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 125/253] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:41.847543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd838e26-5571-4409-ab70-5e9c238b24cd None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:41.874507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea508697-a95e-4a1e-a7b7-e72d515097c5 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:41.906611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:41.911707 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ff57b-35a0-455d-892d-547e02e53be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:41.912383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ff57b-35a0-455d-892d-547e02e53be9 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:41.912691 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ff57b-35a0-455d-892d-547e02e53be9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:41.912994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ff57b-35a0-455d-892d-547e02e53be9 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:41.914236 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ff57b-35a0-455d-892d-547e02e53be9 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:41.915921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ff57b-35a0-455d-892d-547e02e53be9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:41.917649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 129/254] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:41 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:41.918457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ff57b-35a0-455d-892d-547e02e53be9 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:41.964036 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd838e26-5571-4409-ab70-5e9c238b24cd None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:42.014362 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406e40c-4759-4908-9606-1932755e4c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:42.015205 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e40c-4759-4908-9606-1932755e4c26 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.015864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e40c-4759-4908-9606-1932755e4c26 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.016247 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e40c-4759-4908-9606-1932755e4c26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:42.027336 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ff57b-35a0-455d-892d-547e02e53be9 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:42.030876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406e40c-4759-4908-9606-1932755e4c26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-4406e40c-4759-4908-9606-1932755e4c26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:42.038663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:42.039739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:42.040672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 126/255] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:42.041465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10779-5f03-403e-be5d-493645243b45 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.041465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10779-5f03-403e-be5d-493645243b45 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.041611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10779-5f03-403e-be5d-493645243b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:42.053203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:42.064745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-eef10779-5f03-403e-be5d-493645243b45 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:42.066313 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-eef10779-5f03-403e-be5d-493645243b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:42.077093 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:42.077942 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.078415 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.078846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:42.079842 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] RBAC: Adding query filter params (role.id=a0a0e158ed4f4f9496ff04139752f839, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee, user.id=260a9f255f384247b2c1b56c56257a67) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:42.080311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:42.082031 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:42.088686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 130/256] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:41 2022] GET /identity/v3/role_assignments?user.id=260a9f255f384247b2c1b56c56257a67&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=a0a0e158ed4f4f9496ff04139752f839 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:42.089815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:42.188065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 127/257] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:42.188922 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10779-5f03-403e-be5d-493645243b45 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:42.200668 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf9cbf-a2d4-4038-ad7b-f4600dc4b07b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:42.266088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:42.268711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57aab-76e7-4391-a024-4548c6f2665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:42.269478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57aab-76e7-4391-a024-4548c6f2665c None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.269759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57aab-76e7-4391-a024-4548c6f2665c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.270033 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57aab-76e7-4391-a024-4548c6f2665c None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/users/8ed76d53fe964bf790be7083bc868622/roles/0aacd40c75f64f668308391b44dc4e85` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:42.275841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd57aab-76e7-4391-a024-4548c6f2665c None admin] RBAC: Authorizing `identity:create_grant(project_id=e8083f126cd24c4b93c7be83ca71f0ee, user_id=8ed76d53fe964bf790be7083bc868622, role_id=0aacd40c75f64f668308391b44dc4e85)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:42.277422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd57aab-76e7-4391-a024-4548c6f2665c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:42.311237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 131/258] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:42 2022] PUT /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/users/8ed76d53fe964bf790be7083bc868622/roles/0aacd40c75f64f668308391b44dc4e85 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:42.319641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57aab-76e7-4391-a024-4548c6f2665c None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:42.323207 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10779-5f03-403e-be5d-493645243b45 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:42.361666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:42.362528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.362902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.363180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:42.363970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:42.364425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:42.366067 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:42.377124 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 128/259] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:42.391975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:42.392670 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:42.460189 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57aab-76e7-4391-a024-4548c6f2665c None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:42.495942 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c272368-d18f-4422-aada-bd514f09e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:42.496801 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c272368-d18f-4422-aada-bd514f09e8e3 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.497176 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c272368-d18f-4422-aada-bd514f09e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.497549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c272368-d18f-4422-aada-bd514f09e8e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:42.509531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c272368-d18f-4422-aada-bd514f09e8e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-1c272368-d18f-4422-aada-bd514f09e8e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:42.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:42.517342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:42.518565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:42.520614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 132/260] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:42.526175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c272368-d18f-4422-aada-bd514f09e8e3 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:42.577457 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53dbd1-830c-4e4f-b497-04dd2f30980e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:42.589998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:42.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59ea83-fcc1-46b7-bd69-ce0b29f1db89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:42.641812 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59ea83-fcc1-46b7-bd69-ce0b29f1db89 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.642092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59ea83-fcc1-46b7-bd69-ce0b29f1db89 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.642390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59ea83-fcc1-46b7-bd69-ce0b29f1db89 None admin] PATH_INFO: `/v3/users/a2683672ffd64d99823348e2781589fc` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:42.644079 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59ea83-fcc1-46b7-bd69-ce0b29f1db89 None admin] RBAC: Authorizing `identity:get_user(user_id=a2683672ffd64d99823348e2781589fc)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:42.645614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59ea83-fcc1-46b7-bd69-ce0b29f1db89 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:42.650183 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 129/261] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:42 2022] GET /identity/v3/users/a2683672ffd64d99823348e2781589fc => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:42.660866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59ea83-fcc1-46b7-bd69-ce0b29f1db89 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:42.730249 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:42.736243 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c272368-d18f-4422-aada-bd514f09e8e3 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:42.801433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:42.802193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.802480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.803007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:42.803764 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:42.804070 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:42.806041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:42.825333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 133/262] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:42.830542 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:42.908431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59ea83-fcc1-46b7-bd69-ce0b29f1db89 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:42.983767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb06d1f-fd64-4689-930f-f56d28bb379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:42.984694 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb06d1f-fd64-4689-930f-f56d28bb379d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:42.985127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb06d1f-fd64-4689-930f-f56d28bb379d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:42.985547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb06d1f-fd64-4689-930f-f56d28bb379d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:43.015132 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb609bb-7cd5-4006-9334-2f48cb19d37d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:43.072557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:43.087560 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fedbc-9989-4173-91da-881bfbce78a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:43.090091 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fedbc-9989-4173-91da-881bfbce78a8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:43.090548 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fedbc-9989-4173-91da-881bfbce78a8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:43.090968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fedbc-9989-4173-91da-881bfbce78a8 None admin] PATH_INFO: `/v3/users/a2683672ffd64d99823348e2781589fc` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:43.093403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fedbc-9989-4173-91da-881bfbce78a8 None admin] RBAC: Authorizing `identity:get_user(user_id=a2683672ffd64d99823348e2781589fc)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:43.096887 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fedbc-9989-4173-91da-881bfbce78a8 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:43.100570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 134/263] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:42 2022] GET /identity/v3/users/a2683672ffd64d99823348e2781589fc => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:43.106153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fedbc-9989-4173-91da-881bfbce78a8 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:43.133613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb06d1f-fd64-4689-930f-f56d28bb379d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:43.135580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb06d1f-fd64-4689-930f-f56d28bb379d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:43.178492 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:43.286833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 130/264] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:13:43.292905 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb06d1f-fd64-4689-930f-f56d28bb379d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:43.330080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fedbc-9989-4173-91da-881bfbce78a8 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:43.369954 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:43.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb06d1f-fd64-4689-930f-f56d28bb379d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:43.415970 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579e901-c8f2-4228-ad7b-9c330f169354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:43.416628 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579e901-c8f2-4228-ad7b-9c330f169354 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:43.416915 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579e901-c8f2-4228-ad7b-9c330f169354 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:43.417196 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579e901-c8f2-4228-ad7b-9c330f169354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:43.445177 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605cdc-24b2-4396-8c08-748082319c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:43.446198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605cdc-24b2-4396-8c08-748082319c95 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:43.447002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605cdc-24b2-4396-8c08-748082319c95 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:43.447424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605cdc-24b2-4396-8c08-748082319c95 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:43.448833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605cdc-24b2-4396-8c08-748082319c95 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:43.450524 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605cdc-24b2-4396-8c08-748082319c95 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:43.457502 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 131/265] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:43 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:43.459043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605cdc-24b2-4396-8c08-748082319c95 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:43.564781 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605cdc-24b2-4396-8c08-748082319c95 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:43.591382 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579e901-c8f2-4228-ad7b-9c330f169354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:43.592835 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579e901-c8f2-4228-ad7b-9c330f169354 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:43.603131 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:43.604043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:43.604502 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:43.604925 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:43.605897 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee, user.id=a2683672ffd64d99823348e2781589fc) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:43.606505 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:43.608330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:43.617100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 132/266] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:43 2022] GET /identity/v3/role_assignments?user.id=a2683672ffd64d99823348e2781589fc&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=40504e80dbb845aeb07ca258977ecacb => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:43.660488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:43.772171 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 135/267] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:43.777809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:43.786145 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:43.920722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40af21-9859-409e-ae19-51a2ed5c4eef None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:43.958312 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907da21f-4665-4ace-8e86-30685c4642f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:43.959427 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da21f-4665-4ace-8e86-30685c4642f4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:43.959912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da21f-4665-4ace-8e86-30685c4642f4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:43.960388 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da21f-4665-4ace-8e86-30685c4642f4 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:43.961856 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907da21f-4665-4ace-8e86-30685c4642f4 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:43.963609 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907da21f-4665-4ace-8e86-30685c4642f4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:43.965977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 133/268] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:43 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:43.974429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579e901-c8f2-4228-ad7b-9c330f169354 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:44.033148 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069baf86-cb16-410b-a4f2-8a9cd9fd57a4 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.033885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069baf86-cb16-410b-a4f2-8a9cd9fd57a4 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.034478 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069baf86-cb16-410b-a4f2-8a9cd9fd57a4 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.037882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 134/269] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:44.042962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33f603-467c-402f-85d7-8d7beff0325d None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.043536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33f603-467c-402f-85d7-8d7beff0325d None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.044026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33f603-467c-402f-85d7-8d7beff0325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.078249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-1e33f603-467c-402f-85d7-8d7beff0325d None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:44.079476 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-1e33f603-467c-402f-85d7-8d7beff0325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:44.135447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579e901-c8f2-4228-ad7b-9c330f169354 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:44.170690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:44.177103 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943bf0-26c2-47ee-9a39-4a786c71f863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:44.178074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943bf0-26c2-47ee-9a39-4a786c71f863 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.178571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943bf0-26c2-47ee-9a39-4a786c71f863 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.179019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943bf0-26c2-47ee-9a39-4a786c71f863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.179992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28943bf0-26c2-47ee-9a39-4a786c71f863 None admin] RBAC: Adding query filter params (role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee, user.id=a2683672ffd64d99823348e2781589fc) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:44.180459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28943bf0-26c2-47ee-9a39-4a786c71f863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:44.182094 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28943bf0-26c2-47ee-9a39-4a786c71f863 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:44.189145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 136/270] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:43 2022] GET /identity/v3/role_assignments?user.id=a2683672ffd64d99823348e2781589fc&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:44.204659 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 135/271] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:44.227350 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6d0c7-691b-452c-8420-e653c41e0bae None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.227968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6d0c7-691b-452c-8420-e653c41e0bae None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.228471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6d0c7-691b-452c-8420-e653c41e0bae None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.230095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 137/272] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:44.236327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf316b-85ff-4470-90a1-786ae8631eb0 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.236602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf316b-85ff-4470-90a1-786ae8631eb0 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.236878 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf316b-85ff-4470-90a1-786ae8631eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.246534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:44.247451 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6d0c7-691b-452c-8420-e653c41e0bae None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:44.265200 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-babf316b-85ff-4470-90a1-786ae8631eb0 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:44.266445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-babf316b-85ff-4470-90a1-786ae8631eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:44.368895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 136/273] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:44.396307 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf316b-85ff-4470-90a1-786ae8631eb0 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:44.492634 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6d0c7-691b-452c-8420-e653c41e0bae None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:44.526460 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf316b-85ff-4470-90a1-786ae8631eb0 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:44.542136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c638aac-3e82-46c8-aeb5-be2e1592f7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:44.543665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638aac-3e82-46c8-aeb5-be2e1592f7f5 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.544187 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638aac-3e82-46c8-aeb5-be2e1592f7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.544973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638aac-3e82-46c8-aeb5-be2e1592f7f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.546792 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c638aac-3e82-46c8-aeb5-be2e1592f7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:44.548965 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c638aac-3e82-46c8-aeb5-be2e1592f7f5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:44.552038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 138/274] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:44.557265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c638aac-3e82-46c8-aeb5-be2e1592f7f5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:44.566501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:44.573420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9185dfb-cc14-4ba6-ae99-ec5952bc8d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:44.574089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185dfb-cc14-4ba6-ae99-ec5952bc8d55 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.574436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185dfb-cc14-4ba6-ae99-ec5952bc8d55 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.574761 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185dfb-cc14-4ba6-ae99-ec5952bc8d55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.588046 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9185dfb-cc14-4ba6-ae99-ec5952bc8d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-d9185dfb-cc14-4ba6-ae99-ec5952bc8d55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:44.594671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:44.595778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:44.596810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 137/275] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:44.602269 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9185dfb-cc14-4ba6-ae99-ec5952bc8d55 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:44.642614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:44.701490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c638aac-3e82-46c8-aeb5-be2e1592f7f5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:44.715001 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9185dfb-cc14-4ba6-ae99-ec5952bc8d55 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:44.750243 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38010fde-c1bb-4914-a81d-28a60806cada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:44.751885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38010fde-c1bb-4914-a81d-28a60806cada None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.752401 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38010fde-c1bb-4914-a81d-28a60806cada None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.753094 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38010fde-c1bb-4914-a81d-28a60806cada None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.754637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38010fde-c1bb-4914-a81d-28a60806cada None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:44.756840 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38010fde-c1bb-4914-a81d-28a60806cada None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:44.758371 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:44.759003 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.759280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.759548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.760273 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:44.760574 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:44.761960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:44.771242 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 138/276] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:44.777004 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:44.790268 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 139/277] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:13:44 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:44.838453 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c3562-cc68-4d0f-b6e0-56528db5e29b None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.838849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c3562-cc68-4d0f-b6e0-56528db5e29b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.839135 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c3562-cc68-4d0f-b6e0-56528db5e29b None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.840577 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 140/278] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:13:44.846543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7a34a-048f-4435-aef6-172a42695123 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.846906 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7a34a-048f-4435-aef6-172a42695123 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.847188 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7a34a-048f-4435-aef6-172a42695123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.854552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:44.867310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-2ee7a34a-048f-4435-aef6-172a42695123 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:44.869143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-2ee7a34a-048f-4435-aef6-172a42695123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:44.903836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550ac4-005e-4bb7-857d-8b4e61f46613 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:44.944066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a271120-eb74-45bd-84b4-daf52a7e07f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:44.945372 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a271120-eb74-45bd-84b4-daf52a7e07f3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:44.945994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a271120-eb74-45bd-84b4-daf52a7e07f3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:44.947543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 141/279] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:44.947821 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a271120-eb74-45bd-84b4-daf52a7e07f3 None admin] PATH_INFO: `/v3/users/bfc7271bfbc34fed93f3b26fc72ff33d` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:44.964289 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a271120-eb74-45bd-84b4-daf52a7e07f3 None admin] RBAC: Authorizing `identity:get_user(user_id=bfc7271bfbc34fed93f3b26fc72ff33d)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:44.966211 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a271120-eb74-45bd-84b4-daf52a7e07f3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:44.968611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 139/280] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:44 2022] GET /identity/v3/users/bfc7271bfbc34fed93f3b26fc72ff33d => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:44.974563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7a34a-048f-4435-aef6-172a42695123 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:44.975586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a271120-eb74-45bd-84b4-daf52a7e07f3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:45.029187 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:45.102656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a271120-eb74-45bd-84b4-daf52a7e07f3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:45.103231 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7a34a-048f-4435-aef6-172a42695123 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:45.119151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:45.143033 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795b754-1300-4170-8944-64ea2a8e6c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:45.144319 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795b754-1300-4170-8944-64ea2a8e6c5f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.144873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795b754-1300-4170-8944-64ea2a8e6c5f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.145343 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795b754-1300-4170-8944-64ea2a8e6c5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.148303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:45.149052 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.149339 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.149619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.150478 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] RBAC: Adding query filter params (role.id=0aacd40c75f64f668308391b44dc4e85, user.id=bfc7271bfbc34fed93f3b26fc72ff33d, scope.system=all) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:45.150789 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:45.152465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:45.158263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795b754-1300-4170-8944-64ea2a8e6c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:45.159241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 140/281] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:13:44 2022] GET /identity/v3/role_assignments?user.id=bfc7271bfbc34fed93f3b26fc72ff33d&scope.system=all&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-8795b754-1300-4170-8944-64ea2a8e6c5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:45.163549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:45.164619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:45.165851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 142/282] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:45.170887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:45.299644 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e954c-4fdb-4313-9e5f-f76fa07d538c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:45.352336 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd6186-bb92-41e0-bf3d-929736ade2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:45.352993 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6186-bb92-41e0-bf3d-929736ade2bf None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.353273 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6186-bb92-41e0-bf3d-929736ade2bf None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.353546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6186-bb92-41e0-bf3d-929736ade2bf None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.354314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd6186-bb92-41e0-bf3d-929736ade2bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:45.354617 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd6186-bb92-41e0-bf3d-929736ade2bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:45.359436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd6186-bb92-41e0-bf3d-929736ade2bf None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:45.370490 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 141/283] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:45.376193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795b754-1300-4170-8944-64ea2a8e6c5f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:45.397704 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa7ab2-e317-4dff-8656-b8d8b7f08423 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.398522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa7ab2-e317-4dff-8656-b8d8b7f08423 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.399030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa7ab2-e317-4dff-8656-b8d8b7f08423 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.400939 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 142/284] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:45.407417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afd346-cae2-4733-8be2-54f7c9bd8407 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.407701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afd346-cae2-4733-8be2-54f7c9bd8407 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.407961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afd346-cae2-4733-8be2-54f7c9bd8407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.427987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d9afd346-cae2-4733-8be2-54f7c9bd8407 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:45.429201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d9afd346-cae2-4733-8be2-54f7c9bd8407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:45.472757 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:45.506280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795b754-1300-4170-8944-64ea2a8e6c5f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:45.515862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:45.546716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9717853-2a9e-4f3f-9430-9c0f639d62b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:45.548215 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9717853-2a9e-4f3f-9430-9c0f639d62b8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.548713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9717853-2a9e-4f3f-9430-9c0f639d62b8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.549210 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9717853-2a9e-4f3f-9430-9c0f639d62b8 None admin] PATH_INFO: `/v3/users/260a9f255f384247b2c1b56c56257a67` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.551121 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9717853-2a9e-4f3f-9430-9c0f639d62b8 None admin] RBAC: Authorizing `identity:get_user(user_id=260a9f255f384247b2c1b56c56257a67)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:45.553192 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9717853-2a9e-4f3f-9430-9c0f639d62b8 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:45.555781 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 143/285] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:45 2022] GET /identity/v3/users/260a9f255f384247b2c1b56c56257a67 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:45.562570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9717853-2a9e-4f3f-9430-9c0f639d62b8 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:45.576010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 143/286] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:45.615121 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afd346-cae2-4733-8be2-54f7c9bd8407 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:45.756880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afd346-cae2-4733-8be2-54f7c9bd8407 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:45.793714 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8504-f41c-44d1-b165-dbb03b3f3e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:45.794808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8504-f41c-44d1-b165-dbb03b3f3e1f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.795304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8504-f41c-44d1-b165-dbb03b3f3e1f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.795827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8504-f41c-44d1-b165-dbb03b3f3e1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.800407 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9717853-2a9e-4f3f-9430-9c0f639d62b8 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:45.809522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f8504-f41c-44d1-b165-dbb03b3f3e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-1b1f8504-f41c-44d1-b165-dbb03b3f3e1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:45.814942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:45.816047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:45.816972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:45.819936 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 144/287] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:45.826383 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8504-f41c-44d1-b165-dbb03b3f3e1f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:45.861179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:45.874864 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406118d-b969-412e-9668-21f850ca0b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:45.875583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406118d-b969-412e-9668-21f850ca0b31 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.875863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406118d-b969-412e-9668-21f850ca0b31 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.876134 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406118d-b969-412e-9668-21f850ca0b31 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.877330 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406118d-b969-412e-9668-21f850ca0b31 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:45.879020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406118d-b969-412e-9668-21f850ca0b31 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:45.890592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 144/288] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:45 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:45.891525 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406118d-b969-412e-9668-21f850ca0b31 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:45.941535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8504-f41c-44d1-b165-dbb03b3f3e1f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:45.978859 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:45.979964 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:45.980473 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:45.980967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:45.981929 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:45.982489 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:45.984110 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:45.996032 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 145/289] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:45.997242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:46.001441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:46.084071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406118d-b969-412e-9668-21f850ca0b31 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:46.125126 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a2981-828f-425d-8188-f77437f2e5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:46.126390 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a2981-828f-425d-8188-f77437f2e5a5 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:46.126898 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a2981-828f-425d-8188-f77437f2e5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:46.127398 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a2981-828f-425d-8188-f77437f2e5a5 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/users/260a9f255f384247b2c1b56c56257a67/roles/a0a0e158ed4f4f9496ff04139752f839` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:46.129765 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a2981-828f-425d-8188-f77437f2e5a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8083f126cd24c4b93c7be83ca71f0ee, user_id=260a9f255f384247b2c1b56c56257a67, role_id=a0a0e158ed4f4f9496ff04139752f839)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:46.132217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a2981-828f-425d-8188-f77437f2e5a5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:46.158759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 145/290] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:13:45 2022] PUT /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/users/260a9f255f384247b2c1b56c56257a67/roles/a0a0e158ed4f4f9496ff04139752f839 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:46.167693 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:46.182223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7e3d6-d28f-4555-8c57-677d9aa626c4 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:46.223903 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffe32f-f01d-4d23-9b69-190ea5a4795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:46.224969 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffe32f-f01d-4d23-9b69-190ea5a4795c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:46.225465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffe32f-f01d-4d23-9b69-190ea5a4795c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:46.225953 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffe32f-f01d-4d23-9b69-190ea5a4795c None admin] PATH_INFO: `/v3/users/8ed76d53fe964bf790be7083bc868622` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:46.227609 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffe32f-f01d-4d23-9b69-190ea5a4795c None admin] RBAC: Authorizing `identity:get_user(user_id=8ed76d53fe964bf790be7083bc868622)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:46.229347 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffe32f-f01d-4d23-9b69-190ea5a4795c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:46.231699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 146/291] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:46 2022] GET /identity/v3/users/8ed76d53fe964bf790be7083bc868622 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:46.239575 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a2981-828f-425d-8188-f77437f2e5a5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:46.377834 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a2981-828f-425d-8188-f77437f2e5a5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:46.380095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:46.418876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3476c-1c9e-41eb-9a38-2cf355d61350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:46.419924 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3476c-1c9e-41eb-9a38-2cf355d61350 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:46.420418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3476c-1c9e-41eb-9a38-2cf355d61350 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:46.420910 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3476c-1c9e-41eb-9a38-2cf355d61350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:46.494034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3476c-1c9e-41eb-9a38-2cf355d61350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:46.496110 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3476c-1c9e-41eb-9a38-2cf355d61350 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:46.599294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 146/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:46.607295 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffe32f-f01d-4d23-9b69-190ea5a4795c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:46.724415 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffe32f-f01d-4d23-9b69-190ea5a4795c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:46.766349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fd40d-95f2-476d-a133-151444c50abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:46.767419 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd40d-95f2-476d-a133-151444c50abb None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:46.767913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd40d-95f2-476d-a133-151444c50abb None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:46.768448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd40d-95f2-476d-a133-151444c50abb None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:46.769971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fd40d-95f2-476d-a133-151444c50abb None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:46.771792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fd40d-95f2-476d-a133-151444c50abb None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:46.774198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 147/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:46 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:46.783262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3476c-1c9e-41eb-9a38-2cf355d61350 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:46.800222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:46.924334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3476c-1c9e-41eb-9a38-2cf355d61350 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:46.976291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20e453-7ad8-4cec-9697-6bfd740a39c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:46.976939 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20e453-7ad8-4cec-9697-6bfd740a39c2 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:46.977236 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20e453-7ad8-4cec-9697-6bfd740a39c2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:46.977511 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20e453-7ad8-4cec-9697-6bfd740a39c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:46.978350 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20e453-7ad8-4cec-9697-6bfd740a39c2 None admin] RBAC: Adding query filter params (role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee, user.id=8ed76d53fe964bf790be7083bc868622) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:46.978669 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20e453-7ad8-4cec-9697-6bfd740a39c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:46.980156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20e453-7ad8-4cec-9697-6bfd740a39c2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:46.991601 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 147/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:46 2022] GET /identity/v3/role_assignments?user.id=8ed76d53fe964bf790be7083bc868622&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:47.552550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b72880-888d-4f36-a659-fb7da08ff70b None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:47.553652 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b72880-888d-4f36-a659-fb7da08ff70b None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:47.554203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b72880-888d-4f36-a659-fb7da08ff70b None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:47.556322 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 148/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:47.562149 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57085b-440e-42e3-aa82-b8a2a8ec6bda None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:47.562778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57085b-440e-42e3-aa82-b8a2a8ec6bda None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:47.563118 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fff64-7d71-4e23-91a5-eec1bac621a9 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:47.563412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fff64-7d71-4e23-91a5-eec1bac621a9 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:47.563693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fff64-7d71-4e23-91a5-eec1bac621a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:47.563974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57085b-440e-42e3-aa82-b8a2a8ec6bda None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:47.566055 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 149/296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:47.572891 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a758096-114e-4b88-90c3-86a1619c2ad8 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:47.574537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a758096-114e-4b88-90c3-86a1619c2ad8 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:47.575072 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a758096-114e-4b88-90c3-86a1619c2ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:47.592288 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-860fff64-7d71-4e23-91a5-eec1bac621a9 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:47.593551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-860fff64-7d71-4e23-91a5-eec1bac621a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:47.601211 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-1a758096-114e-4b88-90c3-86a1619c2ad8 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:47.602885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-1a758096-114e-4b88-90c3-86a1619c2ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:47.631513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:47.670301 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:47.680393 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 148/297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:47.681727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e5a0-206e-49f5-96c1-477371eb7aa9 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:47.682897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e5a0-206e-49f5-96c1-477371eb7aa9 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:47.683461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e5a0-206e-49f5-96c1-477371eb7aa9 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:47.690259 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 149/298] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:47.693191 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6990f82-77a6-471d-85e3-01cdf3e12a7d None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:47.693984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6990f82-77a6-471d-85e3-01cdf3e12a7d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:47.694538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6990f82-77a6-471d-85e3-01cdf3e12a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:47.702051 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 150/299] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:47.717537 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-f6990f82-77a6-471d-85e3-01cdf3e12a7d None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:47.717810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a758096-114e-4b88-90c3-86a1619c2ad8 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:47.719209 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-f6990f82-77a6-471d-85e3-01cdf3e12a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:47.805639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 150/300] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:47.806438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6990f82-77a6-471d-85e3-01cdf3e12a7d None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:47.861851 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a758096-114e-4b88-90c3-86a1619c2ad8 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:47.910407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39ded5-a117-46bc-8cfb-b0fc67520a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:47.911078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39ded5-a117-46bc-8cfb-b0fc67520a93 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:47.911323 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39ded5-a117-46bc-8cfb-b0fc67520a93 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:47.911642 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39ded5-a117-46bc-8cfb-b0fc67520a93 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:47.925047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39ded5-a117-46bc-8cfb-b0fc67520a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:47.928196 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6990f82-77a6-471d-85e3-01cdf3e12a7d None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-7c39ded5-a117-46bc-8cfb-b0fc67520a93 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:47.930156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:47.931301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 151/301] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:47.934623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39ded5-a117-46bc-8cfb-b0fc67520a93 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:47.959341 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:47.959700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:47.970078 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be9a47-8008-47d2-b080-763926b68333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:47.970777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9a47-8008-47d2-b080-763926b68333 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:47.971050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9a47-8008-47d2-b080-763926b68333 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:47.971414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9a47-8008-47d2-b080-763926b68333 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:47.972533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be9a47-8008-47d2-b080-763926b68333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:47.973997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be9a47-8008-47d2-b080-763926b68333 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:47.978064 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 151/302] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:47.978064 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be9a47-8008-47d2-b080-763926b68333 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:48.080208 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39ded5-a117-46bc-8cfb-b0fc67520a93 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:48.092830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be9a47-8008-47d2-b080-763926b68333 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:48.128135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0b457-ddde-4921-bb74-313ac0d1d0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:48.128816 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0b457-ddde-4921-bb74-313ac0d1d0d6 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.129090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0b457-ddde-4921-bb74-313ac0d1d0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.129371 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0b457-ddde-4921-bb74-313ac0d1d0d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.143054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:48.143383 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0b457-ddde-4921-bb74-313ac0d1d0d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:48.143823 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.144104 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.144442 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.145187 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:48.145493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:48.146945 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-51f0b457-ddde-4921-bb74-313ac0d1d0d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:48.148541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:48.149767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:48.150885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 152/303] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:48.151718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0b457-ddde-4921-bb74-313ac0d1d0d6 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:48.164911 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 152/304] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:48.166361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:48.230217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:48.248064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:48.273346 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0b457-ddde-4921-bb74-313ac0d1d0d6 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:48.278510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27600d35-164a-4a80-9b7b-394d07881d6d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:48.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8b78f-e66f-44b0-94cc-ea08bccd8b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:48.311635 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8b78f-e66f-44b0-94cc-ea08bccd8b10 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.311922 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8b78f-e66f-44b0-94cc-ea08bccd8b10 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.312309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8b78f-e66f-44b0-94cc-ea08bccd8b10 None admin] PATH_INFO: `/v3/groups` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.312924 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8b78f-e66f-44b0-94cc-ea08bccd8b10 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:48.314566 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8b78f-e66f-44b0-94cc-ea08bccd8b10 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:48.314701 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:48.315386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.315671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.315948 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.316680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:48.316989 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:48.318397 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:48.333416 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 153/305] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:48.333416 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:48.342190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 153/306] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:13:48 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:48.342190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8b78f-e66f-44b0-94cc-ea08bccd8b10 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:48.456602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8b78f-e66f-44b0-94cc-ea08bccd8b10 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:48.477495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7090fc-5555-4db7-bd9d-e27e9e6e30a1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:48.496863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb401a-d0bc-4df7-8363-9122176382a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:48.497553 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb401a-d0bc-4df7-8363-9122176382a7 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.497850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb401a-d0bc-4df7-8363-9122176382a7 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.498129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb401a-d0bc-4df7-8363-9122176382a7 None admin] PATH_INFO: `/v3/users/bfc7271bfbc34fed93f3b26fc72ff33d` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.499395 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb401a-d0bc-4df7-8363-9122176382a7 None admin] RBAC: Authorizing `identity:get_user(user_id=bfc7271bfbc34fed93f3b26fc72ff33d)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:48.500892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb401a-d0bc-4df7-8363-9122176382a7 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:48.502533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:48.503096 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 154/307] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:48 2022] GET /identity/v3/users/bfc7271bfbc34fed93f3b26fc72ff33d => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:48.511652 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb401a-d0bc-4df7-8363-9122176382a7 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:48.516870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:48.535439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e08ab-f314-47bb-b969-1d65671eaf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:48.536215 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e08ab-f314-47bb-b969-1d65671eaf09 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.536506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e08ab-f314-47bb-b969-1d65671eaf09 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.536791 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e08ab-f314-47bb-b969-1d65671eaf09 None admin] PATH_INFO: `/v3/users/18af986a5fda41d4af45150cad65b324` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.554351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e08ab-f314-47bb-b969-1d65671eaf09 None admin] RBAC: Authorizing `identity:get_user(user_id=18af986a5fda41d4af45150cad65b324)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:48.555875 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e08ab-f314-47bb-b969-1d65671eaf09 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:48.558021 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 154/308] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:48 2022] GET /identity/v3/users/18af986a5fda41d4af45150cad65b324 => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:48.565713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e08ab-f314-47bb-b969-1d65671eaf09 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:48.664160 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb401a-d0bc-4df7-8363-9122176382a7 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:48.701658 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e08ab-f314-47bb-b969-1d65671eaf09 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:48.707502 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc44ee1-3117-4de1-ae0c-821e95fb197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:48.708198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc44ee1-3117-4de1-ae0c-821e95fb197e None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.708483 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc44ee1-3117-4de1-ae0c-821e95fb197e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.708765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc44ee1-3117-4de1-ae0c-821e95fb197e None admin] PATH_INFO: `/v3/system/users/bfc7271bfbc34fed93f3b26fc72ff33d/roles/0aacd40c75f64f668308391b44dc4e85` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.710321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc44ee1-3117-4de1-ae0c-821e95fb197e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bfc7271bfbc34fed93f3b26fc72ff33d, role_id=0aacd40c75f64f668308391b44dc4e85)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:48.711768 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc44ee1-3117-4de1-ae0c-821e95fb197e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:48.722403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 155/309] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:13:48 2022] PUT /identity/v3/system/users/bfc7271bfbc34fed93f3b26fc72ff33d/roles/0aacd40c75f64f668308391b44dc4e85 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:13:48.724747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17b9a-66e4-43fe-963b-a41e78b15873 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.725403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17b9a-66e4-43fe-963b-a41e78b15873 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.725931 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17b9a-66e4-43fe-963b-a41e78b15873 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.733046 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 156/310] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:48.736054 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add85248-f9b2-4181-8f99-02a69aa2b25c None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.736765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add85248-f9b2-4181-8f99-02a69aa2b25c None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.737410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add85248-f9b2-4181-8f99-02a69aa2b25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.747969 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:48.748639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:48.748913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:48.749174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:48.750000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] RBAC: Adding query filter params (role.id=a0a0e158ed4f4f9496ff04139752f839, user.id=18af986a5fda41d4af45150cad65b324, scope.system=all) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:48.750486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:48.752096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:48.760118 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 155/311] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:13:48 2022] GET /identity/v3/role_assignments?user.id=18af986a5fda41d4af45150cad65b324&scope.system=all&role.id=a0a0e158ed4f4f9496ff04139752f839 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:48.762999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-add85248-f9b2-4181-8f99-02a69aa2b25c None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:48.764253 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-add85248-f9b2-4181-8f99-02a69aa2b25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:48.855308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:48.866595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 157/312] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:48.889925 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:48.949254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:49.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449d11f-bde4-487b-9b7c-2c92222a2e39 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:49.056195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853c189-3efc-40fc-bbaf-98e558cd82c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:49.057245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c189-3efc-40fc-bbaf-98e558cd82c4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:49.057765 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c189-3efc-40fc-bbaf-98e558cd82c4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:49.058350 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c189-3efc-40fc-bbaf-98e558cd82c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:49.069905 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853c189-3efc-40fc-bbaf-98e558cd82c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-e853c189-3efc-40fc-bbaf-98e558cd82c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:49.075914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:49.076995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:49.077954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:49.081116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 156/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:49.087214 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add85248-f9b2-4181-8f99-02a69aa2b25c None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:49.197172 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add85248-f9b2-4181-8f99-02a69aa2b25c None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:49.233932 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:49.234514 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:49.234799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:49.235074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:49.235818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:49.236133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:49.237583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:49.255271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853c189-3efc-40fc-bbaf-98e558cd82c4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:49.257764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 158/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:49.368004 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:49.371997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853c189-3efc-40fc-bbaf-98e558cd82c4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:49.410685 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff0dc4-a42d-4bc9-a977-92040e82bc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:49.411373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0dc4-a42d-4bc9-a977-92040e82bc81 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:49.411657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0dc4-a42d-4bc9-a977-92040e82bc81 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:49.411935 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0dc4-a42d-4bc9-a977-92040e82bc81 None admin] PATH_INFO: `/v3/users/260a9f255f384247b2c1b56c56257a67` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:49.413323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff0dc4-a42d-4bc9-a977-92040e82bc81 None admin] RBAC: Authorizing `identity:get_user(user_id=260a9f255f384247b2c1b56c56257a67)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:49.414891 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff0dc4-a42d-4bc9-a977-92040e82bc81 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:49.417038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 157/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:49 2022] GET /identity/v3/users/260a9f255f384247b2c1b56c56257a67 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:49.422923 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:49.488256 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:49.537936 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d2cda-841e-4c0c-af70-8cbe1eca7ba0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:49.575813 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551451b-56a2-41d6-9f94-df5d1921475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:49.576456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551451b-56a2-41d6-9f94-df5d1921475b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:49.576744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551451b-56a2-41d6-9f94-df5d1921475b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:49.577022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551451b-56a2-41d6-9f94-df5d1921475b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:49.655216 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551451b-56a2-41d6-9f94-df5d1921475b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:49.656864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551451b-56a2-41d6-9f94-df5d1921475b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:49.725330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:49.771005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 159/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:49.779477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff0dc4-a42d-4bc9-a977-92040e82bc81 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:49.883534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff0dc4-a42d-4bc9-a977-92040e82bc81 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:49.921366 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf639e-d270-491a-9178-917689959f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:49.922574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf639e-d270-491a-9178-917689959f4e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:49.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf639e-d270-491a-9178-917689959f4e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:49.923704 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf639e-d270-491a-9178-917689959f4e None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:49.925568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf639e-d270-491a-9178-917689959f4e None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:49.927496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf639e-d270-491a-9178-917689959f4e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:49.930232 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 158/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:49 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:49.937140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551451b-56a2-41d6-9f94-df5d1921475b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:50.052033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551451b-56a2-41d6-9f94-df5d1921475b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:50.089024 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191025fd-5fe5-4b54-b934-b65b819ebd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:50.090131 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191025fd-5fe5-4b54-b934-b65b819ebd30 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:50.090679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191025fd-5fe5-4b54-b934-b65b819ebd30 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:50.091167 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191025fd-5fe5-4b54-b934-b65b819ebd30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:50.092210 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191025fd-5fe5-4b54-b934-b65b819ebd30 None admin] RBAC: Adding query filter params (role.id=a0a0e158ed4f4f9496ff04139752f839, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee, user.id=260a9f255f384247b2c1b56c56257a67) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:50.092764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191025fd-5fe5-4b54-b934-b65b819ebd30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:50.094521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191025fd-5fe5-4b54-b934-b65b819ebd30 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:50.105524 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 160/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:13:49 2022] GET /identity/v3/role_assignments?user.id=260a9f255f384247b2c1b56c56257a67&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=a0a0e158ed4f4f9496ff04139752f839 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:50.594707 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a65e-d6db-4efb-9c44-b335a90b5953 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:50.595018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a65e-d6db-4efb-9c44-b335a90b5953 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:50.595385 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a65e-d6db-4efb-9c44-b335a90b5953 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:50.596731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 159/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:50.602968 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7fe8-7a26-4dd0-a09a-a3681dc531c8 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:50.603271 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7fe8-7a26-4dd0-a09a-a3681dc531c8 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:50.603535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7fe8-7a26-4dd0-a09a-a3681dc531c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:50.622809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-579c7fe8-7a26-4dd0-a09a-a3681dc531c8 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:50.624034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-579c7fe8-7a26-4dd0-a09a-a3681dc531c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:50.645114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:50.720410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 161/320] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:50.746140 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496a65e-d6db-4efb-9c44-b335a90b5953 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:50.785569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:50.843704 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465abee-559f-4930-a69d-8e95b62beca8 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:50.844874 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465abee-559f-4930-a69d-8e95b62beca8 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:50.845056 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465abee-559f-4930-a69d-8e95b62beca8 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:50.846730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 162/321] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:13:50.852150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0de50-d4d5-4daf-b7f5-bfa2704bdbb9 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:50.852674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0de50-d4d5-4daf-b7f5-bfa2704bdbb9 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:50.853185 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0de50-d4d5-4daf-b7f5-bfa2704bdbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:50.859868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496a65e-d6db-4efb-9c44-b335a90b5953 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:50.872537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-9fc0de50-d4d5-4daf-b7f5-bfa2704bdbb9 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:50.874090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-9fc0de50-d4d5-4daf-b7f5-bfa2704bdbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:50.899684 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23564c6-703c-4e62-a5d3-9dd910f4d176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:50.900348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23564c6-703c-4e62-a5d3-9dd910f4d176 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:50.900626 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23564c6-703c-4e62-a5d3-9dd910f4d176 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:50.900895 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23564c6-703c-4e62-a5d3-9dd910f4d176 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:50.902440 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23564c6-703c-4e62-a5d3-9dd910f4d176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:50.903895 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23564c6-703c-4e62-a5d3-9dd910f4d176 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:50.905553 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 160/322] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:13:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:50.911238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23564c6-703c-4e62-a5d3-9dd910f4d176 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:50.956021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 163/323] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:50.978193 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0de50-d4d5-4daf-b7f5-bfa2704bdbb9 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:51.025091 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23564c6-703c-4e62-a5d3-9dd910f4d176 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:51.028954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:51.069766 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c4211-4f87-4c43-aef7-a099bb0b4321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:51.070433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4211-4f87-4c43-aef7-a099bb0b4321 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.070715 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4211-4f87-4c43-aef7-a099bb0b4321 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.070992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4211-4f87-4c43-aef7-a099bb0b4321 None admin] PATH_INFO: `/v3/groups` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.072039 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c4211-4f87-4c43-aef7-a099bb0b4321 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:51.073238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:51.073588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c4211-4f87-4c43-aef7-a099bb0b4321 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:51.090080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0de50-d4d5-4daf-b7f5-bfa2704bdbb9 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:51.090753 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 161/324] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:13:50 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:13:51.092236 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a6a7-61e2-47cc-95af-8e9a1e338fe9 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.092527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a6a7-61e2-47cc-95af-8e9a1e338fe9 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.092804 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a6a7-61e2-47cc-95af-8e9a1e338fe9 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.094146 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 162/325] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:51.101823 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50e967-7357-4dc3-998e-d098abb0b12a None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.102057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50e967-7357-4dc3-998e-d098abb0b12a None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.102390 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50e967-7357-4dc3-998e-d098abb0b12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.131986 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-ea50e967-7357-4dc3-998e-d098abb0b12a None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:51.133250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-ea50e967-7357-4dc3-998e-d098abb0b12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:51.143719 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820775a-8b8b-42c4-b9c3-6555b89c58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:51.144859 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820775a-8b8b-42c4-b9c3-6555b89c58b9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.145187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820775a-8b8b-42c4-b9c3-6555b89c58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.145805 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820775a-8b8b-42c4-b9c3-6555b89c58b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.159439 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820775a-8b8b-42c4-b9c3-6555b89c58b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-7820775a-8b8b-42c4-b9c3-6555b89c58b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:51.164866 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:51.166022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:51.166995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 164/326] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:51.172673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820775a-8b8b-42c4-b9c3-6555b89c58b9 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:51.222377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 163/327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:51.245817 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50e967-7357-4dc3-998e-d098abb0b12a None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:51.310824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820775a-8b8b-42c4-b9c3-6555b89c58b9 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:51.365029 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:51.365613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.365902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.366218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.367004 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:51.367318 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:51.368861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:51.377458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:51.382413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 165/328] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:51.388510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:51.394523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50e967-7357-4dc3-998e-d098abb0b12a None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:51.403255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:51.442791 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2965f-0109-4bda-9cb2-3ec012cf88dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:51.443481 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2965f-0109-4bda-9cb2-3ec012cf88dc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.443769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2965f-0109-4bda-9cb2-3ec012cf88dc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.444043 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2965f-0109-4bda-9cb2-3ec012cf88dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.461981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2965f-0109-4bda-9cb2-3ec012cf88dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-11b2965f-0109-4bda-9cb2-3ec012cf88dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:51.467561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:51.468673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:51.469846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 164/329] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:51.475910 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2965f-0109-4bda-9cb2-3ec012cf88dc None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:51.547349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2019c-7b7b-4aa6-823e-77aadc9e94f3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:51.597107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bdfa-f485-4f0e-a031-abf90cf7ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:51.597787 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bdfa-f485-4f0e-a031-abf90cf7ae90 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.598070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bdfa-f485-4f0e-a031-abf90cf7ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.598449 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bdfa-f485-4f0e-a031-abf90cf7ae90 None admin] PATH_INFO: `/v3/users/18af986a5fda41d4af45150cad65b324` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.600982 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13bdfa-f485-4f0e-a031-abf90cf7ae90 None admin] RBAC: Authorizing `identity:get_user(user_id=18af986a5fda41d4af45150cad65b324)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:51.602525 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13bdfa-f485-4f0e-a031-abf90cf7ae90 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:51.604569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 166/330] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:51 2022] GET /identity/v3/users/18af986a5fda41d4af45150cad65b324 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:51.611340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bdfa-f485-4f0e-a031-abf90cf7ae90 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:51.624555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2965f-0109-4bda-9cb2-3ec012cf88dc None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:51.668750 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:51.669452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.669773 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.670026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.670798 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:51.671092 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:51.672512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:51.682611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 165/331] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:51.696120 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:51.737224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:51.770984 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bdfa-f485-4f0e-a031-abf90cf7ae90 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:51.777451 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:51.831197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af864fae-3826-4ed6-a442-3a65bdc07cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:51.831896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af864fae-3826-4ed6-a442-3a65bdc07cb1 None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.832222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af864fae-3826-4ed6-a442-3a65bdc07cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.832484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af864fae-3826-4ed6-a442-3a65bdc07cb1 None admin] PATH_INFO: `/v3/system/users/18af986a5fda41d4af45150cad65b324/roles/a0a0e158ed4f4f9496ff04139752f839` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.833984 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af864fae-3826-4ed6-a442-3a65bdc07cb1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=18af986a5fda41d4af45150cad65b324, role_id=a0a0e158ed4f4f9496ff04139752f839)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:51.835436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af864fae-3826-4ed6-a442-3a65bdc07cb1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:51.850337 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 167/332] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:13:51 2022] PUT /identity/v3/system/users/18af986a5fda41d4af45150cad65b324/roles/a0a0e158ed4f4f9496ff04139752f839 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:13:51.906711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc09ee-a136-4b1e-abc4-dff46046ff9d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:51.945960 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438efc9-13ca-4513-826d-abc0ac0d9710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:51.947287 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438efc9-13ca-4513-826d-abc0ac0d9710 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:51.947742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438efc9-13ca-4513-826d-abc0ac0d9710 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:51.948172 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438efc9-13ca-4513-826d-abc0ac0d9710 None admin] PATH_INFO: `/v3/users/bfc7271bfbc34fed93f3b26fc72ff33d` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:51.949585 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438efc9-13ca-4513-826d-abc0ac0d9710 None admin] RBAC: Authorizing `identity:get_user(user_id=bfc7271bfbc34fed93f3b26fc72ff33d)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:51.951290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438efc9-13ca-4513-826d-abc0ac0d9710 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:51.953611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 166/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:51 2022] GET /identity/v3/users/bfc7271bfbc34fed93f3b26fc72ff33d => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:51.963301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af864fae-3826-4ed6-a442-3a65bdc07cb1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:52.183754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af864fae-3826-4ed6-a442-3a65bdc07cb1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:52.247765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90dff1-e671-4c7b-b864-ea259ef5c1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:52.248849 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90dff1-e671-4c7b-b864-ea259ef5c1bf None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:52.249349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90dff1-e671-4c7b-b864-ea259ef5c1bf None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:52.249841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90dff1-e671-4c7b-b864-ea259ef5c1bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:52.251322 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90dff1-e671-4c7b-b864-ea259ef5c1bf None admin] RBAC: Adding query filter params (role.id=0aacd40c75f64f668308391b44dc4e85, user.id=bfc7271bfbc34fed93f3b26fc72ff33d, scope.system=all) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:52.251861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90dff1-e671-4c7b-b864-ea259ef5c1bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:52.254708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90dff1-e671-4c7b-b864-ea259ef5c1bf None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:52.263361 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:52.281071 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 168/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:13:51 2022] GET /identity/v3/role_assignments?user.id=bfc7271bfbc34fed93f3b26fc72ff33d&scope.system=all&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 510 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:53.465353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92246aa1-a1d9-4ea3-9a47-2a9f873c372d None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:53.466476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92246aa1-a1d9-4ea3-9a47-2a9f873c372d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:53.466981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92246aa1-a1d9-4ea3-9a47-2a9f873c372d None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:53.468645 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 167/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:53.475699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b34323-4e9e-424a-82da-eef1e5fe92a8 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:53.476672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b34323-4e9e-424a-82da-eef1e5fe92a8 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:53.477182 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b34323-4e9e-424a-82da-eef1e5fe92a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:53.497460 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-58b34323-4e9e-424a-82da-eef1e5fe92a8 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:53.499100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-58b34323-4e9e-424a-82da-eef1e5fe92a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:53.573015 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df421dc4-59e8-450e-8065-7d85509debf4 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:53.573371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df421dc4-59e8-450e-8065-7d85509debf4 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:53.573574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df421dc4-59e8-450e-8065-7d85509debf4 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:53.575210 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 168/336] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:53.581451 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaa052-6a19-47bc-8531-b866eea161f0 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:53.581742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaa052-6a19-47bc-8531-b866eea161f0 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:53.582004 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaa052-6a19-47bc-8531-b866eea161f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:53.583020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 169/337] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:53.607935 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b34323-4e9e-424a-82da-eef1e5fe92a8 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:53.615174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-4bdaa052-6a19-47bc-8531-b866eea161f0 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:53.616392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-4bdaa052-6a19-47bc-8531-b866eea161f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:53.775672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 169/338] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:53.796951 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b34323-4e9e-424a-82da-eef1e5fe92a8 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:53.812892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdaa052-6a19-47bc-8531-b866eea161f0 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:53.834926 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:53.847005 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:53.851331 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c5e47-01b7-4d4b-a664-546228aff48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:53.852493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c5e47-01b7-4d4b-a664-546228aff48b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:53.852493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c5e47-01b7-4d4b-a664-546228aff48b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:53.852698 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c5e47-01b7-4d4b-a664-546228aff48b None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:53.874828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c5e47-01b7-4d4b-a664-546228aff48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-872c5e47-01b7-4d4b-a664-546228aff48b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:53.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:53.882099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:53.883636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:53.887403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 170/339] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:53.889318 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c5e47-01b7-4d4b-a664-546228aff48b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:54.045330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c5e47-01b7-4d4b-a664-546228aff48b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:54.053441 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdaa052-6a19-47bc-8531-b866eea161f0 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:54.093660 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:54.094781 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:54.095292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:54.095789 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:54.096791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:54.097319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:54.099035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:54.113785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 171/340] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:54.120453 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:54.138969 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3b0cf-36be-444c-8491-37e3ef32c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:54.139984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3b0cf-36be-444c-8491-37e3ef32c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:54.140453 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3b0cf-36be-444c-8491-37e3ef32c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:54.140902 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3b0cf-36be-444c-8491-37e3ef32c1f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:54.169174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3b0cf-36be-444c-8491-37e3ef32c1f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-2ba3b0cf-36be-444c-8491-37e3ef32c1f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:54.174979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:54.176113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:54.177071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:54.180086 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 170/341] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:54.185922 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3b0cf-36be-444c-8491-37e3ef32c1f6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:54.203181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:54.277024 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945531-ee31-4103-9523-a8d708c743bf None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:54.310469 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:54.333893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea37cb-5512-493c-b11c-35a5e2363e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:54.335015 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea37cb-5512-493c-b11c-35a5e2363e87 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:54.335522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea37cb-5512-493c-b11c-35a5e2363e87 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:54.336029 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea37cb-5512-493c-b11c-35a5e2363e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:54.342497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3b0cf-36be-444c-8491-37e3ef32c1f6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:54.383575 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:54.384226 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:54.384511 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:54.384784 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:54.385524 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:54.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:54.387282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:54.396555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 171/342] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:54.401771 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:54.442167 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea37cb-5512-493c-b11c-35a5e2363e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:54.443682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea37cb-5512-493c-b11c-35a5e2363e87 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:54.508447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:54.512497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb56e-44b3-448e-b00a-a371d859591b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:54.561489 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344212ac-ed02-4c57-b4e4-cea941647654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:54.562058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344212ac-ed02-4c57-b4e4-cea941647654 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:54.562366 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344212ac-ed02-4c57-b4e4-cea941647654 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:54.562649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344212ac-ed02-4c57-b4e4-cea941647654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:54.569559 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:54.575598 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 172/343] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:13:54.578138 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea37cb-5512-493c-b11c-35a5e2363e87 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:54.649089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344212ac-ed02-4c57-b4e4-cea941647654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:54.650619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344212ac-ed02-4c57-b4e4-cea941647654 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:54.684511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea37cb-5512-493c-b11c-35a5e2363e87 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:54.734933 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabff4e1-419e-4956-ae1f-8a716de64ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:54.735614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff4e1-419e-4956-ae1f-8a716de64ead None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:54.735896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff4e1-419e-4956-ae1f-8a716de64ead None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:54.736171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff4e1-419e-4956-ae1f-8a716de64ead None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:54.737635 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabff4e1-419e-4956-ae1f-8a716de64ead None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:54.739165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabff4e1-419e-4956-ae1f-8a716de64ead None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:54.742283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 173/344] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:54 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:54.746934 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabff4e1-419e-4956-ae1f-8a716de64ead None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:54.772932 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 172/345] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:54.778241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344212ac-ed02-4c57-b4e4-cea941647654 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:54.785905 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:54.830161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:54.973386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344212ac-ed02-4c57-b4e4-cea941647654 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:54.983382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabff4e1-419e-4956-ae1f-8a716de64ead None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:55.037066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c665d4-9d81-4986-9d08-61428f8a802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:55.038161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c665d4-9d81-4986-9d08-61428f8a802d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.038701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c665d4-9d81-4986-9d08-61428f8a802d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.039238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c665d4-9d81-4986-9d08-61428f8a802d None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.046858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c665d4-9d81-4986-9d08-61428f8a802d None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:55.048751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c665d4-9d81-4986-9d08-61428f8a802d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:55.050987 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabecd8-1673-4dce-99bd-aaf026b736e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:55.051546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 174/346] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:13:54 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:55.051752 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabecd8-1673-4dce-99bd-aaf026b736e6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.052027 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabecd8-1673-4dce-99bd-aaf026b736e6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.052300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabecd8-1673-4dce-99bd-aaf026b736e6 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.053808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a54b-0771-4816-8b07-5241a443e4ce None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.054367 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a54b-0771-4816-8b07-5241a443e4ce None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.054642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabecd8-1673-4dce-99bd-aaf026b736e6 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:55.055101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a54b-0771-4816-8b07-5241a443e4ce None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.056150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabecd8-1673-4dce-99bd-aaf026b736e6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:55.056960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 175/347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:55.061068 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 173/348] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:54 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:55.062990 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d258-9667-444b-a770-e265595e6ac9 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.063544 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d258-9667-444b-a770-e265595e6ac9 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.064043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d258-9667-444b-a770-e265595e6ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.064165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabecd8-1673-4dce-99bd-aaf026b736e6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:55.103345 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-51a8d258-9667-444b-a770-e265595e6ac9 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:55.104496 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-51a8d258-9667-444b-a770-e265595e6ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:55.211738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabecd8-1673-4dce-99bd-aaf026b736e6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:55.214161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:55.253589 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:55.254251 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.254532 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.254806 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.255611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] RBAC: Adding query filter params (group.id=84b7d8ea31a9475eb4e398e697cb3cae, role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:55.255918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:55.257411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:55.265291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 176/349] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:55.266189 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8d258-9667-444b-a770-e265595e6ac9 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:55.272776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 174/350] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:13:55 2022] GET /identity/v3/role_assignments?group.id=84b7d8ea31a9475eb4e398e697cb3cae&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:55.302511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:55.322485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:55.418834 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8d258-9667-444b-a770-e265595e6ac9 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:55.467345 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83655d2e-7ce2-4840-844a-d0a505518596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:55.468043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83655d2e-7ce2-4840-844a-d0a505518596 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.468320 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83655d2e-7ce2-4840-844a-d0a505518596 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.468592 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83655d2e-7ce2-4840-844a-d0a505518596 None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.469810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83655d2e-7ce2-4840-844a-d0a505518596 None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:55.471413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83655d2e-7ce2-4840-844a-d0a505518596 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:55.473154 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 177/351] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:13:55 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:55.481551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83655d2e-7ce2-4840-844a-d0a505518596 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:55.545978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d4a58-1566-4a2e-8f39-2db693eb187a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:55.614699 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce5a69-512f-4784-9105-cfbd9d99eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:55.615376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce5a69-512f-4784-9105-cfbd9d99eed8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.615657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce5a69-512f-4784-9105-cfbd9d99eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.615935 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce5a69-512f-4784-9105-cfbd9d99eed8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.627731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce5a69-512f-4784-9105-cfbd9d99eed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:55.635444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-acce5a69-512f-4784-9105-cfbd9d99eed8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:55.638345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:55.639679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:55.640620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:55.641582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 175/352] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:55.645095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92208240-2f35-4cf9-8a1d-322e4a43e14e None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.645472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92208240-2f35-4cf9-8a1d-322e4a43e14e None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.645814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92208240-2f35-4cf9-8a1d-322e4a43e14e None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.647674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 176/353] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:55.650791 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92208240-2f35-4cf9-8a1d-322e4a43e14e None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:55.668787 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83655d2e-7ce2-4840-844a-d0a505518596 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:55.724272 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:55.728194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c4ab4-2938-480f-8d12-1c504a47b0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:55.729066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c4ab4-2938-480f-8d12-1c504a47b0e2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.729447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c4ab4-2938-480f-8d12-1c504a47b0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.729729 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c4ab4-2938-480f-8d12-1c504a47b0e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.731554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c4ab4-2938-480f-8d12-1c504a47b0e2 None admin] RBAC: Adding query filter params (group.id=84b7d8ea31a9475eb4e398e697cb3cae, role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:55.731865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c4ab4-2938-480f-8d12-1c504a47b0e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:55.733335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c4ab4-2938-480f-8d12-1c504a47b0e2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:55.740251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 178/354] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:13:55 2022] GET /identity/v3/role_assignments?group.id=84b7d8ea31a9475eb4e398e697cb3cae&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:55.741722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc48dc-7393-4012-827b-0c0d54c42d22 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.742002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc48dc-7393-4012-827b-0c0d54c42d22 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.742317 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc48dc-7393-4012-827b-0c0d54c42d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.775510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d3dc48dc-7393-4012-827b-0c0d54c42d22 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:55.776800 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d3dc48dc-7393-4012-827b-0c0d54c42d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:55.865917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92208240-2f35-4cf9-8a1d-322e4a43e14e None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:55.883105 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 179/355] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:55.904347 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:55.905179 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:55.905465 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:55.905749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:55.906566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:55.906874 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:55.908312 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:55.918696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 177/356] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:55.924556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc48dc-7393-4012-827b-0c0d54c42d22 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:55.931147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:56.018076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:56.056150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc48dc-7393-4012-827b-0c0d54c42d22 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:56.102029 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ba16-d6ed-4f8c-a02e-3b92623db89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:56.102702 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ba16-d6ed-4f8c-a02e-3b92623db89f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.102983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ba16-d6ed-4f8c-a02e-3b92623db89f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.103257 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ba16-d6ed-4f8c-a02e-3b92623db89f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.120011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:56.146066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd64e3b-f700-49ed-b426-9eab9a53bdcf None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:56.180893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259ba16-d6ed-4f8c-a02e-3b92623db89f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:56.182836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259ba16-d6ed-4f8c-a02e-3b92623db89f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:56.223820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e2519-7b29-483c-a381-9155f0fc8479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:56.224946 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e2519-7b29-483c-a381-9155f0fc8479 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.225331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e2519-7b29-483c-a381-9155f0fc8479 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.225728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e2519-7b29-483c-a381-9155f0fc8479 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.259124 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e2519-7b29-483c-a381-9155f0fc8479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:56.260476 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-739e2519-7b29-483c-a381-9155f0fc8479 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:56.264632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:56.267360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:56.268512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 178/357] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:13:56.272228 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e2519-7b29-483c-a381-9155f0fc8479 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:56.300202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 180/358] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:56.305425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ba16-d6ed-4f8c-a02e-3b92623db89f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:56.420776 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ba16-d6ed-4f8c-a02e-3b92623db89f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:56.433492 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e2519-7b29-483c-a381-9155f0fc8479 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:56.460349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59028-a2e8-47c3-b563-fddbd1fa8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:56.461443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59028-a2e8-47c3-b563-fddbd1fa8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.461726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59028-a2e8-47c3-b563-fddbd1fa8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.461994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59028-a2e8-47c3-b563-fddbd1fa8bd4 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.463254 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb59028-a2e8-47c3-b563-fddbd1fa8bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:56.464735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb59028-a2e8-47c3-b563-fddbd1fa8bd4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:56.470130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 181/359] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:56 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:56.470130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc775baf-7c7c-4dea-ad56-286c4ff9fe97 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.470130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc775baf-7c7c-4dea-ad56-286c4ff9fe97 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.470130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc775baf-7c7c-4dea-ad56-286c4ff9fe97 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.470549 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 182/360] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:56.472064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc775baf-7c7c-4dea-ad56-286c4ff9fe97 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:56.494288 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851367c-8cb3-4a1d-8dcc-b6c622f07f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:56.495408 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851367c-8cb3-4a1d-8dcc-b6c622f07f39 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.497692 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851367c-8cb3-4a1d-8dcc-b6c622f07f39 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.498311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851367c-8cb3-4a1d-8dcc-b6c622f07f39 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.499355 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851367c-8cb3-4a1d-8dcc-b6c622f07f39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:56.501867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851367c-8cb3-4a1d-8dcc-b6c622f07f39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:56.503840 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851367c-8cb3-4a1d-8dcc-b6c622f07f39 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:56.521731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 179/361] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:56.528569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb292e-8a90-45d5-9c0e-bcf2cfa4cf0a None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.529912 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb292e-8a90-45d5-9c0e-bcf2cfa4cf0a None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.530221 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb292e-8a90-45d5-9c0e-bcf2cfa4cf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.551870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-ceeb292e-8a90-45d5-9c0e-bcf2cfa4cf0a None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:56.554033 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-ceeb292e-8a90-45d5-9c0e-bcf2cfa4cf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:56.556255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:56.574651 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:56.630428 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc775baf-7c7c-4dea-ad56-286c4ff9fe97 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:56.644408 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 180/362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:56.645837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be9fd-b461-44c2-938c-0a5a257b18f2 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.647348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be9fd-b461-44c2-938c-0a5a257b18f2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.647638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be9fd-b461-44c2-938c-0a5a257b18f2 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.649210 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 181/363] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:56.649977 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8be9fd-b461-44c2-938c-0a5a257b18f2 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:56.684608 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943549b4-ccba-47c1-9a48-ca70b5aafd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:56.685223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943549b4-ccba-47c1-9a48-ca70b5aafd00 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.685525 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943549b4-ccba-47c1-9a48-ca70b5aafd00 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.685803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943549b4-ccba-47c1-9a48-ca70b5aafd00 None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.691181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943549b4-ccba-47c1-9a48-ca70b5aafd00 None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:56.693088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943549b4-ccba-47c1-9a48-ca70b5aafd00 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:56.695599 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 183/364] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:13:56 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:56.697042 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2947f-7bb9-43a3-8cd3-785031a422b2 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.697342 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2947f-7bb9-43a3-8cd3-785031a422b2 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.700303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2947f-7bb9-43a3-8cd3-785031a422b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.733660 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-37d2947f-7bb9-43a3-8cd3-785031a422b2 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:56.748180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-37d2947f-7bb9-43a3-8cd3-785031a422b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:56.849443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 184/365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:56.850204 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2947f-7bb9-43a3-8cd3-785031a422b2 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:56.850624 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8be9fd-b461-44c2-938c-0a5a257b18f2 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:56.899777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:56.924166 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e0f6e-7138-49de-8d86-b85b9e2b8ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:56.925344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e0f6e-7138-49de-8d86-b85b9e2b8ccf None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:56.925893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e0f6e-7138-49de-8d86-b85b9e2b8ccf None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:56.926414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e0f6e-7138-49de-8d86-b85b9e2b8ccf None admin] PATH_INFO: `/v3/users/18af986a5fda41d4af45150cad65b324` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:56.928069 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e0f6e-7138-49de-8d86-b85b9e2b8ccf None admin] RBAC: Authorizing `identity:get_user(user_id=18af986a5fda41d4af45150cad65b324)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:56.929762 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e0f6e-7138-49de-8d86-b85b9e2b8ccf None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:56.932245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 182/366] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:13:56 2022] GET /identity/v3/users/18af986a5fda41d4af45150cad65b324 => generated 293 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:56.933323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e0f6e-7138-49de-8d86-b85b9e2b8ccf None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:56.947238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:56.990417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2947f-7bb9-43a3-8cd3-785031a422b2 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:57.029496 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d99df5-f974-49eb-99ec-0be207e56b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:57.031863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d99df5-f974-49eb-99ec-0be207e56b90 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:57.032166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d99df5-f974-49eb-99ec-0be207e56b90 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:57.032446 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d99df5-f974-49eb-99ec-0be207e56b90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:57.046922 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d99df5-f974-49eb-99ec-0be207e56b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-a4d99df5-f974-49eb-99ec-0be207e56b90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:57.052152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:57.053218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 185/367] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:57.057597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d99df5-f974-49eb-99ec-0be207e56b90 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:57.071772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e0f6e-7138-49de-8d86-b85b9e2b8ccf None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:57.125443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:57.126704 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:57.128161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:57.130554 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:57.131701 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] RBAC: Adding query filter params (group.id=84b7d8ea31a9475eb4e398e697cb3cae, role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:57.132030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:57.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:57.145857 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 183/368] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:13:56 2022] GET /identity/v3/role_assignments?group.id=84b7d8ea31a9475eb4e398e697cb3cae&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:57.147793 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:57.241844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:57.247455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d99df5-f974-49eb-99ec-0be207e56b90 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:57.311928 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3ab54-e2d3-4aff-8176-fef8cf681aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:57.312947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3ab54-e2d3-4aff-8176-fef8cf681aa9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:57.313829 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3ab54-e2d3-4aff-8176-fef8cf681aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:57.314162 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3ab54-e2d3-4aff-8176-fef8cf681aa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:57.344868 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3ab54-e2d3-4aff-8176-fef8cf681aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-9dd3ab54-e2d3-4aff-8176-fef8cf681aa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:57.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:57.351806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:57.352743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:57.355887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 186/369] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:57.359353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3ab54-e2d3-4aff-8176-fef8cf681aa9 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:57.385034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:57.462443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06032b-aa79-4c4f-9a18-e9a30a3cb8dc None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:57.536092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3ab54-e2d3-4aff-8176-fef8cf681aa9 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:57.591540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:57.592208 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:57.592490 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:57.592766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:57.593522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:57.593827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:57.596202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:57.609967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 187/370] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:57.611983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:57.626867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:57.628338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:57.628859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:57.629361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:57.630452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] RBAC: Adding query filter params (role.id=a0a0e158ed4f4f9496ff04139752f839, user.id=18af986a5fda41d4af45150cad65b324, scope.system=all) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:57.630987 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:57.632679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:57.641786 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:57.661093 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 184/371] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:13:57 2022] GET /identity/v3/role_assignments?user.id=18af986a5fda41d4af45150cad65b324&scope.system=all&role.id=a0a0e158ed4f4f9496ff04139752f839 => generated 510 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:57.662385 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:57.757169 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de0f9e-1ed7-403a-87e7-6808a03276df None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:57.840566 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:57.841285 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:57.841564 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:57.841834 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:57.842599 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:57.842902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:57.844413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:57.860244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 188/372] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:57.860969 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64038a4-760d-450e-8661-2ef94d3c96ed None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:57.871520 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:57.915992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22aa54-15f4-46c5-9266-b7faa60d2a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:57.916642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22aa54-15f4-46c5-9266-b7faa60d2a7b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:57.916917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22aa54-15f4-46c5-9266-b7faa60d2a7b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:57.917188 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22aa54-15f4-46c5-9266-b7faa60d2a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:57.918389 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:58.023684 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22aa54-15f4-46c5-9266-b7faa60d2a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:58.025213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22aa54-15f4-46c5-9266-b7faa60d2a7b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:58.068494 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:58.103617 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bff6e-88cd-4c1b-988c-cb2f6a1758e7 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:58.157843 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c26ec9-837f-45ef-90ad-00d93b8469f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:58.159096 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26ec9-837f-45ef-90ad-00d93b8469f2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.159562 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26ec9-837f-45ef-90ad-00d93b8469f2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.159999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26ec9-837f-45ef-90ad-00d93b8469f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.172515 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 185/373] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:58.178165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22aa54-15f4-46c5-9266-b7faa60d2a7b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:58.236126 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:58.305206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22aa54-15f4-46c5-9266-b7faa60d2a7b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:58.347975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60278d-19ff-487b-9677-7bca750d3ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:58.348670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60278d-19ff-487b-9677-7bca750d3ea1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.348956 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60278d-19ff-487b-9677-7bca750d3ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.349237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60278d-19ff-487b-9677-7bca750d3ea1 None admin] PATH_INFO: `/v3/projects/c6d6cb8b74c64466888b7108b567aa82` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.350555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60278d-19ff-487b-9677-7bca750d3ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d6cb8b74c64466888b7108b567aa82)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:58.352621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60278d-19ff-487b-9677-7bca750d3ea1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:58.354885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 186/374] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:58 2022] GET /identity/v3/projects/c6d6cb8b74c64466888b7108b567aa82 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:58.360975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60278d-19ff-487b-9677-7bca750d3ea1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:58.395115 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c26ec9-837f-45ef-90ad-00d93b8469f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:58.396681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c26ec9-837f-45ef-90ad-00d93b8469f2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:58.478697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60278d-19ff-487b-9677-7bca750d3ea1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:58.480617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:58.523984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6a979-cb71-46e1-a663-273300ed88a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:58.524789 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6a979-cb71-46e1-a663-273300ed88a6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.525166 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6a979-cb71-46e1-a663-273300ed88a6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.525555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6a979-cb71-46e1-a663-273300ed88a6 None admin] PATH_INFO: `/v3/groups/2ff52158b39d4790ac5f3cb49d9b37cd` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.532900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6a979-cb71-46e1-a663-273300ed88a6 None admin] RBAC: Authorizing `identity:get_group(group_id=2ff52158b39d4790ac5f3cb49d9b37cd)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:58.534581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6a979-cb71-46e1-a663-273300ed88a6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:58.537338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 187/375] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:13:58 2022] GET /identity/v3/groups/2ff52158b39d4790ac5f3cb49d9b37cd => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:58.543273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6a979-cb71-46e1-a663-273300ed88a6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:58.642598 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:58.652702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6a979-cb71-46e1-a663-273300ed88a6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:58.685325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 189/376] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 814 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:13:58.687085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ca4d-09a2-4ae0-9762-b8f98658b538 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.687085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ca4d-09a2-4ae0-9762-b8f98658b538 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.687410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ca4d-09a2-4ae0-9762-b8f98658b538 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.690498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 190/377] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:58.691967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e99b2-9a0b-4eaf-8998-81fa7b4fefc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:58.692678 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e99b2-9a0b-4eaf-8998-81fa7b4fefc7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.693026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e99b2-9a0b-4eaf-8998-81fa7b4fefc7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.693308 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e99b2-9a0b-4eaf-8998-81fa7b4fefc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.694420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e99b2-9a0b-4eaf-8998-81fa7b4fefc7 None admin] RBAC: Adding query filter params (group.id=2ff52158b39d4790ac5f3cb49d9b37cd, role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=c6d6cb8b74c64466888b7108b567aa82) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:58.694730 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e99b2-9a0b-4eaf-8998-81fa7b4fefc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:58.694972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ca4d-09a2-4ae0-9762-b8f98658b538 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:58.696202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e99b2-9a0b-4eaf-8998-81fa7b4fefc7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:58.703827 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 188/378] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:13:58 2022] GET /identity/v3/role_assignments?group.id=2ff52158b39d4790ac5f3cb49d9b37cd&scope.project.id=c6d6cb8b74c64466888b7108b567aa82&role.id=40504e80dbb845aeb07ca258977ecacb => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:58.704405 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8614e-95c4-4186-8e52-1f0d1e1db342 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.704697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8614e-95c4-4186-8e52-1f0d1e1db342 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.704967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8614e-95c4-4186-8e52-1f0d1e1db342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.724611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-d4c8614e-95c4-4186-8e52-1f0d1e1db342 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:58.725864 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-d4c8614e-95c4-4186-8e52-1f0d1e1db342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:58.760916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:58.829478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 189/379] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:13:58.830997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe019f-18fa-43b5-909e-e8f28bff1114 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.831283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe019f-18fa-43b5-909e-e8f28bff1114 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.831552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe019f-18fa-43b5-909e-e8f28bff1114 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.832901 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 190/380] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:58.839026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba635d-6b6b-4442-9d0c-08591e9eb8ee None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.839308 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba635d-6b6b-4442-9d0c-08591e9eb8ee None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.839585 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba635d-6b6b-4442-9d0c-08591e9eb8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.867304 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-daba635d-6b6b-4442-9d0c-08591e9eb8ee None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:58.869369 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-daba635d-6b6b-4442-9d0c-08591e9eb8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:58.905164 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ca4d-09a2-4ae0-9762-b8f98658b538 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:58.967009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746c4a7-a21c-4f35-b5d7-118a631ff2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:58.967928 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746c4a7-a21c-4f35-b5d7-118a631ff2e9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:58.968314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746c4a7-a21c-4f35-b5d7-118a631ff2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:58.968990 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746c4a7-a21c-4f35-b5d7-118a631ff2e9 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:58.970804 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746c4a7-a21c-4f35-b5d7-118a631ff2e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:58.974076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746c4a7-a21c-4f35-b5d7-118a631ff2e9 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:58.978469 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 191/381] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:58 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:58.978469 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746c4a7-a21c-4f35-b5d7-118a631ff2e9 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:58.978752 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 191/382] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:13:58.983460 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba635d-6b6b-4442-9d0c-08591e9eb8ee None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:59.044553 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:59.086521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:59.098762 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba635d-6b6b-4442-9d0c-08591e9eb8ee None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:59.116934 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746c4a7-a21c-4f35-b5d7-118a631ff2e9 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:59.139765 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404825c0-3963-4e90-a46a-6d53ad080ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:59.140446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404825c0-3963-4e90-a46a-6d53ad080ce0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.140769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404825c0-3963-4e90-a46a-6d53ad080ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.140988 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404825c0-3963-4e90-a46a-6d53ad080ce0 None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.142268 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404825c0-3963-4e90-a46a-6d53ad080ce0 None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:59.143942 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404825c0-3963-4e90-a46a-6d53ad080ce0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:59.150255 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 192/383] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:13:58 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:59.151235 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404825c0-3963-4e90-a46a-6d53ad080ce0 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:59.166535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb735990-4518-483d-9c32-ba776c6da57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:59.167899 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb735990-4518-483d-9c32-ba776c6da57a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.168397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb735990-4518-483d-9c32-ba776c6da57a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.168898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb735990-4518-483d-9c32-ba776c6da57a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.188666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb735990-4518-483d-9c32-ba776c6da57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-cb735990-4518-483d-9c32-ba776c6da57a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:59.194089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:59.195970 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 192/384] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:13:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:59.200777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb735990-4518-483d-9c32-ba776c6da57a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:59.330707 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb735990-4518-483d-9c32-ba776c6da57a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:59.344299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:59.369193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404825c0-3963-4e90-a46a-6d53ad080ce0 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:59.371672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:59.372319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.373358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.373358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.373677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] RBAC: Adding query filter params (group.id=84b7d8ea31a9475eb4e398e697cb3cae, role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:59.373965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:59.375463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:59.382608 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 193/385] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:13:59 2022] GET /identity/v3/role_assignments?group.id=84b7d8ea31a9475eb4e398e697cb3cae&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:59.387184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:59.420551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91bb0-8b36-4dbb-9f08-8d39eecd61d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:59.421141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91bb0-8b36-4dbb-9f08-8d39eecd61d6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.421424 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91bb0-8b36-4dbb-9f08-8d39eecd61d6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.421696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91bb0-8b36-4dbb-9f08-8d39eecd61d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.444739 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd91bb0-8b36-4dbb-9f08-8d39eecd61d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-5bd91bb0-8b36-4dbb-9f08-8d39eecd61d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:13:59.450299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:13:59.451363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:13:59.452311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:13:59.458139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 193/386] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:13:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:13:59.461475 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91bb0-8b36-4dbb-9f08-8d39eecd61d6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:59.492883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:59.560504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d5ac-eacc-49d5-bec8-530a11bd7734 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:59.613101 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91bb0-8b36-4dbb-9f08-8d39eecd61d6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:59.636784 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:59.637633 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.638444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.638444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.639358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:59.639776 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:59.641306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:59.656112 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:59.657390 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.657838 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.658950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.659869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:13:59.660325 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:59.661878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:59.667292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 194/387] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:13:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:59.674015 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 194/388] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:13:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:59.674302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:59.678720 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:59.761887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:59.774770 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:13:59.803012 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4862-dff1-4df9-9b91-f502ed67008f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:59.843342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7f649-7723-43a5-b641-0e9a47852520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:59.845386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f649-7723-43a5-b641-0e9a47852520 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.845851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f649-7723-43a5-b641-0e9a47852520 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.846414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f649-7723-43a5-b641-0e9a47852520 None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.847849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7f649-7723-43a5-b641-0e9a47852520 None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:59.849485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7f649-7723-43a5-b641-0e9a47852520 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:59.852326 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 195/389] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:13:59 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:13:59.853951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51f2c5-b873-4173-a14a-247f72c76e7f None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.858625 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51f2c5-b873-4173-a14a-247f72c76e7f None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.859133 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51f2c5-b873-4173-a14a-247f72c76e7f None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.860724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 196/390] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:13:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:13:59.862511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bcc42-64c7-4e28-ac3d-aa1df6781d8d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:13:59.862855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51f2c5-b873-4173-a14a-247f72c76e7f None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:13:59.898828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc2dfc-2f11-43d9-bd32-0b7286a85200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:13:59.899503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc2dfc-2f11-43d9-bd32-0b7286a85200 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.899791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc2dfc-2f11-43d9-bd32-0b7286a85200 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.900329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc2dfc-2f11-43d9-bd32-0b7286a85200 None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.901334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc2dfc-2f11-43d9-bd32-0b7286a85200 None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:13:59.902927 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc2dfc-2f11-43d9-bd32-0b7286a85200 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:13:59.911702 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 195/391] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:13:59 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:13:59.912946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916ca3b-bb02-49dd-9d53-18e09be802a8 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:13:59.913239 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916ca3b-bb02-49dd-9d53-18e09be802a8 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:13:59.913522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916ca3b-bb02-49dd-9d53-18e09be802a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:13:59.932867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-f916ca3b-bb02-49dd-9d53-18e09be802a8 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:13:59.934074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f916ca3b-bb02-49dd-9d53-18e09be802a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:13:59.982764 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51f2c5-b873-4173-a14a-247f72c76e7f None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:00.009739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 196/392] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:00.010809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916ca3b-bb02-49dd-9d53-18e09be802a8 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:00.022421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a0480-f30f-4d05-bec3-1b0a21d00d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:00.023439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0480-f30f-4d05-bec3-1b0a21d00d02 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:00.023891 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0480-f30f-4d05-bec3-1b0a21d00d02 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:00.024339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0480-f30f-4d05-bec3-1b0a21d00d02 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:00.025713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a0480-f30f-4d05-bec3-1b0a21d00d02 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:00.028757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a0480-f30f-4d05-bec3-1b0a21d00d02 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:00.031323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 197/393] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:13:59 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:00.037396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a0480-f30f-4d05-bec3-1b0a21d00d02 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:00.048658 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:00.057378 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:00.124161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916ca3b-bb02-49dd-9d53-18e09be802a8 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:00.156085 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a0480-f30f-4d05-bec3-1b0a21d00d02 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:00.170863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7504-d93e-4faf-b347-a0cd538f2a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:00.171541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7504-d93e-4faf-b347-a0cd538f2a1f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:00.171824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7504-d93e-4faf-b347-a0cd538f2a1f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:00.172111 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7504-d93e-4faf-b347-a0cd538f2a1f None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:00.173608 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7504-d93e-4faf-b347-a0cd538f2a1f None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:00.175910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7504-d93e-4faf-b347-a0cd538f2a1f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:00.178256 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 197/394] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:00 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:00.179188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7504-d93e-4faf-b347-a0cd538f2a1f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:00.212416 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2afdc-0692-4df7-b269-3341b4d68045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:00.213048 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2afdc-0692-4df7-b269-3341b4d68045 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:00.213334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2afdc-0692-4df7-b269-3341b4d68045 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:00.213617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2afdc-0692-4df7-b269-3341b4d68045 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/groups/84b7d8ea31a9475eb4e398e697cb3cae/roles/0aacd40c75f64f668308391b44dc4e85` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:00.215693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2afdc-0692-4df7-b269-3341b4d68045 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ac986a4d83f4f07af22c95818f1b5bb, group_id=84b7d8ea31a9475eb4e398e697cb3cae, role_id=0aacd40c75f64f668308391b44dc4e85)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:00.217285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2afdc-0692-4df7-b269-3341b4d68045 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:00.244853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 198/395] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:14:00 2022] PUT /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/groups/84b7d8ea31a9475eb4e398e697cb3cae/roles/0aacd40c75f64f668308391b44dc4e85 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:00.246086 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2afdc-0692-4df7-b269-3341b4d68045 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:00.344120 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7504-d93e-4faf-b347-a0cd538f2a1f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:00.346340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:00.390546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a6d76-e37e-45f7-a84c-26fbcf932e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:00.392253 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a6d76-e37e-45f7-a84c-26fbcf932e36 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:00.392778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a6d76-e37e-45f7-a84c-26fbcf932e36 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:00.393376 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a6d76-e37e-45f7-a84c-26fbcf932e36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:00.409014 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:00.422486 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a6d76-e37e-45f7-a84c-26fbcf932e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-982a6d76-e37e-45f7-a84c-26fbcf932e36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:00.428221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:00.429350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:14:00.430348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:00.438497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 198/396] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:00.444661 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a6d76-e37e-45f7-a84c-26fbcf932e36 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:00.475645 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2afdc-0692-4df7-b269-3341b4d68045 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:00.521882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c928d-e2bc-454d-acad-b92c23938653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:00.523050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c928d-e2bc-454d-acad-b92c23938653 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:00.523555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c928d-e2bc-454d-acad-b92c23938653 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:00.524069 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c928d-e2bc-454d-acad-b92c23938653 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/groups/84b7d8ea31a9475eb4e398e697cb3cae/roles/0f002225dde34ef1a2adda2f583b8520` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:00.526664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c928d-e2bc-454d-acad-b92c23938653 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ac986a4d83f4f07af22c95818f1b5bb, group_id=84b7d8ea31a9475eb4e398e697cb3cae, role_id=0f002225dde34ef1a2adda2f583b8520)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:00.530940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c928d-e2bc-454d-acad-b92c23938653 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:00.558895 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 199/397] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:14:00 2022] PUT /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb/groups/84b7d8ea31a9475eb4e398e697cb3cae/roles/0f002225dde34ef1a2adda2f583b8520 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:00.625423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a6d76-e37e-45f7-a84c-26fbcf932e36 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:00.666484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:00.667214 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:00.667501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:00.667774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:00.668534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:00.668842 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:00.670568 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:00.681319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 199/398] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:00.687182 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c928d-e2bc-454d-acad-b92c23938653 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:00.835197 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c928d-e2bc-454d-acad-b92c23938653 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:00.872531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b498c-e066-4f3e-9f51-4c6ce66d2271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:00.873216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b498c-e066-4f3e-9f51-4c6ce66d2271 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:00.873495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b498c-e066-4f3e-9f51-4c6ce66d2271 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:00.873771 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b498c-e066-4f3e-9f51-4c6ce66d2271 None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:00.875130 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b498c-e066-4f3e-9f51-4c6ce66d2271 None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:00.876606 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b498c-e066-4f3e-9f51-4c6ce66d2271 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:00.878332 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 200/399] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:14:00 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:00.884240 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:00.919304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:01.009780 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9586de-9648-4c8f-ac69-f830c13126e0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:01.045470 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726386a-4bc1-4d3f-969a-14f334df00de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:01.046162 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726386a-4bc1-4d3f-969a-14f334df00de None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:01.046455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726386a-4bc1-4d3f-969a-14f334df00de None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:01.046738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726386a-4bc1-4d3f-969a-14f334df00de None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:01.049902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726386a-4bc1-4d3f-969a-14f334df00de None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:01.051481 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726386a-4bc1-4d3f-969a-14f334df00de None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:01.053552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 200/400] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:00 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:01.059508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b498c-e066-4f3e-9f51-4c6ce66d2271 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:01.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:01.169493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b498c-e066-4f3e-9f51-4c6ce66d2271 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:01.205776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018532bd-7054-4fa7-b68e-ddf5ea184fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:01.206510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018532bd-7054-4fa7-b68e-ddf5ea184fb2 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:01.206843 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018532bd-7054-4fa7-b68e-ddf5ea184fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:01.207137 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018532bd-7054-4fa7-b68e-ddf5ea184fb2 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/groups/84b7d8ea31a9475eb4e398e697cb3cae/roles/0aacd40c75f64f668308391b44dc4e85` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:01.210065 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018532bd-7054-4fa7-b68e-ddf5ea184fb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8083f126cd24c4b93c7be83ca71f0ee, group_id=84b7d8ea31a9475eb4e398e697cb3cae, role_id=0aacd40c75f64f668308391b44dc4e85)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:01.211633 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018532bd-7054-4fa7-b68e-ddf5ea184fb2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:01.240981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 201/401] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:14:01 2022] PUT /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/groups/84b7d8ea31a9475eb4e398e697cb3cae/roles/0aacd40c75f64f668308391b44dc4e85 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:01.561102 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e855-ad15-4b71-9846-1adbb9f25dc1 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:01.561821 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e855-ad15-4b71-9846-1adbb9f25dc1 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:01.562460 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e855-ad15-4b71-9846-1adbb9f25dc1 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:01.564562 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 201/402] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:01.570346 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787d3f0-fdbc-42ae-aaf0-34f8943e95c4 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:01.570978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787d3f0-fdbc-42ae-aaf0-34f8943e95c4 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:01.571958 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787d3f0-fdbc-42ae-aaf0-34f8943e95c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:01.597894 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-1787d3f0-fdbc-42ae-aaf0-34f8943e95c4 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:01.599154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-1787d3f0-fdbc-42ae-aaf0-34f8943e95c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:01.683026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:01.702477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 202/403] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:01.724928 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e855-ad15-4b71-9846-1adbb9f25dc1 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:01.872452 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:01.884726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e855-ad15-4b71-9846-1adbb9f25dc1 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:01.924243 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c55588-2f6d-48ef-99d3-5fcfe09aa268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:01.924932 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55588-2f6d-48ef-99d3-5fcfe09aa268 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:01.925271 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55588-2f6d-48ef-99d3-5fcfe09aa268 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:01.925542 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55588-2f6d-48ef-99d3-5fcfe09aa268 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:01.937072 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c55588-2f6d-48ef-99d3-5fcfe09aa268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-a3c55588-2f6d-48ef-99d3-5fcfe09aa268 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:01.942971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:01.944034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:01.944997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:01.945973 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 202/404] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:01.952844 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787d3f0-fdbc-42ae-aaf0-34f8943e95c4 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:02.106809 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787d3f0-fdbc-42ae-aaf0-34f8943e95c4 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:02.150927 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31720a9-8247-440d-b995-7d738b6203ec None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.151627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31720a9-8247-440d-b995-7d738b6203ec None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.152274 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31720a9-8247-440d-b995-7d738b6203ec None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.154829 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 203/405] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:14:02.158208 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:02.158208 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.158208 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.158208 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.158713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:02.158713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:02.160636 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:02.162425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876158a6-e0c8-48f2-976d-2bdfb0a809d0 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.163014 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876158a6-e0c8-48f2-976d-2bdfb0a809d0 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.163500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876158a6-e0c8-48f2-976d-2bdfb0a809d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.170801 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 203/406] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:02.178418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:02.189949 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-876158a6-e0c8-48f2-976d-2bdfb0a809d0 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:02.191601 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-876158a6-e0c8-48f2-976d-2bdfb0a809d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:02.270247 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 204/407] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:02.284352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:02.293127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876158a6-e0c8-48f2-976d-2bdfb0a809d0 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:02.375256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddde48d-12c6-4bdb-81ce-018d53818022 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:02.382491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:02.401557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876158a6-e0c8-48f2-976d-2bdfb0a809d0 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:02.438350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7e14-5691-4075-b6f4-c23d36003466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:02.439605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7e14-5691-4075-b6f4-c23d36003466 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.440178 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7e14-5691-4075-b6f4-c23d36003466 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.441021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7e14-5691-4075-b6f4-c23d36003466 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.453423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea7e14-5691-4075-b6f4-c23d36003466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-10ea7e14-5691-4075-b6f4-c23d36003466 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:02.459867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:02.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:14:02.461883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:02.468579 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 205/408] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:02.469791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7e14-5691-4075-b6f4-c23d36003466 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:02.492066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae973ee6-299a-4186-a9a1-5edc8965397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:02.492744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae973ee6-299a-4186-a9a1-5edc8965397f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.493031 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae973ee6-299a-4186-a9a1-5edc8965397f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.493309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae973ee6-299a-4186-a9a1-5edc8965397f None admin] PATH_INFO: `/v3/groups/2ff52158b39d4790ac5f3cb49d9b37cd` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.494571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae973ee6-299a-4186-a9a1-5edc8965397f None admin] RBAC: Authorizing `identity:get_group(group_id=2ff52158b39d4790ac5f3cb49d9b37cd)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:02.496564 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae973ee6-299a-4186-a9a1-5edc8965397f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:02.498336 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 204/409] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:14:02 2022] GET /identity/v3/groups/2ff52158b39d4790ac5f3cb49d9b37cd => generated 229 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:02.507144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae973ee6-299a-4186-a9a1-5edc8965397f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:02.610967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7e14-5691-4075-b6f4-c23d36003466 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:02.661751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:02.662694 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.663122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.663571 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.664498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:02.664979 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:02.670805 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:02.672482 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae973ee6-299a-4186-a9a1-5edc8965397f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:02.684954 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 206/410] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:02.691099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:02.695224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:02.705101 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:02.722341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a3bda-43b2-4c68-8b6f-ba90e414d223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:02.722341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3bda-43b2-4c68-8b6f-ba90e414d223 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.722341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3bda-43b2-4c68-8b6f-ba90e414d223 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.722341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3bda-43b2-4c68-8b6f-ba90e414d223 None admin] PATH_INFO: `/v3/projects/c6d6cb8b74c64466888b7108b567aa82` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.724092 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a3bda-43b2-4c68-8b6f-ba90e414d223 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d6cb8b74c64466888b7108b567aa82)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:02.726609 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a3bda-43b2-4c68-8b6f-ba90e414d223 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:02.728415 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 205/411] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:02 2022] GET /identity/v3/projects/c6d6cb8b74c64466888b7108b567aa82 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:02.736213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a3bda-43b2-4c68-8b6f-ba90e414d223 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:02.877346 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8cae9-c506-43e3-b743-66afe5287ea6 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:02.915324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8549a5e-85f7-4d8d-b074-75624e68fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:02.916170 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8549a5e-85f7-4d8d-b074-75624e68fdfc None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.916454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8549a5e-85f7-4d8d-b074-75624e68fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.916730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8549a5e-85f7-4d8d-b074-75624e68fdfc None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.917923 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8549a5e-85f7-4d8d-b074-75624e68fdfc None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:02.920144 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8549a5e-85f7-4d8d-b074-75624e68fdfc None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:02.923667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 207/412] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:14:02 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:02.929253 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8549a5e-85f7-4d8d-b074-75624e68fdfc None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:02.937508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a3bda-43b2-4c68-8b6f-ba90e414d223 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:02.989966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fd401-a967-4291-a954-7392e68f343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:02.990705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd401-a967-4291-a954-7392e68f343a None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:02.990990 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd401-a967-4291-a954-7392e68f343a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:02.991640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd401-a967-4291-a954-7392e68f343a None admin] PATH_INFO: `/v3/projects/c6d6cb8b74c64466888b7108b567aa82/groups/2ff52158b39d4790ac5f3cb49d9b37cd/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:02.994461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fd401-a967-4291-a954-7392e68f343a None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d6cb8b74c64466888b7108b567aa82, group_id=2ff52158b39d4790ac5f3cb49d9b37cd, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:02.996131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fd401-a967-4291-a954-7392e68f343a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:03.015129 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 206/413] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:14:02 2022] PUT /identity/v3/projects/c6d6cb8b74c64466888b7108b567aa82/groups/2ff52158b39d4790ac5f3cb49d9b37cd/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:03.016488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7285d4-bcd9-486a-acdf-422a9238b3c0 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.016780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7285d4-bcd9-486a-acdf-422a9238b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.017333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7285d4-bcd9-486a-acdf-422a9238b3c0 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.019952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 207/414] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:03.025895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:03.030154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec2e4-b6e9-4b2c-955b-ee2e9032156a None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.030852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec2e4-b6e9-4b2c-955b-ee2e9032156a None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.031436 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec2e4-b6e9-4b2c-955b-ee2e9032156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.054455 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-376ec2e4-b6e9-4b2c-955b-ee2e9032156a None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:03.056224 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-376ec2e4-b6e9-4b2c-955b-ee2e9032156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:03.067594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8549a5e-85f7-4d8d-b074-75624e68fdfc None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:03.113672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:03.118199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf885fe-5898-47e9-be50-63c74ea564c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:03.118199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf885fe-5898-47e9-be50-63c74ea564c3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.118199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf885fe-5898-47e9-be50-63c74ea564c3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.118199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf885fe-5898-47e9-be50-63c74ea564c3 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.118686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf885fe-5898-47e9-be50-63c74ea564c3 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:03.119467 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf885fe-5898-47e9-be50-63c74ea564c3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:03.122324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 208/415] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:02 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:03.128064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf885fe-5898-47e9-be50-63c74ea564c3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:03.156703 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 208/416] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:03.181173 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ec2e4-b6e9-4b2c-955b-ee2e9032156a None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:03.307078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf885fe-5898-47e9-be50-63c74ea564c3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:03.351715 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ec2e4-b6e9-4b2c-955b-ee2e9032156a None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:03.379795 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f65de6-7872-455c-8842-7f00394af067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:03.381802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f65de6-7872-455c-8842-7f00394af067 None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.382085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f65de6-7872-455c-8842-7f00394af067 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.382383 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f65de6-7872-455c-8842-7f00394af067 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/groups/84b7d8ea31a9475eb4e398e697cb3cae/roles/0f002225dde34ef1a2adda2f583b8520` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.386387 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f65de6-7872-455c-8842-7f00394af067 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8083f126cd24c4b93c7be83ca71f0ee, group_id=84b7d8ea31a9475eb4e398e697cb3cae, role_id=0f002225dde34ef1a2adda2f583b8520)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:03.387882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f65de6-7872-455c-8842-7f00394af067 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:03.399981 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:03.407278 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 209/417] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:14:03 2022] PUT /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee/groups/84b7d8ea31a9475eb4e398e697cb3cae/roles/0f002225dde34ef1a2adda2f583b8520 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:03.409129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9aea3-d191-4d76-a4a2-218e23c54293 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.409815 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9aea3-d191-4d76-a4a2-218e23c54293 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.410211 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9aea3-d191-4d76-a4a2-218e23c54293 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.411803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 210/418] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:03.414979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6848770-5bbf-4e77-af1d-827c40436807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:03.415676 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6848770-5bbf-4e77-af1d-827c40436807 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.415955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6848770-5bbf-4e77-af1d-827c40436807 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.416303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6848770-5bbf-4e77-af1d-827c40436807 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.418172 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbaa0-6b1e-40b7-b04d-0bc3d9ffeb45 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.418883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbaa0-6b1e-40b7-b04d-0bc3d9ffeb45 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.419172 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbaa0-6b1e-40b7-b04d-0bc3d9ffeb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.430655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6848770-5bbf-4e77-af1d-827c40436807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-c6848770-5bbf-4e77-af1d-827c40436807 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:03.436418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:03.437444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:03.438419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 209/419] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:03.441351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-b21cbaa0-6b1e-40b7-b04d-0bc3d9ffeb45 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:03.443069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-b21cbaa0-6b1e-40b7-b04d-0bc3d9ffeb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:03.444475 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6848770-5bbf-4e77-af1d-827c40436807 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:03.510649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:03.569528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 211/420] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:03.610599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6848770-5bbf-4e77-af1d-827c40436807 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:03.620969 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cbaa0-6b1e-40b7-b04d-0bc3d9ffeb45 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:03.657545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:03.658449 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.658833 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.659204 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.660921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] RBAC: Adding query filter params (name=member) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:03.664040 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:03.664040 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:03.674745 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 210/421] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:14:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:03.684837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:03.779183 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cbaa0-6b1e-40b7-b04d-0bc3d9ffeb45 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:03.815756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef89d1-13a3-4999-ace1-7b2e37453580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:03.816950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef89d1-13a3-4999-ace1-7b2e37453580 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:03.817490 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef89d1-13a3-4999-ace1-7b2e37453580 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:03.818013 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef89d1-13a3-4999-ace1-7b2e37453580 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:03.822301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:03.846707 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef89d1-13a3-4999-ace1-7b2e37453580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-31ef89d1-13a3-4999-ace1-7b2e37453580 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:03.852075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:03.853101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:03.854680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 212/422] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:03.859492 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef89d1-13a3-4999-ace1-7b2e37453580 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:03.914811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ae4d2-bbe9-4033-8b63-81d24c3b8c33 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:03.916643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:03.989044 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef89d1-13a3-4999-ace1-7b2e37453580 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:04.030976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:04.031324 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3409e20-3ff9-420b-8ce4-3832500a5b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:04.031620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.031792 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409e20-3ff9-420b-8ce4-3832500a5b10 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.031949 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.032100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409e20-3ff9-420b-8ce4-3832500a5b10 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.032257 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.032422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409e20-3ff9-420b-8ce4-3832500a5b10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.032768 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:04.033076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:04.034622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:04.044393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 213/423] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:04.055202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:04.114812 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:04.182489 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8a48d-61ea-47b6-b87f-359f9e0592bc None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:04.195063 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3409e20-3ff9-420b-8ce4-3832500a5b10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:04.196945 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3409e20-3ff9-420b-8ce4-3832500a5b10 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:04.223783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080e812-16ce-4bf2-aeee-3a5d77252348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:04.224423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080e812-16ce-4bf2-aeee-3a5d77252348 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.224710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080e812-16ce-4bf2-aeee-3a5d77252348 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.224981 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080e812-16ce-4bf2-aeee-3a5d77252348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.337630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 211/424] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:04.343897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3409e20-3ff9-420b-8ce4-3832500a5b10 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:04.346013 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:04.380390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080e812-16ce-4bf2-aeee-3a5d77252348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:04.382701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080e812-16ce-4bf2-aeee-3a5d77252348 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:04.479811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3409e20-3ff9-420b-8ce4-3832500a5b10 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:04.480433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:04.501282 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 214/425] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:04.502920 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8df3c-8d18-4c19-9afb-d3818267684b None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.503309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8df3c-8d18-4c19-9afb-d3818267684b None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.503705 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8df3c-8d18-4c19-9afb-d3818267684b None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.505283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 215/426] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:04.510472 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8df3c-8d18-4c19-9afb-d3818267684b None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:04.519277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf893737-fd7a-4045-a2fa-34fcfc3c9077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:04.519935 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf893737-fd7a-4045-a2fa-34fcfc3c9077 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.520216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf893737-fd7a-4045-a2fa-34fcfc3c9077 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.520487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf893737-fd7a-4045-a2fa-34fcfc3c9077 None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.522091 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf893737-fd7a-4045-a2fa-34fcfc3c9077 None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:04.523974 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf893737-fd7a-4045-a2fa-34fcfc3c9077 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:04.526593 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 212/427] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:04 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:04.527912 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c17dd-086c-4c34-b5b5-0147a89bde75 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.528567 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c17dd-086c-4c34-b5b5-0147a89bde75 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.528851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c17dd-086c-4c34-b5b5-0147a89bde75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.553406 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-c15c17dd-086c-4c34-b5b5-0147a89bde75 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:04.554649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-c15c17dd-086c-4c34-b5b5-0147a89bde75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:04.622575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8df3c-8d18-4c19-9afb-d3818267684b None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:04.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:04.638550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 213/428] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:04.640509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c17dd-086c-4c34-b5b5-0147a89bde75 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:04.672990 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416902a-f8c2-4a12-874f-18d98fd3a28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:04.674054 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416902a-f8c2-4a12-874f-18d98fd3a28f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.674737 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416902a-f8c2-4a12-874f-18d98fd3a28f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.675248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416902a-f8c2-4a12-874f-18d98fd3a28f None admin] PATH_INFO: `/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.676966 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416902a-f8c2-4a12-874f-18d98fd3a28f None admin] RBAC: Authorizing `identity:get_project(project_id=8ac986a4d83f4f07af22c95818f1b5bb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:04.679041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416902a-f8c2-4a12-874f-18d98fd3a28f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:04.682991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 216/429] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:04 2022] GET /identity/v3/projects/8ac986a4d83f4f07af22c95818f1b5bb => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:04.690704 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416902a-f8c2-4a12-874f-18d98fd3a28f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:04.721183 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:04.760728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c17dd-086c-4c34-b5b5-0147a89bde75 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:04.798481 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cfad0-166e-476a-b883-1ed145adee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:04.799213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cfad0-166e-476a-b883-1ed145adee25 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.799522 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cfad0-166e-476a-b883-1ed145adee25 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.800187 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cfad0-166e-476a-b883-1ed145adee25 None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.801053 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cfad0-166e-476a-b883-1ed145adee25 None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:04.802625 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cfad0-166e-476a-b883-1ed145adee25 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:04.809314 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 214/430] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:14:04 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:04.809314 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cfad0-166e-476a-b883-1ed145adee25 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:04.834961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416902a-f8c2-4a12-874f-18d98fd3a28f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:04.875397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1071d-458c-42bb-96ff-0539bcf29a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:04.876466 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1071d-458c-42bb-96ff-0539bcf29a0d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.877027 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1071d-458c-42bb-96ff-0539bcf29a0d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.877634 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1071d-458c-42bb-96ff-0539bcf29a0d None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.879270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1071d-458c-42bb-96ff-0539bcf29a0d None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:04.880019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:04.882047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1071d-458c-42bb-96ff-0539bcf29a0d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:04.884760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 217/431] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:14:04 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:04.886264 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1071d-458c-42bb-96ff-0539bcf29a0d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:04.922948 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cfad0-166e-476a-b883-1ed145adee25 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:04.960482 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacfa36-46c3-4e9c-b873-c30197792f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:04.961143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacfa36-46c3-4e9c-b873-c30197792f66 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:04.961423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacfa36-46c3-4e9c-b873-c30197792f66 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:04.961702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacfa36-46c3-4e9c-b873-c30197792f66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:04.974447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacfa36-46c3-4e9c-b873-c30197792f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-5dacfa36-46c3-4e9c-b873-c30197792f66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:04.979654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:04.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 215/432] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:04.982278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacfa36-46c3-4e9c-b873-c30197792f66 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:04.997306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:05.013526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1071d-458c-42bb-96ff-0539bcf29a0d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:05.052584 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:05.053650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.054186 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.054665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.055797 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] RBAC: Adding query filter params (group.id=84b7d8ea31a9475eb4e398e697cb3cae, role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:05.056307 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:05.058020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:05.064797 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 218/433] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:14:04 2022] GET /identity/v3/role_assignments?group.id=84b7d8ea31a9475eb4e398e697cb3cae&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:05.066077 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:05.121693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacfa36-46c3-4e9c-b873-c30197792f66 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:05.166712 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:05.183370 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3879aa-e6ce-4a69-bccf-4bd0be7ffc43 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:05.191020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:05.191687 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.191972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.192245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.193071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] RBAC: Adding query filter params (group.id=84b7d8ea31a9475eb4e398e697cb3cae, role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:05.193375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:05.194859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:05.202014 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 216/434] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:14:04 2022] GET /identity/v3/role_assignments?group.id=84b7d8ea31a9475eb4e398e697cb3cae&scope.project.id=8ac986a4d83f4f07af22c95818f1b5bb&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:05.227930 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f1291-054f-4436-8c02-2f417d2c7243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:05.229069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f1291-054f-4436-8c02-2f417d2c7243 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.229574 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f1291-054f-4436-8c02-2f417d2c7243 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.230067 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f1291-054f-4436-8c02-2f417d2c7243 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.231097 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f1291-054f-4436-8c02-2f417d2c7243 None admin] RBAC: Adding query filter params (name=member) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:05.231610 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f1291-054f-4436-8c02-2f417d2c7243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:05.233334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f1291-054f-4436-8c02-2f417d2c7243 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:05.244323 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 219/435] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:05.250756 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:05.376964 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa840-6574-452c-aa4a-8403efe9c2ac None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.377733 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa840-6574-452c-aa4a-8403efe9c2ac None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.378369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa840-6574-452c-aa4a-8403efe9c2ac None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.380066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 220/436] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:14:05.386081 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4c18-9a0d-45cc-81d9-b9a9fc4969bc None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.386784 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4c18-9a0d-45cc-81d9-b9a9fc4969bc None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.387267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4c18-9a0d-45cc-81d9-b9a9fc4969bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.393748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc257c19-c48a-4bfa-b22c-0576ac03f7a1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:05.420110 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-0d7f4c18-9a0d-45cc-81d9-b9a9fc4969bc None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:05.421270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-0d7f4c18-9a0d-45cc-81d9-b9a9fc4969bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:05.432006 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43f46-b342-42b3-9a4f-8bfcdd4a4094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:05.432759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43f46-b342-42b3-9a4f-8bfcdd4a4094 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.433110 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43f46-b342-42b3-9a4f-8bfcdd4a4094 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.433445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43f46-b342-42b3-9a4f-8bfcdd4a4094 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.442528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:05.509316 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 221/437] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:05.529175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:05.535282 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f4c18-9a0d-45cc-81d9-b9a9fc4969bc None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:05.537134 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b43f46-b342-42b3-9a4f-8bfcdd4a4094 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:05.539830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b43f46-b342-42b3-9a4f-8bfcdd4a4094 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:05.676184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f4c18-9a0d-45cc-81d9-b9a9fc4969bc None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:05.681131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 217/438] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:05.687678 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43f46-b342-42b3-9a4f-8bfcdd4a4094 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:05.729882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e53193-4bb3-4c25-8862-80a7d54dfb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:05.730704 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e53193-4bb3-4c25-8862-80a7d54dfb7f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.730983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e53193-4bb3-4c25-8862-80a7d54dfb7f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.731259 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e53193-4bb3-4c25-8862-80a7d54dfb7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.744791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e53193-4bb3-4c25-8862-80a7d54dfb7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-b5e53193-4bb3-4c25-8862-80a7d54dfb7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:05.750266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:05.751325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:05.752368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 222/439] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:05.753166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f801b6-05ed-4ca6-96c9-622ed10fc358 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.753248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f801b6-05ed-4ca6-96c9-622ed10fc358 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.753520 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f801b6-05ed-4ca6-96c9-622ed10fc358 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.759778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 223/440] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:05.760087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f801b6-05ed-4ca6-96c9-622ed10fc358 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:05.766199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:05.800621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:05.810996 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43f46-b342-42b3-9a4f-8bfcdd4a4094 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:05.854731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23a631-4491-4e12-9dd8-151ba5f2e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:05.855436 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23a631-4491-4e12-9dd8-151ba5f2e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.855719 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23a631-4491-4e12-9dd8-151ba5f2e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.855998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23a631-4491-4e12-9dd8-151ba5f2e8c2 None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.857205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23a631-4491-4e12-9dd8-151ba5f2e8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:05.858716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23a631-4491-4e12-9dd8-151ba5f2e8c2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:05.860434 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 218/441] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:05 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:05.862214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58081390-5d15-4f6d-80ee-51d427329d04 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.862214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58081390-5d15-4f6d-80ee-51d427329d04 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.862429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58081390-5d15-4f6d-80ee-51d427329d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.883795 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-58081390-5d15-4f6d-80ee-51d427329d04 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:05.885123 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-58081390-5d15-4f6d-80ee-51d427329d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:05.891188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f801b6-05ed-4ca6-96c9-622ed10fc358 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:05.928336 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:05.928889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:05.929173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:05.929448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:05.930214 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:05.930522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:05.931923 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:05.940643 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 224/442] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:05.941550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:05.966854 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 219/443] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:05.967581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58081390-5d15-4f6d-80ee-51d427329d04 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.020192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:06.056308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d706-5968-40a5-85d3-e35927348ac7 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.064349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:06.074876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58081390-5d15-4f6d-80ee-51d427329d04 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.090397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c2738-3b83-46f4-89dc-a881836db3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.091055 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c2738-3b83-46f4-89dc-a881836db3f7 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.091334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c2738-3b83-46f4-89dc-a881836db3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.091607 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c2738-3b83-46f4-89dc-a881836db3f7 None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.092827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c2738-3b83-46f4-89dc-a881836db3f7 None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.094279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c2738-3b83-46f4-89dc-a881836db3f7 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.095886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 225/444] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:14:05 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:06.096644 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c2738-3b83-46f4-89dc-a881836db3f7 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.115381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc76ba-d965-4363-be92-6bdb36e6926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.116021 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc76ba-d965-4363-be92-6bdb36e6926c None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.116304 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc76ba-d965-4363-be92-6bdb36e6926c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.116581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc76ba-d965-4363-be92-6bdb36e6926c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.182475 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc76ba-d965-4363-be92-6bdb36e6926c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.183855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc76ba-d965-4363-be92-6bdb36e6926c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.198515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c2738-3b83-46f4-89dc-a881836db3f7 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.233044 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b66fae-8276-4b5c-8181-f92ccd83e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.233701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66fae-8276-4b5c-8181-f92ccd83e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.233999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66fae-8276-4b5c-8181-f92ccd83e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.234318 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66fae-8276-4b5c-8181-f92ccd83e4a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.236232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:06.245957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b66fae-8276-4b5c-8181-f92ccd83e4a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-50b66fae-8276-4b5c-8181-f92ccd83e4a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:06.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:06.252118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:06.253098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 226/445] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:06.253857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b66fae-8276-4b5c-8181-f92ccd83e4a4 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:06.284012 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 220/446] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:06.285007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc76ba-d965-4363-be92-6bdb36e6926c None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.365611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b66fae-8276-4b5c-8181-f92ccd83e4a4 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.387483 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc76ba-d965-4363-be92-6bdb36e6926c None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.400956 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.401591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.401867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.402166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.402957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] RBAC: Adding query filter params (group.id=84b7d8ea31a9475eb4e398e697cb3cae, role.id=0aacd40c75f64f668308391b44dc4e85, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:06.403260 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.404706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.410865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 227/447] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/role_assignments?group.id=84b7d8ea31a9475eb4e398e697cb3cae&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=0aacd40c75f64f668308391b44dc4e85 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:06.411910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.422505 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.423163 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.423440 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.423744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.424580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:06.424887 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.426365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.436433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 221/448] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:06.442343 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.491224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:06.507853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:06.540584 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a0e7a-d3e9-40e1-87df-a349b109ce29 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.564672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79996f59-d8e7-45f7-a4f9-6d4ebc75fe93 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.580532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bb3b3-1661-473c-8b06-bee6825a01a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.581206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb3b3-1661-473c-8b06-bee6825a01a6 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.581492 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb3b3-1661-473c-8b06-bee6825a01a6 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.581771 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb3b3-1661-473c-8b06-bee6825a01a6 None admin] PATH_INFO: `/v3/projects/c6d6cb8b74c64466888b7108b567aa82` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.583865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bb3b3-1661-473c-8b06-bee6825a01a6 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d6cb8b74c64466888b7108b567aa82)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.585592 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bb3b3-1661-473c-8b06-bee6825a01a6 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.587315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 228/449] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/projects/c6d6cb8b74c64466888b7108b567aa82 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:06.593016 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bb3b3-1661-473c-8b06-bee6825a01a6 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.608124 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3d345-1691-44bb-bb49-d0dd8781afe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.608775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d345-1691-44bb-bb49-d0dd8781afe0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.609062 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d345-1691-44bb-bb49-d0dd8781afe0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.609339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d345-1691-44bb-bb49-d0dd8781afe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.698664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3d345-1691-44bb-bb49-d0dd8781afe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.700065 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3d345-1691-44bb-bb49-d0dd8781afe0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.719618 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bb3b3-1661-473c-8b06-bee6825a01a6 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.749194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:06.760500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c720951-d40e-44ac-b39d-6fbd19e7d152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.761169 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c720951-d40e-44ac-b39d-6fbd19e7d152 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.761456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c720951-d40e-44ac-b39d-6fbd19e7d152 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.761737 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c720951-d40e-44ac-b39d-6fbd19e7d152 None admin] PATH_INFO: `/v3/groups/2ff52158b39d4790ac5f3cb49d9b37cd` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.763131 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c720951-d40e-44ac-b39d-6fbd19e7d152 None admin] RBAC: Authorizing `identity:get_group(group_id=2ff52158b39d4790ac5f3cb49d9b37cd)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.764695 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c720951-d40e-44ac-b39d-6fbd19e7d152 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.771499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 229/450] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/groups/2ff52158b39d4790ac5f3cb49d9b37cd => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:06.772621 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c720951-d40e-44ac-b39d-6fbd19e7d152 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.776772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:06.828378 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 222/451] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:14:06.834141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3d345-1691-44bb-bb49-d0dd8781afe0 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:06.874126 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c720951-d40e-44ac-b39d-6fbd19e7d152 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.908295 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.908967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.909307 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.909617 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.910451 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] RBAC: Adding query filter params (group.id=2ff52158b39d4790ac5f3cb49d9b37cd, role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=c6d6cb8b74c64466888b7108b567aa82) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:06.910744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.912200 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.917998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 230/452] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/role_assignments?group.id=2ff52158b39d4790ac5f3cb49d9b37cd&scope.project.id=c6d6cb8b74c64466888b7108b567aa82&role.id=40504e80dbb845aeb07ca258977ecacb => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:06.933207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3d345-1691-44bb-bb49-d0dd8781afe0 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:06.974059 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a06098-1e2b-4508-ba2e-3a0735e4835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:06.974822 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a06098-1e2b-4508-ba2e-3a0735e4835a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:06.974962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a06098-1e2b-4508-ba2e-3a0735e4835a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:06.975310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a06098-1e2b-4508-ba2e-3a0735e4835a None admin] PATH_INFO: `/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:06.976656 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a06098-1e2b-4508-ba2e-3a0735e4835a None admin] RBAC: Authorizing `identity:get_project(project_id=e8083f126cd24c4b93c7be83ca71f0ee)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:06.978204 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a06098-1e2b-4508-ba2e-3a0735e4835a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:06.980019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 223/453] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/projects/e8083f126cd24c4b93c7be83ca71f0ee => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:06.986067 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:07.054097 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:07.102999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f002980-c0d9-47b5-9122-fce67b36698f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:07.137960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849aa3ff-047d-4e43-9452-60d47b49066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:07.138650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aa3ff-047d-4e43-9452-60d47b49066e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:07.138914 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aa3ff-047d-4e43-9452-60d47b49066e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:07.139206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aa3ff-047d-4e43-9452-60d47b49066e None admin] PATH_INFO: `/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:07.140409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aa3ff-047d-4e43-9452-60d47b49066e None admin] RBAC: Authorizing `identity:get_group(group_id=84b7d8ea31a9475eb4e398e697cb3cae)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:07.141864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aa3ff-047d-4e43-9452-60d47b49066e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:07.143614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 231/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:14:06 2022] GET /identity/v3/groups/84b7d8ea31a9475eb4e398e697cb3cae => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:07.149928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a06098-1e2b-4508-ba2e-3a0735e4835a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:07.172674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:07.259377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a06098-1e2b-4508-ba2e-3a0735e4835a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:07.296916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e034e-80a1-4992-8650-7b0ea30d611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:07.297539 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e034e-80a1-4992-8650-7b0ea30d611b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:07.297819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e034e-80a1-4992-8650-7b0ea30d611b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:07.298095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e034e-80a1-4992-8650-7b0ea30d611b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:07.298924 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e034e-80a1-4992-8650-7b0ea30d611b None admin] RBAC: Adding query filter params (group.id=84b7d8ea31a9475eb4e398e697cb3cae, role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:07.299322 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e034e-80a1-4992-8650-7b0ea30d611b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:07.300761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e034e-80a1-4992-8650-7b0ea30d611b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:07.306821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 224/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:14:07 2022] GET /identity/v3/role_assignments?group.id=84b7d8ea31a9475eb4e398e697cb3cae&scope.project.id=e8083f126cd24c4b93c7be83ca71f0ee&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:10.179037 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94618e7-a038-4099-97b2-a80a2472b38d None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.180092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94618e7-a038-4099-97b2-a80a2472b38d None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.180598 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94618e7-a038-4099-97b2-a80a2472b38d None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.182521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 232/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:10.188327 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ebfe9-6e9e-4a1d-a27a-62392ae8d07e None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.189204 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ebfe9-6e9e-4a1d-a27a-62392ae8d07e None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.189813 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ebfe9-6e9e-4a1d-a27a-62392ae8d07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.209060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237f6f-37ab-4af7-b4c5-a1f0dd4450dc None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.209735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237f6f-37ab-4af7-b4c5-a1f0dd4450dc None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.210266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237f6f-37ab-4af7-b4c5-a1f0dd4450dc None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.212219 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 233/457] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:10.214494 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-5d4ebfe9-6e9e-4a1d-a27a-62392ae8d07e None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:10.215707 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-5d4ebfe9-6e9e-4a1d-a27a-62392ae8d07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:10.218071 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbe8fb-d9e6-47c3-993e-78a35529cb11 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.218628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbe8fb-d9e6-47c3-993e-78a35529cb11 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.219122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbe8fb-d9e6-47c3-993e-78a35529cb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.246418 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-23bbe8fb-d9e6-47c3-993e-78a35529cb11 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:10.247637 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-23bbe8fb-d9e6-47c3-993e-78a35529cb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:10.334148 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 225/458] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:10.341100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40aa6d-942a-415e-a783-ff9208e379a7 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.341390 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40aa6d-942a-415e-a783-ff9208e379a7 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.341665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40aa6d-942a-415e-a783-ff9208e379a7 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.343251 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 226/459] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:10.349621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf6364-6bac-4832-a720-37836dd5c2c3 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.349958 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf6364-6bac-4832-a720-37836dd5c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.350260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf6364-6bac-4832-a720-37836dd5c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.369620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 234/460] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:10.379335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbe8fb-d9e6-47c3-993e-78a35529cb11 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:10.382454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-4adf6364-6bac-4832-a720-37836dd5c2c3 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:10.383862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-4adf6364-6bac-4832-a720-37836dd5c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:10.394552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:10.464542 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 227/461] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:10.465771 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf6364-6bac-4832-a720-37836dd5c2c3 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:10.467333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:10.527523 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbe8fb-d9e6-47c3-993e-78a35529cb11 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:10.565358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:10.565974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.566294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.566549 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.573266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf6364-6bac-4832-a720-37836dd5c2c3 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:10.578361 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:10.579847 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:10.596534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:10.597597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:10.598623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:10.598623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:10.598623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:10.598623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:10.598623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 235/462] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:10.598623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:10.619522 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe284d-a3a1-4b53-a12e-8bf421f3cf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:10.620562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe284d-a3a1-4b53-a12e-8bf421f3cf52 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.621069 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe284d-a3a1-4b53-a12e-8bf421f3cf52 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.621576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe284d-a3a1-4b53-a12e-8bf421f3cf52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.633467 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe284d-a3a1-4b53-a12e-8bf421f3cf52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:10.634977 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe284d-a3a1-4b53-a12e-8bf421f3cf52 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:10.637972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-90fe284d-a3a1-4b53-a12e-8bf421f3cf52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:10.655892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:10.657057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:10.658082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:10.658082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:10.658082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:10.658082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:10.662422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 228/463] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:10.663672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d9ccd-1767-4eb8-9416-b57c2a6d1849 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.663967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d9ccd-1767-4eb8-9416-b57c2a6d1849 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.664232 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d9ccd-1767-4eb8-9416-b57c2a6d1849 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.668131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 229/464] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:10.668131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d9ccd-1767-4eb8-9416-b57c2a6d1849 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:10.716825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10299db-1879-4fe0-9adb-927cdde49fb9 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:10.727494 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:10.754904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:10.755840 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.756123 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.756402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.768578 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:10.770403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:10.786176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:10.787251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:10.788288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:10.788288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:10.788288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:10.788288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:10.790574 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 236/465] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:10.790805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d9ccd-1767-4eb8-9416-b57c2a6d1849 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:10.791691 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:10.833726 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99751234-914f-4129-9fd7-d3d7534122d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:10.835112 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751234-914f-4129-9fd7-d3d7534122d0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.835112 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751234-914f-4129-9fd7-d3d7534122d0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.835112 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751234-914f-4129-9fd7-d3d7534122d0 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.835620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99751234-914f-4129-9fd7-d3d7534122d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:10.835927 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99751234-914f-4129-9fd7-d3d7534122d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:10.837316 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99751234-914f-4129-9fd7-d3d7534122d0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:10.851752 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 230/466] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:10.857426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05975082-c99b-4602-ac03-532f1e755069 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.857709 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05975082-c99b-4602-ac03-532f1e755069 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.857981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05975082-c99b-4602-ac03-532f1e755069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.876258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-05975082-c99b-4602-ac03-532f1e755069 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:10.877547 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-05975082-c99b-4602-ac03-532f1e755069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:10.903619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d95aa-ea8c-4064-931d-f1915d05e9f1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:10.930649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:10.942913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:10.943560 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:10.943858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:10.944103 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:10.944814 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:10.945121 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:10.946946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:10.951229 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 231/467] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:10.951921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05975082-c99b-4602-ac03-532f1e755069 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:10.982146 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 237/468] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:10.983444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:11.015181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:11.078670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05975082-c99b-4602-ac03-532f1e755069 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:11.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdd64f-742e-4768-8ecb-537c5d278536 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:11.118437 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:11.119090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.119373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.119650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.120389 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:11.120699 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:11.122173 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:11.135001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 232/469] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:11.135900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:11.149067 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d9132-7322-46ff-b1df-46bc7185b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:11.149710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9132-7322-46ff-b1df-46bc7185b39a None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.149999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9132-7322-46ff-b1df-46bc7185b39a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.150319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9132-7322-46ff-b1df-46bc7185b39a None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.151023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d9132-7322-46ff-b1df-46bc7185b39a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:11.152525 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d9132-7322-46ff-b1df-46bc7185b39a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:11.175123 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 238/470] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:14:10 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:11.176007 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d9132-7322-46ff-b1df-46bc7185b39a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:11.199319 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:11.269746 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037accaa-1e01-4177-8eff-8154251b4f08 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:11.273655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:11.305447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d9132-7322-46ff-b1df-46bc7185b39a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:11.311266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f89b5-ce7c-4ec1-990d-423f65c4536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:11.311954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f89b5-ce7c-4ec1-990d-423f65c4536d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.312238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f89b5-ce7c-4ec1-990d-423f65c4536d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.312510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f89b5-ce7c-4ec1-990d-423f65c4536d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.330185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f89b5-ce7c-4ec1-990d-423f65c4536d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:11.331743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f89b5-ce7c-4ec1-990d-423f65c4536d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-374f89b5-ce7c-4ec1-990d-423f65c4536d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:11.347908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:11.349001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:11.349982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:11.349982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:11.349982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:11.349982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:11.349982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 233/471] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:11.350668 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b641361-857a-4297-bf10-fd9e50b4c54d None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.350967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b641361-857a-4297-bf10-fd9e50b4c54d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.351254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b641361-857a-4297-bf10-fd9e50b4c54d None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.352101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e966d-bd87-4232-acf2-6f4079afa4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:11.352778 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 234/472] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:11.353453 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b641361-857a-4297-bf10-fd9e50b4c54d None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:11.357102 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e966d-bd87-4232-acf2-6f4079afa4f5 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.357585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e966d-bd87-4232-acf2-6f4079afa4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.357864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e966d-bd87-4232-acf2-6f4079afa4f5 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.358644 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e966d-bd87-4232-acf2-6f4079afa4f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:11.360197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e966d-bd87-4232-acf2-6f4079afa4f5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:11.392017 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 239/473] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:14:11 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:11.397653 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e966d-bd87-4232-acf2-6f4079afa4f5 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:11.485858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:11.492187 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b641361-857a-4297-bf10-fd9e50b4c54d None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:11.539508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4063a7-fc34-413f-ab5b-91102fc98a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:11.540189 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4063a7-fc34-413f-ab5b-91102fc98a71 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.540467 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4063a7-fc34-413f-ab5b-91102fc98a71 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.540839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4063a7-fc34-413f-ab5b-91102fc98a71 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.541561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4063a7-fc34-413f-ab5b-91102fc98a71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:11.543224 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4063a7-fc34-413f-ab5b-91102fc98a71 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:11.550481 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e966d-bd87-4232-acf2-6f4079afa4f5 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:11.578383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 235/474] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:14:11 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:11.580158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b725b8d-0c72-47aa-a99b-a6a8ba16acd7 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.580530 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b725b8d-0c72-47aa-a99b-a6a8ba16acd7 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.580808 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b725b8d-0c72-47aa-a99b-a6a8ba16acd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.586264 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:11.601350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:11.602518 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.603045 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.603557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.604525 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:11.605093 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:11.606978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:11.611212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-8b725b8d-0c72-47aa-a99b-a6a8ba16acd7 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:11.612513 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-8b725b8d-0c72-47aa-a99b-a6a8ba16acd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:11.623490 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 240/475] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:11.629328 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:11.716000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 236/476] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:11.751082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b725b8d-0c72-47aa-a99b-a6a8ba16acd7 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:11.778906 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23833538-bbd6-452f-858f-70c83dc9f01a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:11.831997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d38acd-bb65-490a-9e43-f4cea0743484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:11.832556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d38acd-bb65-490a-9e43-f4cea0743484 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.832842 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d38acd-bb65-490a-9e43-f4cea0743484 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.833124 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d38acd-bb65-490a-9e43-f4cea0743484 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.833822 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d38acd-bb65-490a-9e43-f4cea0743484 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:11.837148 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d38acd-bb65-490a-9e43-f4cea0743484 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:11.871950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 241/477] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:14:11 2022] POST /identity/v3/users => generated 249 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:11.883818 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:11.910076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b725b8d-0c72-47aa-a99b-a6a8ba16acd7 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:11.961541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:11.962327 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:11.962611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:11.962884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:11.963636 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:11.965116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:11.982509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.sql.core [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:14:11.982509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:14:11.982509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [parameters: {'id': '88595727b8974f5e8b127eccff3efd6a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:14:11.982509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=144042) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result.read() Jul 28 08:14:12.000820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:14:12.001979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application yield resource Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise self.value Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application session.commit() Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:14:12.003839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:14:12.005848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:14:12.009457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result.read() Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:14:12.011607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [parameters: {'id': '88595727b8974f5e8b127eccff3efd6a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:12.013702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:14:12.015849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:12.018090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 237/478] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:14:11 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:12.018294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d38acd-bb65-490a-9e43-f4cea0743484 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:12.070728 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:12.128474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d38acd-bb65-490a-9e43-f4cea0743484 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:12.164820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e688e4-ec6b-4d83-86ea-b6537f33ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:12.165894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e688e4-ec6b-4d83-86ea-b6537f33ef61 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:12.166409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e688e4-ec6b-4d83-86ea-b6537f33ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:12.166887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e688e4-ec6b-4d83-86ea-b6537f33ef61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:12.179623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e688e4-ec6b-4d83-86ea-b6537f33ef61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-62e688e4-ec6b-4d83-86ea-b6537f33ef61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:12.185463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:12.186551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:14:12.187510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:12.190436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 242/479] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:12.196024 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:12.297794 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fa521-6742-43c5-a370-864b09b3171d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:12.339908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efcf27-2be8-4869-81a7-a231626e56be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:12.340985 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efcf27-2be8-4869-81a7-a231626e56be None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:12.341475 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efcf27-2be8-4869-81a7-a231626e56be None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:12.341952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efcf27-2be8-4869-81a7-a231626e56be None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:12.342971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efcf27-2be8-4869-81a7-a231626e56be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:12.343560 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efcf27-2be8-4869-81a7-a231626e56be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:12.345297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efcf27-2be8-4869-81a7-a231626e56be None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:12.355802 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 238/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:13.521425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0812b0-9875-4b4b-a0f1-2cf6891a913b None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:13.522012 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0812b0-9875-4b4b-a0f1-2cf6891a913b None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:13.522012 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0812b0-9875-4b4b-a0f1-2cf6891a913b None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:13.523628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 243/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:13.538223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c2c20-2ab9-4e18-9574-1cf79baa450b None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:13.538768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c2c20-2ab9-4e18-9574-1cf79baa450b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:13.538768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c2c20-2ab9-4e18-9574-1cf79baa450b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:13.563303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-349c2c20-2ab9-4e18-9574-1cf79baa450b None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:13.564592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-349c2c20-2ab9-4e18-9574-1cf79baa450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:13.567948 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:13.655978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 239/482] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:13.693439 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0812b0-9875-4b4b-a0f1-2cf6891a913b None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:13.820845 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54817dd-c59c-4623-9bb6-8ba123f3458d None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:13.821874 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54817dd-c59c-4623-9bb6-8ba123f3458d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:13.822402 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54817dd-c59c-4623-9bb6-8ba123f3458d None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:13.825394 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 240/483] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:13.830309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d689a-7455-4367-ba86-b071f65b50ff None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:13.830888 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d689a-7455-4367-ba86-b071f65b50ff None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:13.831405 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d689a-7455-4367-ba86-b071f65b50ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:13.835793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0812b0-9875-4b4b-a0f1-2cf6891a913b None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:13.849996 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:13.855538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-150d689a-7455-4367-ba86-b071f65b50ff None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:13.857074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-150d689a-7455-4367-ba86-b071f65b50ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:13.910809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62912c9-cd40-440d-a58b-e5ddde8aeb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:13.911942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62912c9-cd40-440d-a58b-e5ddde8aeb5b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:13.912479 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62912c9-cd40-440d-a58b-e5ddde8aeb5b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:13.912974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62912c9-cd40-440d-a58b-e5ddde8aeb5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:13.935611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62912c9-cd40-440d-a58b-e5ddde8aeb5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-e62912c9-cd40-440d-a58b-e5ddde8aeb5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:13.943722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:13.944785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:13.945726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:13.949997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 244/484] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:13.956816 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62912c9-cd40-440d-a58b-e5ddde8aeb5b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:13.964199 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 241/485] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:14.009091 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d689a-7455-4367-ba86-b071f65b50ff None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:14.068937 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62912c9-cd40-440d-a58b-e5ddde8aeb5b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:14.110820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:14.112238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.113215 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.115659 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.116706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:14.118075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:14.119771 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:14.135912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 245/486] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:14.146165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:14.173403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:14.198978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:14.234493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d689a-7455-4367-ba86-b071f65b50ff None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:14.297793 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940646ff-801a-4033-90cf-6462d6c3402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:14.298501 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940646ff-801a-4033-90cf-6462d6c3402e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.298920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940646ff-801a-4033-90cf-6462d6c3402e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.299252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940646ff-801a-4033-90cf-6462d6c3402e None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.318795 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940646ff-801a-4033-90cf-6462d6c3402e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:14.323577 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136719d7-b249-4093-a1de-a3f49fefb0e8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-940646ff-801a-4033-90cf-6462d6c3402e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:14.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:14.325338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:14.326323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:14.330484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 242/487] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:14.332366 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0fea5-2a9b-400c-b130-8e7087ea8fdb None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.332656 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0fea5-2a9b-400c-b130-8e7087ea8fdb None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.332938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0fea5-2a9b-400c-b130-8e7087ea8fdb None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.338453 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 243/488] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:14.339266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0fea5-2a9b-400c-b130-8e7087ea8fdb None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:14.360417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af1af-bfec-481d-a419-b115ecc993e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:14.361156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af1af-bfec-481d-a419-b115ecc993e4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.361474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af1af-bfec-481d-a419-b115ecc993e4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.361765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af1af-bfec-481d-a419-b115ecc993e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.375225 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713af1af-bfec-481d-a419-b115ecc993e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:14.376755 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713af1af-bfec-481d-a419-b115ecc993e4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-713af1af-bfec-481d-a419-b115ecc993e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:14.393802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:14.396021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:14.398266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:14.398266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:14.398266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:14.398266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:14.398266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 246/489] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:14.401566 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496b6a-5435-48cd-a576-80455a3a4362 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.406139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496b6a-5435-48cd-a576-80455a3a4362 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.406139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496b6a-5435-48cd-a576-80455a3a4362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.422135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-e3496b6a-5435-48cd-a576-80455a3a4362 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:14.423439 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-e3496b6a-5435-48cd-a576-80455a3a4362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:14.454525 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0fea5-2a9b-400c-b130-8e7087ea8fdb None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:14.490209 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:14.490847 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.491130 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.491409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.492147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:14.492454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:14.493872 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:14.495546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:14.499125 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 247/490] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:14.500018 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496b6a-5435-48cd-a576-80455a3a4362 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:14.510814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 244/491] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:14.515588 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:14.516864 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:14.610629 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496b6a-5435-48cd-a576-80455a3a4362 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:14.649696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932410f-6a42-40d6-a8c0-845de46ab3ba None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:14.652179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:14.653057 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.653527 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.653980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.654904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:14.655405 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:14.657022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:14.671158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 248/492] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:14.676661 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:14.695681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:14.696726 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.697571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.697571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.711410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:14.713040 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:14.730045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:14.731162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:14.732172 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:14.732172 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:14.732172 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:14.732172 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:14.736769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 245/493] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:14.741640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:14.769579 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:14.797860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:14.809564 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f38dee-0c98-488a-95f2-9913b8ee59c1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:14.859304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e95ca4-6080-47ce-a12a-16c0695d186c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:14.860107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95ca4-6080-47ce-a12a-16c0695d186c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.860458 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95ca4-6080-47ce-a12a-16c0695d186c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.860803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95ca4-6080-47ce-a12a-16c0695d186c None admin] PATH_INFO: `/v3/roles/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.861871 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939871a6-1d15-4cc5-8a83-ef4c428d8b53 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:14.872855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e95ca4-6080-47ce-a12a-16c0695d186c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-57e95ca4-6080-47ce-a12a-16c0695d186c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:14.878616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:14.879715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 249/494] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:14.880708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e95ca4-6080-47ce-a12a-16c0695d186c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:14.909844 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0112343-6f76-4d84-a7cd-86a31102f68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:14.910583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112343-6f76-4d84-a7cd-86a31102f68a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.911066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112343-6f76-4d84-a7cd-86a31102f68a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.911564 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112343-6f76-4d84-a7cd-86a31102f68a None admin] PATH_INFO: `/v3/users/nova` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.921842 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0112343-6f76-4d84-a7cd-86a31102f68a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:14.923519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0112343-6f76-4d84-a7cd-86a31102f68a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-d0112343-6f76-4d84-a7cd-86a31102f68a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:14.940395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:14.941819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:14.943252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:14.946649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 246/495] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:14.947973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b96ed-fd08-435f-a614-b829cb983839 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:14.948259 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b96ed-fd08-435f-a614-b829cb983839 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:14.948541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b96ed-fd08-435f-a614-b829cb983839 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:14.954123 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 247/496] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:14.954123 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b96ed-fd08-435f-a614-b829cb983839 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.007688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e95ca4-6080-47ce-a12a-16c0695d186c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.041912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.047736 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.048638 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.049357 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.049840 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.051201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:15.051510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.053049 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.061968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b96ed-fd08-435f-a614-b829cb983839 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.065936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.068317 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 250/497] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:15.069350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.104885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73612613-ed22-4871-90f2-53e90183640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.105532 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612613-ed22-4871-90f2-53e90183640b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.105810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612613-ed22-4871-90f2-53e90183640b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.106084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612613-ed22-4871-90f2-53e90183640b None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.106855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73612613-ed22-4871-90f2-53e90183640b None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:15.107160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73612613-ed22-4871-90f2-53e90183640b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.108543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73612613-ed22-4871-90f2-53e90183640b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.117464 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 248/498] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:15.120228 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b6ce-fa62-4cd1-b1a7-6394d3bc7a89 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.120983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b6ce-fa62-4cd1-b1a7-6394d3bc7a89 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.121474 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b6ce-fa62-4cd1-b1a7-6394d3bc7a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.143642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-9982b6ce-fa62-4cd1-b1a7-6394d3bc7a89 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:15.144856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-9982b6ce-fa62-4cd1-b1a7-6394d3bc7a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:15.176282 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a1e58-d30d-4d25-8890-5ef93c5a95f4 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.212239 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.212857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.213139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.213557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.215171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:15.215667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.217290 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.221963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 249/499] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:15.222877 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982b6ce-fa62-4cd1-b1a7-6394d3bc7a89 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.256466 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 251/500] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:15.257341 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.310188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.325966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.332371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982b6ce-fa62-4cd1-b1a7-6394d3bc7a89 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.363088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53844f0-db1f-4b8b-b12c-2cca10668dbc None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.367588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470663fb-1a9d-4394-afed-74f3cef1bd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.368244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470663fb-1a9d-4394-afed-74f3cef1bd8f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.368525 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470663fb-1a9d-4394-afed-74f3cef1bd8f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.368799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470663fb-1a9d-4394-afed-74f3cef1bd8f None admin] PATH_INFO: `/v3/users/glance` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.376016 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470663fb-1a9d-4394-afed-74f3cef1bd8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.377534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470663fb-1a9d-4394-afed-74f3cef1bd8f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-470663fb-1a9d-4394-afed-74f3cef1bd8f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:15.389365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:15.390672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 250/501] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:15.391738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c176-db1d-400c-987e-98060da94b7f None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.392534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c176-db1d-400c-987e-98060da94b7f None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.392534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c176-db1d-400c-987e-98060da94b7f None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.393578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 251/502] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:15.394378 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c176-db1d-400c-987e-98060da94b7f None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.401639 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.402341 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.402630 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.402910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.413437 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.414957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:15.429422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:15.430619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:15.431841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:15.431841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:15.431841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:15.431841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:15.431841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 252/503] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:15.431841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.500336 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c176-db1d-400c-987e-98060da94b7f None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.534480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece687-2bc6-4a03-adbc-cbe7aaa89276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.535144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece687-2bc6-4a03-adbc-cbe7aaa89276 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.535426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece687-2bc6-4a03-adbc-cbe7aaa89276 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.535691 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c532a-a64a-43f8-8617-ff2f4e1bdff1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.535807 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece687-2bc6-4a03-adbc-cbe7aaa89276 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.537275 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.546412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ece687-2bc6-4a03-adbc-cbe7aaa89276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-a9ece687-2bc6-4a03-adbc-cbe7aaa89276 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:15.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:15.553292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 252/504] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:15.554478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece687-2bc6-4a03-adbc-cbe7aaa89276 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.574491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c967-469e-4f81-8dd7-acbb59281912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.575122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c967-469e-4f81-8dd7-acbb59281912 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.575401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c967-469e-4f81-8dd7-acbb59281912 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.575673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c967-469e-4f81-8dd7-acbb59281912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.576158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.645928 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448c967-469e-4f81-8dd7-acbb59281912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.647461 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448c967-469e-4f81-8dd7-acbb59281912 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.664398 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece687-2bc6-4a03-adbc-cbe7aaa89276 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.703080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811eff0-5fd2-4d7d-92b4-64d42f3b5c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.703663 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811eff0-5fd2-4d7d-92b4-64d42f3b5c3d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.703944 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811eff0-5fd2-4d7d-92b4-64d42f3b5c3d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.704238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811eff0-5fd2-4d7d-92b4-64d42f3b5c3d None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.705152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811eff0-5fd2-4d7d-92b4-64d42f3b5c3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:15.705462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811eff0-5fd2-4d7d-92b4-64d42f3b5c3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.706981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811eff0-5fd2-4d7d-92b4-64d42f3b5c3d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.745368 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 253/505] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:15.746762 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69c323-07dd-41aa-9f02-a915ae472e4f None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.747051 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69c323-07dd-41aa-9f02-a915ae472e4f None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.747322 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69c323-07dd-41aa-9f02-a915ae472e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.760302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 253/506] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:15.761168 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c967-469e-4f81-8dd7-acbb59281912 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.763519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.765785 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-1a69c323-07dd-41aa-9f02-a915ae472e4f None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:15.767157 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-1a69c323-07dd-41aa-9f02-a915ae472e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:15.836113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 254/507] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:15.837083 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69c323-07dd-41aa-9f02-a915ae472e4f None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.846599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.865100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c967-469e-4f81-8dd7-acbb59281912 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.904376 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.905038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.905319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.905602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.906341 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:15.906650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.908061 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.919690 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 254/508] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:15.920515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:15.950871 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69c323-07dd-41aa-9f02-a915ae472e4f None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:15.983096 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:15.985534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:15.986217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:15.986501 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:15.986773 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:15.987503 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:15.987805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:15.989193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:15.997486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 255/509] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:15.998347 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.026409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565f9b8-8612-43dd-966d-72df85562027 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.060915 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096d3d-5f9e-4f93-a5dd-9a717b8fcc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.061555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096d3d-5f9e-4f93-a5dd-9a717b8fcc02 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.061839 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096d3d-5f9e-4f93-a5dd-9a717b8fcc02 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.062143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096d3d-5f9e-4f93-a5dd-9a717b8fcc02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.062749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:16.099770 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b8270-8815-4be5-96a7-1dbf4fc7a9ca None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.132596 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b096d3d-5f9e-4f93-a5dd-9a717b8fcc02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.134080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b096d3d-5f9e-4f93-a5dd-9a717b8fcc02 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.134231 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.134774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.135055 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.135322 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.146341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.147829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:16.162313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:16.163438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:16.164487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:16.164487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:16.164487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:16.164487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:16.164487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 256/510] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:16.164487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.201531 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:16.243627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 255/511] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:16.244448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096d3d-5f9e-4f93-a5dd-9a717b8fcc02 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.266889 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14cd5b-a865-4a4c-99d1-0ad9533de17f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.284491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:16.303420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.304079 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.304359 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.304628 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.315532 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.317010 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:16.332263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:16.333306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:16.334352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:16.334352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:16.334352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:16.334352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:16.334352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 257/512] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:16.334352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.349583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096d3d-5f9e-4f93-a5dd-9a717b8fcc02 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.383822 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.384487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.384768 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.385038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.393083 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.394613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:16.407616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:16.408724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 256/513] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:16.409777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.435338 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:16.446099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c539002-9418-47dd-82d6-247fa39a212b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.481301 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.481953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.482275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.482563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.494048 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.495520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:16.513447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:16.514757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:16.516603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:16.516603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:16.516603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:16.516603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:16.516603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 258/514] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:16.516603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.520315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8797c9f-6c55-406a-9aaa-74c00912ea64 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.541421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:16.558624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.559256 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.559535 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.559806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.560504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:16.560808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.562230 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.575154 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 257/515] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:16.576114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.625218 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937a64a-def0-412c-81d6-fffaebfc35dc None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.659761 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:16.661066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.661728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.662009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.662323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.673458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.674999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.680837 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15a81a-3a6f-40be-9f94-13406074a923 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:16.689961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:16.691335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:16.692758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:16.692758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:16.692758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:16.692758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:16.692758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 259/516] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:16.692758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.721544 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.722214 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.722497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.722773 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.723471 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:16.723777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.725192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.737047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 258/517] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:16.737680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.774174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:16.801422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f06b95-9131-4f38-bb4b-6b20c830d5b6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.835630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.836252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.836534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.836807 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.837609 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:16.837913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.839392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.841649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b7291-661c-4daa-97c0-a7faa01df408 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:16.858360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 260/518] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:16.859283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.879230 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:16.879886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:16.880168 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:16.880552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:16.881422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:16.881422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:16.884047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:16.894795 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 259/519] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:16.895711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:16.898422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:16.971827 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fac2d-c81a-4425-8e5c-91643f1458e4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.001665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:17.004059 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0d465-bd6b-4852-86c0-60357f617302 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.007893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.008593 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.008871 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.009143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.017590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.019144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:17.032896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:17.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 261/520] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:17.035570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.047577 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.048219 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.048839 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.048839 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.049543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:17.049842 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.051270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.063105 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 260/521] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:17.063946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.123930 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:17.140210 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bd295-b783-4111-956d-460a6b2f3d78 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.165432 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a503fa-ddc5-4ce4-b0de-beb903fad929 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.175788 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dfb31-c4e8-4c20-9164-e6fd78fe6436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.176421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfb31-c4e8-4c20-9164-e6fd78fe6436 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.176702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfb31-c4e8-4c20-9164-e6fd78fe6436 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.176978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfb31-c4e8-4c20-9164-e6fd78fe6436 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.177664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dfb31-c4e8-4c20-9164-e6fd78fe6436 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.179162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dfb31-c4e8-4c20-9164-e6fd78fe6436 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.200792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074faead-c23e-419d-9a43-322c8a825a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.201394 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 262/522] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:14:17 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:17.201547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074faead-c23e-419d-9a43-322c8a825a79 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.201717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074faead-c23e-419d-9a43-322c8a825a79 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.201975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074faead-c23e-419d-9a43-322c8a825a79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.202302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dfb31-c4e8-4c20-9164-e6fd78fe6436 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.265546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:17.292686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074faead-c23e-419d-9a43-322c8a825a79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.294389 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074faead-c23e-419d-9a43-322c8a825a79 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.329834 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dfb31-c4e8-4c20-9164-e6fd78fe6436 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.369843 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:17.370256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.371602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.371875 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.372147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.384108 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.384572 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:17.396846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:17.399544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 263/523] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:17.400836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.418607 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 261/524] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:17.419821 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074faead-c23e-419d-9a43-322c8a825a79 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.522386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bf439-ccf2-473e-b30b-82ad31559e60 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.529673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074faead-c23e-419d-9a43-322c8a825a79 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.532966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:17.562364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.563271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.563569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.563834 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.564657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:17.564961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.566430 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.567614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.568338 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.568689 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.569031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.576363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.577916 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.591108 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 264/525] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:17.591108 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:17.591430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:17.592541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 262/526] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:17.594511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.622890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:17.707433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71af4d-74ce-47ca-a689-6878cfeaa19d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.708122 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e54c-3a56-4e3c-b157-87f569cfd8a9 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.747147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.747782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.747961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.748065 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.748328 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.748701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.749142 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:17.749334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.749443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.750921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.762143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.763640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.773959 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 265/527] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:17.773959 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:17.778716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:17.779862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:17.781073 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:17.781073 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:17.781073 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:17.781073 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:17.783089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 263/528] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:17.783918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.809697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:17.868003 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:17.892446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913fee7-ad72-46cc-95c2-7e5424d48190 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.893563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364902f5-ddcb-4809-8857-acd13538f54f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:17.928559 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.929202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.929485 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.929759 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.930562 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:17.930877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.932185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:17.932343 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.933342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:17.933854 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:17.934467 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:17.935517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=45cd9a93c342491889ca8260bc4510c6) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:17.936054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:17.937860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:17.945710 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 266/529] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/role_assignments?user.id=45cd9a93c342491889ca8260bc4510c6&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:17.947413 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:17.970160 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 264/530] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:17.970160 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:18.083957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:18.091604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae3689-98bf-41df-937c-0cd373206a7f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:18.098436 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e1933-4dc3-458e-ae3a-0ad58234f911 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:18.132321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:18.133333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:18.133875 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:18.134400 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:18.135315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:18.135820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:18.137459 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:18.139493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:18.143834 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ad597-df7a-492c-971e-b70ac578afcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:18.144558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad597-df7a-492c-971e-b70ac578afcf None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:18.144914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad597-df7a-492c-971e-b70ac578afcf None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:18.145256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad597-df7a-492c-971e-b70ac578afcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:18.153791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 265/531] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:18.155469 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:18.224674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ad597-df7a-492c-971e-b70ac578afcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:18.226406 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ad597-df7a-492c-971e-b70ac578afcf None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:18.263413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550032b-3dac-444c-bf20-5caab17f9ee2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:18.299406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:18.300474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:18.300981 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:18.301492 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:18.302548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=07d2ec2ce99a42cda7652878d2919f0a) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:18.303190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:18.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:18.311777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 266/532] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:18 2022] GET /identity/v3/role_assignments?user.id=07d2ec2ce99a42cda7652878d2919f0a&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:18.313044 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:18.338498 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:18.345036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 267/533] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:18.362237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ad597-df7a-492c-971e-b70ac578afcf None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:18.416207 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:18.495684 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ad597-df7a-492c-971e-b70ac578afcf None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:18.538077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:18.539722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:18.540411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:18.540912 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:18.548493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e431f2-51c4-40b8-8342-22dac21f5473 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:18.554156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:18.557027 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:18.589351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:18.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:18.591478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:18.591478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:18.591478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:18.591478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:18.591478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 268/534] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:18.603416 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:18.607637 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:18.608754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:18.609243 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:18.609741 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:18.622987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:18.626488 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:18.643303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:18.644391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:18.645409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:18.645409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:18.645409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:18.645409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:18.645409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:18.645409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:18.645409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:18.645409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:18.649237 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 267/535] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:18.655516 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:18.712660 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:18.800318 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ed59-70bb-4c6c-9e98-5ec1d7e42177 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:18.824675 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:18.847995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:18.849072 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:18.849586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:18.850071 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:18.851131 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:18.851665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:18.854213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:18.870533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 268/536] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:18.876404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:18.892653 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539d1df-aa72-45af-a750-57c0836b997f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:19.005191 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83c52e-efce-417e-936d-d799932e9232 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:19.015845 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:19.016974 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:19.017507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:19.017998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:19.018918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:19.019229 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:19.020657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:19.048631 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 269/537] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:19.053999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:19.057383 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:19.058299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:19.060270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:19.060795 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:19.061827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=d24756347c554b45bfa30698306608be) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:19.063481 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:19.070765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:19.080472 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 269/538] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:18 2022] GET /identity/v3/role_assignments?user.id=d24756347c554b45bfa30698306608be&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:19.184085 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc052f1-b1c1-44ff-865d-3da3ba450573 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:19.186215 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:19.225321 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5824e-f1bf-4759-9aac-4a1baef09016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:19.226441 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5824e-f1bf-4759-9aac-4a1baef09016 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:19.226950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5824e-f1bf-4759-9aac-4a1baef09016 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:19.227445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5824e-f1bf-4759-9aac-4a1baef09016 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:19.235923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5824e-f1bf-4759-9aac-4a1baef09016 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:19.237942 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5824e-f1bf-4759-9aac-4a1baef09016 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-08f5824e-f1bf-4759-9aac-4a1baef09016 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:19.251082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:19.252238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:19.253237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:19.253237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:19.253237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:19.253237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:19.253237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:19.253237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:19.253237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:19.253237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:19.256078 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 270/539] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:19.263344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:19.317213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703370-ec0a-4ecd-8af4-7b5a69ad2be9 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:19.317477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703370-ec0a-4ecd-8af4-7b5a69ad2be9 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:19.317760 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703370-ec0a-4ecd-8af4-7b5a69ad2be9 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:19.319607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 271/540] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:19.325725 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60782e8-09ff-4b65-a69d-5b3575cd64b7 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:19.326075 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60782e8-09ff-4b65-a69d-5b3575cd64b7 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:19.326926 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60782e8-09ff-4b65-a69d-5b3575cd64b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:19.335002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:19.350019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-f60782e8-09ff-4b65-a69d-5b3575cd64b7 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:19.351666 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-f60782e8-09ff-4b65-a69d-5b3575cd64b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:19.430761 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1d5c-91dd-4d1d-ba35-e53582fde608 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:19.484148 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:19.484808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:19.485087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:19.485355 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:19.486211 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:19.486552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:19.488034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:19.491641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 272/541] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:19.503823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 270/542] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:19.515365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60782e8-09ff-4b65-a69d-5b3575cd64b7 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:19.519699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:19.723869 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189ec6b-a756-4adb-87ae-3091f1866167 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:19.726461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:19.754521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:19.763047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb507dec-1a60-41fd-85f3-445b3c0c65b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:19.763718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507dec-1a60-41fd-85f3-445b3c0c65b2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:19.764000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507dec-1a60-41fd-85f3-445b3c0c65b2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:19.764124 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60782e8-09ff-4b65-a69d-5b3575cd64b7 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:19.764294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507dec-1a60-41fd-85f3-445b3c0c65b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:19.777931 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb507dec-1a60-41fd-85f3-445b3c0c65b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-fb507dec-1a60-41fd-85f3-445b3c0c65b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:19.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:19.784344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:19.785286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:19.786261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 271/543] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:19.792992 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb507dec-1a60-41fd-85f3-445b3c0c65b2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:19.844472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:19.845117 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:19.845400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:19.845805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:19.846674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=05afd25e30764af0819436cc91dd85c5) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:19.846998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:19.848469 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:19.864998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 273/544] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:19 2022] GET /identity/v3/role_assignments?user.id=05afd25e30764af0819436cc91dd85c5&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:19.963945 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb507dec-1a60-41fd-85f3-445b3c0c65b2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:20.014384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:20.015017 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:20.015295 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:20.015574 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:20.016297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] RBAC: Adding query filter params (name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:20.016599 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:20.018033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:20.033539 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 272/545] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:20.039772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:20.154298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f588-e88c-46fd-84cc-a88a71b45194 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:20.189135 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bb5d8-1423-49fe-b1bf-06ab6a6424a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:20.189856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bb5d8-1423-49fe-b1bf-06ab6a6424a8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:20.191025 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bb5d8-1423-49fe-b1bf-06ab6a6424a8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:20.191308 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bb5d8-1423-49fe-b1bf-06ab6a6424a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:20.203317 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bb5d8-1423-49fe-b1bf-06ab6a6424a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:20.204841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bb5d8-1423-49fe-b1bf-06ab6a6424a8 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-0b2bb5d8-1423-49fe-b1bf-06ab6a6424a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:20.225563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:20.226677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:20.227665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:20.227665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:20.227665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:20.227665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:20.231246 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 274/546] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:20.237168 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:20.290463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:20.365999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18b20f-8a38-42ec-8967-5e09e348fb93 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:20.366403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18b20f-8a38-42ec-8967-5e09e348fb93 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:20.366624 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18b20f-8a38-42ec-8967-5e09e348fb93 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:20.368176 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 275/547] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:20.373917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c1930-4b9d-4e05-ba19-c1f7ecdb22d9 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:20.374523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c1930-4b9d-4e05-ba19-c1f7ecdb22d9 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:20.375018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c1930-4b9d-4e05-ba19-c1f7ecdb22d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:20.394817 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-936c1930-4b9d-4e05-ba19-c1f7ecdb22d9 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:20.396709 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-936c1930-4b9d-4e05-ba19-c1f7ecdb22d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:20.400613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:20.413929 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d46bd-da99-48b4-8fa1-58d95b61bb6c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:20.485747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:20.486873 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:20.487379 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:20.488785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:20.489703 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:20.490427 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:20.493333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:20.518283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 273/548] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:20.524113 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:20.526073 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 276/549] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:20.575360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c1930-4b9d-4e05-ba19-c1f7ecdb22d9 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:20.649631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:20.679847 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c703-4bf9-4229-a742-03e564fee6d5 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:20.756127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:20.756810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:20.757093 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:20.757368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] PATH_INFO: `/v3/users/swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:20.769761 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:20.774077 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:20.778862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c1930-4b9d-4e05-ba19-c1f7ecdb22d9 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:20.801334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:20.802536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:20.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:20.806917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 274/550] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:20.811639 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:20.861261 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09c62a-c353-45bb-8cc4-d9a152cdf6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:20.861852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09c62a-c353-45bb-8cc4-d9a152cdf6f6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:20.862207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09c62a-c353-45bb-8cc4-d9a152cdf6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:20.862484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09c62a-c353-45bb-8cc4-d9a152cdf6f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:20.876487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:20.886350 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09c62a-c353-45bb-8cc4-d9a152cdf6f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-8a09c62a-c353-45bb-8cc4-d9a152cdf6f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:20.891486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:20.893776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:20.895883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:20.902584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 277/551] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:20.903561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09c62a-c353-45bb-8cc4-d9a152cdf6f6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:20.992384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce446-d45f-4a66-8fc0-de9310640b1e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.036125 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da613be-cbfb-48b2-af36-50b349db64a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.037031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da613be-cbfb-48b2-af36-50b349db64a8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.037314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da613be-cbfb-48b2-af36-50b349db64a8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.037590 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da613be-cbfb-48b2-af36-50b349db64a8 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.038904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da613be-cbfb-48b2-af36-50b349db64a8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:21.039223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da613be-cbfb-48b2-af36-50b349db64a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:21.041123 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da613be-cbfb-48b2-af36-50b349db64a8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:21.044722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09c62a-c353-45bb-8cc4-d9a152cdf6f6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.064101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 275/552] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:14:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:21.065514 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25bcb9-ce7c-4e53-9b5c-8fc6b86c3fb4 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.065803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25bcb9-ce7c-4e53-9b5c-8fc6b86c3fb4 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.066082 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25bcb9-ce7c-4e53-9b5c-8fc6b86c3fb4 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.072443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 276/553] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:21.073311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25bcb9-ce7c-4e53-9b5c-8fc6b86c3fb4 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:21.086763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e25059-897a-4dc4-963e-fef71a229ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.087482 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e25059-897a-4dc4-963e-fef71a229ee9 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.087832 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e25059-897a-4dc4-963e-fef71a229ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.088174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e25059-897a-4dc4-963e-fef71a229ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.088971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e25059-897a-4dc4-963e-fef71a229ee9 None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:21.089349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e25059-897a-4dc4-963e-fef71a229ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:21.090849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e25059-897a-4dc4-963e-fef71a229ee9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:21.100880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:21.106380 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 278/554] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:21.108773 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d42c3-bfba-422d-a3d3-5a8cb09a2e8b None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.109294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d42c3-bfba-422d-a3d3-5a8cb09a2e8b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.109836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d42c3-bfba-422d-a3d3-5a8cb09a2e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.130894 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-801d42c3-bfba-422d-a3d3-5a8cb09a2e8b None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:21.132197 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-801d42c3-bfba-422d-a3d3-5a8cb09a2e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:21.190799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:21.201499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25bcb9-ce7c-4e53-9b5c-8fc6b86c3fb4 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.204592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 279/555] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:21.205708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d42c3-bfba-422d-a3d3-5a8cb09a2e8b None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:21.255300 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84865034-c307-4846-845e-29c125c330be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.256416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84865034-c307-4846-845e-29c125c330be None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.256941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84865034-c307-4846-845e-29c125c330be None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.257556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84865034-c307-4846-845e-29c125c330be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.333327 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d42c3-bfba-422d-a3d3-5a8cb09a2e8b None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.350347 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84865034-c307-4846-845e-29c125c330be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:21.352491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84865034-c307-4846-845e-29c125c330be None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:21.354900 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:21.375056 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.376118 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.376638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.377209 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.388517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:21.390346 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:21.409473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:21.410599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:21.411605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:21.411605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:21.411605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:21.411605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:21.411849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 280/556] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:21.417109 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:21.462531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:21.484706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 277/557] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:21.484706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84865034-c307-4846-845e-29c125c330be None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:21.529729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4d4a9-d8c3-4727-b663-62b4341837bf None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.568514 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ac320-d9e1-443e-adee-6f9b74542777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.569182 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ac320-d9e1-443e-adee-6f9b74542777 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.569463 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ac320-d9e1-443e-adee-6f9b74542777 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.569733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ac320-d9e1-443e-adee-6f9b74542777 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.582837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ac320-d9e1-443e-adee-6f9b74542777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-036ac320-d9e1-443e-adee-6f9b74542777 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:21.588484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:21.589836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:21.590904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 281/558] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:21.591748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792767-3f0e-4855-8b4a-bd9f2ae34feb None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.591748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792767-3f0e-4855-8b4a-bd9f2ae34feb None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.591748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792767-3f0e-4855-8b4a-bd9f2ae34feb None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.592940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 282/559] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:21.593487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86792767-3f0e-4855-8b4a-bd9f2ae34feb None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:21.600918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84865034-c307-4846-845e-29c125c330be None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.603278 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:21.641029 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.641747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.642014 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.642388 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.643091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:21.643401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:21.644799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:21.657484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 278/560] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:21.658037 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:21.713310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86792767-3f0e-4855-8b4a-bd9f2ae34feb None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.714936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:21.755393 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4746703-184a-4344-a437-a10174911459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.756407 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746703-184a-4344-a437-a10174911459 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.756707 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746703-184a-4344-a437-a10174911459 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.757078 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746703-184a-4344-a437-a10174911459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.767190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a2a99-2877-42ab-bc8e-d4500430e469 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.769272 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4746703-184a-4344-a437-a10174911459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:21.770996 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4746703-184a-4344-a437-a10174911459 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-e4746703-184a-4344-a437-a10174911459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:21.788034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:21.789799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:21.790884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:21.790884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:21.790884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:21.790884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:21.790884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 283/561] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:21.790884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5edd-1c4f-421c-806c-fa1be443fae3 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.791275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5edd-1c4f-421c-806c-fa1be443fae3 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.791360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5edd-1c4f-421c-806c-fa1be443fae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.811677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-393a5edd-1c4f-421c-806c-fa1be443fae3 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:21.811932 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.812320 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.812600 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.812872 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.812985 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-393a5edd-1c4f-421c-806c-fa1be443fae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:21.813595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] RBAC: Adding query filter params (name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:21.813893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:21.815643 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:21.830175 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 279/562] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:21.830175 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:21.857598 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:21.896212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 284/563] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:21.897413 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a5edd-1c4f-421c-806c-fa1be443fae3 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:21.945399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8af1cb-efbe-4729-93c7-8028da159d55 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:21.984827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:21.985487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:21.985938 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:21.986213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] PATH_INFO: `/v3/users/nova` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:21.992185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:21.994355 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:21.996312 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:22.007947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:22.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:22.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:22.010830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a5edd-1c4f-421c-806c-fa1be443fae3 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.013844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 280/564] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:14:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:22.014734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.049017 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.049656 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.049940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.050244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.050953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:22.051262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.052656 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.064874 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 285/565] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:22.065533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.108820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:22.127023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5066df-9e2e-470c-baf5-6a02a45a597d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.163913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.164606 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.164862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.165140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.173762 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280bf8e-2d70-4d02-b138-6cac16d2985e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.177844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.179370 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:22.194823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:22.195941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:22.197351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:22.197351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:22.197351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:22.197351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:22.197351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 281/566] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:22.197351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.216657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76c78-5284-4066-ab83-694948d7a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.217333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76c78-5284-4066-ab83-694948d7a38a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.217614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76c78-5284-4066-ab83-694948d7a38a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.217894 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76c78-5284-4066-ab83-694948d7a38a None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.229523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd76c78-5284-4066-ab83-694948d7a38a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-cfd76c78-5284-4066-ab83-694948d7a38a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:22.235054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:22.238473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 286/567] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:22.239414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76c78-5284-4066-ab83-694948d7a38a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.244812 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:22.303614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bae01a-2c29-4df4-81ea-8f08de6f322b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.338534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba27bff-6e73-47ee-a34b-c1fb4146c6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.339163 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba27bff-6e73-47ee-a34b-c1fb4146c6c5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.339446 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba27bff-6e73-47ee-a34b-c1fb4146c6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.339721 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba27bff-6e73-47ee-a34b-c1fb4146c6c5 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.340572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba27bff-6e73-47ee-a34b-c1fb4146c6c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:22.340875 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba27bff-6e73-47ee-a34b-c1fb4146c6c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.341920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76c78-5284-4066-ab83-694948d7a38a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.342351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba27bff-6e73-47ee-a34b-c1fb4146c6c5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.344917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:22.360427 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 282/568] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:22.362039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a0217-330d-4fd5-8419-34a66353b846 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.362363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a0217-330d-4fd5-8419-34a66353b846 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.362638 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a0217-330d-4fd5-8419-34a66353b846 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.363885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 283/569] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:22.364585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a0217-330d-4fd5-8419-34a66353b846 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.380215 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.380877 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.381157 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.381429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.388492 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.389976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:22.401582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:22.407716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 287/570] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:22.408966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.469182 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:22.473974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a0217-330d-4fd5-8419-34a66353b846 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.507450 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9f597-809b-4971-88d8-1bd6643b83f7 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.508710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.509338 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.509615 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.509889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.510622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:22.510919 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.512304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.523857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 284/571] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:22.524678 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.545741 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b84203-1de5-45b5-969a-14024a85dc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.546407 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84203-1de5-45b5-969a-14024a85dc28 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.546690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84203-1de5-45b5-969a-14024a85dc28 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.546956 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84203-1de5-45b5-969a-14024a85dc28 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.547680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b84203-1de5-45b5-969a-14024a85dc28 None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:22.547985 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b84203-1de5-45b5-969a-14024a85dc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.549440 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b84203-1de5-45b5-969a-14024a85dc28 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.557448 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 288/572] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:22.558702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618742f-31c4-4561-9d01-b929f11c0b3c None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.558985 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618742f-31c4-4561-9d01-b929f11c0b3c None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.559269 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618742f-31c4-4561-9d01-b929f11c0b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.578716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-f618742f-31c4-4561-9d01-b929f11c0b3c None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:22.579962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-f618742f-31c4-4561-9d01-b929f11c0b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:22.580115 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:22.625870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6f569-409e-4925-a870-e5c6ba03d302 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.651306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 289/573] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:22.652157 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618742f-31c4-4561-9d01-b929f11c0b3c None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.676358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.677022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.677314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.677587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.688803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.690396 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:22.705774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:22.711718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:22.713031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:22.713031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:22.713031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:22.713031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:22.713031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 285/574] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:22.713031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.714233 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:22.757061 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618742f-31c4-4561-9d01-b929f11c0b3c None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.791694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.792387 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.792629 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.792867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.793621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:22.793937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.795437 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.807188 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 290/575] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:22.808004 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.808989 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd03a4-e56a-4ebc-92bf-94670c5afc45 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.816732 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:22.844440 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.845100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.845403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.845716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] PATH_INFO: `/v3/users/glance` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.853339 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.854876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:22.866401 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:22.867519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 286/576] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:22.868580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:22.913925 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653e1f4-3173-4f92-8cb1-09f49aab1e6b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.929759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:22.948526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:22.949458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:22.949755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:22.950030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:22.961343 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:22.962867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:22.967590 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e0e93-43d1-4ac9-81df-f2dc89b78d42 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:22.978497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:22.979744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:22.980810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:22.980810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:22.980810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:22.980810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:22.980810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 291/577] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:22.980810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.005499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854f6c1-3475-4d9d-8c9b-d3831d343edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.006168 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854f6c1-3475-4d9d-8c9b-d3831d343edd None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.006457 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854f6c1-3475-4d9d-8c9b-d3831d343edd None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.006734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854f6c1-3475-4d9d-8c9b-d3831d343edd None admin] PATH_INFO: `/v3/roles/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.017507 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854f6c1-3475-4d9d-8c9b-d3831d343edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-d854f6c1-3475-4d9d-8c9b-d3831d343edd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:23.022977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:23.024231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 287/578] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:23.025202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854f6c1-3475-4d9d-8c9b-d3831d343edd None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.051962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:23.089380 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872e872-be5c-453f-abe8-45132c9643f3 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.126870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.127528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.127812 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.128087 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.128242 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854f6c1-3475-4d9d-8c9b-d3831d343edd None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.128811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:23.129120 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.130586 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.143944 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 292/579] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:23.144909 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.165823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.166498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.166785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.167060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.167863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=da81aff79a9a467da6d8bf372da9faf2) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:23.168180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.169648 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.174311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:23.190135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 288/580] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/role_assignments?user.id=da81aff79a9a467da6d8bf372da9faf2&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:23.190135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.259426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95b06-e158-4894-ab81-99d4aceaa2c8 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.285222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:23.295765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079925bd-f231-472a-9a77-72449ba368ca None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.299785 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.300440 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.300725 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.300997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.301823 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:23.302157 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.303674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.327850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 293/581] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:23.328758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.339321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.339962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.339962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.339962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.341203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:23.341515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.342956 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.356716 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 289/582] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:23.357587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.431320 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:23.446956 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ecb2-c2ae-478d-8a39-26029b52aba6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.471666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ac338-44f5-4e31-b1ac-9c194d07cb30 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.482123 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.482763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.483045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.483318 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.484052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:23.484359 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.485755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.494553 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 294/583] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:23.495394 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.517000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.517965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.518470 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.518904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.526959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.528726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:23.540918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:23.542194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:23.543232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:23.543232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:23.543232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:23.543232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:23.543232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:23.543232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:23.543232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:23.543232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:23.544008 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 290/584] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:23.544599 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.558511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:23.603959 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52622b-be86-4b4a-953f-2d30ffc367b4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.645593 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.646299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.646585 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.646865 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.655559 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32136c6-d244-411b-aef5-2196c3c8a93e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.658288 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.659768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.668202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:23.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:23.677472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:23.683616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:23.683616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:23.683616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:23.683616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:23.683616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 295/585] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:23.683616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.704034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.704699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.705013 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.705292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.713248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.714757 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:23.726475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:23.727627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 291/586] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:23.728717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.804051 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8d6fb-0f8e-4c08-83ec-32a2da38c2a5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.805130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:23.842901 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.843615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.843766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55467bd-f9cf-48fb-b57b-7b0d47d1ca51 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:23.843883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.844130 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.857185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.858753 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:23.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:23.877617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:23.878670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:23.878670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:23.878670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:23.878670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:23.878670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 296/587] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:23.878670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.889341 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:23.890009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:23.890326 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:23.890623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:23.891412 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:23.891727 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:23.893324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:23.909208 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 292/588] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:23.909941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:23.948288 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:24.005768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa19b-164d-4fe1-b6b5-22e4807d70c7 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.027295 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a4a2b-a642-442c-bba1-23766a0f81cf None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.045568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.046244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.046532 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.046810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.047541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:24.047842 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.049375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.063510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 297/589] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:24.064410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.073464 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:24.082440 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.083680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.084395 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.084684 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.085656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:24.085967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.087497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.111823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 293/590] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:14:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:24.111823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.193839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4386c-001c-4632-9ec0-161a307e65f4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.210744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:24.222326 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a51375-c833-4c5b-bce3-1de8f4bc5e3e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.240630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.241270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.241548 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.241821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.242555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:24.242856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.244244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.259420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 298/591] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:24.260684 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.269807 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1dfa-d1dd-4805-b07d-058ed9ed3c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.270550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1dfa-d1dd-4805-b07d-058ed9ed3c6b None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.270843 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1dfa-d1dd-4805-b07d-058ed9ed3c6b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.271603 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1dfa-d1dd-4805-b07d-058ed9ed3c6b None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/45cd9a93c342491889ca8260bc4510c6/roles/55d67b43cde54213983e27412c2a314c` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.291402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca1dfa-d1dd-4805-b07d-058ed9ed3c6b None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=45cd9a93c342491889ca8260bc4510c6, role_id=55d67b43cde54213983e27412c2a314c)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.292998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca1dfa-d1dd-4805-b07d-058ed9ed3c6b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.308563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 294/592] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:14:24 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/45cd9a93c342491889ca8260bc4510c6/roles/55d67b43cde54213983e27412c2a314c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:24.309434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1dfa-d1dd-4805-b07d-058ed9ed3c6b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.363882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:24.411269 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea5aa7-9d62-48a5-8a5c-ee9b72d3b5dd None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.457556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1dfa-d1dd-4805-b07d-058ed9ed3c6b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.467065 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.469322 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.469852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.471228 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.483443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.485919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:24.500371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:24.502334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:24.503641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:24.503641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:24.503641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:24.503641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:24.503641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:24.503641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:24.503641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:24.503641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:24.504542 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 299/593] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:24.505348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.509146 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.510729 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.511130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.511509 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.526470 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.528204 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:24.546556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:24.547730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:24.548877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:24.548877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:24.548877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:24.548877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:24.548877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 295/594] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:24.549225 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.554661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:24.653023 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cbd8c-69fe-4fd8-904d-7aa90f67ceb3 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.670021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:24.682627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc07b7-7033-4f37-b633-58eef1caf794 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.707467 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.708582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.709070 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.709554 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.718570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.720446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.726855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.727512 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.727798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.728076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.729186 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:24.729501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.731037 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:24.735000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:24.736181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:14:24.737282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:24.744928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 300/595] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:24.746286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.750218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 296/596] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:24.751479 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.864736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:24.882860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953598c1-ad03-46b4-b138-85effa59d111 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.899511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937a823-0ccf-4196-9626-9fc73ffe7278 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:24.921980 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.924574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.925122 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.925724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.926778 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:24.927304 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.929105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.937837 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:24.938513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:24.938802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:24.939130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:24.939990 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:24.940308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:24.941930 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:24.949596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 301/597] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:24.950969 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.969569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 297/598] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:14:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:24.969569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:24.977229 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:25.075107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7ff1c-4684-4894-aebc-4ea45f9f227b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:25.075843 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b77991-155d-464b-9ad4-1550303b061e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:25.113920 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:25.114655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.114937 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.115679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.116141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c575173-03bf-4f38-b92d-a56545016f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:25.116835 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575173-03bf-4f38-b92d-a56545016f0d None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.117112 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575173-03bf-4f38-b92d-a56545016f0d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.117385 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575173-03bf-4f38-b92d-a56545016f0d None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/07d2ec2ce99a42cda7652878d2919f0a/roles/55d67b43cde54213983e27412c2a314c` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.125594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:25.127485 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:25.133964 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c575173-03bf-4f38-b92d-a56545016f0d None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=07d2ec2ce99a42cda7652878d2919f0a, role_id=55d67b43cde54213983e27412c2a314c)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:25.135579 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c575173-03bf-4f38-b92d-a56545016f0d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:25.140679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:25.142090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 298/599] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:25.143411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:25.151432 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 302/600] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:14:24 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/07d2ec2ce99a42cda7652878d2919f0a/roles/55d67b43cde54213983e27412c2a314c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:25.152294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c575173-03bf-4f38-b92d-a56545016f0d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:25.157888 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:25.262578 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:25.285272 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c575173-03bf-4f38-b92d-a56545016f0d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:25.318870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c956ec1-7a4b-48f1-b259-b645729bfa91 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:25.325790 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c6cae-6cd2-435c-be75-41d8b0bcba2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:25.326471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c6cae-6cd2-435c-be75-41d8b0bcba2d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.326749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c6cae-6cd2-435c-be75-41d8b0bcba2d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.327024 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c6cae-6cd2-435c-be75-41d8b0bcba2d None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.327795 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c6cae-6cd2-435c-be75-41d8b0bcba2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:25.328100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c6cae-6cd2-435c-be75-41d8b0bcba2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:25.329816 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c6cae-6cd2-435c-be75-41d8b0bcba2d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:25.345624 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 303/601] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:25.348362 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7e50-21e4-419e-93f3-2eeefb4c6413 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.348896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7e50-21e4-419e-93f3-2eeefb4c6413 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.349388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7e50-21e4-419e-93f3-2eeefb4c6413 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.355565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 304/602] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:25.355565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7e50-21e4-419e-93f3-2eeefb4c6413 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:25.380746 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dc669-6086-4884-b5e0-bb64f339f0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:25.381790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dc669-6086-4884-b5e0-bb64f339f0da None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.382303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dc669-6086-4884-b5e0-bb64f339f0da None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.382813 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dc669-6086-4884-b5e0-bb64f339f0da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.396322 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233dc669-6086-4884-b5e0-bb64f339f0da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:25.398207 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233dc669-6086-4884-b5e0-bb64f339f0da None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-233dc669-6086-4884-b5e0-bb64f339f0da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:25.414025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:25.415180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:25.416203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:25.416203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:25.416203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:25.416203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:25.416786 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 299/603] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:25.417810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8c101-873b-4237-b390-eb41f8be9cf5 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.418119 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8c101-873b-4237-b390-eb41f8be9cf5 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.418402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8c101-873b-4237-b390-eb41f8be9cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.441988 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-c1b8c101-873b-4237-b390-eb41f8be9cf5 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:25.442637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:25.443773 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-c1b8c101-873b-4237-b390-eb41f8be9cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:25.497452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7e50-21e4-419e-93f3-2eeefb4c6413 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:25.538244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 300/604] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:25.538244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8c101-873b-4237-b390-eb41f8be9cf5 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:25.555455 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb00932-a0ab-426b-b75a-98edffa9561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:25.556160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb00932-a0ab-426b-b75a-98edffa9561c None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.556432 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb00932-a0ab-426b-b75a-98edffa9561c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.556708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb00932-a0ab-426b-b75a-98edffa9561c None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/d24756347c554b45bfa30698306608be/roles/55d67b43cde54213983e27412c2a314c` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.585066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb00932-a0ab-426b-b75a-98edffa9561c None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=d24756347c554b45bfa30698306608be, role_id=55d67b43cde54213983e27412c2a314c)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:25.587109 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb00932-a0ab-426b-b75a-98edffa9561c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:25.604429 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:25.609260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 305/605] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:14:25 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/d24756347c554b45bfa30698306608be/roles/55d67b43cde54213983e27412c2a314c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:25.611275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb00932-a0ab-426b-b75a-98edffa9561c None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:25.700653 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8c101-873b-4237-b390-eb41f8be9cf5 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:25.786850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:25.787508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.787790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.788068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.788846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:25.789099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:25.790705 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:25.802512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb00932-a0ab-426b-b75a-98edffa9561c None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:25.816582 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 301/606] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:25.822280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:25.823893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:25.861471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a25e7-09d4-402a-9ffb-9eaa7a86162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:25.862223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a25e7-09d4-402a-9ffb-9eaa7a86162e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.862511 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a25e7-09d4-402a-9ffb-9eaa7a86162e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.862874 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a25e7-09d4-402a-9ffb-9eaa7a86162e None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:25.875878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a25e7-09d4-402a-9ffb-9eaa7a86162e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-cd1a25e7-09d4-402a-9ffb-9eaa7a86162e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:25.881535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:25.882689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:25.883679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 306/607] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:25.889410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a25e7-09d4-402a-9ffb-9eaa7a86162e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:25.958884 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776bc44-4e63-4fd2-9d0e-9d26104c277a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:25.971048 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:25.997427 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:25.998574 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:25.999136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:25.999626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.007704 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:26.010543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:26.023889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:26.024962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:26.025975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:26.025975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:26.025975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:26.025975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:26.025975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:26.025975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:26.025975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:26.025975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:26.026614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a25e7-09d4-402a-9ffb-9eaa7a86162e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:26.028360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 302/608] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:26.033844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:26.101606 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:26.104040 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:26.104503 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:26.106606 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.107542 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:26.110186 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:26.111973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:26.131082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 307/609] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:26.137682 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:26.186345 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f186b0-2b09-487a-ab15-455597d40466 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:26.211544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:26.224457 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:26.225466 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:26.225905 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:26.226423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.227416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:26.227890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:26.229501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:26.243267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 303/610] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:26.251367 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:26.299482 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:26.306493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8a6e5-e0d4-4a82-9084-38a3149246f2 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:26.379282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:26.381398 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:26.384149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:26.384859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.404366 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:26.406372 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:26.425918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:26.427058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:26.428105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:26.428105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:26.428105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:26.428105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:26.430582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 308/611] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:26.439660 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0085cc-739b-41ad-9ec1-5c5d216ad131 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:26.443561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:26.505921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb1bad-f521-4eed-96b0-8bc6a8a4c763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:26.507092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb1bad-f521-4eed-96b0-8bc6a8a4c763 None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:26.508450 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb1bad-f521-4eed-96b0-8bc6a8a4c763 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:26.509197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb1bad-f521-4eed-96b0-8bc6a8a4c763 None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/05afd25e30764af0819436cc91dd85c5/roles/55d67b43cde54213983e27412c2a314c` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.544262 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb1bad-f521-4eed-96b0-8bc6a8a4c763 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=05afd25e30764af0819436cc91dd85c5, role_id=55d67b43cde54213983e27412c2a314c)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:26.547740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb1bad-f521-4eed-96b0-8bc6a8a4c763 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:26.571655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 304/612] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:14:26 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/05afd25e30764af0819436cc91dd85c5/roles/55d67b43cde54213983e27412c2a314c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:26.627944 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:26.643254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160776c8-4d4b-47a6-a9c0-c9c95f2dd242 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:26.695938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ef4b4-81e8-456d-8816-de09ecc8c49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:26.697295 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ef4b4-81e8-456d-8816-de09ecc8c49e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:26.697582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ef4b4-81e8-456d-8816-de09ecc8c49e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:26.697850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ef4b4-81e8-456d-8816-de09ecc8c49e None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.698607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ef4b4-81e8-456d-8816-de09ecc8c49e None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:26.698910 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ef4b4-81e8-456d-8816-de09ecc8c49e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:26.700334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ef4b4-81e8-456d-8816-de09ecc8c49e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:26.724576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 309/613] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:26.730815 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb1bad-f521-4eed-96b0-8bc6a8a4c763 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:26.762355 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7767-3e89-4728-9e92-22295ced8315 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:26.763033 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7767-3e89-4728-9e92-22295ced8315 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:26.763538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7767-3e89-4728-9e92-22295ced8315 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.765520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 310/614] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:14:26.771420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfb2b3-9a40-4331-ac09-bd73c41a734d None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:26.772048 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfb2b3-9a40-4331-ac09-bd73c41a734d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:26.772549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfb2b3-9a40-4331-ac09-bd73c41a734d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.793022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-27cfb2b3-9a40-4331-ac09-bd73c41a734d None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:26.794671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-27cfb2b3-9a40-4331-ac09-bd73c41a734d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:26.848010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:26.871152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb1bad-f521-4eed-96b0-8bc6a8a4c763 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:26.872591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 311/615] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:26.907280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfb2b3-9a40-4331-ac09-bd73c41a734d None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:26.922944 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:26.923798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:26.924084 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:26.924367 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:26.942770 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:26.944272 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:26.965619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:26.966809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:26.967887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:26.976389 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 305/616] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:26.983208 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:27.046496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:27.081955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfb2b3-9a40-4331-ac09-bd73c41a734d None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:27.151333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123827a-2088-4112-a197-846b6dbfd1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:27.152007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123827a-2088-4112-a197-846b6dbfd1c4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.152292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123827a-2088-4112-a197-846b6dbfd1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.152568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123827a-2088-4112-a197-846b6dbfd1c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.170275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123827a-2088-4112-a197-846b6dbfd1c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-3123827a-2088-4112-a197-846b6dbfd1c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:27.175454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:27.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:27.177550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:27.183850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 312/617] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:27.188135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b96b2c-eca3-4857-bcae-332087d346aa None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:27.190795 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123827a-2088-4112-a197-846b6dbfd1c4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:27.238199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:27.239226 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.239730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.240202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.241340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:27.241875 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:27.243611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:27.272474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:27.276586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 306/618] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:14:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:27.278542 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:27.373237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123827a-2088-4112-a197-846b6dbfd1c4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:27.428175 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39e384-ba59-41db-9fb5-89a2b00b745b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:27.448079 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:27.448839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.449035 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.449297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.450019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:27.450369 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:27.451798 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:27.454429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:27.465035 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 313/619] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:27.468554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:27.469331 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.469711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.470083 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.470694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:27.481747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:27.483403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:27.501857 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:27.503019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:27.504476 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:27.504476 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:27.504476 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:27.504476 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:27.504476 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 307/620] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:27.509236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:27.567367 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:27.622280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2901-59e7-4060-8049-4e1d0074be1a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:27.680150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf3b7-db15-42f2-9dda-cb54f1ac6475 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:27.689375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d18c4b-4812-4f58-bcab-3a94abbb47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:27.690029 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d18c4b-4812-4f58-bcab-3a94abbb47ed None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.690346 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d18c4b-4812-4f58-bcab-3a94abbb47ed None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.690628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d18c4b-4812-4f58-bcab-3a94abbb47ed None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.691356 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d18c4b-4812-4f58-bcab-3a94abbb47ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:27.691672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d18c4b-4812-4f58-bcab-3a94abbb47ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:27.693651 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d18c4b-4812-4f58-bcab-3a94abbb47ed None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:27.715732 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 308/621] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:27.715732 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143cfb-bed4-404e-929d-8918313c417d None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.715732 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143cfb-bed4-404e-929d-8918313c417d None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.715732 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143cfb-bed4-404e-929d-8918313c417d None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.718198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 309/622] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:27.719434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b143cfb-bed4-404e-929d-8918313c417d None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:27.728905 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529883f4-acca-4875-874f-ad006d2176f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:27.729844 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529883f4-acca-4875-874f-ad006d2176f1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.730327 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529883f4-acca-4875-874f-ad006d2176f1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.730771 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529883f4-acca-4875-874f-ad006d2176f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.749121 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529883f4-acca-4875-874f-ad006d2176f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:27.751138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529883f4-acca-4875-874f-ad006d2176f1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-529883f4-acca-4875-874f-ad006d2176f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:27.767576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:27.768848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:27.770136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:27.770136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:27.770136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:27.770136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:27.772231 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 314/623] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:27.773694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbbe43-f9b3-4000-ab84-e0818997c3fb None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.774164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbbe43-f9b3-4000-ab84-e0818997c3fb None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.774627 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbbe43-f9b3-4000-ab84-e0818997c3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.794476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-c8bbbe43-f9b3-4000-ab84-e0818997c3fb None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:27.795998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-c8bbbe43-f9b3-4000-ab84-e0818997c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:27.850497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:27.854517 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b143cfb-bed4-404e-929d-8918313c417d None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:27.871464 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:27.875073 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 315/624] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:27.876212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbbe43-f9b3-4000-ab84-e0818997c3fb None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:27.916103 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:27.917170 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:27.917717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:27.918273 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:27.926269 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:27.928091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:27.940980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:27.942118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:27.943165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:27.943165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:27.943165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:27.943165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:27.943165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:27.943165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:27.943165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:27.943165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:27.945708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 310/625] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:27.952070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.027298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbbe43-f9b3-4000-ab84-e0818997c3fb None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.068198 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.069011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.069713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.070472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:28.070785 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:28.073250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:28.082409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041f7a-339c-478a-a399-2751d5ea64c8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.093888 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 316/626] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:28.095205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.132033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d2087-00eb-4fc5-ba47-e9fad2afdd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.132800 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d2087-00eb-4fc5-ba47-e9fad2afdd19 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.133088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d2087-00eb-4fc5-ba47-e9fad2afdd19 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.133362 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d2087-00eb-4fc5-ba47-e9fad2afdd19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.143263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:28.144517 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d2087-00eb-4fc5-ba47-e9fad2afdd19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-286d2087-00eb-4fc5-ba47-e9fad2afdd19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:28.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:28.151327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 311/627] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:28.152281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d2087-00eb-4fc5-ba47-e9fad2afdd19 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.190150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:28.214534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49ec84-4f68-430a-a177-0bc3839977ea None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.251659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976a63-5762-454b-a5c0-a35aa59b8bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.252296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976a63-5762-454b-a5c0-a35aa59b8bf1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.252579 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976a63-5762-454b-a5c0-a35aa59b8bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.252860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976a63-5762-454b-a5c0-a35aa59b8bf1 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.253619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976a63-5762-454b-a5c0-a35aa59b8bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:28.253930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976a63-5762-454b-a5c0-a35aa59b8bf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:28.255437 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976a63-5762-454b-a5c0-a35aa59b8bf1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:28.262537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d2087-00eb-4fc5-ba47-e9fad2afdd19 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.268447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 317/628] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:28.270075 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714a9de-2976-41d8-8d15-9a03f898adee None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.270392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714a9de-2976-41d8-8d15-9a03f898adee None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.270670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714a9de-2976-41d8-8d15-9a03f898adee None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.272076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 318/629] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:28.272605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714a9de-2976-41d8-8d15-9a03f898adee None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.303380 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.304107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.304393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.304698 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.312815 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:28.314368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:28.327087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:28.328224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 312/630] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:14:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:28.329946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.394621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:28.400131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714a9de-2976-41d8-8d15-9a03f898adee None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.429921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:28.440912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8a1d9-14bc-46fe-bf0f-4c1539b211ad None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.441163 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7d036-4899-4c73-a8b9-42c84de8349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.441818 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7d036-4899-4c73-a8b9-42c84de8349e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.442224 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7d036-4899-4c73-a8b9-42c84de8349e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.442573 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7d036-4899-4c73-a8b9-42c84de8349e None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.443377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7d036-4899-4c73-a8b9-42c84de8349e None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:28.443761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7d036-4899-4c73-a8b9-42c84de8349e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:28.445267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7d036-4899-4c73-a8b9-42c84de8349e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:28.460602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 319/631] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:28.460602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34631e4-0091-4421-a7b6-9236f965b6cd None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.460602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34631e4-0091-4421-a7b6-9236f965b6cd None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.460602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34631e4-0091-4421-a7b6-9236f965b6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.476777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-c34631e4-0091-4421-a7b6-9236f965b6cd None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:28.477880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-c34631e4-0091-4421-a7b6-9236f965b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:28.484995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1420f-d5d3-443f-903b-edb2ca8bb62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.485696 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1420f-d5d3-443f-903b-edb2ca8bb62a None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.486197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1420f-d5d3-443f-903b-edb2ca8bb62a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.486560 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1420f-d5d3-443f-903b-edb2ca8bb62a None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/da81aff79a9a467da6d8bf372da9faf2/roles/55d67b43cde54213983e27412c2a314c` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.502137 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1420f-d5d3-443f-903b-edb2ca8bb62a None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=da81aff79a9a467da6d8bf372da9faf2, role_id=55d67b43cde54213983e27412c2a314c)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:28.503925 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1420f-d5d3-443f-903b-edb2ca8bb62a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:28.518176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 313/632] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:14:28 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/da81aff79a9a467da6d8bf372da9faf2/roles/55d67b43cde54213983e27412c2a314c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:28.527197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1420f-d5d3-443f-903b-edb2ca8bb62a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.562657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 320/633] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:28.564019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34631e4-0091-4421-a7b6-9236f965b6cd None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.676899 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1420f-d5d3-443f-903b-edb2ca8bb62a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.707044 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:28.717250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34631e4-0091-4421-a7b6-9236f965b6cd None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.724882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.725527 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.725810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.726089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.726997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:28.727444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:28.728811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:28.739259 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:28.755495 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 314/634] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:14:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:28.756653 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.771172 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.772470 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.772922 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.773622 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.787191 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:28.788671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:28.806552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:28.807654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:28.809156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:28.809156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:28.809156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:28.809156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:28.809156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 321/635] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:28.809156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.902618 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12852d-054f-404e-9e93-1d0d375c2f5b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.923725 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1c46-2362-4dc1-8368-e6e25129d1ba None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:28.938157 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f51d4-5343-48d6-9c41-b3d64acef8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.938809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f51d4-5343-48d6-9c41-b3d64acef8d5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.939480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f51d4-5343-48d6-9c41-b3d64acef8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.939480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f51d4-5343-48d6-9c41-b3d64acef8d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.954809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f51d4-5343-48d6-9c41-b3d64acef8d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-248f51d4-5343-48d6-9c41-b3d64acef8d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:28.960064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:28.961133 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 315/636] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:28.962224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f51d4-5343-48d6-9c41-b3d64acef8d5 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:28.973309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9d3e-927b-4e9d-8317-0a564b9e230e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:28.973558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9d3e-927b-4e9d-8317-0a564b9e230e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:28.973842 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9d3e-927b-4e9d-8317-0a564b9e230e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:28.974149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9d3e-927b-4e9d-8317-0a564b9e230e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:28.993640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:29.035269 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:29.056120 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d9d3e-927b-4e9d-8317-0a564b9e230e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.057699 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d9d3e-927b-4e9d-8317-0a564b9e230e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.082608 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f51d4-5343-48d6-9c41-b3d64acef8d5 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.123455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99785f-1866-46b0-9813-820735df0a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.124084 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99785f-1866-46b0-9813-820735df0a66 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.124370 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99785f-1866-46b0-9813-820735df0a66 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.124641 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99785f-1866-46b0-9813-820735df0a66 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.125335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99785f-1866-46b0-9813-820735df0a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:29.125637 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99785f-1866-46b0-9813-820735df0a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.127127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99785f-1866-46b0-9813-820735df0a66 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.139158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 316/637] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:29.140534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ec79-3289-49df-8c28-71121e09cc88 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.140819 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ec79-3289-49df-8c28-71121e09cc88 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.141090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ec79-3289-49df-8c28-71121e09cc88 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.142646 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 317/638] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:29.143226 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ec79-3289-49df-8c28-71121e09cc88 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.178445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 322/639] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:29.179921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9d3e-927b-4e9d-8317-0a564b9e230e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.228618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:29.256369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ec79-3289-49df-8c28-71121e09cc88 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.275477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:29.286659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9d3e-927b-4e9d-8317-0a564b9e230e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.294593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a784fb-5301-43cb-b273-a8b5d0ca1bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.295235 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a784fb-5301-43cb-b273-a8b5d0ca1bdb None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.295510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a784fb-5301-43cb-b273-a8b5d0ca1bdb None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.295780 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a784fb-5301-43cb-b273-a8b5d0ca1bdb None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.296497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a784fb-5301-43cb-b273-a8b5d0ca1bdb None admin] RBAC: Adding query filter params (name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:29.296790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a784fb-5301-43cb-b273-a8b5d0ca1bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.298530 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a784fb-5301-43cb-b273-a8b5d0ca1bdb None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.307619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 318/640] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:29.313459 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78da02f-21d6-4715-a944-7c525cdfb915 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.313749 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78da02f-21d6-4715-a944-7c525cdfb915 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.314076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78da02f-21d6-4715-a944-7c525cdfb915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.327578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.328238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.328520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.328796 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.332377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-f78da02f-21d6-4715-a944-7c525cdfb915 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:29.333573 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f78da02f-21d6-4715-a944-7c525cdfb915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:29.337356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.338861 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:29.352155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:29.353341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 323/641] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:29.355018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.406451 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 319/642] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:29.407880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78da02f-21d6-4715-a944-7c525cdfb915 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.469026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a33f6-529b-4f58-8a88-08cf3d6192c1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.472323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:29.508934 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.509494 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.509771 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.510033 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.522145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.522565 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78da02f-21d6-4715-a944-7c525cdfb915 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.523664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.536212 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:29.540700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:29.541733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:29.542775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:29.542775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:29.542775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:29.542775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:29.542775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 324/643] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:29.542775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.568877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.569543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.569829 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.570127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.582873 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.584440 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:29.600844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:29.601921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:29.602978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:29.602978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:29.602978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:29.602978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:29.602978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 320/644] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:29.602978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.661676 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc2863-a9c4-4c60-b4e5-6fce26caf4d6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.702658 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.703295 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.703573 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.703845 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.704700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:29.705004 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.706470 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.725856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 325/645] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:29.727099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.733969 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0b101-c41b-4b8a-b409-233092f414a1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.747774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:29.770530 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb67464-e427-48bc-b5eb-69cf4ec22f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.771086 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb67464-e427-48bc-b5eb-69cf4ec22f21 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.771360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb67464-e427-48bc-b5eb-69cf4ec22f21 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.771629 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb67464-e427-48bc-b5eb-69cf4ec22f21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.782140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb67464-e427-48bc-b5eb-69cf4ec22f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-8eb67464-e427-48bc-b5eb-69cf4ec22f21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:29.787597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:29.788683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:29.789626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:29.793694 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 321/646] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:29.794483 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb67464-e427-48bc-b5eb-69cf4ec22f21 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.820513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:29.841541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6f4b7-af6d-479c-a274-dae701fdb03d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.878446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.879346 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.879632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.879901 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.880766 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:29.881436 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.882845 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.894829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 326/647] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:29.895620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.904700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb67464-e427-48bc-b5eb-69cf4ec22f21 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:29.943666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:29.944310 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:29.944590 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:29.944863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:29.945575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:29.945879 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:29.947746 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:29.963643 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 322/648] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:29.969335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:29.981194 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:30.013069 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654a3cb-3ead-47ac-9f51-99f95152fbfb None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.054688 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a7677-3fa1-4a73-a3ae-07f6b8e2a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.055251 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a7677-3fa1-4a73-a3ae-07f6b8e2a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.055530 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a7677-3fa1-4a73-a3ae-07f6b8e2a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.055806 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a7677-3fa1-4a73-a3ae-07f6b8e2a7b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.067240 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:30.094404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78855ba6-37ae-4fbb-a81c-db19600f8899 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.133050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a7677-3fa1-4a73-a3ae-07f6b8e2a7b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.134572 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a7677-3fa1-4a73-a3ae-07f6b8e2a7b7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.141731 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.142559 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.142844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.143118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.143862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] RBAC: Adding query filter params (name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:30.144167 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.145597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.154947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 323/649] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:30.155854 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:30.230522 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:30.264702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 327/650] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:30.266007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a7677-3fa1-4a73-a3ae-07f6b8e2a7b7 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:30.275379 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d33b1-59e3-4829-bba1-b2243aec595b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.311899 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.312460 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.312740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.313016 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.320785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.322715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:30.335526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:30.337381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 324/651] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:30.339241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:30.346510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:30.408608 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a7677-3fa1-4a73-a3ae-07f6b8e2a7b7 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.458372 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0436625-f814-4581-9221-99063a712f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.459182 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0436625-f814-4581-9221-99063a712f22 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.459472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0436625-f814-4581-9221-99063a712f22 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.459748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0436625-f814-4581-9221-99063a712f22 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.468987 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0436625-f814-4581-9221-99063a712f22 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.470812 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0436625-f814-4581-9221-99063a712f22 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.475798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb5a3a-1ad7-4d76-a6e2-51683eb24e0c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-a0436625-f814-4581-9221-99063a712f22 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:30.484278 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:30.485540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 328/652] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:30.486879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0436625-f814-4581-9221-99063a712f22 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:30.520455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.521393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.521674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.521946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.533220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:30.534780 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.537144 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:30.555850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:30.556947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:30.558005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:30.558005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:30.558005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:30.558005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:30.562057 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 325/653] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:30.563007 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:30.616079 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0436625-f814-4581-9221-99063a712f22 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.631374 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:30.656005 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.656639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.656914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.657192 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.668988 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.670694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.682410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd7a5d-963b-4231-9128-e5f055e02585 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:30.686383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:30.688230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:30.690245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:30.690245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:30.690245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:30.690245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:30.690245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 329/654] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:30.690245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:30.732066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.732768 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.733100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.733540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.734688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:30.735193 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.736830 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.752434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 326/655] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:30.753298 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:30.798870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:30.816106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ae023-6ef7-47c2-aa7a-19750a99a62e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.857305 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8b99e-b86e-417a-b336-898090a4f53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.857603 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf0799-1399-4108-8200-dca61bb60705 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:30.857947 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8b99e-b86e-417a-b336-898090a4f53a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.858260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8b99e-b86e-417a-b336-898090a4f53a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.858538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8b99e-b86e-417a-b336-898090a4f53a None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.859358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8b99e-b86e-417a-b336-898090a4f53a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:30.859672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8b99e-b86e-417a-b336-898090a4f53a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.861254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8b99e-b86e-417a-b336-898090a4f53a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.876162 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:30.881696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 330/656] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:30.882981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe792-0d0b-4fcc-90fd-193604361fe9 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.883266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe792-0d0b-4fcc-90fd-193604361fe9 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.883542 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe792-0d0b-4fcc-90fd-193604361fe9 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.885122 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 331/657] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:30.885691 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe792-0d0b-4fcc-90fd-193604361fe9 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:30.901943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:30.903032 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:30.903499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:30.903941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:30.904766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:30.905080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:30.906512 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:30.919950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 327/658] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:30.920976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.014420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe792-0d0b-4fcc-90fd-193604361fe9 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.044510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39d42f-fc17-47b6-9f8d-85c78cc5536d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.058787 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.059414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.059690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.059961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.060694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:31.060996 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:31.062412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:31.075663 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 332/659] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:31.076429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.081144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:31.090383 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c63936-9117-42ea-8226-c4aaf0429c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.091169 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c63936-9117-42ea-8226-c4aaf0429c3a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.091636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c63936-9117-42ea-8226-c4aaf0429c3a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.091980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c63936-9117-42ea-8226-c4aaf0429c3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.092954 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c63936-9117-42ea-8226-c4aaf0429c3a None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=45cd9a93c342491889ca8260bc4510c6) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:31.093298 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c63936-9117-42ea-8226-c4aaf0429c3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:31.095012 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c63936-9117-42ea-8226-c4aaf0429c3a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:31.112390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 328/660] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:30 2022] GET /identity/v3/role_assignments?user.id=45cd9a93c342491889ca8260bc4510c6&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:31.112390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94a51-1434-4917-8594-41490b71d0ce None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.112390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94a51-1434-4917-8594-41490b71d0ce None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.112390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94a51-1434-4917-8594-41490b71d0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.131402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-6ee94a51-1434-4917-8594-41490b71d0ce None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:31.132685 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-6ee94a51-1434-4917-8594-41490b71d0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:31.185746 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:31.203155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d0467-64c7-439d-8d1e-6b7b0a7be107 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.225953 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 329/661] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:31.227190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee94a51-1434-4917-8594-41490b71d0ce None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.254649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e76622-a5a3-414e-b39a-992b0880cf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.255588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e76622-a5a3-414e-b39a-992b0880cf11 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.255870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e76622-a5a3-414e-b39a-992b0880cf11 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.256143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e76622-a5a3-414e-b39a-992b0880cf11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.347566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e76622-a5a3-414e-b39a-992b0880cf11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:31.349043 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e76622-a5a3-414e-b39a-992b0880cf11 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:31.353841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:31.355959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee94a51-1434-4917-8594-41490b71d0ce None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.396036 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.396715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.396976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.397252 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] PATH_INFO: `/v3/users/neutron` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.405245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:31.406760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:31.420053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:31.421910 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 330/662] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:31.423898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.458691 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:31.504339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 333/663] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:31.509185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e76622-a5a3-414e-b39a-992b0880cf11 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.540936 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30282f-8120-4dc2-b962-8171656b63db None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.585700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.586723 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.587319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.587765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.596054 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:31.597912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:31.609901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:31.611101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 331/664] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:31.612136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.635778 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e76622-a5a3-414e-b39a-992b0880cf11 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.638421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:31.679046 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3fcbe-e63b-45ff-8739-78f6874b60fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.679733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3fcbe-e63b-45ff-8739-78f6874b60fd None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.680023 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3fcbe-e63b-45ff-8739-78f6874b60fd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.680310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3fcbe-e63b-45ff-8739-78f6874b60fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.691848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3fcbe-e63b-45ff-8739-78f6874b60fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-1ed3fcbe-e63b-45ff-8739-78f6874b60fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:31.697271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:31.698571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 334/665] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:31.699528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3fcbe-e63b-45ff-8739-78f6874b60fd None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.720234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:31.733636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7213f-2c72-44c8-9a34-bf8d884d9cfe None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.772019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.772650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.772930 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.773209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.774010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:31.774920 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:31.775813 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:31.799144 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 332/666] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:31.800136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.807904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3fcbe-e63b-45ff-8739-78f6874b60fd None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.848901 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.849622 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.849904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.850203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.861965 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:31.864055 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:31.880077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:31.881149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:31.882591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:31.882591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:31.882591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:31.882591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:31.886256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 335/667] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:31.887030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.907270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:31.912833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19864554-6dd2-49df-9fef-b60af710b6b7 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:31.949743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:31.950424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:31.950710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:31.950989 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:31.951793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:31.952104 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:31.953575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:31.973406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 333/668] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:31.973406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:31.985330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:31.999076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8eb2c-8d90-4636-b782-6b534db2af9f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.034411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.035058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.035319 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.035600 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.036433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:32.036848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.038454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:32.052113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 336/669] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:32.052113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:32.084468 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efc3e7-3cb2-48e7-9c63-23ac51141e56 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.121289 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c9606-2a7e-4b69-86ee-899f189648f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.121933 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c9606-2a7e-4b69-86ee-899f189648f1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.122174 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c9606-2a7e-4b69-86ee-899f189648f1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.122387 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c9606-2a7e-4b69-86ee-899f189648f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.130162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:32.152294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ada283-c35f-44b3-9450-2b9c82bbc147 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.186651 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.187283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.187566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.187841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.188539 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:32.188843 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.190298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:32.196849 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c9606-2a7e-4b69-86ee-899f189648f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.198675 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c9606-2a7e-4b69-86ee-899f189648f1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:32.202749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 337/670] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:32.203590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:32.210245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:32.307826 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 334/671] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:32.309051 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c9606-2a7e-4b69-86ee-899f189648f1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:32.309176 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a76888-f4cf-499f-9814-e24f5270f657 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.348373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.349010 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.349294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.349572 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.350411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=07d2ec2ce99a42cda7652878d2919f0a) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:32.350724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.352205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:32.359039 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 338/672] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:32 2022] GET /identity/v3/role_assignments?user.id=07d2ec2ce99a42cda7652878d2919f0a&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:32.359933 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:32.373430 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:32.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41a8cf-1b67-4d04-89e2-03afe8a9394a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.546696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.547721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.548195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.548796 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.552654 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c9606-2a7e-4b69-86ee-899f189648f1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.558373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.559951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:32.565223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:32.580077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:32.581290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:32.582563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:32.582563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:32.582563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:32.582563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:32.582563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:32.582563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:32.582563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:32.582563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:32.584872 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 339/673] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:32.585708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:32.597895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.598631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.598895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.599169 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.613707 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.615640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:32.633135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:32.634245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:32.635279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:32.635279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:32.635279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:32.635279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:32.635279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 335/674] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:32.635279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:32.681323 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:32.733807 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a843-43cd-4660-ad63-ab44ba0e0087 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.764818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9f06b-03a4-4a49-becd-d3e5c9be7476 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.781076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.782193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.782795 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.783296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.795252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.797113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:32.802753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.803396 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.803680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.803956 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.804839 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:32.805198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.807219 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:32.814992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:32.816127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:32.817144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:32.817144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:32.817144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:32.817144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:32.819491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 340/675] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:32.826166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 336/676] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:32.826166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:32.827397 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:32.849262 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:32.945155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9caf3-64a0-42a0-a976-2bf709a5cf80 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.965356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7888ebc-8c03-4faf-a249-c1fadfd9eabd None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:32.984461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:32.986313 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:32.987287 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:32.987785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:32.988301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:32.989267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:32.989801 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:32.991498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:33.006411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 337/677] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:33.007824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.008400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:33.009112 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.009398 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.009672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.010436 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:33.010957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:33.012658 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:33.037588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 341/678] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:33.037588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.113636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:33.143664 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73d20e-2a43-43ba-87e2-412ecb517283 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:33.174580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed5f60-314b-4b07-91ba-d31420e79539 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:33.185025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:33.185681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.185963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.186302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.187653 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=d24756347c554b45bfa30698306608be) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:33.187653 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:33.188988 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:33.202240 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 338/679] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/role_assignments?user.id=d24756347c554b45bfa30698306608be&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:33.202240 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.234931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b4dca-e444-4d32-a446-ab907cb5102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:33.236534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b4dca-e444-4d32-a446-ab907cb5102e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.237030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b4dca-e444-4d32-a446-ab907cb5102e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.237534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b4dca-e444-4d32-a446-ab907cb5102e None admin] PATH_INFO: `/v3/users/swift` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.247125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b4dca-e444-4d32-a446-ab907cb5102e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:33.249200 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b4dca-e444-4d32-a446-ab907cb5102e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-272b4dca-e444-4d32-a446-ab907cb5102e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:33.264089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:33.265306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:33.266400 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:33.269477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 342/680] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:33.271302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a959de-4fd0-4ca7-9c6b-0554503d14f5 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.271830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a959de-4fd0-4ca7-9c6b-0554503d14f5 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.272424 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a959de-4fd0-4ca7-9c6b-0554503d14f5 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.274499 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 343/681] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:33.275671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a959de-4fd0-4ca7-9c6b-0554503d14f5 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.317833 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:33.323135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8406d-2144-4ee3-b050-9ba99439873b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:33.363378 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6f9e1-e51d-467b-8797-3ffab5be544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:33.364098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6f9e1-e51d-467b-8797-3ffab5be544f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.364386 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6f9e1-e51d-467b-8797-3ffab5be544f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.364712 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6f9e1-e51d-467b-8797-3ffab5be544f None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.372899 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6f9e1-e51d-467b-8797-3ffab5be544f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:33.374602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6f9e1-e51d-467b-8797-3ffab5be544f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-7ae6f9e1-e51d-467b-8797-3ffab5be544f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:33.388118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:33.389236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:33.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:33.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:33.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:33.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:33.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:33.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:33.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:33.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:33.394385 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 339/682] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:33.396017 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c690b9e-7e21-4db2-aa92-809219ef6ab0 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.396165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c690b9e-7e21-4db2-aa92-809219ef6ab0 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.396415 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c690b9e-7e21-4db2-aa92-809219ef6ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.404313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a959de-4fd0-4ca7-9c6b-0554503d14f5 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:33.405603 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:33.420070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-9c690b9e-7e21-4db2-aa92-809219ef6ab0 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:33.421314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-9c690b9e-7e21-4db2-aa92-809219ef6ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:33.451580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:33.453303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.453959 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.454545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.455549 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:33.455859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:33.457496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:33.479764 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 344/683] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:33.480646 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.501775 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 340/684] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:33.502682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c690b9e-7e21-4db2-aa92-809219ef6ab0 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.603688 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:33.621404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f144a-2ee6-4ae0-925f-33dbffa71b16 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:33.623854 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c690b9e-7e21-4db2-aa92-809219ef6ab0 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:33.684928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:33.686321 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.686928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.687376 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bea01-bb5b-4441-9fed-2a853016d230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:33.687509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.688332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bea01-bb5b-4441-9fed-2a853016d230 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.688776 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bea01-bb5b-4441-9fed-2a853016d230 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.688937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:33.689267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bea01-bb5b-4441-9fed-2a853016d230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.689434 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:33.692273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:33.693633 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:33.724342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 345/685] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:33.726092 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.804886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bea01-bb5b-4441-9fed-2a853016d230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:33.806864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bea01-bb5b-4441-9fed-2a853016d230 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:33.854135 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01337cd0-6b0f-4e0f-a575-17d3bf6022fd None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:33.898584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a847800-3432-4e0e-8c6e-4aca84bd7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:33.899500 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a847800-3432-4e0e-8c6e-4aca84bd7765 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:33.899787 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a847800-3432-4e0e-8c6e-4aca84bd7765 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:33.900073 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a847800-3432-4e0e-8c6e-4aca84bd7765 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:33.910611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 341/686] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:33.912003 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bea01-bb5b-4441-9fed-2a853016d230 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.913728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a847800-3432-4e0e-8c6e-4aca84bd7765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-9a847800-3432-4e0e-8c6e-4aca84bd7765 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:33.920445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:33.921520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:33.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:33.926201 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 346/687] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:33.927558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a847800-3432-4e0e-8c6e-4aca84bd7765 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:33.950913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:33.998656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:34.040522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bea01-bb5b-4441-9fed-2a853016d230 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:34.060260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a847800-3432-4e0e-8c6e-4aca84bd7765 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:34.087498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:34.088653 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.089466 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.090186 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.091177 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=05afd25e30764af0819436cc91dd85c5) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:34.091917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:34.094379 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:34.097431 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:34.098258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.098619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.098992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.102788 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 342/688] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/role_assignments?user.id=05afd25e30764af0819436cc91dd85c5&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:34.103889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:34.111837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:34.113605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:34.130555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:34.131893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:34.133076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:34.133076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:34.133076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:34.133076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:34.134253 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 347/689] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:34.146167 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:34.299802 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:34.300432 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044521e-b58b-4215-b99f-fd5367a78e58 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:34.330608 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:34.342402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:34.343060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.343344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.343622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.344372 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:34.344687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:34.347406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:34.359911 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 343/690] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:34.361124 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89f07f-4af6-4592-8235-d6366e9a2d2e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:34.365035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:34.406405 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:34.407242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.407918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.408275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.409529 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:34.409917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:34.411793 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:34.430202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 348/691] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:34.436424 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:34.517053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918804c-13f3-490c-9476-cadb9fb9c8f0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:34.566990 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:34.567672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.567870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.568147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.595635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d5b50-a292-4239-aed5-e2a6446dd504 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:34.599307 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:34.600812 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:34.628145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:34.633681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:34.635296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:34.636561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:34.636561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:34.636561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:34.636561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:34.639589 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 344/692] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:34.640978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:34.655827 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:34.656888 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.657452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.657979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.674139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:34.675246 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:34.679578 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:34.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:34.700736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:34.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:34.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:34.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:34.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:34.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:34.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:34.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:34.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:34.707117 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 349/693] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:34.712465 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:34.816888 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67160335-f810-44c7-9dcd-b5a0e27d4c54 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:34.852916 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b3e58-ef5f-4252-8eeb-f55c4743bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:34.853463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3e58-ef5f-4252-8eeb-f55c4743bc68 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.853743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3e58-ef5f-4252-8eeb-f55c4743bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.854020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3e58-ef5f-4252-8eeb-f55c4743bc68 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.854774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b3e58-ef5f-4252-8eeb-f55c4743bc68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:34.855079 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b3e58-ef5f-4252-8eeb-f55c4743bc68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:34.856472 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b3e58-ef5f-4252-8eeb-f55c4743bc68 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:34.865254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e93bc9-c1c9-452e-88a4-bb932f07a04e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:34.869288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 345/694] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:34.870773 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a1b1f-cdbe-4c5f-9541-132357f9e170 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.871054 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a1b1f-cdbe-4c5f-9541-132357f9e170 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.871330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a1b1f-cdbe-4c5f-9541-132357f9e170 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.876863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 346/695] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:34.877720 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a1b1f-cdbe-4c5f-9541-132357f9e170 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:34.907321 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe2d03-f87c-4d04-9470-ec217c756b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:34.908246 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe2d03-f87c-4d04-9470-ec217c756b05 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.908835 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe2d03-f87c-4d04-9470-ec217c756b05 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.909114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe2d03-f87c-4d04-9470-ec217c756b05 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.910330 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe2d03-f87c-4d04-9470-ec217c756b05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:34.910779 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe2d03-f87c-4d04-9470-ec217c756b05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:34.912798 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe2d03-f87c-4d04-9470-ec217c756b05 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:34.930212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 350/696] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:34.931543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55057a-d458-4637-9284-a323668d7026 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:34.931878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55057a-d458-4637-9284-a323668d7026 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:34.932155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55057a-d458-4637-9284-a323668d7026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:34.951266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-3a55057a-d458-4637-9284-a323668d7026 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:34.952674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-3a55057a-d458-4637-9284-a323668d7026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:34.962214 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:35.001532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a1b1f-cdbe-4c5f-9541-132357f9e170 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:35.001875 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:35.038026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 351/697] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:35.039144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55057a-d458-4637-9284-a323668d7026 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:35.041283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:35.042147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.042449 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.042726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] PATH_INFO: `/v3/users/nova` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.051858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:35.053333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:35.072627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:35.073754 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:35.074870 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:35.075466 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 347/698] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:14:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:35.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:35.171639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55057a-d458-4637-9284-a323668d7026 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:35.207049 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8de3-83cf-47cf-aa50-b57fbb043153 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:35.208970 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:35.209638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.209921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.210227 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.211334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=da81aff79a9a467da6d8bf372da9faf2) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:35.211470 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:35.212877 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:35.226133 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 352/699] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:35 2022] GET /identity/v3/role_assignments?user.id=da81aff79a9a467da6d8bf372da9faf2&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:35.226133 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:35.269841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:35.280063 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:35.280963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.282741 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.283025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.306384 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:35.309387 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:35.312735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:35.342454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:35.343529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:35.344575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:35.344575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:35.344575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:35.344575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:35.344575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 348/700] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:35.356573 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:35.408977 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f4355-21af-4259-8506-a564390df631 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:35.459729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:35.460396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.460746 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.461099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.462081 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:35.462483 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:35.464108 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:35.496138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 353/701] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:14:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:35.501443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:35.530249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5529-fe53-4617-b61f-4cd6d1953479 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:35.582149 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0520d9-ea3b-4c52-89a2-63f522090998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:35.582759 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0520d9-ea3b-4c52-89a2-63f522090998 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.583036 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0520d9-ea3b-4c52-89a2-63f522090998 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.583305 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0520d9-ea3b-4c52-89a2-63f522090998 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.584069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0520d9-ea3b-4c52-89a2-63f522090998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:35.584375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0520d9-ea3b-4c52-89a2-63f522090998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:35.585777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0520d9-ea3b-4c52-89a2-63f522090998 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:35.601647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 349/702] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:35.601647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27d994-b4e9-471a-8bc3-68f61afb425e None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.601647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27d994-b4e9-471a-8bc3-68f61afb425e None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.601647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27d994-b4e9-471a-8bc3-68f61afb425e None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.602089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 350/703] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:35.607487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6ad1-36f5-4316-8f08-8d2044826ebc None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.607864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6ad1-36f5-4316-8f08-8d2044826ebc None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.608234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6ad1-36f5-4316-8f08-8d2044826ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.626637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:35.628230 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-f3fb6ad1-36f5-4316-8f08-8d2044826ebc None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:35.629024 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f3fb6ad1-36f5-4316-8f08-8d2044826ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:35.635854 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a4d95-7a20-4b4d-bdd0-b5556c235370 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:35.674473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aed636-f820-402e-935a-fbe4ab4980c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:35.675165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aed636-f820-402e-935a-fbe4ab4980c0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.675455 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aed636-f820-402e-935a-fbe4ab4980c0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.675737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aed636-f820-402e-935a-fbe4ab4980c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.676441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:35.704480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 351/704] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:35.711043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb6ad1-36f5-4316-8f08-8d2044826ebc None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:35.762241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aed636-f820-402e-935a-fbe4ab4980c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:35.764159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aed636-f820-402e-935a-fbe4ab4980c0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:35.838190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb6ad1-36f5-4316-8f08-8d2044826ebc None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:35.880167 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b2230-5ea5-4a6f-a621-efc3ca92a8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:35.880819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:35.882657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b2230-5ea5-4a6f-a621-efc3ca92a8b9 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:35.883159 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b2230-5ea5-4a6f-a621-efc3ca92a8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:35.883711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b2230-5ea5-4a6f-a621-efc3ca92a8b9 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:35.884624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b2230-5ea5-4a6f-a621-efc3ca92a8b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:35.886651 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b2230-5ea5-4a6f-a621-efc3ca92a8b9 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:35.888168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 354/705] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:14:35.889311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aed636-f820-402e-935a-fbe4ab4980c0 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:35.918413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 352/706] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:14:35 2022] POST /identity/v3/users => generated 254 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:35.919700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b2230-5ea5-4a6f-a621-efc3ca92a8b9 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:35.978074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:36.008243 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aed636-f820-402e-935a-fbe4ab4980c0 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:36.048707 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:36.049537 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.049883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.050267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.051003 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:36.052659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:36.065480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:36.066967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:36.068392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 355/707] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:14:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:36.076620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:36.085642 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b2230-5ea5-4a6f-a621-efc3ca92a8b9 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:36.131854 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:36.133212 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.134555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.135186 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.164052 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:36.166791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:36.188340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:36.189666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:36.190958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:36.190958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:36.190958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:36.190958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:36.192918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 353/708] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:36.198921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:36.199979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:36.249627 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770be0ad-4f21-44aa-9b2d-5a45636a478d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:36.287994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:36.288591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.288869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.289142 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.289870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:36.290202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:36.292377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:36.297073 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 356/709] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:14:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:36.302806 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:36.306797 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:36.337915 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba7209-4f79-45e1-a4dd-48e5820a3f48 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:36.384901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:36.386127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.386585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.387023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.388048 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:36.389492 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:36.391346 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:36.408164 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 354/710] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:36.413371 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:36.439472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25a611-963b-4ba0-adbb-525f9d56a1e4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:36.473334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:36.492205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074872-e6a7-49b8-b613-ea76a2a8eb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:36.493122 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074872-e6a7-49b8-b613-ea76a2a8eb16 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.493984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074872-e6a7-49b8-b613-ea76a2a8eb16 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.494387 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074872-e6a7-49b8-b613-ea76a2a8eb16 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.496061 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074872-e6a7-49b8-b613-ea76a2a8eb16 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:36.496386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074872-e6a7-49b8-b613-ea76a2a8eb16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:36.498680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074872-e6a7-49b8-b613-ea76a2a8eb16 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:36.509899 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 357/711] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:14:36 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:36.514141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e037bd-bbc5-4e7b-a5a1-6216f89fcca8 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.514141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e037bd-bbc5-4e7b-a5a1-6216f89fcca8 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.514141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e037bd-bbc5-4e7b-a5a1-6216f89fcca8 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.514141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 358/712] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:36.520665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196966-9fd7-42c7-b1a0-13c52a932317 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.520665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196966-9fd7-42c7-b1a0-13c52a932317 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.520665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196966-9fd7-42c7-b1a0-13c52a932317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.558066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-b4196966-9fd7-42c7-b1a0-13c52a932317 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:36.561981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-b4196966-9fd7-42c7-b1a0-13c52a932317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:36.580130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd81c1f-a6d9-41fa-a2a0-e113c86fcea1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:36.633436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:36.634097 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.634409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.634693 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.642859 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:36.644356 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:36.657176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:36.658391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:36.659391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 355/713] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:36.664671 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:36.670257 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:36.716721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 359/714] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:36.739826 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4196966-9fd7-42c7-b1a0-13c52a932317 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:36.812409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66054e5d-ece5-4914-bc20-52990db29ceb None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:36.859735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:36.860386 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:36.860740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:36.861043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:36.861824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:36.862151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:36.863664 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:36.896955 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 356/715] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:36.902398 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4196966-9fd7-42c7-b1a0-13c52a932317 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:36.904737 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:36.959446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:36.998609 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:37.026156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048a334-d7dd-4e38-9a57-c408ad86e29a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:37.050899 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:37.051592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:37.051880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:37.052158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] PATH_INFO: `/v3/services/network` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:37.052816 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:37.054281 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:37.064515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:37.065069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:37.065347 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:37.065621 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:37.066470 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=45cd9a93c342491889ca8260bc4510c6) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:37.066777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:37.068764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:37.075811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 357/716] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:36 2022] GET /identity/v3/role_assignments?user.id=45cd9a93c342491889ca8260bc4510c6&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=40504e80dbb845aeb07ca258977ecacb => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:37.081099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:37.082265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:37.083695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:37.091042 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 360/717] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:36 2022] GET /identity/v3/services/network => generated 88 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:37.099357 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:37.257255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f57ed-6e9f-40e0-80d8-7ae30be0ae18 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:37.290475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:37.294003 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb3c4-cd03-4c10-8693-8fe54fda5909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:37.294751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb3c4-cd03-4c10-8693-8fe54fda5909 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:37.295464 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb3c4-cd03-4c10-8693-8fe54fda5909 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:37.296017 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb3c4-cd03-4c10-8693-8fe54fda5909 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:37.297054 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfb3c4-cd03-4c10-8693-8fe54fda5909 None admin] RBAC: Adding query filter params (name=network) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:37.297776 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfb3c4-cd03-4c10-8693-8fe54fda5909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:37.299268 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfb3c4-cd03-4c10-8693-8fe54fda5909 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:37.305596 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 358/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:14:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:37.315381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:37.472388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a050-1310-4ca6-bf03-02239a531451 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:37.542922 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc69e0-2fbd-4f8f-ba36-7ac797763202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:37.543583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc69e0-2fbd-4f8f-ba36-7ac797763202 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:37.543866 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc69e0-2fbd-4f8f-ba36-7ac797763202 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:37.544143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc69e0-2fbd-4f8f-ba36-7ac797763202 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:37.544868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc69e0-2fbd-4f8f-ba36-7ac797763202 None admin] RBAC: Adding query filter params (type=network) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:37.545176 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc69e0-2fbd-4f8f-ba36-7ac797763202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:37.546648 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc69e0-2fbd-4f8f-ba36-7ac797763202 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:37.552622 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 361/719] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:14:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:37.970947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71366e2a-9dee-4286-b052-1161b43f36fe None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:37.971499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71366e2a-9dee-4286-b052-1161b43f36fe None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:37.971499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71366e2a-9dee-4286-b052-1161b43f36fe None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:37.973345 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 359/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:37.980548 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b1fd5-2dee-4757-8fe3-cc8f05a52558 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:37.980894 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b1fd5-2dee-4757-8fe3-cc8f05a52558 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:37.981142 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b1fd5-2dee-4757-8fe3-cc8f05a52558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:38.003379 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-881b1fd5-2dee-4757-8fe3-cc8f05a52558 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:38.004778 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-881b1fd5-2dee-4757-8fe3-cc8f05a52558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:38.067845 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:38.113545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 362/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:38.163428 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71366e2a-9dee-4286-b052-1161b43f36fe None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:38.311906 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71366e2a-9dee-4286-b052-1161b43f36fe None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:38.355410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4df730-ec10-4dc3-9b1a-19158d42efb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:38.356077 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df730-ec10-4dc3-9b1a-19158d42efb8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:38.356357 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df730-ec10-4dc3-9b1a-19158d42efb8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:38.356628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df730-ec10-4dc3-9b1a-19158d42efb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:38.399681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4df730-ec10-4dc3-9b1a-19158d42efb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-8f4df730-ec10-4dc3-9b1a-19158d42efb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:38.409706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:38.410893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:38.412455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 360/722] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:38.417295 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b1fd5-2dee-4757-8fe3-cc8f05a52558 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:38.569350 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b1fd5-2dee-4757-8fe3-cc8f05a52558 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:38.630552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3687d3-d9c7-4d96-8c36-af4be170bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:38.631251 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3687d3-d9c7-4d96-8c36-af4be170bc22 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:38.631537 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3687d3-d9c7-4d96-8c36-af4be170bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:38.631813 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3687d3-d9c7-4d96-8c36-af4be170bc22 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:38.632571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3687d3-d9c7-4d96-8c36-af4be170bc22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:38.632878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3687d3-d9c7-4d96-8c36-af4be170bc22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:38.634345 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3687d3-d9c7-4d96-8c36-af4be170bc22 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:38.654663 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 363/723] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:38.656364 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4df730-ec10-4dc3-9b1a-19158d42efb8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:38.657304 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceda41-4133-4c1d-9261-7ffc84f00bc4 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:38.657657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceda41-4133-4c1d-9261-7ffc84f00bc4 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:38.657963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceda41-4133-4c1d-9261-7ffc84f00bc4 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:38.659922 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 364/724] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:38.666870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365822-138b-4db4-857c-5d78e223ce60 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:38.667217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365822-138b-4db4-857c-5d78e223ce60 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:38.667457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365822-138b-4db4-857c-5d78e223ce60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:38.707255 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-48365822-138b-4db4-857c-5d78e223ce60 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:38.708650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-48365822-138b-4db4-857c-5d78e223ce60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:38.780920 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:38.821390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4df730-ec10-4dc3-9b1a-19158d42efb8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:38.864761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:38.866686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:38.867687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:38.867868 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:38.868148 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:38.881756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:38.883551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:38.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:38.903766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:38.905701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:38.905701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:38.905701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:38.905701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:38.905701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 361/725] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:38.910519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:39.011313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 365/726] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:14:39.051970 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48365822-138b-4db4-857c-5d78e223ce60 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:39.066704 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6819b10-51ed-46b0-a7e3-ee9be7141319 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:39.068756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:39.183358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48365822-138b-4db4-857c-5d78e223ce60 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:39.191070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:39.192219 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.193006 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.193495 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.194457 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:39.194993 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:39.197131 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:39.225189 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c2d0e-8d2c-4f27-988d-64321c3d9d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:39.225903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c2d0e-8d2c-4f27-988d-64321c3d9d06 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.226240 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c2d0e-8d2c-4f27-988d-64321c3d9d06 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.226536 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c2d0e-8d2c-4f27-988d-64321c3d9d06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.236696 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 362/727] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:39.246596 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:39.254601 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c2d0e-8d2c-4f27-988d-64321c3d9d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-b37c2d0e-8d2c-4f27-988d-64321c3d9d06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:39.274619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:39.276721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:39.278556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:39.294466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 366/728] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:39.309896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c2d0e-8d2c-4f27-988d-64321c3d9d06 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:39.353252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:39.447052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c2d0e-8d2c-4f27-988d-64321c3d9d06 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:39.489168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277638f8-45dc-474c-b7ce-d18f33684056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:39.489829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277638f8-45dc-474c-b7ce-d18f33684056 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.490141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277638f8-45dc-474c-b7ce-d18f33684056 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.490441 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277638f8-45dc-474c-b7ce-d18f33684056 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.491209 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277638f8-45dc-474c-b7ce-d18f33684056 None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:39.491518 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277638f8-45dc-474c-b7ce-d18f33684056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:39.492966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277638f8-45dc-474c-b7ce-d18f33684056 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:39.511174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 367/729] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:39.512700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc53d0-65ff-44cc-89e0-62e6c3f1f79d None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.513019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc53d0-65ff-44cc-89e0-62e6c3f1f79d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.513269 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc53d0-65ff-44cc-89e0-62e6c3f1f79d None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.515101 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 368/730] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:39.516003 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc53d0-65ff-44cc-89e0-62e6c3f1f79d None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:39.535020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a0ae5-2311-4b87-aa79-9971129c485c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:39.605895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41900aea-b511-41df-8eae-b4def0918851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:39.607120 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41900aea-b511-41df-8eae-b4def0918851 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.607415 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41900aea-b511-41df-8eae-b4def0918851 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.607961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41900aea-b511-41df-8eae-b4def0918851 None admin] PATH_INFO: `/v3/users/swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.622216 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41900aea-b511-41df-8eae-b4def0918851 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:39.625268 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41900aea-b511-41df-8eae-b4def0918851 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:39.639574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-41900aea-b511-41df-8eae-b4def0918851 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:39.649665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:39.650855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:39.651959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 363/731] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:39.655321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf748-72c8-4b21-b083-e2dadad95630 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.655826 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf748-72c8-4b21-b083-e2dadad95630 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.656257 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf748-72c8-4b21-b083-e2dadad95630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.660933 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc53d0-65ff-44cc-89e0-62e6c3f1f79d None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:39.679307 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-e9cdf748-72c8-4b21-b083-e2dadad95630 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:39.680563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-e9cdf748-72c8-4b21-b083-e2dadad95630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:39.703580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:39.708911 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033cde6-ec53-414c-8b3c-7bcab77e5490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:39.710003 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033cde6-ec53-414c-8b3c-7bcab77e5490 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.710376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033cde6-ec53-414c-8b3c-7bcab77e5490 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.710664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033cde6-ec53-414c-8b3c-7bcab77e5490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.725430 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033cde6-ec53-414c-8b3c-7bcab77e5490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:39.727024 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033cde6-ec53-414c-8b3c-7bcab77e5490 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-9033cde6-ec53-414c-8b3c-7bcab77e5490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:39.752273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:39.753531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:39.754850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:39.754850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:39.754850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:39.754850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:39.758496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 369/732] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:39.759975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accf41b-6ff3-4c2b-864e-d76d95d7b8b2 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.760262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accf41b-6ff3-4c2b-864e-d76d95d7b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.760530 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accf41b-6ff3-4c2b-864e-d76d95d7b8b2 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.762261 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 370/733] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:39.762751 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accf41b-6ff3-4c2b-864e-d76d95d7b8b2 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:39.791065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 364/734] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:39.793034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf748-72c8-4b21-b083-e2dadad95630 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:39.933951 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf748-72c8-4b21-b083-e2dadad95630 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:39.940649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accf41b-6ff3-4c2b-864e-d76d95d7b8b2 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:39.986038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022e99a-c8d4-4b8c-aafd-2a0c7080917d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:39.986616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e99a-c8d4-4b8c-aafd-2a0c7080917d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.986905 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e99a-c8d4-4b8c-aafd-2a0c7080917d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.987177 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e99a-c8d4-4b8c-aafd-2a0c7080917d None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.987878 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022e99a-c8d4-4b8c-aafd-2a0c7080917d None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:39.988180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022e99a-c8d4-4b8c-aafd-2a0c7080917d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:39.989606 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022e99a-c8d4-4b8c-aafd-2a0c7080917d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:39.995946 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:39.996836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:39.997314 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:39.997755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:39.999020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:39.999744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:40.002137 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:40.004980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 365/735] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:40.006303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b808f2-4c61-46fd-9db9-32dd041721bd None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:40.006597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b808f2-4c61-46fd-9db9-32dd041721bd None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:40.006887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b808f2-4c61-46fd-9db9-32dd041721bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:40.025876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:40.031390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d4b808f2-4c61-46fd-9db9-32dd041721bd None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:40.032688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d4b808f2-4c61-46fd-9db9-32dd041721bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:40.036574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 371/736] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:14:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:40.037873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:40.045511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:40.138563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 366/737] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:40.143895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b808f2-4c61-46fd-9db9-32dd041721bd None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:40.207118 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0191c-e9ff-413b-95ed-b2c29a46d505 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:40.258143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d93959-d407-4cab-8de7-186e3c106a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:40.259719 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93959-d407-4cab-8de7-186e3c106a51 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:40.260529 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93959-d407-4cab-8de7-186e3c106a51 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:40.261271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93959-d407-4cab-8de7-186e3c106a51 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:40.262895 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d93959-d407-4cab-8de7-186e3c106a51 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:40.265763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d93959-d407-4cab-8de7-186e3c106a51 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:40.295371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 372/738] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:14:40 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:40.297448 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d93959-d407-4cab-8de7-186e3c106a51 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:40.324280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b808f2-4c61-46fd-9db9-32dd041721bd None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:40.389132 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:40.389896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:40.390258 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:40.390580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:40.402144 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:40.404118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:40.406765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:40.421644 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:40.428572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:40.429962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:14:40.431593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:40.438477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 367/739] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:14:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:40.440137 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:40.495940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d93959-d407-4cab-8de7-186e3c106a51 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:40.548195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090b431-7310-46f0-a549-b4dd12dcc0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:40.550985 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090b431-7310-46f0-a549-b4dd12dcc0a6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:40.551307 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090b431-7310-46f0-a549-b4dd12dcc0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:40.551583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090b431-7310-46f0-a549-b4dd12dcc0a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:40.633881 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9050a-754e-47a9-9197-af1f954bd689 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:40.659622 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090b431-7310-46f0-a549-b4dd12dcc0a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:40.661812 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090b431-7310-46f0-a549-b4dd12dcc0a6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:40.696600 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a9c96-0753-4b70-a396-0650dc5f34ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:40.697868 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a9c96-0753-4b70-a396-0650dc5f34ee None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:40.698378 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a9c96-0753-4b70-a396-0650dc5f34ee None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:40.698886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a9c96-0753-4b70-a396-0650dc5f34ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:40.718512 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a9c96-0753-4b70-a396-0650dc5f34ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-ec6a9c96-0753-4b70-a396-0650dc5f34ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:40.727188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:40.729198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:40.731540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 368/740] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:40.732999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19602-bc3e-4391-a456-a142acbf3047 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:40.732999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19602-bc3e-4391-a456-a142acbf3047 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:40.732999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19602-bc3e-4391-a456-a142acbf3047 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:40.738258 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 369/741] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:40.738258 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19602-bc3e-4391-a456-a142acbf3047 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:40.852020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:40.929191 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19602-bc3e-4391-a456-a142acbf3047 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:40.953357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:40.994419 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:40.994947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:40.995261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:40.995518 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:40.996344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:40.996652 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:40.998322 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:41.021278 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 370/742] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:14:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:41.022792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:41.023145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 373/743] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:40 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:41.025288 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091177c-d7ea-4c4b-af32-29edf8852227 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:41.025982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091177c-d7ea-4c4b-af32-29edf8852227 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:41.026557 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091177c-d7ea-4c4b-af32-29edf8852227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:41.054011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-9091177c-d7ea-4c4b-af32-29edf8852227 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:41.055311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-9091177c-d7ea-4c4b-af32-29edf8852227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:41.142184 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 374/744] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:14:41.142184 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091177c-d7ea-4c4b-af32-29edf8852227 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:41.243232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294045de-cd2d-43eb-b60d-3e5e5461798a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:41.253213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:41.267509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091177c-d7ea-4c4b-af32-29edf8852227 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:41.275164 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:41.308657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:41.309803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:41.310275 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:41.310755 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:41.312181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:41.312733 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:41.314380 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768751cc-a820-41e6-bef7-b6659d2490f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:41.315159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768751cc-a820-41e6-bef7-b6659d2490f7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:41.315539 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768751cc-a820-41e6-bef7-b6659d2490f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:41.315687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:41.316005 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768751cc-a820-41e6-bef7-b6659d2490f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:41.332674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 371/745] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:41.333937 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:41.428318 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768751cc-a820-41e6-bef7-b6659d2490f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:41.429826 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768751cc-a820-41e6-bef7-b6659d2490f7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:41.492654 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4f1ce-5b5d-4046-aea5-1d8d8f7e7907 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:41.536099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:41.537342 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:41.537718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:41.538065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:41.551503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:41.552999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:41.560786 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:41.572974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:41.574963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:41.576692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:41.576692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:41.576692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:41.576692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:41.577175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 375/746] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:41 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:14:41.577175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768751cc-a820-41e6-bef7-b6659d2490f7 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:41.579356 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 372/747] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:41.580264 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:41.634584 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:41.728431 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768751cc-a820-41e6-bef7-b6659d2490f7 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:41.768186 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c222704-7e63-4c17-8beb-554b930cdc7c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:41.798319 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1277c-9038-48d2-96ac-6b9b48ea06c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:41.800103 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1277c-9038-48d2-96ac-6b9b48ea06c7 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:41.800397 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1277c-9038-48d2-96ac-6b9b48ea06c7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:41.800826 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1277c-9038-48d2-96ac-6b9b48ea06c7 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:41.801433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1277c-9038-48d2-96ac-6b9b48ea06c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:41.802942 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1277c-9038-48d2-96ac-6b9b48ea06c7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:41.819926 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:41.820592 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:41.820886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:41.821540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:41.822533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 376/748] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:14:41 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:41.823411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1277c-9038-48d2-96ac-6b9b48ea06c7 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:41.855934 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:41.859834 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:41.894085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:41.896382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:41.898802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:41.898802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:41.898802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:41.898802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:41.900621 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 373/749] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:41.902827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:41.945465 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:42.048401 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1277c-9038-48d2-96ac-6b9b48ea06c7 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.057642 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:42.087483 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65dc9f-5a10-454a-805a-fe948720721f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.095256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:42.095920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:42.096202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:42.096512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:42.110667 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:42.113193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:42.134708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:42.136306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:42.137580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:42.137580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:42.137580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:42.137580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:42.138861 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 377/750] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:42.139864 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:42.147563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:42.148423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:42.148829 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:42.149363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:42.150034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:42.150465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:42.152294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:42.166330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 374/751] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:42.171252 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:42.300794 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:42.309017 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fd25a-2e55-4b80-a19d-d83c710be305 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.335974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a6b51-e052-4955-8368-d607b0327154 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.359697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:42.360521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:42.360819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:42.361136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:42.362370 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:42.362722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:42.364518 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:42.384046 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 378/752] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:42.385415 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:42.406009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:42.406702 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:42.406988 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:42.407533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:42.408565 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:42.408565 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:42.410007 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:42.423746 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 375/753] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:42.424807 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:42.428457 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:42.557942 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bda77-5312-4fce-aa87-cd0809180106 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.592544 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1af4a3-d9ea-44e9-9bab-e2756a37cec0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.605302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:42.605981 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:42.606316 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:42.606592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:42.614831 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:42.616365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:42.629772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:42.630900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:42.631913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 379/754] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:42.632960 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:42.659033 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:42.676793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:42.678383 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:42.679095 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:42.679800 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] PATH_INFO: `/v3/users/nova` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:42.694949 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:42.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:42.723699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:42.725692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:42.728491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:42.733016 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 376/755] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:14:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:42.734995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:42.760727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e2854-2a49-4e03-a71d-f3b7ad38ed54 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.802277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:42.802965 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:42.803249 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:42.803527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:42.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:42.813774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:42.827497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:42.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:42.831300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:42.831300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:42.831300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:42.831300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:42.831300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:42.831300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:42.831300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:42.831300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:42.833856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 380/756] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:42.834922 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:42.870998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:42.923478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:42.947225 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d5bde-4015-4e15-9cc5-cf6c48cccfcc None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.947525 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33fec5-904a-48b4-a10e-0c3b27e92a72 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:42.989459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:42.990755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:42.990755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:42.990755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:42.991623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:42.991933 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:42.993453 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.012916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 381/757] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:14:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:43.013920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:43.019426 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:43.020879 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.021661 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.022494 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.024334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:43.025355 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:43.028447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.055387 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 377/758] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:43.057619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:43.129464 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c162a2-506b-47f5-8841-0be5aaf15e53 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:43.168179 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:43.168852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.169135 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.169452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.170225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:43.170530 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:43.172122 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.175405 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:43.192383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 382/759] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:43.194380 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:43.234356 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3511d44-d24b-427d-8170-e5c737facec9 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:43.259903 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:43.273550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:43.274263 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.274548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.274824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.286508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:43.288070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:43.303718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:43.304892 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:43.306023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:43.306023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:43.306023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:43.306023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:43.306023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 378/760] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:43.306023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:43.311742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf27ff-40df-48f5-8e92-beb9dc4f2f62 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:43.357540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2aa59-d73e-4222-a732-dcc7c8767ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:43.358198 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2aa59-d73e-4222-a732-dcc7c8767ec2 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.358521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2aa59-d73e-4222-a732-dcc7c8767ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.358729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2aa59-d73e-4222-a732-dcc7c8767ec2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.359535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2aa59-d73e-4222-a732-dcc7c8767ec2 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=da81aff79a9a467da6d8bf372da9faf2) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:43.359928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2aa59-d73e-4222-a732-dcc7c8767ec2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:43.361420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2aa59-d73e-4222-a732-dcc7c8767ec2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.367478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 383/761] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/role_assignments?user.id=da81aff79a9a467da6d8bf372da9faf2&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=40504e80dbb845aeb07ca258977ecacb => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:43.368508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c13c2-5032-4cf7-9fb2-f94da58dc363 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.368784 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c13c2-5032-4cf7-9fb2-f94da58dc363 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.369057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c13c2-5032-4cf7-9fb2-f94da58dc363 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.374737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 384/762] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:43.374737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c13c2-5032-4cf7-9fb2-f94da58dc363 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:43.454422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735198e-33cb-456f-b742-37bae05388fb None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:43.459223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:43.494611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d72057-fc67-48b5-ba74-7fa0dc27e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:43.495268 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72057-fc67-48b5-ba74-7fa0dc27e28b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.495549 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72057-fc67-48b5-ba74-7fa0dc27e28b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.496342 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72057-fc67-48b5-ba74-7fa0dc27e28b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.496811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d72057-fc67-48b5-ba74-7fa0dc27e28b None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=74f4cb06ce4548aea8e61a57c984fe70) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:43.497149 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d72057-fc67-48b5-ba74-7fa0dc27e28b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:43.498721 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d72057-fc67-48b5-ba74-7fa0dc27e28b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.513293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 379/763] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/role_assignments?user.id=74f4cb06ce4548aea8e61a57c984fe70&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:43.514991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c518a06-706b-4343-997b-8134f2e7835d None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.515282 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c518a06-706b-4343-997b-8134f2e7835d None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.515562 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c518a06-706b-4343-997b-8134f2e7835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.522989 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c13c2-5032-4cf7-9fb2-f94da58dc363 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:43.546534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-7c518a06-706b-4343-997b-8134f2e7835d None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:43.547848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-7c518a06-706b-4343-997b-8134f2e7835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:43.576149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:43.577438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.578056 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.579936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:43.581097 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:43.583433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.588552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:43.603491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 385/764] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:43.608955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:43.645724 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 380/765] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:43.676290 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c518a06-706b-4343-997b-8134f2e7835d None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:43.810568 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c518a06-706b-4343-997b-8134f2e7835d None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:43.853975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adac94-8787-4f70-ad00-b334926a9d43 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:43.868418 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:43.869920 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.870361 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.871090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.872301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:43.875089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.882401 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:43.902136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:43.903294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:43.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 381/766] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:43.911426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:43.912374 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:43.912836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:43.913179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:43.913443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:43.935834 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:43.939383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:43.967867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:43.969968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:43.972222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:43.972222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:43.972222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:43.972222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:43.972222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:43.972222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:43.972222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:43.972222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:43.975113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 386/767] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:43.980060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:43.981949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:44.162213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100a76-f5e8-4f7a-822e-85d3824aa509 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:44.230948 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:44.237677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:44.238542 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:44.239276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:44.240798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:44.241520 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:44.246962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:44.267243 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 382/768] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:44.275077 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:44.286074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9c80a-34f8-4693-8278-706ed3d36dec None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:44.382842 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:44.383542 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:44.384546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:44.384546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:44.384883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:44.385158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:44.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:44.400304 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 387/769] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:44.406240 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:44.470813 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:44.471800 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:44.485873 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330e0e7-3612-4d35-afa8-f05d54920069 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:44.535541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6edb4-5480-46ff-9bd8-c73cd248acda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:44.536283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6edb4-5480-46ff-9bd8-c73cd248acda None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:44.536667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6edb4-5480-46ff-9bd8-c73cd248acda None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:44.536955 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6edb4-5480-46ff-9bd8-c73cd248acda None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:44.537631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6edb4-5480-46ff-9bd8-c73cd248acda None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:44.537935 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6edb4-5480-46ff-9bd8-c73cd248acda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:44.539632 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6edb4-5480-46ff-9bd8-c73cd248acda None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:44.542521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d006e26-4335-4ec6-afcb-b6f5eee4d97a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:44.546697 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 383/770] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:14:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:44.548136 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a82d-a5a4-4d43-b521-1f6d5781b8eb None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:44.548443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a82d-a5a4-4d43-b521-1f6d5781b8eb None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:44.548717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a82d-a5a4-4d43-b521-1f6d5781b8eb None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:44.554508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 384/771] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:44.555552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a82d-a5a4-4d43-b521-1f6d5781b8eb None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:44.581712 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae9e48-854b-4f42-b460-8d2a4daedaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:44.583084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae9e48-854b-4f42-b460-8d2a4daedaef None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:44.583084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae9e48-854b-4f42-b460-8d2a4daedaef None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:44.583084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae9e48-854b-4f42-b460-8d2a4daedaef None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/45cd9a93c342491889ca8260bc4510c6/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:44.585247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae9e48-854b-4f42-b460-8d2a4daedaef None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=45cd9a93c342491889ca8260bc4510c6, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:44.586072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae9e48-854b-4f42-b460-8d2a4daedaef None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:44.603865 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 388/772] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:14:44 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/45cd9a93c342491889ca8260bc4510c6/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:44.609532 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dac85-686c-4745-bb19-84b08bf73e19 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:44.609863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dac85-686c-4745-bb19-84b08bf73e19 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:44.610056 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dac85-686c-4745-bb19-84b08bf73e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:44.645149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-833dac85-686c-4745-bb19-84b08bf73e19 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:44.646457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-833dac85-686c-4745-bb19-84b08bf73e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:44.767745 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 389/773] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:44.793656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a82d-a5a4-4d43-b521-1f6d5781b8eb None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:44.818917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dac85-686c-4745-bb19-84b08bf73e19 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:44.869285 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:44.870459 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:44.871008 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:44.871523 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:44.873513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] RBAC: Adding query filter params (interface=public, service_id=0778e458fab04b9d8a2bd825d64419e1, region_id=RegionOne) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:44.874093 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:44.877063 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:44.881011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:44.894418 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 385/774] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:14:44 2022] GET /identity/v3/endpoints?service_id=0778e458fab04b9d8a2bd825d64419e1&interface=public®ion_id=RegionOne => generated 194 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:44.904281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:44.963416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:44.986700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dac85-686c-4745-bb19-84b08bf73e19 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:45.075472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:45.076553 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:45.077087 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:45.077587 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] PATH_INFO: `/v3/services/network` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:45.078477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:45.080203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:45.096860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:45.098052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:45.099072 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:45.107110 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292bc86-ff13-4416-ac49-5108d60094ab None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:45.111180 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 390/775] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:44 2022] GET /identity/v3/services/network => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:45.119543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:45.169101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf06ad2-880a-4a3b-8ac4-402bf4c472b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:45.170244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf06ad2-880a-4a3b-8ac4-402bf4c472b5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:45.170765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf06ad2-880a-4a3b-8ac4-402bf4c472b5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:45.171278 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf06ad2-880a-4a3b-8ac4-402bf4c472b5 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:45.172595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf06ad2-880a-4a3b-8ac4-402bf4c472b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:45.174395 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf06ad2-880a-4a3b-8ac4-402bf4c472b5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:45.182466 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 386/776] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:14:44 2022] GET /identity/v3/services => generated 772 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:45.323051 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:45.337880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20da882-5788-400e-a650-64baa57813d7 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:45.400976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:45.401723 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:45.402016 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:45.402334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:45.403153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] RBAC: Adding query filter params (name=network) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:45.403472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:45.405289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:45.413039 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 391/777] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:14:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:45.418963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf06ad2-880a-4a3b-8ac4-402bf4c472b5 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:45.539121 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:45.555853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf06ad2-880a-4a3b-8ac4-402bf4c472b5 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:45.599152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:45.599697 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:45.599968 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:45.600252 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:45.600960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] RBAC: Adding query filter params (type=network) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:45.601272 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:45.602723 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:45.608810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 387/778] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:14:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:45.614679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:45.770377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907638c0-0189-4c24-8d6f-f750f94e40a5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:45.856657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8efe4-120e-47f6-b83b-94f5efdb17fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:45.857676 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8efe4-120e-47f6-b83b-94f5efdb17fc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:45.858297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8efe4-120e-47f6-b83b-94f5efdb17fc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:45.858761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8efe4-120e-47f6-b83b-94f5efdb17fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:45.860381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8efe4-120e-47f6-b83b-94f5efdb17fc None admin] RBAC: Adding query filter params (interface=public, service_id=fb69dc388dba4929b7654051d50d9cc4, region_id=RegionOne) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:45.861105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8efe4-120e-47f6-b83b-94f5efdb17fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:45.863906 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8efe4-120e-47f6-b83b-94f5efdb17fc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:45.888376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 392/779] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:14:45 2022] GET /identity/v3/endpoints?service_id=fb69dc388dba4929b7654051d50d9cc4&interface=public®ion_id=RegionOne => generated 194 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:45.894342 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:45.916603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224dd14-ae28-4325-91bb-24523c082ba7 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:45.917189 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224dd14-ae28-4325-91bb-24523c082ba7 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:45.917686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224dd14-ae28-4325-91bb-24523c082ba7 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:45.920735 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 393/780] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:45.925975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c7615-318e-455d-9131-e93be163bde3 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:45.926322 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c7615-318e-455d-9131-e93be163bde3 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:45.926608 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c7615-318e-455d-9131-e93be163bde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:45.946795 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-f14c7615-318e-455d-9131-e93be163bde3 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:45.948036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-f14c7615-318e-455d-9131-e93be163bde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:45.986500 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:46.075465 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 394/781] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:14:46.100399 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c7615-318e-455d-9131-e93be163bde3 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:46.159800 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e4a04-c126-4d3a-ad99-fd5b9bbbc74a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:46.210962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c60ca-717c-4402-92bd-51e2e27e0eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:46.211616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c60ca-717c-4402-92bd-51e2e27e0eef None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:46.211887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c60ca-717c-4402-92bd-51e2e27e0eef None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:46.212187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c60ca-717c-4402-92bd-51e2e27e0eef None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:46.212815 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c60ca-717c-4402-92bd-51e2e27e0eef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:46.214284 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c60ca-717c-4402-92bd-51e2e27e0eef None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:46.221263 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 388/782] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:14:45 2022] GET /identity/v3/services => generated 772 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:46.298404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c7615-318e-455d-9131-e93be163bde3 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:46.353868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ca6eb-3191-4edd-b1fd-c40bef197998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:46.354597 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ca6eb-3191-4edd-b1fd-c40bef197998 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:46.354888 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ca6eb-3191-4edd-b1fd-c40bef197998 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:46.355169 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ca6eb-3191-4edd-b1fd-c40bef197998 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:46.369808 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ca6eb-3191-4edd-b1fd-c40bef197998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-496ca6eb-3191-4edd-b1fd-c40bef197998 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:46.375903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:46.379613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:46.382983 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 395/783] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:46.390080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c60ca-717c-4402-92bd-51e2e27e0eef None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:46.423228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:46.532518 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e2bc4-9f34-4ade-a054-20942a0cae66 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:46.532917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e2bc4-9f34-4ade-a054-20942a0cae66 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:46.533153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e2bc4-9f34-4ade-a054-20942a0cae66 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:46.537543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 396/784] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:46.543193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c15d9-904f-4dca-b2c3-978de3f93919 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:46.543515 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c15d9-904f-4dca-b2c3-978de3f93919 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:46.543750 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c15d9-904f-4dca-b2c3-978de3f93919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:46.571862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-ad7c15d9-904f-4dca-b2c3-978de3f93919 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:46.573168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-ad7c15d9-904f-4dca-b2c3-978de3f93919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:46.578308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c60ca-717c-4402-92bd-51e2e27e0eef None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:46.583990 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:46.657344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:46.659021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:46.659556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:46.660266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:46.661880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:46.662777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:46.665657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:46.677882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 389/785] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:46.684045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 397/786] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:46.685961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:46.731757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c15d9-904f-4dca-b2c3-978de3f93919 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:46.794976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:46.820156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf853d-0f03-4083-960d-2abac08b9d0e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:46.870998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:46.871809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:46.872258 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:46.872612 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:46.888583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:46.890609 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:46.913620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:46.914783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:46.915997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:46.915997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:46.915997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:46.915997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:46.915997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 390/787] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:46.922736 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:47.009427 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c15d9-904f-4dca-b2c3-978de3f93919 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:47.073420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904619ef-0cc7-4493-9e0c-d2472d402fad None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:47.096498 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29571b-1cd8-4821-879b-a9d1b917ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:47.097328 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29571b-1cd8-4821-879b-a9d1b917ad29 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:47.097616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29571b-1cd8-4821-879b-a9d1b917ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:47.097894 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29571b-1cd8-4821-879b-a9d1b917ad29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:47.107161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:47.114019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:47.114798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:47.115232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:47.115508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:47.116225 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:47.116609 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:47.118334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:47.127379 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29571b-1cd8-4821-879b-a9d1b917ad29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:47.132559 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 391/788] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-8c29571b-1cd8-4821-879b-a9d1b917ad29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:47.134728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:47.135774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:47.136749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:47.137485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 398/789] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:47.142170 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:47.145500 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29571b-1cd8-4821-879b-a9d1b917ad29 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:47.156043 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:47.252893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ede5b-abcf-4c18-b342-163c19459d23 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:47.294308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:47.294851 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:47.295123 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:47.295398 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] PATH_INFO: `/v3/users/swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:47.304397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:47.305914 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:47.320504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:47.323363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:47.325745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 392/790] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:47.331465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:47.406578 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:47.448171 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29571b-1cd8-4821-879b-a9d1b917ad29 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:47.523336 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1879fb-7ef0-4047-ad6f-21f3a9eee68c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:47.557179 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:47.557883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:47.558095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:47.558442 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:47.559270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:47.559575 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:47.561055 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:47.589967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 399/791] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:47.594351 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:47.595099 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:47.595384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:47.595656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:47.597269 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:47.623643 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:47.623643 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:47.623643 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:47.662147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 393/792] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:14:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:47.671256 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:47.795993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:47.877723 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5929020-df8e-4b5c-8291-93da170cfdfa None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:47.916410 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbce33-720f-4c14-8cc6-3bfd6fe51a52 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:47.931295 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbd702-6a55-48e9-97e6-d0af49c8513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:47.932579 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd702-6a55-48e9-97e6-d0af49c8513f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:47.933095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd702-6a55-48e9-97e6-d0af49c8513f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:47.933752 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd702-6a55-48e9-97e6-d0af49c8513f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:47.935531 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:47.953987 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbd702-6a55-48e9-97e6-d0af49c8513f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:47.955716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbd702-6a55-48e9-97e6-d0af49c8513f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-dfbbd702-6a55-48e9-97e6-d0af49c8513f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:47.981160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:47.983011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:47.984869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:47.984869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:47.984869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:47.984869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:47.984869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 400/793] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:47.985495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1e4f-3f01-4361-b212-08b7f43b7805 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:47.985771 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1e4f-3f01-4361-b212-08b7f43b7805 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:47.986260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1e4f-3f01-4361-b212-08b7f43b7805 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:47.987605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7207143-8876-49c1-b407-0c1540a58a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:47.989201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7207143-8876-49c1-b407-0c1540a58a08 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:47.989908 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7207143-8876-49c1-b407-0c1540a58a08 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:47.990583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7207143-8876-49c1-b407-0c1540a58a08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:47.993845 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 401/794] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:47.995591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a1e4f-3f01-4361-b212-08b7f43b7805 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:48.011070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7207143-8876-49c1-b407-0c1540a58a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:48.013868 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7207143-8876-49c1-b407-0c1540a58a08 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-e7207143-8876-49c1-b407-0c1540a58a08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:48.039404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:48.040583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:48.041349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:48.041349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:48.041349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:48.041349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:48.042206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 394/795] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:48.043713 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b1ef-d2e8-4f5e-80b3-b44e396f9964 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.044019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b1ef-d2e8-4f5e-80b3-b44e396f9964 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.044295 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b1ef-d2e8-4f5e-80b3-b44e396f9964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.069273 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d1a1b1ef-d2e8-4f5e-80b3-b44e396f9964 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:48.070557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d1a1b1ef-d2e8-4f5e-80b3-b44e396f9964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:48.176571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a1e4f-3f01-4361-b212-08b7f43b7805 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:48.177953 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 395/796] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:48.179683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1b1ef-d2e8-4f5e-80b3-b44e396f9964 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:48.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:48.254034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:48.255720 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.256453 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.257248 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.258509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:48.259223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:48.261380 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:48.280652 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 402/797] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:48.282560 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:48.335492 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:48.350224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1b1ef-d2e8-4f5e-80b3-b44e396f9964 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:48.399089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:48.399789 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.400071 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.400345 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.401055 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:48.401354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:48.402811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:48.420319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 396/798] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:48.421834 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:48.446911 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4847-4302-4e31-ab63-b19a5f82339c None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:48.494620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25067918-5034-4c5a-ac70-4d34281cb86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:48.495491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067918-5034-4c5a-ac70-4d34281cb86b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.495968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067918-5034-4c5a-ac70-4d34281cb86b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.496301 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067918-5034-4c5a-ac70-4d34281cb86b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.514440 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25067918-5034-4c5a-ac70-4d34281cb86b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-25067918-5034-4c5a-ac70-4d34281cb86b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:48.522277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:48.523237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:48.524296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:48.530496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 403/799] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:48.532809 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b69fdf-b0ff-4215-a1c9-55b39ad7ebcb None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.533177 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b69fdf-b0ff-4215-a1c9-55b39ad7ebcb None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.533753 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b69fdf-b0ff-4215-a1c9-55b39ad7ebcb None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.535741 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 404/800] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:48.536505 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b69fdf-b0ff-4215-a1c9-55b39ad7ebcb None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:48.561129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de784f-2547-4809-a0fc-9d9bf6faecb2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:48.605303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:48.621082 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:48.622373 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.623530 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.639045 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:48.641092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:48.663297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:48.664335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 397/801] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:14:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:48.665283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:48.689487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:48.717215 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b69fdf-b0ff-4215-a1c9-55b39ad7ebcb None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:48.764131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e5547-4d80-4a14-bc3f-d0bf83564f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:48.764821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e5547-4d80-4a14-bc3f-d0bf83564f18 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.765103 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e5547-4d80-4a14-bc3f-d0bf83564f18 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.765433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e5547-4d80-4a14-bc3f-d0bf83564f18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.773868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e5547-4d80-4a14-bc3f-d0bf83564f18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:48.775543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e5547-4d80-4a14-bc3f-d0bf83564f18 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-7d3e5547-4d80-4a14-bc3f-d0bf83564f18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:48.790761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:48.791815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:48.793155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 405/802] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:48.793937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1321d5-4712-4f56-9164-b9b5d2a75d77 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.793937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1321d5-4712-4f56-9164-b9b5d2a75d77 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.793937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1321d5-4712-4f56-9164-b9b5d2a75d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.814657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-9b1321d5-4712-4f56-9164-b9b5d2a75d77 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:48.815884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-9b1321d5-4712-4f56-9164-b9b5d2a75d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:48.845568 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb3af1-372c-4223-8b02-9bdd08580564 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:48.888523 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:48.889288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:48.889809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:48.890245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:48.891233 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:48.892288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:48.894543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:48.902452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 406/803] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:48.903337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1321d5-4712-4f56-9164-b9b5d2a75d77 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:48.912689 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 398/804] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:48.912689 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:48.946889 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:49.010853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:49.042700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8b2d-a143-4555-bce1-64001636d860 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.068578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1321d5-4712-4f56-9164-b9b5d2a75d77 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.086344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea28800-398a-42f4-95e9-10dc60dfd7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.087087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28800-398a-42f4-95e9-10dc60dfd7f7 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.087464 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28800-398a-42f4-95e9-10dc60dfd7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.087783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28800-398a-42f4-95e9-10dc60dfd7f7 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.088675 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea28800-398a-42f4-95e9-10dc60dfd7f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:49.089079 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea28800-398a-42f4-95e9-10dc60dfd7f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.090946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea28800-398a-42f4-95e9-10dc60dfd7f7 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.106625 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 399/805] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:49.108118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd742-36f8-4b82-b233-01e69382397e None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.108413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd742-36f8-4b82-b233-01e69382397e None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.108687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd742-36f8-4b82-b233-01e69382397e None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.109988 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 400/806] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:49.110799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dd742-36f8-4b82-b233-01e69382397e None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:49.116526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.117733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.118308 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.118777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.120376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:49.121628 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.124771 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.147388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 407/807] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:14:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:49.148425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:49.229371 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dd742-36f8-4b82-b233-01e69382397e None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.259285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:49.267429 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:49.275052 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.275790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.276075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.276354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.288330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.289978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.291461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1363f-c188-4a0a-8520-a617ed2d8256 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:49.312141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:49.313820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:49.315329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:49.315329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:49.315329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:49.315329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:49.315329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 401/808] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:49.315329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:49.352046 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828a9e-cecc-4f96-939a-339415a175b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.353060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828a9e-cecc-4f96-939a-339415a175b4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.353512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828a9e-cecc-4f96-939a-339415a175b4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.353856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828a9e-cecc-4f96-939a-339415a175b4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.354606 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828a9e-cecc-4f96-939a-339415a175b4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.356435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828a9e-cecc-4f96-939a-339415a175b4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-e1828a9e-cecc-4f96-939a-339415a175b4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:49.368938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:49.370032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:49.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 408/809] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:14:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:49.372816 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fa7d-0754-40c1-92cc-4546879d3ef2 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.373239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fa7d-0754-40c1-92cc-4546879d3ef2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.373648 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fa7d-0754-40c1-92cc-4546879d3ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.397266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-36b4fa7d-0754-40c1-92cc-4546879d3ef2 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:49.399435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-36b4fa7d-0754-40c1-92cc-4546879d3ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:49.440305 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb714c-7f3c-46c9-80c6-b6103a7eddc8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.482079 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 409/810] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:49.482688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f56772-a22f-431d-9116-1d575057759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.483056 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4fa7d-0754-40c1-92cc-4546879d3ef2 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:49.483447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f56772-a22f-431d-9116-1d575057759b None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.483674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f56772-a22f-431d-9116-1d575057759b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.483974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f56772-a22f-431d-9116-1d575057759b None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/da81aff79a9a467da6d8bf372da9faf2/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.486496 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f56772-a22f-431d-9116-1d575057759b None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=da81aff79a9a467da6d8bf372da9faf2, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.488126 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f56772-a22f-431d-9116-1d575057759b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.503426 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 402/811] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:14:49 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/da81aff79a9a467da6d8bf372da9faf2/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:49.504299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f56772-a22f-431d-9116-1d575057759b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:49.588072 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:49.629592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:49.678194 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4fa7d-0754-40c1-92cc-4546879d3ef2 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.703981 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f56772-a22f-431d-9116-1d575057759b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.731409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.732089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.732372 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.732648 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.746133 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.749324 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.757031 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.757723 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.757966 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.758279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.759005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:49.759323 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.760772 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.775187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 403/812] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:49.775827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:49.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:49.781551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:49.782592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:49.782592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:49.782592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:49.782592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:49.782592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 410/813] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:49.784152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:49.928982 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39543c90-b6c2-4589-bfc5-7d38dd869043 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.934333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:49.936524 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a98412-9861-469a-917f-d12b2b665811 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:49.979056 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:49.985947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.986903 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.987210 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:49.987391 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.987869 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.988015 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:49.988139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:49.988454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] PATH_INFO: `/v3/services/network` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:49.989153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:49.989303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.989725 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:49.991049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.991787 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:49.998447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 404/814] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:14:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:49.999147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:50.008456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:50.009527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:50.010623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 411/815] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:49 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:50.013246 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:50.162737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe0c4f-cf21-4a87-bfef-40fa27a92078 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:50.174228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d13b0d-b56f-47c5-a070-244d00c4aea0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:50.212720 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:50.213286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:50.213571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:50.214590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:50.215973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:50.216604 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:50.219336 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:50.233245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:50.233944 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:50.234269 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:50.234549 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:50.248705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 412/816] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:50.248705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:50.250921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:50.252576 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:50.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:50.268441 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:50.269255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 405/817] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:14:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:50.269768 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:50.331121 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:50.338946 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:50.408404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d7fa5-e1e7-412d-9a5f-18811d661fb5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:50.437092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819c353-e825-49f9-8b06-3a7658388f7b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:50.468932 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:50.469471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:50.469837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:50.470402 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:50.471109 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:50.471804 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:50.473276 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:50.479839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 413/818] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:14:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:50.480708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:50.488858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:50.489630 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:50.489922 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:50.490291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:50.490963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] RBAC: Adding query filter params (name=network) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:50.491269 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:50.492714 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:50.498679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 406/819] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:14:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:50.499577 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:50.618623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1706be-41b2-4b8c-b66b-2218b5ebbaff None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:50.640077 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:50.645002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:50.671091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884b0f6-0272-42fd-97da-6a4a1a503325 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:50.679688 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:50.680849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:50.681336 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:50.681822 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:50.695111 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:50.695111 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:50.707639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:50.708882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 414/820] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:50.710147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:50.729515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:50.731630 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:50.732002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:50.732417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:50.733901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:50.734445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:50.736864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:50.773547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 407/821] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:14:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:50.775324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:50.844327 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821e127-4d48-4da0-bbf1-18d5aa37a85c None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:50.888303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1587114-3033-4e77-8263-c88fc9b0ebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:50.889353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1587114-3033-4e77-8263-c88fc9b0ebde None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:50.889704 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1587114-3033-4e77-8263-c88fc9b0ebde None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:50.890183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1587114-3033-4e77-8263-c88fc9b0ebde None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:50.891207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1587114-3033-4e77-8263-c88fc9b0ebde None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:50.893390 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1587114-3033-4e77-8263-c88fc9b0ebde None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:50.930017 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72820b-0559-470f-9132-1147cec8351d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:50.942534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 415/822] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:14:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:50.942534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1587114-3033-4e77-8263-c88fc9b0ebde None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:51.001543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:51.043431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:51.044712 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:51.045367 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:51.045798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:51.046975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] RBAC: Adding query filter params (type=network) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:51.047680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:51.050370 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:51.073002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 408/823] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:14:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:51.074480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:51.091326 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:51.124796 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1587114-3033-4e77-8263-c88fc9b0ebde None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:51.190475 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:51.191815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:51.192368 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:51.192919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:51.194141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:51.194690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:51.197101 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:51.204678 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a44b7-4ffd-4d3f-82d1-6a5cd794dbea None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:51.224084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 416/824] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:51.224084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:51.265355 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7797e-bed8-47d2-8e05-d857db98a90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:51.265355 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7797e-bed8-47d2-8e05-d857db98a90c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:51.265820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7797e-bed8-47d2-8e05-d857db98a90c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:51.265820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7797e-bed8-47d2-8e05-d857db98a90c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:51.376310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:51.382769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd553f7f-5256-4e92-a155-6e8fc606de2b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:51.410766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7797e-bed8-47d2-8e05-d857db98a90c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:51.413393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7797e-bed8-47d2-8e05-d857db98a90c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:51.432733 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:51.432949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9990fe-3895-4fcd-bab2-3fb5b3586dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:51.434407 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9990fe-3895-4fcd-bab2-3fb5b3586dd2 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:51.434856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9990fe-3895-4fcd-bab2-3fb5b3586dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:51.435161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9990fe-3895-4fcd-bab2-3fb5b3586dd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:51.437157 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9990fe-3895-4fcd-bab2-3fb5b3586dd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:51.438712 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9990fe-3895-4fcd-bab2-3fb5b3586dd2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:51.478343 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 417/825] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:14:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:51.481390 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9990fe-3895-4fcd-bab2-3fb5b3586dd2 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:51.702341 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 409/826] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:14:51.708426 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7797e-bed8-47d2-8e05-d857db98a90c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:51.795187 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9990fe-3895-4fcd-bab2-3fb5b3586dd2 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:51.838571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102cf50-d2ea-4251-ae5f-9760987a1025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:51.839300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102cf50-d2ea-4251-ae5f-9760987a1025 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:51.839584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102cf50-d2ea-4251-ae5f-9760987a1025 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:51.839863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102cf50-d2ea-4251-ae5f-9760987a1025 None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/74f4cb06ce4548aea8e61a57c984fe70/roles/55d67b43cde54213983e27412c2a314c` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:51.856971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102cf50-d2ea-4251-ae5f-9760987a1025 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=74f4cb06ce4548aea8e61a57c984fe70, role_id=55d67b43cde54213983e27412c2a314c)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:51.858748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102cf50-d2ea-4251-ae5f-9760987a1025 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:51.863045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:51.875454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 418/827] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:14:51 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/74f4cb06ce4548aea8e61a57c984fe70/roles/55d67b43cde54213983e27412c2a314c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:14:51.876832 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dcb0b-d78e-496d-82f8-a94e5c659314 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:51.877117 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dcb0b-d78e-496d-82f8-a94e5c659314 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:51.877389 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dcb0b-d78e-496d-82f8-a94e5c659314 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:51.878911 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 419/828] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:51.884846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df6e3-0b94-42c9-897f-546b5afa5d6b None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:51.885174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df6e3-0b94-42c9-897f-546b5afa5d6b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:51.885507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df6e3-0b94-42c9-897f-546b5afa5d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:51.907385 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7797e-bed8-47d2-8e05-d857db98a90c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:51.907937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-5c5df6e3-0b94-42c9-897f-546b5afa5d6b None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:51.909156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-5c5df6e3-0b94-42c9-897f-546b5afa5d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:51.925948 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:51.961334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:51.962076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:51.962405 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:51.962684 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:51.979312 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:51.982051 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:52.014601 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 420/829] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:52.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:52.015677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:52.016501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:52.016501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:52.016501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:52.016501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:52.022506 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 410/830] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:52.027043 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5df6e3-0b94-42c9-897f-546b5afa5d6b None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:52.037637 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:52.173786 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5df6e3-0b94-42c9-897f-546b5afa5d6b None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:52.176740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638970de-4104-45ad-a3ff-20bb15707dfa None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:52.210170 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:52.220114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f51f16-65e6-44b1-afa7-ecc03131ce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:52.220789 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51f16-65e6-44b1-afa7-ecc03131ce50 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:52.221068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51f16-65e6-44b1-afa7-ecc03131ce50 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:52.221347 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51f16-65e6-44b1-afa7-ecc03131ce50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:52.223387 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:52.224488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:52.225330 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:52.225849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:52.226809 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:52.227551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:52.229472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:52.236098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f51f16-65e6-44b1-afa7-ecc03131ce50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-f9f51f16-65e6-44b1-afa7-ecc03131ce50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:52.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:52.243276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:52.244074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 411/831] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:52.248541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 421/832] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:52.253390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f51f16-65e6-44b1-afa7-ecc03131ce50 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:52.254596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:52.288378 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:52.388356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b87da-00e8-46af-a97a-e1b10c584193 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:52.413026 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f51f16-65e6-44b1-afa7-ecc03131ce50 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:52.434467 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:52.435836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:52.436368 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:52.436841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:52.447360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:52.449986 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:52.467447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:52.467681 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:52.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:52.469207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 422/833] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:52.471131 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:52.471429 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:52.471710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:52.472468 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:52.472777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:52.474237 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:52.475041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:52.490314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 412/834] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:52.492790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:52.538853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:52.609331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9d220-d3d8-4cf9-828f-4494fdc4779b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:52.624412 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:52.651976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbafc2-d7dd-4a33-b91d-aa450d23bf2b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:52.654446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:52.655309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:52.655595 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:52.655871 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:52.657979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:52.658594 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:52.661109 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:52.683793 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 423/835] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:52.693307 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:52.700629 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:52.701324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:52.701616 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:52.701895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:52.716806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:52.718515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:52.744445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:52.745434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:52.746546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:52.746546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:52.746546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:52.746546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:52.750202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 413/836] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:52.756332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:52.848639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee43fa-0692-4a12-9593-2347a5c9af42 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:52.869722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:52.909541 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e4e2e-c61f-4927-9ecc-441d07bdae47 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:52.910896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:52.911907 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:52.912473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:52.913056 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:52.914260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=45cd9a93c342491889ca8260bc4510c6) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:52.914862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:52.916975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:52.934640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 424/837] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/role_assignments?user.id=45cd9a93c342491889ca8260bc4510c6&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=40504e80dbb845aeb07ca258977ecacb => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:52.956251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:52.956819 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:52.957104 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:52.957448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:52.958198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:52.958556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:52.960007 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:52.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:52.982201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 414/838] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:52.987847 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:53.151348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd79511-3b2a-4f9e-ad69-312ec1628b13 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:53.191905 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1b894-4315-4199-b21a-c48d434bb51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:53.192651 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1b894-4315-4199-b21a-c48d434bb51e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:53.192940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1b894-4315-4199-b21a-c48d434bb51e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:53.193221 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1b894-4315-4199-b21a-c48d434bb51e None admin] PATH_INFO: `/v3/users/swift` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:53.203203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1b894-4315-4199-b21a-c48d434bb51e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:53.204813 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1b894-4315-4199-b21a-c48d434bb51e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6da1b894-4315-4199-b21a-c48d434bb51e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:53.218596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:53.219782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:53.220814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 425/839] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:14:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:53.226059 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:53.338396 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8063b15-2017-46bb-bb92-37b773ce64e9 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:53.373133 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e729073-ab6a-4cee-9d80-e7333df25005 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:53.373468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e729073-ab6a-4cee-9d80-e7333df25005 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:53.373687 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e729073-ab6a-4cee-9d80-e7333df25005 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:53.375602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 426/840] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:53.378248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:53.378902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:53.379189 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:53.379463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:53.380286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:53.380595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:53.382217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:53.383613 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbac4e-0af9-4301-b34d-404ce5a290e5 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:53.384409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbac4e-0af9-4301-b34d-404ce5a290e5 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:53.384923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbac4e-0af9-4301-b34d-404ce5a290e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:53.402371 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 415/841] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:14:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:53.408588 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:53.415931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-52dbac4e-0af9-4301-b34d-404ce5a290e5 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:53.418480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-52dbac4e-0af9-4301-b34d-404ce5a290e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:53.454259 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:53.481954 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:53.518612 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 427/842] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:14:53.542881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbac4e-0af9-4301-b34d-404ce5a290e5 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:53.552032 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511906b-d470-429c-a740-799bfe4b3ef3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:53.594016 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950385f-ac3b-4766-928c-08a4a7ad09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:53.595479 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950385f-ac3b-4766-928c-08a4a7ad09fd None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:53.596182 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950385f-ac3b-4766-928c-08a4a7ad09fd None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:53.596863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950385f-ac3b-4766-928c-08a4a7ad09fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:53.677237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbac4e-0af9-4301-b34d-404ce5a290e5 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:53.692693 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950385f-ac3b-4766-928c-08a4a7ad09fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:53.694914 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950385f-ac3b-4766-928c-08a4a7ad09fd None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:53.718432 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:53.719102 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:53.719382 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:53.719815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:53.720470 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:53.722386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:53.733638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:53.735002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:14:53.735881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:53.746090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 428/843] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:53.747194 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:53.767066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:53.778376 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:53.829596 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 416/844] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:53 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:14:53.840029 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950385f-ac3b-4766-928c-08a4a7ad09fd None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:53.867279 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acecb2c-41e8-4fcf-8f3d-0d052fa791b1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:53.913791 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:53.914508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:53.915153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:53.915489 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:53.916790 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:53.917202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:53.920149 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:53.929295 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 429/845] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:53.936659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:53.986317 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950385f-ac3b-4766-928c-08a4a7ad09fd None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:54.036041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:54.050271 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:54.052119 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.052614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.053360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.056383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df227c1d-aa54-488b-a01c-074497eb4006 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:54.068058 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:54.070486 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:54.093060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:54.093998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:54.094897 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:54.094897 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:54.094897 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:54.094897 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:54.098190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 417/846] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:54.105498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:54.111775 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad059bf5-9e3f-4212-8b31-697c09901857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:54.112437 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad059bf5-9e3f-4212-8b31-697c09901857 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.112719 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad059bf5-9e3f-4212-8b31-697c09901857 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.112999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad059bf5-9e3f-4212-8b31-697c09901857 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.113700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad059bf5-9e3f-4212-8b31-697c09901857 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:54.114006 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad059bf5-9e3f-4212-8b31-697c09901857 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:54.115449 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad059bf5-9e3f-4212-8b31-697c09901857 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:54.118991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:54.130651 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 430/847] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:54.255672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdfc2-d10a-4ea3-9fb2-f7f81b8e18b1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:54.291732 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad395c1-4b81-4e5f-9e33-4893029f4e85 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.292360 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad395c1-4b81-4e5f-9e33-4893029f4e85 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.292653 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad395c1-4b81-4e5f-9e33-4893029f4e85 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.294583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 431/848] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:54.297355 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:54.298261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.298676 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.299084 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.300007 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:54.300533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:54.300641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb8e86-c781-4a99-8881-9a2a4886b63e None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.300919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb8e86-c781-4a99-8881-9a2a4886b63e None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.301202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb8e86-c781-4a99-8881-9a2a4886b63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.302518 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:54.319371 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 418/849] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:54.325988 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:54.326486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-a3cb8e86-c781-4a99-8881-9a2a4886b63e None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:54.327794 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-a3cb8e86-c781-4a99-8881-9a2a4886b63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:54.411028 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 432/850] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:54.418912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:54.439685 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb8e86-c781-4a99-8881-9a2a4886b63e None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:54.458161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95bf8e-62d8-4cf7-aa48-5435927cb713 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:54.497013 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:54.497752 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.498036 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.498344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.506851 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:54.508515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:54.522281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:54.523304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:54.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:54.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:54.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:54.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:54.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:54.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:54.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:54.524221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:54.528641 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 419/851] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:54.529762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:54.562165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:54.590079 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb8e86-c781-4a99-8881-9a2a4886b63e None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:54.636473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2288eee-7100-4d0f-a073-1f5418f7a7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:54.637484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2288eee-7100-4d0f-a073-1f5418f7a7e3 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.637896 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2288eee-7100-4d0f-a073-1f5418f7a7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.638472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2288eee-7100-4d0f-a073-1f5418f7a7e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.654882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2288eee-7100-4d0f-a073-1f5418f7a7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:14:54.660772 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec439fb6-ad1e-4019-8109-49b1bcc19d90 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-c2288eee-7100-4d0f-a073-1f5418f7a7e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:14:54.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:54.664916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:14:54.666136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:54.672221 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 433/852] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:14:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:54.678409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2288eee-7100-4d0f-a073-1f5418f7a7e3 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:54.707550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:54.721554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:54.722629 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.722984 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.723321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.724327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:54.724646 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:54.727061 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:54.753999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 420/853] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:54.755636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:54.835363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2288eee-7100-4d0f-a073-1f5418f7a7e3 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:54.874783 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:54.875848 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.876326 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.876793 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.877716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:54.878272 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:54.879868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:54.889841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 434/854] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:14:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:54.902230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:54.907777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:54.915636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86f426-ec5a-4624-a60d-a5e5bce2dff2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:54.968217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:54.969961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:54.970495 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:54.971225 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:54.972384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=da81aff79a9a467da6d8bf372da9faf2) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:54.972886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:54.975763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:54.986614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 421/855] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:54 2022] GET /identity/v3/role_assignments?user.id=da81aff79a9a467da6d8bf372da9faf2&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=40504e80dbb845aeb07ca258977ecacb => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:55.053455 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f4de3-2254-4f56-a0d7-592148471f27 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:55.104431 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262780c-c1e8-493f-b8b9-73c3a9eae1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:55.105032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262780c-c1e8-493f-b8b9-73c3a9eae1a7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.105372 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262780c-c1e8-493f-b8b9-73c3a9eae1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:55.105659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262780c-c1e8-493f-b8b9-73c3a9eae1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:55.119256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262780c-c1e8-493f-b8b9-73c3a9eae1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:55.120921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262780c-c1e8-493f-b8b9-73c3a9eae1a7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-7262780c-c1e8-493f-b8b9-73c3a9eae1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:55.138724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:55.139615 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:55.140435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:55.140435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:55.140435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:55.140435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:55.140435 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 435/856] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:55.146530 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:55.213521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:55.279819 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1ced3-9453-4c16-a55a-c51b54f97f71 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:55.308329 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533272b-789f-4036-8c90-084a0499a8c2 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.308654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533272b-789f-4036-8c90-084a0499a8c2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:55.308885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533272b-789f-4036-8c90-084a0499a8c2 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:55.310398 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 436/857] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:14:55.316534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65501784-65fb-4d26-bf46-6f2fbd401b16 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.316818 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65501784-65fb-4d26-bf46-6f2fbd401b16 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:55.317091 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65501784-65fb-4d26-bf46-6f2fbd401b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:55.324512 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:55.325464 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.325933 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:55.326299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:55.327236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:55.327550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:55.329308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:55.338180 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-65501784-65fb-4d26-bf46-6f2fbd401b16 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:55.339834 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-65501784-65fb-4d26-bf46-6f2fbd401b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:55.344431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 422/858] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:55.350766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:55.424399 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:55.464676 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 437/859] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:14:55.493527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65501784-65fb-4d26-bf46-6f2fbd401b16 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:55.515595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1128e00-98af-4b31-abf9-1dfa832595a7 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:55.528333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:55.556656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:55.557521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.557934 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:55.558331 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:55.567414 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:55.569193 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:14:55.583484 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:55.584647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:14:55.585756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:55.587844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 423/860] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:14:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:55.593563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:55.616396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65501784-65fb-4d26-bf46-6f2fbd401b16 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:55.658322 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:55.659367 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.659843 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:55.660266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:55.661241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:55.662987 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:55.679931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:55.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:14:55.681955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:55.684861 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 438/861] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:14:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:14:55.691452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:55.718148 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd72b7-84f9-45b7-bd7d-5b8f26cea952 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:55.754032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:55.778945 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:55.781107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.781584 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:55.782063 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:55.783064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:55.783513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:55.785148 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:55.809233 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 424/862] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:14:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:55.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:55.829611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86dfd-88f3-4afd-a80c-27b41f529571 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:55.853712 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:55.886170 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:55.886170 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.886170 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:55.886170 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:55.886625 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:55.886625 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:55.887976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:55.898255 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 439/863] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:14:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:55.900218 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:55.944925 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fcdaa-ffae-4c11-b8d8-b190d6f85e7f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:55.997775 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d52f0-aff1-4420-a1b6-30f77b24e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:55.999070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d52f0-aff1-4420-a1b6-30f77b24e623 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:55.999502 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d52f0-aff1-4420-a1b6-30f77b24e623 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:56.000329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d52f0-aff1-4420-a1b6-30f77b24e623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:56.012782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975920b-d5cf-4de3-9848-a9d337d8ba73 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:56.029107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:56.057459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:56.058552 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:56.059040 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:56.059557 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:56.060453 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:56.060976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:56.062632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:56.074271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 440/864] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:14:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:14:56.121458 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d52f0-aff1-4420-a1b6-30f77b24e623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:56.123193 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d52f0-aff1-4420-a1b6-30f77b24e623 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:56.158622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:56.249860 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 425/865] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:14:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:14:56.256048 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:56.385992 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43827ef-eddc-48e9-9957-a7cc8e44b541 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:56.444281 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbb366-3f46-472a-9e89-bb9b49f550e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:56.445403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb366-3f46-472a-9e89-bb9b49f550e0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:56.445918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb366-3f46-472a-9e89-bb9b49f550e0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:56.446614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb366-3f46-472a-9e89-bb9b49f550e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:56.464994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbb366-3f46-472a-9e89-bb9b49f550e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:56.466543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbb366-3f46-472a-9e89-bb9b49f550e0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-61fbb366-3f46-472a-9e89-bb9b49f550e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:14:56.483168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:56.484335 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:56.485727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:14:56.485727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:14:56.485727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:14:56.485727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:56.485727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 441/866] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:56.495352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d52f0-aff1-4420-a1b6-30f77b24e623 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:56.548958 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edb3d3-c434-4248-a890-69b2f2dd9558 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:56.549312 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edb3d3-c434-4248-a890-69b2f2dd9558 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:56.550309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edb3d3-c434-4248-a890-69b2f2dd9558 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:56.551577 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 442/867] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:56.558810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a123ca-1cef-4fb8-8f49-f7c10c9abf0c None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:56.559150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a123ca-1cef-4fb8-8f49-f7c10c9abf0c None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:56.559521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a123ca-1cef-4fb8-8f49-f7c10c9abf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:56.582485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-06a123ca-1cef-4fb8-8f49-f7c10c9abf0c None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:56.583771 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-06a123ca-1cef-4fb8-8f49-f7c10c9abf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:56.619050 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:56.670179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d52f0-aff1-4420-a1b6-30f77b24e623 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:56.686617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 443/868] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:14:56.710787 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a123ca-1cef-4fb8-8f49-f7c10c9abf0c None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:56.720886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:56.726581 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:56.727337 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:56.727692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:56.728039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:56.728887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:56.729288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:56.730909 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:56.745194 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 426/869] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:56.752129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:56.853021 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a123ca-1cef-4fb8-8f49-f7c10c9abf0c None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:56.879628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c98b2-a6c3-4469-81b4-ef05fcd7ca08 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:56.903237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083191d-fb06-41ee-9d22-7d43d41a28e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:56.904822 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083191d-fb06-41ee-9d22-7d43d41a28e9 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:56.905275 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083191d-fb06-41ee-9d22-7d43d41a28e9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:56.905703 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083191d-fb06-41ee-9d22-7d43d41a28e9 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:56.906842 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083191d-fb06-41ee-9d22-7d43d41a28e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:56.909106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083191d-fb06-41ee-9d22-7d43d41a28e9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:56.920934 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:56.922025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:56.922554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:56.923075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:56.930860 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:56.934146 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 444/870] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:14:56 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:56.939386 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:14:56.963657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:56.965286 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:56.967763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:14:56.967763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:14:56.967763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:14:56.967763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:14:56.967763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:14:56.967763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:14:56.967763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:14:56.967763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:14:56.971912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 427/871] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:56.978499 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083191d-fb06-41ee-9d22-7d43d41a28e9 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:57.006508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:57.148391 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083191d-fb06-41ee-9d22-7d43d41a28e9 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:57.194065 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3f3b3-ead5-4746-acff-cb99a9d55567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:57.194799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f3b3-ead5-4746-acff-cb99a9d55567 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:57.195083 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f3b3-ead5-4746-acff-cb99a9d55567 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:57.195358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f3b3-ead5-4746-acff-cb99a9d55567 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:57.196546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3f3b3-ead5-4746-acff-cb99a9d55567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:57.196854 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3f3b3-ead5-4746-acff-cb99a9d55567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:57.198361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3f3b3-ead5-4746-acff-cb99a9d55567 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:57.212964 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 445/872] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:57.220975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:57.285901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:57.344053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e596-1388-42e7-993a-ccdaac7dcd23 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:57.381821 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21494c4d-0c07-4344-b62f-fc790ed27280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:57.382487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21494c4d-0c07-4344-b62f-fc790ed27280 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:57.382770 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21494c4d-0c07-4344-b62f-fc790ed27280 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:57.383119 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21494c4d-0c07-4344-b62f-fc790ed27280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:57.383968 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21494c4d-0c07-4344-b62f-fc790ed27280 None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=74f4cb06ce4548aea8e61a57c984fe70) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:57.384384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21494c4d-0c07-4344-b62f-fc790ed27280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:57.385902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21494c4d-0c07-4344-b62f-fc790ed27280 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:57.398668 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 428/873] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:14:57 2022] GET /identity/v3/role_assignments?user.id=74f4cb06ce4548aea8e61a57c984fe70&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:57.416945 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb0ee0-f20b-4541-ba5e-31dfab3acdff None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:57.416945 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb0ee0-f20b-4541-ba5e-31dfab3acdff None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:57.416945 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb0ee0-f20b-4541-ba5e-31dfab3acdff None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:57.422749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 446/874] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:14:57.424893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b25c3-64c8-4e85-8502-aaac89665a60 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:57.425175 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b25c3-64c8-4e85-8502-aaac89665a60 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:57.425445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b25c3-64c8-4e85-8502-aaac89665a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:57.448008 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-756b25c3-64c8-4e85-8502-aaac89665a60 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:57.449421 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-756b25c3-64c8-4e85-8502-aaac89665a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:57.533511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 429/875] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:57.558167 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb0ee0-f20b-4541-ba5e-31dfab3acdff None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:57.670697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:57.730134 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb0ee0-f20b-4541-ba5e-31dfab3acdff None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:57.776897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:57.776897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:57.777294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:57.777461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:57.782134 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:57.784296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:57.798000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:57.799412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:57.800286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 447/876] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:14:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:57.806015 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b25c3-64c8-4e85-8502-aaac89665a60 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:57.922643 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:57.941569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b25c3-64c8-4e85-8502-aaac89665a60 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:58.006141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc539a17-c26b-4f63-820f-58cf8d03d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:58.006141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc539a17-c26b-4f63-820f-58cf8d03d661 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:58.006141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc539a17-c26b-4f63-820f-58cf8d03d661 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:58.006141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc539a17-c26b-4f63-820f-58cf8d03d661 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:58.006904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc539a17-c26b-4f63-820f-58cf8d03d661 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:58.011869 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc539a17-c26b-4f63-820f-58cf8d03d661 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:58.012938 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc539a17-c26b-4f63-820f-58cf8d03d661 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:58.033185 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 430/877] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:14:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:58.036891 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:58.214570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ed575-e799-469c-8416-a168b3b031d1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:58.260290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c21b4-4fc7-4348-9b1b-23035a0a221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:58.260934 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c21b4-4fc7-4348-9b1b-23035a0a221e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:58.261212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c21b4-4fc7-4348-9b1b-23035a0a221e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:58.261486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c21b4-4fc7-4348-9b1b-23035a0a221e None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:58.262218 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c21b4-4fc7-4348-9b1b-23035a0a221e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:58.262527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c21b4-4fc7-4348-9b1b-23035a0a221e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:58.263930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c21b4-4fc7-4348-9b1b-23035a0a221e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:58.269686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 448/878] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:14:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:58.501698 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91c79b-3849-4293-8826-98ca424b8bab None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:58.502775 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91c79b-3849-4293-8826-98ca424b8bab None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:58.503397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91c79b-3849-4293-8826-98ca424b8bab None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:58.505705 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 431/879] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:58.512438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035b2f-f617-4300-b006-cf8296ba3d72 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:58.512794 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035b2f-f617-4300-b006-cf8296ba3d72 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:58.513101 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035b2f-f617-4300-b006-cf8296ba3d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:58.521488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:58.535675 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-57035b2f-f617-4300-b006-cf8296ba3d72 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:58.537300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-57035b2f-f617-4300-b006-cf8296ba3d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:58.623976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 449/880] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:14:58.649399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91c79b-3849-4293-8826-98ca424b8bab None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:58.765265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91c79b-3849-4293-8826-98ca424b8bab None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:58.773538 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:58.804658 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585d6ef-1bcf-4be4-ad88-8269398d6aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:58.805292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585d6ef-1bcf-4be4-ad88-8269398d6aed None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:58.805575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585d6ef-1bcf-4be4-ad88-8269398d6aed None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:58.805853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585d6ef-1bcf-4be4-ad88-8269398d6aed None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:58.806672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585d6ef-1bcf-4be4-ad88-8269398d6aed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:58.808098 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585d6ef-1bcf-4be4-ad88-8269398d6aed None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:58.827175 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 432/881] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:14:58 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:14:59.158974 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5be9ff-d4e1-4ea5-b843-ffc258b96863 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:59.159378 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5be9ff-d4e1-4ea5-b843-ffc258b96863 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:59.159553 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5be9ff-d4e1-4ea5-b843-ffc258b96863 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:59.161213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 450/882] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:14:59.166668 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e508c8-53ef-4af0-abb8-0c973ccd6f89 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:59.166965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e508c8-53ef-4af0-abb8-0c973ccd6f89 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:59.167248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e508c8-53ef-4af0-abb8-0c973ccd6f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:59.186278 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-b3e508c8-53ef-4af0-abb8-0c973ccd6f89 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:14:59.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-b3e508c8-53ef-4af0-abb8-0c973ccd6f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:14:59.271130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 433/883] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:14:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:14:59.293943 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5be9ff-d4e1-4ea5-b843-ffc258b96863 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:59.425563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5be9ff-d4e1-4ea5-b843-ffc258b96863 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:59.467516 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:59.468591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:59.469130 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:59.469767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:59.470718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:59.472418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:59.474713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:14:59.485022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:14:59.486018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:14:59.487979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 451/884] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:14:59.492794 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e508c8-53ef-4af0-abb8-0c973ccd6f89 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:59.602714 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:14:59.612764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e508c8-53ef-4af0-abb8-0c973ccd6f89 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:59.652308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:59.652961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:59.653243 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:59.653513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:59.654251 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:59.654564 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:59.656028 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:59.663094 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 434/885] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:59.669867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:59.806099 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f3a39-9129-4041-a240-98fc945746f5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:14:59.855883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:14:59.856859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:14:59.857456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:14:59.857809 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:14:59.858747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:14:59.859073 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:14:59.861241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:14:59.868189 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 452/886] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:14:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:14:59.873861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:14:59.989081 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf0e49-1456-4f88-83fe-9d9320a65e38 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:00.035345 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529374e-bf4f-4635-b7eb-752c04fd5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:00.036139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529374e-bf4f-4635-b7eb-752c04fd5b3e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.036482 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529374e-bf4f-4635-b7eb-752c04fd5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.036757 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529374e-bf4f-4635-b7eb-752c04fd5b3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.037701 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529374e-bf4f-4635-b7eb-752c04fd5b3e None admin] RBAC: Adding query filter params (interface=public, service_id=3731dfc569fc435da93b1d24d8cf4cf6, region_id=RegionOne) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:00.038130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529374e-bf4f-4635-b7eb-752c04fd5b3e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:00.039792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529374e-bf4f-4635-b7eb-752c04fd5b3e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:00.049644 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 435/887] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:14:59 2022] GET /identity/v3/endpoints?service_id=3731dfc569fc435da93b1d24d8cf4cf6&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:00.055304 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:00.144780 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c765349-ec85-4a20-b721-0c03e19df21d None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.145293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c765349-ec85-4a20-b721-0c03e19df21d None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.145740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c765349-ec85-4a20-b721-0c03e19df21d None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.149972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 436/888] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:00.158318 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc679d-9e49-4f49-9a65-b648b3d9c260 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.158971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc679d-9e49-4f49-9a65-b648b3d9c260 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.159197 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:00.159710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc679d-9e49-4f49-9a65-b648b3d9c260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.183142 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-01bc679d-9e49-4f49-9a65-b648b3d9c260 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:00.184416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-01bc679d-9e49-4f49-9a65-b648b3d9c260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:00.205828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:00.235273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1e260-fe24-47d1-96d3-8a47665d2675 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:00.282189 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db160e1b-330e-4cdc-a267-b25c050fde3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:00.282445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 437/889] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:00.286975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db160e1b-330e-4cdc-a267-b25c050fde3a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.287490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db160e1b-330e-4cdc-a267-b25c050fde3a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.288095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db160e1b-330e-4cdc-a267-b25c050fde3a None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.288977 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db160e1b-330e-4cdc-a267-b25c050fde3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:00.290836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db160e1b-330e-4cdc-a267-b25c050fde3a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:00.300569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 453/890] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:15:00 2022] GET /identity/v3/services => generated 1268 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:15:00.309994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc679d-9e49-4f49-9a65-b648b3d9c260 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:00.453416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc679d-9e49-4f49-9a65-b648b3d9c260 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:00.498234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608a145-3e79-412e-8320-dccb5b17f320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:00.499606 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a145-3e79-412e-8320-dccb5b17f320 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.500255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a145-3e79-412e-8320-dccb5b17f320 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.500894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a145-3e79-412e-8320-dccb5b17f320 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.514738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608a145-3e79-412e-8320-dccb5b17f320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-c608a145-3e79-412e-8320-dccb5b17f320 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:00.520817 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:00.521890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:00.522836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:00.525403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 438/891] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:15:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:00.532224 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db160e1b-330e-4cdc-a267-b25c050fde3a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:00.598777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dcfb2-2547-4f82-b9b5-b69bb9cd50d4 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dcfb2-2547-4f82-b9b5-b69bb9cd50d4 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.600494 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dcfb2-2547-4f82-b9b5-b69bb9cd50d4 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.602674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 439/892] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:00.608524 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b84d7-2de0-402f-bc1b-f38649fbf8fd None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.609139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b84d7-2de0-402f-bc1b-f38649fbf8fd None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.609139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b84d7-2de0-402f-bc1b-f38649fbf8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.629930 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d60b84d7-2de0-402f-bc1b-f38649fbf8fd None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:00.631706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d60b84d7-2de0-402f-bc1b-f38649fbf8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:00.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:00.691965 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db160e1b-330e-4cdc-a267-b25c050fde3a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:00.726202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:00.735222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 440/893] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:00.740777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:00.741747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.742297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.742822 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.743853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:00.744407 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:00.746167 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:00.756518 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 454/894] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:15:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:00.763111 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b84d7-2de0-402f-bc1b-f38649fbf8fd None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:00.764697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:00.897678 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235cae68-b135-4b0f-b644-3ed2931f7f4d None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:00.915166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b84d7-2de0-402f-bc1b-f38649fbf8fd None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:00.949469 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0f047-1f35-47de-a05e-cc0297e28a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:00.950150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f047-1f35-47de-a05e-cc0297e28a41 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:00.950438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f047-1f35-47de-a05e-cc0297e28a41 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:00.950715 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f047-1f35-47de-a05e-cc0297e28a41 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:00.951650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0f047-1f35-47de-a05e-cc0297e28a41 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:00.953073 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0f047-1f35-47de-a05e-cc0297e28a41 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:00.982258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 455/895] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:15:00 2022] POST /identity/v3/services => generated 241 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:01.005854 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:01.007297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.007865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.008368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.012304 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:01.024499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:01.026451 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:01.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:01.045480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:01.046533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:01.046533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:01.046533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:01.046533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:01.048747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 441/896] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:01.054982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0f047-1f35-47de-a05e-cc0297e28a41 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:01.115386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:01.182597 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0f047-1f35-47de-a05e-cc0297e28a41 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:01.222006 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9379ecb-d4f9-4542-9072-616f43575fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:01.222686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379ecb-d4f9-4542-9072-616f43575fdc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.222974 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379ecb-d4f9-4542-9072-616f43575fdc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.223254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379ecb-d4f9-4542-9072-616f43575fdc None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.224119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9379ecb-d4f9-4542-9072-616f43575fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:01.224437 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9379ecb-d4f9-4542-9072-616f43575fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:01.225889 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9379ecb-d4f9-4542-9072-616f43575fdc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:01.240411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 456/897] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:01.246278 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:01.292289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8a8a-bf07-4e45-a5b1-6747b4a9e2be None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.292619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8a8a-bf07-4e45-a5b1-6747b4a9e2be None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.292829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8a8a-bf07-4e45-a5b1-6747b4a9e2be None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.294530 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 457/898] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:01.304166 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14eeab-5b8c-4d01-b629-1371c9727011 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.304673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14eeab-5b8c-4d01-b629-1371c9727011 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.305054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14eeab-5b8c-4d01-b629-1371c9727011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.333386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-1b14eeab-5b8c-4d01-b629-1371c9727011 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:01.334859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-1b14eeab-5b8c-4d01-b629-1371c9727011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:01.395643 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019daab-901e-4d31-81bd-573635e98bf6 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:01.426266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 458/899] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:01.441092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:01.441877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.442350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.442576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14eeab-5b8c-4d01-b629-1371c9727011 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:01.442673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.453540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:01.455594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:15:01.470010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:01.470985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:01.472399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 442/900] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:15:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:01.479163 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:01.497892 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:01.569684 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:01.598013 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14eeab-5b8c-4d01-b629-1371c9727011 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:01.638270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:01.639109 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.639396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.639671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.640314 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:01.641807 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:01.647881 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fd4cd-8928-4d81-bee5-4d3f09ff4d73 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:01.651566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:01.652643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:01.653621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 459/901] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:15:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:01.660237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:01.716201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:01.718028 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.718976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.719827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.721867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:01.722883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:01.726618 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:01.761819 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 443/902] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:15:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:01.770382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:01.797146 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb718ce-45e1-4051-af29-1a9d6cde7cd8 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:01.799002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:01.837518 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:01.838032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.838371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.838651 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.839356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:01.839669 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:01.841231 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:01.850519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 460/903] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:15:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:01.853366 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:01.906493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900ba09-27e8-4789-ad02-cafd72a14524 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:01.915983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:01.949647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcc3af-e597-4984-8927-3db0ff408414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:01.950705 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcc3af-e597-4984-8927-3db0ff408414 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:01.951198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcc3af-e597-4984-8927-3db0ff408414 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:01.951762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcc3af-e597-4984-8927-3db0ff408414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:01.966522 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fadb08-d331-45c4-9695-f4c4c1c7256b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:02.004318 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:02.004972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:02.005258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:02.005550 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:02.006294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:02.006610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:02.008090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:02.018605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 461/904] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:15:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:02.020469 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:02.034049 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcc3af-e597-4984-8927-3db0ff408414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:02.035951 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcc3af-e597-4984-8927-3db0ff408414 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:02.054386 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:02.133758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be92ef-f512-4b12-ba7e-a6ee18b0907f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:02.163803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 444/905] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:01 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:02.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcc3af-e597-4984-8927-3db0ff408414 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:02.178261 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:02.179302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:02.179412 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:02.180342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:02.180971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:02.182216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] RBAC: Adding query filter params (interface=public, service_id=3f65b27a4dfe4030aa940d01dabf8f30, region_id=RegionOne) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:02.182749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:02.184699 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:02.196439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 462/906] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:15:02 2022] GET /identity/v3/endpoints?service_id=3f65b27a4dfe4030aa940d01dabf8f30&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:02.197798 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:02.291610 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcc3af-e597-4984-8927-3db0ff408414 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:02.306555 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:02.315873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e37191-f990-43ff-b64d-04f7e85c4b2f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:02.331252 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:02.332065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:02.332244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:02.332528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:02.345924 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:02.347777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:02.356947 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd9655-6f00-458f-844b-9082c1c2d1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:02.357964 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9655-6f00-458f-844b-9082c1c2d1a1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:02.358473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9655-6f00-458f-844b-9082c1c2d1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:02.358949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9655-6f00-458f-844b-9082c1c2d1a1 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:02.359827 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd9655-6f00-458f-844b-9082c1c2d1a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:02.361525 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd9655-6f00-458f-844b-9082c1c2d1a1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:02.366880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:02.367846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:02.368744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:02.368744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:02.368744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:02.368744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:02.368744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 445/907] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:02.374234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 463/908] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:15:02 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:15:02.379434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:02.460561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:02.522209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906db887-2944-46cf-948e-f808f88716e8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:02.573175 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa35bac-d64c-4ff2-87da-f16f99da9556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:02.574244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa35bac-d64c-4ff2-87da-f16f99da9556 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:02.574751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa35bac-d64c-4ff2-87da-f16f99da9556 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:02.575214 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa35bac-d64c-4ff2-87da-f16f99da9556 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:02.576533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa35bac-d64c-4ff2-87da-f16f99da9556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:02.577088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa35bac-d64c-4ff2-87da-f16f99da9556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:02.579757 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa35bac-d64c-4ff2-87da-f16f99da9556 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:02.599393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 446/909] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:02.605580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd9655-6f00-458f-844b-9082c1c2d1a1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:02.675058 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91064a6a-6c23-4d79-81ce-12cf6505b2af None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:02.675608 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91064a6a-6c23-4d79-81ce-12cf6505b2af None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:02.675930 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91064a6a-6c23-4d79-81ce-12cf6505b2af None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:02.678603 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 447/910] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:15:02.684617 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5619fa-e371-4d34-a553-da5843664827 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:02.684985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5619fa-e371-4d34-a553-da5843664827 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:02.685294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5619fa-e371-4d34-a553-da5843664827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:02.710475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-0e5619fa-e371-4d34-a553-da5843664827 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:02.711772 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-0e5619fa-e371-4d34-a553-da5843664827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:02.760517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd9655-6f00-458f-844b-9082c1c2d1a1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:02.832148 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:02.832799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:02.833082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:02.838132 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:02.838299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 448/911] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:02.849792 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:02.852605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:02.862450 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5619fa-e371-4d34-a553-da5843664827 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:15:02.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:02.880526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:02.881459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:15:02.881459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:15:02.881459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:15:02.881459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:15:02.881459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:15:02.881459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:15:02.881459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:02.881459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:02.885484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 464/912] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:02.889898 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:02.925666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:02.933253 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:03.014376 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5619fa-e371-4d34-a553-da5843664827 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:03.063978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:03.064995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.065687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.066139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.067306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:03.069615 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:03.073409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96193b-8ba6-4a0a-b652-badec3626bac None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:03.084972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:03.086150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:15:03.086994 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:03.096978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 449/913] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:15:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:03.098534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:03.133972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:03.135283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.135769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.136521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.137774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:03.138331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:03.140477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:03.165768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 465/914] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:03.167110 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:03.256420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e207c-a7cc-428c-8151-0186242dac57 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:03.258182 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:03.276868 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:03.286185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fbabc-23e1-48ce-aec3-ac2206e14081 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:03.310265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746029c-5994-44f1-8102-3b8044482bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:03.311280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746029c-5994-44f1-8102-3b8044482bbf None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.311811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746029c-5994-44f1-8102-3b8044482bbf None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.312282 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746029c-5994-44f1-8102-3b8044482bbf None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.313327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746029c-5994-44f1-8102-3b8044482bbf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:03.313862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746029c-5994-44f1-8102-3b8044482bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:03.315315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746029c-5994-44f1-8102-3b8044482bbf None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:03.326126 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 450/915] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:15:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:03.327327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c5477-26eb-47c6-869a-9067eec54551 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.327622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c5477-26eb-47c6-869a-9067eec54551 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.327898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c5477-26eb-47c6-869a-9067eec54551 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.333389 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 451/916] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:03.333389 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c5477-26eb-47c6-869a-9067eec54551 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:03.348511 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff699bd4-c98e-4cbf-a3cf-0c24905e6f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:03.349734 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff699bd4-c98e-4cbf-a3cf-0c24905e6f1d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.350012 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff699bd4-c98e-4cbf-a3cf-0c24905e6f1d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.350582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff699bd4-c98e-4cbf-a3cf-0c24905e6f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.351984 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff699bd4-c98e-4cbf-a3cf-0c24905e6f1d None admin] RBAC: Adding query filter params (role.id=8c3b00e9cab2495aaeef5f03675c34e0, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=74f4cb06ce4548aea8e61a57c984fe70) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:03.352340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff699bd4-c98e-4cbf-a3cf-0c24905e6f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:03.353833 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff699bd4-c98e-4cbf-a3cf-0c24905e6f1d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:03.360501 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 466/917] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:03 2022] GET /identity/v3/role_assignments?user.id=74f4cb06ce4548aea8e61a57c984fe70&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=8c3b00e9cab2495aaeef5f03675c34e0 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:03.361589 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049524-8c58-45b6-9300-b340379e3e49 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.361871 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049524-8c58-45b6-9300-b340379e3e49 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.362187 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049524-8c58-45b6-9300-b340379e3e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.392453 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-fb049524-8c58-45b6-9300-b340379e3e49 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:03.397090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-fb049524-8c58-45b6-9300-b340379e3e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:03.506056 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 467/918] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:03.520179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c5477-26eb-47c6-869a-9067eec54551 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:03.539826 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049524-8c58-45b6-9300-b340379e3e49 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:03.575824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:03.577348 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.578141 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.578995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.580516 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:03.581332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:03.584177 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:03.594761 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 452/919] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:15:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:03.600894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:03.644582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:03.688682 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049524-8c58-45b6-9300-b340379e3e49 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:03.702744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:03.753572 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:03.754392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.754750 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.755094 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.755825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:03.757718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:03.772131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:03.773114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:03.774160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 468/920] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:15:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:03.779649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:03.793740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757cc40-e04b-454e-a3b7-b80cfa51483b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:03.835329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94500c6-5125-45b9-beff-cec538e64ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:03.836004 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94500c6-5125-45b9-beff-cec538e64ec2 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.836244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94500c6-5125-45b9-beff-cec538e64ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.836454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94500c6-5125-45b9-beff-cec538e64ec2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.837181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94500c6-5125-45b9-beff-cec538e64ec2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:03.838885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94500c6-5125-45b9-beff-cec538e64ec2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:03.878743 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 453/921] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:15:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:15:03.933670 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75101164-b744-44df-aa6a-1b154b2485f6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:03.984709 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:03.991124 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:03.991776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:03.992060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:03.992336 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:03.993039 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:03.993342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:03.994811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:04.001344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 469/922] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:15:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:04.010296 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94500c6-5125-45b9-beff-cec538e64ec2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:04.131655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:04.138520 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94500c6-5125-45b9-beff-cec538e64ec2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:04.178576 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:04.179223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:04.179509 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:04.179793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:04.180501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:04.180829 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:04.182363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:04.188878 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 454/923] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:15:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:04.195752 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:04.344629 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb10d6-0f22-4a55-8d8c-d46b51b2c5fa None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:04.404866 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3640a2-032b-4377-8316-c59ef596bd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:04.405785 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3640a2-032b-4377-8316-c59ef596bd5b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:04.406087 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3640a2-032b-4377-8316-c59ef596bd5b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:04.406389 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3640a2-032b-4377-8316-c59ef596bd5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:04.407406 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3640a2-032b-4377-8316-c59ef596bd5b None admin] RBAC: Adding query filter params (interface=public, service_id=115f5c3d23004ad3907bc38469c30f8d, region_id=RegionOne) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:04.407713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3640a2-032b-4377-8316-c59ef596bd5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:04.409450 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3640a2-032b-4377-8316-c59ef596bd5b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:04.421614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 470/924] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:15:04 2022] GET /identity/v3/endpoints?service_id=115f5c3d23004ad3907bc38469c30f8d&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:04.430909 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:04.572069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dbde7-8bfe-4c9a-a8e3-2ac6947627f3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:04.620775 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aec73e-4aa0-482f-b530-9e95fa459593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:04.621497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aec73e-4aa0-482f-b530-9e95fa459593 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:04.621784 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aec73e-4aa0-482f-b530-9e95fa459593 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:04.622065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aec73e-4aa0-482f-b530-9e95fa459593 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:04.623035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aec73e-4aa0-482f-b530-9e95fa459593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:04.624914 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aec73e-4aa0-482f-b530-9e95fa459593 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:04.633409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 455/925] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:15:04 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:15:04.777158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04b49f-0f54-4197-8e5a-129dab08bf1d None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:04.777525 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04b49f-0f54-4197-8e5a-129dab08bf1d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:04.777710 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04b49f-0f54-4197-8e5a-129dab08bf1d None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:04.779425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 471/926] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:04.785544 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084ff3-25eb-4036-8c55-ab5f04781809 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:04.786151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084ff3-25eb-4036-8c55-ab5f04781809 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:04.786151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084ff3-25eb-4036-8c55-ab5f04781809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:04.810520 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-72084ff3-25eb-4036-8c55-ab5f04781809 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:04.811961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-72084ff3-25eb-4036-8c55-ab5f04781809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:04.834594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:04.897849 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 456/927] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:04.920427 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04b49f-0f54-4197-8e5a-129dab08bf1d None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:04.989096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:05.040106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04b49f-0f54-4197-8e5a-129dab08bf1d None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:05.083252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:05.084014 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:05.084297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:05.084574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:05.085256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:05.086832 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:05.096863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:05.097776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:05.099041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 472/928] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:15:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:05.107495 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72084ff3-25eb-4036-8c55-ab5f04781809 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:05.225408 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72084ff3-25eb-4036-8c55-ab5f04781809 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:05.271047 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:05.272119 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:05.272607 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:05.273083 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:05.273981 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:05.274700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:05.276621 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:05.284178 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 457/929] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:15:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:05.289993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:05.429995 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5148-e6f7-46f5-874c-7872dee57fb7 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:05.472721 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:05.477479 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e4f47-e3e4-4d4a-8a54-180440cae6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:05.478146 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e4f47-e3e4-4d4a-8a54-180440cae6a4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:05.478438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e4f47-e3e4-4d4a-8a54-180440cae6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:05.478713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e4f47-e3e4-4d4a-8a54-180440cae6a4 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:05.479411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e4f47-e3e4-4d4a-8a54-180440cae6a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:05.479719 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e4f47-e3e4-4d4a-8a54-180440cae6a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:05.481124 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e4f47-e3e4-4d4a-8a54-180440cae6a4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:05.487353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 473/930] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:15:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:05.495255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:05.516600 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:05.603375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3e921-a82d-4d28-99d1-eacdae01a5b5 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:05.604037 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3e921-a82d-4d28-99d1-eacdae01a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:05.604514 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3e921-a82d-4d28-99d1-eacdae01a5b5 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:05.606303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7b5d3-a673-4e15-a80a-c83cb0d34b6c None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:05.607162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 474/931] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:15:05.613447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0cd0a-10d7-4886-8950-179f2e1053a2 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:05.614252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0cd0a-10d7-4886-8950-179f2e1053a2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:05.614728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0cd0a-10d7-4886-8950-179f2e1053a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:05.637829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-e1e0cd0a-10d7-4886-8950-179f2e1053a2 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:05.639620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-e1e0cd0a-10d7-4886-8950-179f2e1053a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:05.660374 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b55bd-b89b-43bb-bb1f-a0e483379ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:05.661517 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b55bd-b89b-43bb-bb1f-a0e483379ffd None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:05.662043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b55bd-b89b-43bb-bb1f-a0e483379ffd None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:05.662629 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b55bd-b89b-43bb-bb1f-a0e483379ffd None admin] PATH_INFO: `/v3/endpoints` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:05.663760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b55bd-b89b-43bb-bb1f-a0e483379ffd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:05.665585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b55bd-b89b-43bb-bb1f-a0e483379ffd None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:05.695673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 458/932] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:15:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:05.764948 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 475/933] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:05.792342 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b55bd-b89b-43bb-bb1f-a0e483379ffd None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:05.899368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:05.936820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b55bd-b89b-43bb-bb1f-a0e483379ffd None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:05.988921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d041cad-5fdd-4972-ba60-47dd0ae8f004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:05.989622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d041cad-5fdd-4972-ba60-47dd0ae8f004 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:05.989907 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d041cad-5fdd-4972-ba60-47dd0ae8f004 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:05.990222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d041cad-5fdd-4972-ba60-47dd0ae8f004 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.003465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d041cad-5fdd-4972-ba60-47dd0ae8f004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-2d041cad-5fdd-4972-ba60-47dd0ae8f004 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:06.011184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:06.012330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:06.014363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 459/934] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:15:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:06.020264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0cd0a-10d7-4886-8950-179f2e1053a2 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:06.143016 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0cd0a-10d7-4886-8950-179f2e1053a2 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:06.171472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:06.184028 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9784-f77d-4e01-bbdc-2a6f5a88a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:06.184673 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9784-f77d-4e01-bbdc-2a6f5a88a070 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:06.184959 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9784-f77d-4e01-bbdc-2a6f5a88a070 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:06.185238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9784-f77d-4e01-bbdc-2a6f5a88a070 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.185988 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9784-f77d-4e01-bbdc-2a6f5a88a070 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:06.186329 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9784-f77d-4e01-bbdc-2a6f5a88a070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:06.187731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9784-f77d-4e01-bbdc-2a6f5a88a070 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:06.198643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 476/935] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:15:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:06.205605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d041cad-5fdd-4972-ba60-47dd0ae8f004 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:06.253726 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e09ac6-8a0c-42f7-a455-b28b9191dba1 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:06.254153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e09ac6-8a0c-42f7-a455-b28b9191dba1 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:06.254709 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e09ac6-8a0c-42f7-a455-b28b9191dba1 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.257331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 477/936] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:06.263726 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa8fa-20ba-4c3c-b7ee-b6bfecb41c53 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:06.264314 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa8fa-20ba-4c3c-b7ee-b6bfecb41c53 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:06.264891 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa8fa-20ba-4c3c-b7ee-b6bfecb41c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.284502 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-4e3aa8fa-20ba-4c3c-b7ee-b6bfecb41c53 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:06.286404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-4e3aa8fa-20ba-4c3c-b7ee-b6bfecb41c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:06.324339 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d041cad-5fdd-4972-ba60-47dd0ae8f004 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:06.371595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:06.372650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:06.373399 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:06.373963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.377707 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:06.389416 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 478/937] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:06.390702 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:06.392649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:06.412125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3aa8fa-20ba-4c3c-b7ee-b6bfecb41c53 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:06.413818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:06.414808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:06.415638 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:06.415638 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:06.415638 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:06.415638 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:06.423856 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 460/938] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:06.426298 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:06.534368 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3aa8fa-20ba-4c3c-b7ee-b6bfecb41c53 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:06.543151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:06.569862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75453947-9a47-4830-8b7f-e53ec97642d0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:06.593512 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:06.595201 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:06.596245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:06.597165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.598621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:06.601065 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:06.605242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: WARNING keystone.server.flask.common [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 08:15:06.614157 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:06.614913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:06.615218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:06.615495 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.616350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:06.616658 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:06.618280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:06.634298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.sql.core [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:15:06.634298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:15:06.634298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [parameters: {'id': '8150e6f312444bd8be6f4d27a8c2824d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:15:06.634298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=144042) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:15:06.640557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 461/939] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:06.642208 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8150e6f312444bd8be6f4d27a8c2824d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8150e6f312444bd8be6f4d27a8c2824d. Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:15:06.647761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result.read() Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 08:15:06.648950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application yield resource Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise self.value Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application session.commit() Jul 28 08:15:06.649876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:15:06.651803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:15:06.652930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result.read() Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:15:06.653769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [parameters: {'id': '8150e6f312444bd8be6f4d27a8c2824d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:06.656294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:06.658767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8150e6f312444bd8be6f4d27a8c2824d. Jul 28 08:15:06.669523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:06.672163 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 479/940] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:15:06 2022] POST /identity/v3/projects => generated 262 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:06.683509 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:06.711030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:06.809481 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586023-304f-40aa-bce5-d659ad6cd68d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:06.839762 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ca6f-7245-43a0-b690-02377d0e0bb3 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:06.850894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:06.851623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:06.851939 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:06.852234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.861091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:06.862998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:15:06.877106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:06.878147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:15:06.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:06.883837 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 462/941] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:15:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:06.885334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:06.892679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:06.893750 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:06.894313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:06.894783 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:06.903639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:06.905642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:15:06.919825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:06.920690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:06.921803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:15:06.921803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:15:06.921803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:15:06.921803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:15:06.921803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:15:06.921803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:15:06.921803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:06.921803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:06.929373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 480/942] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:06.930705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:06.959502 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:07.013440 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e606f-cae6-4e86-b11c-12928ddd35a3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:07.026583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:07.059421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563ead8-72b3-4c8a-b758-472fc0f1cbee None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:07.065362 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1464d-cbea-4b4f-9c33-48107eb794e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:07.066238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1464d-cbea-4b4f-9c33-48107eb794e3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.066639 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1464d-cbea-4b4f-9c33-48107eb794e3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.066971 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1464d-cbea-4b4f-9c33-48107eb794e3 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.068142 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1464d-cbea-4b4f-9c33-48107eb794e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:07.069269 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1464d-cbea-4b4f-9c33-48107eb794e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:07.070195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1464d-cbea-4b4f-9c33-48107eb794e3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:07.102180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 463/943] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:15:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:07.102180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d631-4ffa-41aa-90a5-d257a97ab421 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.102180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d631-4ffa-41aa-90a5-d257a97ab421 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.102180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d631-4ffa-41aa-90a5-d257a97ab421 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.102938 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 464/944] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:07.104332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294d631-4ffa-41aa-90a5-d257a97ab421 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:07.119185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8224551-dce0-4616-af0d-908c2020a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:07.119977 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8224551-dce0-4616-af0d-908c2020a44b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.120349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8224551-dce0-4616-af0d-908c2020a44b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.120654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8224551-dce0-4616-af0d-908c2020a44b None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.121496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8224551-dce0-4616-af0d-908c2020a44b None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:07.121967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8224551-dce0-4616-af0d-908c2020a44b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:07.123709 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8224551-dce0-4616-af0d-908c2020a44b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:07.166152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 481/945] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:15:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:07.166152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abddca3-54a9-4ccf-842f-e2b1f3d71737 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.166152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abddca3-54a9-4ccf-842f-e2b1f3d71737 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.166152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abddca3-54a9-4ccf-842f-e2b1f3d71737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.201765 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-8abddca3-54a9-4ccf-842f-e2b1f3d71737 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:07.204802 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-8abddca3-54a9-4ccf-842f-e2b1f3d71737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:07.259750 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294d631-4ffa-41aa-90a5-d257a97ab421 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:07.314338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 482/946] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:07.323680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:07.324469 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.325028 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.325407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.338563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abddca3-54a9-4ccf-842f-e2b1f3d71737 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:07.343942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:07.345640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:07.364987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:07.366060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:07.367221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:07.367221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:07.367221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:07.367221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:07.375204 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 465/947] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:07.376989 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:07.378034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:07.404377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:07.515076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abddca3-54a9-4ccf-842f-e2b1f3d71737 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:07.527882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfff592-4e0b-4ad5-83df-ada94fe4e157 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:07.574809 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:07.575710 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.576114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.576493 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.577428 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:07.579297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:07.587124 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:07.588034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.588456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.588868 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.589709 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:07.590186 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:07.591216 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:07.592306 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:07.593247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:07.594846 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:07.595837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 483/948] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:15:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:07.602157 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:07.625950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 466/949] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:07.631483 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:07.744667 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:07.779191 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:07.783697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857e0ad-48ed-4963-9547-5cb5588cad2a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:07.811294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18ef42-9564-45fa-8c15-d7b2e0cfd217 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:07.837799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:07.838485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.838768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.839765 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.840497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:07.840807 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:07.842749 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:07.851633 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 484/950] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:15:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:07.857141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:07.860709 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:07.861975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:07.862621 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:07.863093 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:07.874097 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:07.876474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:15:07.892631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:07.893734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:07.894803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:15:07.894803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:15:07.894803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:15:07.894803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:15:07.894803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:15:07.894803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:15:07.894803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:07.894803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:07.897165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 467/951] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:07.907368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:08.038829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b55944-cf43-45d7-90de-5db311917fde None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:08.083127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3dc29-f12f-4dad-a821-53a5e9407dc4 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:08.093612 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:08.094397 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:08.094753 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:08.095100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:08.095884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:08.096571 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:08.097780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:08.104802 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 485/952] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:15:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:08.111226 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:08.118459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:08.141349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053f463-190e-4065-974f-064706e67468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:08.142328 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053f463-190e-4065-974f-064706e67468 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:08.143524 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053f463-190e-4065-974f-064706e67468 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:08.144016 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053f463-190e-4065-974f-064706e67468 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:08.145853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053f463-190e-4065-974f-064706e67468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:08.146670 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053f463-190e-4065-974f-064706e67468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:08.150642 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053f463-190e-4065-974f-064706e67468 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:08.155766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:08.174143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 468/953] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:08.174143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c71d-e70c-48df-a933-faef3b48e48d None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:08.174143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c71d-e70c-48df-a933-faef3b48e48d None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:08.174143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c71d-e70c-48df-a933-faef3b48e48d None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:08.174143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 469/954] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:08.178726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9c71d-e70c-48df-a933-faef3b48e48d None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:08.248200 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c329a9-eec5-4054-b6a4-e9bbe5e6a819 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:08.295403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c678d-e8e5-4dcb-971e-6e48345d94fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:08.295951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c678d-e8e5-4dcb-971e-6e48345d94fa None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:08.296233 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c678d-e8e5-4dcb-971e-6e48345d94fa None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:08.296551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c678d-e8e5-4dcb-971e-6e48345d94fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:08.297258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c678d-e8e5-4dcb-971e-6e48345d94fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:08.298783 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c678d-e8e5-4dcb-971e-6e48345d94fa None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:08.334198 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 486/955] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:15:08 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:08.336482 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4b63-8b77-49e1-86af-3e76ace03eaa None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:08.337028 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4b63-8b77-49e1-86af-3e76ace03eaa None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:08.337358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4b63-8b77-49e1-86af-3e76ace03eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:08.360604 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-13cb4b63-8b77-49e1-86af-3e76ace03eaa None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:08.362332 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-13cb4b63-8b77-49e1-86af-3e76ace03eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:08.369973 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9c71d-e70c-48df-a933-faef3b48e48d None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:08.422146 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38be652-e18a-4d6e-a175-4eb28cc07968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:08.423436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38be652-e18a-4d6e-a175-4eb28cc07968 None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:08.423961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38be652-e18a-4d6e-a175-4eb28cc07968 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:08.424412 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38be652-e18a-4d6e-a175-4eb28cc07968 None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/74f4cb06ce4548aea8e61a57c984fe70/roles/8c3b00e9cab2495aaeef5f03675c34e0` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:08.427270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38be652-e18a-4d6e-a175-4eb28cc07968 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=74f4cb06ce4548aea8e61a57c984fe70, role_id=8c3b00e9cab2495aaeef5f03675c34e0)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:08.429169 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38be652-e18a-4d6e-a175-4eb28cc07968 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:08.462583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 470/956] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:15:08 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/74f4cb06ce4548aea8e61a57c984fe70/roles/8c3b00e9cab2495aaeef5f03675c34e0 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:15:08.463365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:08.478998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 487/957] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:08.517632 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38be652-e18a-4d6e-a175-4eb28cc07968 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:08.634422 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:08.703475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38be652-e18a-4d6e-a175-4eb28cc07968 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:08.754460 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:08.755306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:08.755614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:08.756034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] PATH_INFO: `/v3/services/compute` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:08.756693 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:08.758333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:08.770454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:08.772281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:08.773831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 471/958] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:08.779581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb4b63-8b77-49e1-86af-3e76ace03eaa None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:08.927919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb4b63-8b77-49e1-86af-3e76ace03eaa None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:08.991362 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f698d-7a44-430c-aca8-315cd613bd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:08.992019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f698d-7a44-430c-aca8-315cd613bd89 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:08.992304 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f698d-7a44-430c-aca8-315cd613bd89 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:08.992578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f698d-7a44-430c-aca8-315cd613bd89 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:08.993286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f698d-7a44-430c-aca8-315cd613bd89 None admin] RBAC: Adding query filter params (name=compute) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:08.993599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f698d-7a44-430c-aca8-315cd613bd89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:08.995127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f698d-7a44-430c-aca8-315cd613bd89 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:09.001298 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 488/959] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:15:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:09.009195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:09.128432 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a6014-5417-409e-89d6-01d42622cede None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:09.146586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:09.166080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae785367-090f-4492-ac22-f87f1dc4e60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:09.166760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785367-090f-4492-ac22-f87f1dc4e60c None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:09.167041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785367-090f-4492-ac22-f87f1dc4e60c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:09.167313 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785367-090f-4492-ac22-f87f1dc4e60c None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:09.168012 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae785367-090f-4492-ac22-f87f1dc4e60c None admin] RBAC: Adding query filter params (type=compute) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:09.168310 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae785367-090f-4492-ac22-f87f1dc4e60c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:09.169733 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae785367-090f-4492-ac22-f87f1dc4e60c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:09.175607 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 472/960] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:15:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:09.687638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cae6d2-1046-4bed-bb83-1d98fb23ae9f None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:09.688241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cae6d2-1046-4bed-bb83-1d98fb23ae9f None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:09.688241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cae6d2-1046-4bed-bb83-1d98fb23ae9f None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:09.690008 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 489/961] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:09.696871 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4ac87-0ad7-40fd-8ba9-d3e1e1f80ede None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:09.697310 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4ac87-0ad7-40fd-8ba9-d3e1e1f80ede None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:09.697729 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4ac87-0ad7-40fd-8ba9-d3e1e1f80ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:09.723552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-70e4ac87-0ad7-40fd-8ba9-d3e1e1f80ede None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:09.724959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-70e4ac87-0ad7-40fd-8ba9-d3e1e1f80ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:09.817297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 473/962] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:09.843339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cae6d2-1046-4bed-bb83-1d98fb23ae9f None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:09.907584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:09.962459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cae6d2-1046-4bed-bb83-1d98fb23ae9f None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:10.000696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:10.001340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:10.001628 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:10.001910 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:10.002649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:10.004110 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:10.020403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.sql.core [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:15:10.020403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:15:10.020403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [parameters: {'user_id': 'd643e1e4d6864b16a5c657ca9639b552', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:15:10.020403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=144042) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result.read() Jul 28 08:15:10.028202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:15:10.029162 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application yield resource Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise self.value Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application session.commit() Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:15:10.031267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:15:10.032496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:15:10.033340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application result.read() Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:15:10.034339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd643e1e4d6864b16a5c657ca9639b552', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:10.041851 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 08:15:10.044206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 08:15:10.047285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 08:15:10.047285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 08:15:10.047285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:15:10.047285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:15:10.047285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:15:10.047285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:10.047285 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 490/963] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:15:09 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:10.048152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4ac87-0ad7-40fd-8ba9-d3e1e1f80ede None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:10.170722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4ac87-0ad7-40fd-8ba9-d3e1e1f80ede None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:10.186711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:10.214323 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45db15-9a6f-42b0-b219-cb3018d48218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:10.214875 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45db15-9a6f-42b0-b219-cb3018d48218 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:10.215164 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45db15-9a6f-42b0-b219-cb3018d48218 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:10.215447 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45db15-9a6f-42b0-b219-cb3018d48218 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:10.223769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45db15-9a6f-42b0-b219-cb3018d48218 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:10.225266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45db15-9a6f-42b0-b219-cb3018d48218 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-1c45db15-9a6f-42b0-b219-cb3018d48218 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:15:10.238431 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:10.239543 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:10.241306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 474/964] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:15:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:10.246272 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:10.354978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a6323-d591-4dce-9bbd-13ec77c41fe5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:10.379585 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:10.394485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c22b3a-224b-4074-b083-e916ac6e1866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:10.395217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c22b3a-224b-4074-b083-e916ac6e1866 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:10.395873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c22b3a-224b-4074-b083-e916ac6e1866 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:10.396161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c22b3a-224b-4074-b083-e916ac6e1866 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:10.396974 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c22b3a-224b-4074-b083-e916ac6e1866 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:10.397724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c22b3a-224b-4074-b083-e916ac6e1866 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:10.399954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c22b3a-224b-4074-b083-e916ac6e1866 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:10.441331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 491/965] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:15:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:10.709684 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263b5a-5907-413d-ad87-72c58345f628 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:10.710040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263b5a-5907-413d-ad87-72c58345f628 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:10.710287 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263b5a-5907-413d-ad87-72c58345f628 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:10.712038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 475/966] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:10.718036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65674-2658-49ba-af4f-a19a8c747d37 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:10.718564 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65674-2658-49ba-af4f-a19a8c747d37 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:10.718640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65674-2658-49ba-af4f-a19a8c747d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:10.743171 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-83a65674-2658-49ba-af4f-a19a8c747d37 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:10.744740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-83a65674-2658-49ba-af4f-a19a8c747d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:10.845747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 492/967] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:10.874189 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39263b5a-5907-413d-ad87-72c58345f628 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.002507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e322c3-c04b-43c4-9b2e-d7011b5c2cf8 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.002700 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e322c3-c04b-43c4-9b2e-d7011b5c2cf8 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.003092 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e322c3-c04b-43c4-9b2e-d7011b5c2cf8 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.004665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 493/968] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:15:11.010028 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974558a5-453c-4c64-a9c3-906c0b8d0491 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.010352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974558a5-453c-4c64-a9c3-906c0b8d0491 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.010637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974558a5-453c-4c64-a9c3-906c0b8d0491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.022079 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39263b5a-5907-413d-ad87-72c58345f628 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:11.030635 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-974558a5-453c-4c64-a9c3-906c0b8d0491 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:11.031936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-974558a5-453c-4c64-a9c3-906c0b8d0491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:11.065091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:11.066122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.066517 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.066908 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.067880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:11.069652 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:11.080649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:11.087033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:11.088604 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 476/969] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:15:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:11.100405 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.126153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 494/970] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:11.148861 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974558a5-453c-4c64-a9c3-906c0b8d0491 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.161940 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:11.201792 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:11.255114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ce26-dae2-4b6d-9ae5-e06c1b339c5f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:11.285143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974558a5-453c-4c64-a9c3-906c0b8d0491 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:11.298488 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:11.299294 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.299587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.299931 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.300634 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:11.300946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:11.302421 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:11.308021 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 477/971] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:15:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:11.314414 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.326793 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8e70d-c343-471a-9bd1-07162f819056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:11.327586 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e70d-c343-471a-9bd1-07162f819056 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.327868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e70d-c343-471a-9bd1-07162f819056 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.328141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e70d-c343-471a-9bd1-07162f819056 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.340433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8e70d-c343-471a-9bd1-07162f819056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-eda8e70d-c343-471a-9bd1-07162f819056 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:11.345776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:11.346954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:11.351242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 495/972] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:15:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:11.354159 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8e70d-c343-471a-9bd1-07162f819056 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.457731 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:11.467816 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad47c42-ccdd-45e7-bd02-50de8a4e92b5 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:11.484358 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:11.492397 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8e70d-c343-471a-9bd1-07162f819056 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:11.519365 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:11.520130 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.520432 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.521043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.521323 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:11.521631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:11.523420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:11.529017 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 478/973] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:15:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:11.536549 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.541456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:11.541456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.541680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.541923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.542621 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:11.542936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:11.544402 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:11.554860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 496/974] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:15:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:11.560602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.669342 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8cb24-21d7-4c78-b9dc-7550f5cbe019 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:11.692743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f60e4-6816-401e-ab76-bba42a6b64a0 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:11.708811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c46eeb-48c8-4030-9cc3-7f632f20aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:11.709446 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46eeb-48c8-4030-9cc3-7f632f20aeff None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.709751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46eeb-48c8-4030-9cc3-7f632f20aeff None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.710030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46eeb-48c8-4030-9cc3-7f632f20aeff None admin] PATH_INFO: `/v3/endpoints` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.710841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c46eeb-48c8-4030-9cc3-7f632f20aeff None admin] RBAC: Adding query filter params (interface=admin, service_id=115f5c3d23004ad3907bc38469c30f8d, region_id=RegionOne) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:11.711156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c46eeb-48c8-4030-9cc3-7f632f20aeff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:11.712563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c46eeb-48c8-4030-9cc3-7f632f20aeff None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:11.719434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 479/975] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jul 28 08:15:11 2022] GET /identity/v3/endpoints?service_id=115f5c3d23004ad3907bc38469c30f8d&interface=admin®ion_id=RegionOne => generated 193 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:11.721055 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32971393-4c2e-4811-ae40-d6844f9e0ae7 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.721349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32971393-4c2e-4811-ae40-d6844f9e0ae7 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.721792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32971393-4c2e-4811-ae40-d6844f9e0ae7 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.725606 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 480/976] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:11.726697 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32971393-4c2e-4811-ae40-d6844f9e0ae7 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.735353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45168e-3dc4-4e68-8270-0dccffed1b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:11.736231 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45168e-3dc4-4e68-8270-0dccffed1b78 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.736519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45168e-3dc4-4e68-8270-0dccffed1b78 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.736769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45168e-3dc4-4e68-8270-0dccffed1b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.751575 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45168e-3dc4-4e68-8270-0dccffed1b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:11.753069 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45168e-3dc4-4e68-8270-0dccffed1b78 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-ad45168e-3dc4-4e68-8270-0dccffed1b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:11.769139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:11.770254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:11.771113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:11.771113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:11.771113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:11.771113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:11.771325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:11.771396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 497/977] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:11.771798 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def6645-f916-4956-b262-d3312e6fcda6 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.772016 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def6645-f916-4956-b262-d3312e6fcda6 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.772297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def6645-f916-4956-b262-d3312e6fcda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.794030 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-9def6645-f916-4956-b262-d3312e6fcda6 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:11.795362 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-9def6645-f916-4956-b262-d3312e6fcda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:11.797659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:11.874683 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 498/978] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:11.875747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def6645-f916-4956-b262-d3312e6fcda6 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:11.893180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32971393-4c2e-4811-ae40-d6844f9e0ae7 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:11.930881 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa11602-9b77-48fb-898f-6e2c5a028d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:11.931548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa11602-9b77-48fb-898f-6e2c5a028d31 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:11.931853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa11602-9b77-48fb-898f-6e2c5a028d31 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:11.932097 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa11602-9b77-48fb-898f-6e2c5a028d31 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:11.932766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa11602-9b77-48fb-898f-6e2c5a028d31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:11.934329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa11602-9b77-48fb-898f-6e2c5a028d31 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:11.940875 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 481/979] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:15:11 2022] GET /identity/v3/services => generated 1497 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:15:11.941707 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa11602-9b77-48fb-898f-6e2c5a028d31 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:12.022682 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def6645-f916-4956-b262-d3312e6fcda6 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:12.076967 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:12.077602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:12.077880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:12.078185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:12.079038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:12.079343 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:12.080005 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:12.080769 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:12.089130 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:12.100723 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 499/980] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:12.103141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:12.106568 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa11602-9b77-48fb-898f-6e2c5a028d31 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:12.161089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae648-7371-4b90-a239-45cc33b595f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:12.161733 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae648-7371-4b90-a239-45cc33b595f2 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:12.162011 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae648-7371-4b90-a239-45cc33b595f2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:12.162306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae648-7371-4b90-a239-45cc33b595f2 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:12.163019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bae648-7371-4b90-a239-45cc33b595f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:12.164426 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bae648-7371-4b90-a239-45cc33b595f2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:12.183919 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 482/981] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:15:11 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:12.237303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2427d1-0324-4c86-82df-24028e4400fc None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:12.299639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:12.300237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:12.300510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:12.300780 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:12.310056 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:12.311638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:15:12.325353 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:12.326584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:15:12.327476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:12.329069 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 500/982] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:15:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:12.332991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae648-7371-4b90-a239-45cc33b595f2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:12.472064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae648-7371-4b90-a239-45cc33b595f2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:12.522838 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:12.523876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:12.524374 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:12.524896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:12.525921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:12.526453 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:12.528129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:12.550917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 483/983] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:15:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:12.558202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:12.640520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:12.709934 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175f922-f4d1-4c5c-ab5b-b2efaf9d1fa4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:12.758125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba136a-d8d7-4ecf-a446-a9917b275968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:12.759027 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba136a-d8d7-4ecf-a446-a9917b275968 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:12.759105 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba136a-d8d7-4ecf-a446-a9917b275968 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:12.759396 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba136a-d8d7-4ecf-a446-a9917b275968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:12.853841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba136a-d8d7-4ecf-a446-a9917b275968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:12.861542 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba136a-d8d7-4ecf-a446-a9917b275968 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:12.944102 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:13.017011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 501/984] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:12 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:13.024393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:13.036679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:13.139518 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d06a-bef6-4886-b912-0abbb636e359 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:13.184470 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:13.185162 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:13.185446 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:13.185725 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:13.198936 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:13.200480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:13.218074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:13.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:13.226000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:13.226000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:13.226000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:13.226000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:13.226000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 484/985] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:13.226587 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba136a-d8d7-4ecf-a446-a9917b275968 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:13.359485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba136a-d8d7-4ecf-a446-a9917b275968 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:13.407106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:13.407689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:13.408100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:13.408240 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:13.408951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:13.409262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:13.410719 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:13.424630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 502/986] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:13.431162 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:13.509753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:13.562374 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0677973-c634-4214-82ed-1f1e7a74f311 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:13.607404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:13.608460 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:13.608753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:13.609033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:13.617575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:13.619129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:15:13.632315 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:13.640726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:13.641742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 485/987] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:13.642244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:13.668742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:13.814085 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9973a5-7b3d-48e7-a5ca-171aa8f9e07e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:13.863686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220df8a-15b0-4caa-b56c-51cc26c70c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:13.864336 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220df8a-15b0-4caa-b56c-51cc26c70c16 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:13.864614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220df8a-15b0-4caa-b56c-51cc26c70c16 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:13.864865 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220df8a-15b0-4caa-b56c-51cc26c70c16 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:13.865640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220df8a-15b0-4caa-b56c-51cc26c70c16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:13.865953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220df8a-15b0-4caa-b56c-51cc26c70c16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:13.867462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220df8a-15b0-4caa-b56c-51cc26c70c16 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:13.882535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 503/988] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:13.888607 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:14.025007 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22482d-be43-49fc-94f4-d6ff8c0c5836 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:14.049071 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:14.066891 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e65f3-e647-4e37-9594-9750317305bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:14.067531 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e65f3-e647-4e37-9594-9750317305bb None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.067818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e65f3-e647-4e37-9594-9750317305bb None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.068459 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e65f3-e647-4e37-9594-9750317305bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.068908 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e65f3-e647-4e37-9594-9750317305bb None admin] RBAC: Adding query filter params (role.id=8c3b00e9cab2495aaeef5f03675c34e0, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=74f4cb06ce4548aea8e61a57c984fe70) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:14.069222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e65f3-e647-4e37-9594-9750317305bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:14.071248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e65f3-e647-4e37-9594-9750317305bb None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:14.077944 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 486/989] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:13 2022] GET /identity/v3/role_assignments?user.id=74f4cb06ce4548aea8e61a57c984fe70&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=8c3b00e9cab2495aaeef5f03675c34e0 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:14.113422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf69a3-38aa-4ef9-b2bb-8fe571fe2310 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.113768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf69a3-38aa-4ef9-b2bb-8fe571fe2310 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.113953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf69a3-38aa-4ef9-b2bb-8fe571fe2310 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.115731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 504/990] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:14.121486 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb3ad-2749-4cf7-933a-1291f45eda77 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.121824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb3ad-2749-4cf7-933a-1291f45eda77 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.122048 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb3ad-2749-4cf7-933a-1291f45eda77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.145035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-95deb3ad-2749-4cf7-933a-1291f45eda77 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:14.146363 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-95deb3ad-2749-4cf7-933a-1291f45eda77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:14.234279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 487/991] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:14.257633 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf69a3-38aa-4ef9-b2bb-8fe571fe2310 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:14.337807 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd646d-8a87-4324-a6da-d066f8b8405b None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.337983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd646d-8a87-4324-a6da-d066f8b8405b None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.338325 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd646d-8a87-4324-a6da-d066f8b8405b None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.339881 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 488/992] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:15:14.345324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fb534-f571-4ecd-9aa9-1ac529409eeb None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.345627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fb534-f571-4ecd-9aa9-1ac529409eeb None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.345873 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fb534-f571-4ecd-9aa9-1ac529409eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.356564 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:14.367321 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-2f9fb534-f571-4ecd-9aa9-1ac529409eeb None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:14.368551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-2f9fb534-f571-4ecd-9aa9-1ac529409eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:14.388392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf69a3-38aa-4ef9-b2bb-8fe571fe2310 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:14.428954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:14.429625 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.429909 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.430223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.431348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:14.432739 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:14.442614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:14.443610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:14.444404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 505/993] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:15:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:14.450393 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:14.461628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 489/994] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:14.474674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fb534-f571-4ecd-9aa9-1ac529409eeb None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:14.525156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:14.563189 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b43ab-3d8c-4e66-bbf2-4edfea48c47a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:14.587350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fb534-f571-4ecd-9aa9-1ac529409eeb None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:14.599668 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:14.600232 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.600517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.600798 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.601496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:14.601919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:14.603463 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:14.609205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 506/995] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:15:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:14.614893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:14.617936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:14.623155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:14.623822 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.624104 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.624378 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.625015 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:14.627006 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:14.637302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:14.638358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:14.639152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 490/996] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:14.644712 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:14.752129 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae741d73-821f-4658-8c1f-e10a32168084 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:14.783833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256371-8a26-4db4-a24a-22dc3fcd80c3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:14.792690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:14.793421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.793904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.794012 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.794544 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:14.794672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:14.794968 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:14.796716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:14.807648 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 507/997] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:15:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:14.808903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:14.829117 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:14.829755 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.830041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.830349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.831045 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] RBAC: Adding query filter params (name=compute) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:14.831353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:14.832748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:14.838182 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 491/998] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:15:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:14.843725 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:14.880432 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:14.919276 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bde4-18ab-462a-86df-f3be38e3c372 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:14.951666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1424f-ec77-49c6-a700-24879312c613 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:14.955598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cab93-d65c-4022-bb5d-c089f2773061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:14.956241 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cab93-d65c-4022-bb5d-c089f2773061 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:14.956524 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cab93-d65c-4022-bb5d-c089f2773061 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:14.956802 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cab93-d65c-4022-bb5d-c089f2773061 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:14.957563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cab93-d65c-4022-bb5d-c089f2773061 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:14.958996 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cab93-d65c-4022-bb5d-c089f2773061 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:14.978951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 508/999] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:15:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:15:15.009467 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:15.010274 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:15.010629 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:15.010972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:15.011753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] RBAC: Adding query filter params (type=compute) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:15.012129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:15.013649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:15.020123 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 492/1000] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:15:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:15.026175 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cab93-d65c-4022-bb5d-c089f2773061 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:15.147001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cab93-d65c-4022-bb5d-c089f2773061 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:15.177172 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:15.186862 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e31f4-dec0-4d70-8061-61de1e0332f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:15.187504 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e31f4-dec0-4d70-8061-61de1e0332f6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:15.187847 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e31f4-dec0-4d70-8061-61de1e0332f6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:15.188120 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e31f4-dec0-4d70-8061-61de1e0332f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:15.188893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e31f4-dec0-4d70-8061-61de1e0332f6 None admin] RBAC: Adding query filter params (interface=public, service_id=d90f935d343e4fa7961fbf76d7bc4f9a, region_id=RegionOne) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:15.189204 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e31f4-dec0-4d70-8061-61de1e0332f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:15.191015 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e31f4-dec0-4d70-8061-61de1e0332f6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:15.197349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 509/1001] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:15:15 2022] GET /identity/v3/endpoints?service_id=d90f935d343e4fa7961fbf76d7bc4f9a&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:15.203009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:15.237416 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:15.314288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e4fa5-d6d2-4331-9182-9b868efb0b47 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:15.354741 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8c910-5440-4b3c-b36c-3490da1d4acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:15.355397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8c910-5440-4b3c-b36c-3490da1d4acf None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:15.355687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8c910-5440-4b3c-b36c-3490da1d4acf None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:15.355967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8c910-5440-4b3c-b36c-3490da1d4acf None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:15.356719 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8c910-5440-4b3c-b36c-3490da1d4acf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:15.358344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8c910-5440-4b3c-b36c-3490da1d4acf None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:15.365666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 493/1002] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:15:15 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:15:16.158991 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796260-7e59-4829-8134-2f4d2f061472 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:16.159690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796260-7e59-4829-8134-2f4d2f061472 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:16.159690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796260-7e59-4829-8134-2f4d2f061472 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:16.161397 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 510/1003] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:16.167085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808516b0-a82f-44c4-b3ce-fe3e6668b87e None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:16.167336 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808516b0-a82f-44c4-b3ce-fe3e6668b87e None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:16.167649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808516b0-a82f-44c4-b3ce-fe3e6668b87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:16.187420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-808516b0-a82f-44c4-b3ce-fe3e6668b87e None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:16.188704 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-808516b0-a82f-44c4-b3ce-fe3e6668b87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:16.257107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 494/1004] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:16.278843 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c796260-7e59-4829-8134-2f4d2f061472 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:16.387808 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c796260-7e59-4829-8134-2f4d2f061472 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:16.424280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:16.424854 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:16.425138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:16.425418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] PATH_INFO: `/v3/services/image` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:16.426054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:16.427474 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:16.436949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:16.437993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:16.439203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:16.442466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 511/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:15:16 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:16.444553 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808516b0-a82f-44c4-b3ce-fe3e6668b87e None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:16.473885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:16.560151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808516b0-a82f-44c4-b3ce-fe3e6668b87e None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:16.595991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e3946-4110-4d66-9abc-4cf20b3a0c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:16.596621 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e3946-4110-4d66-9abc-4cf20b3a0c8e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:16.596906 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e3946-4110-4d66-9abc-4cf20b3a0c8e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:16.597179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e3946-4110-4d66-9abc-4cf20b3a0c8e None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:16.597869 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e3946-4110-4d66-9abc-4cf20b3a0c8e None admin] RBAC: Adding query filter params (name=image) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:16.598200 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e3946-4110-4d66-9abc-4cf20b3a0c8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:16.599596 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e3946-4110-4d66-9abc-4cf20b3a0c8e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:16.604874 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 495/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:16.610815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:16.688103 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:16.757434 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f24cdc-eb22-45f4-8132-59542acd05a6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:16.794784 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a55c8e-c11c-4715-9d3a-6d1922e75477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:16.795426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a55c8e-c11c-4715-9d3a-6d1922e75477 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:16.795711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a55c8e-c11c-4715-9d3a-6d1922e75477 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:16.795996 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a55c8e-c11c-4715-9d3a-6d1922e75477 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:16.796697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a55c8e-c11c-4715-9d3a-6d1922e75477 None admin] RBAC: Adding query filter params (type=image) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:16.797045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a55c8e-c11c-4715-9d3a-6d1922e75477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:16.798557 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a55c8e-c11c-4715-9d3a-6d1922e75477 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:16.803697 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 512/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:17.053253 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47edb0-ad8a-493c-8eef-dea3ad6e78f5 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:17.053436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47edb0-ad8a-493c-8eef-dea3ad6e78f5 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:17.053710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47edb0-ad8a-493c-8eef-dea3ad6e78f5 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:17.055423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 496/1008] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:17.061855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88533d13-be69-4f31-b713-e4961c12a6c4 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:17.062161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88533d13-be69-4f31-b713-e4961c12a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:17.062430 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88533d13-be69-4f31-b713-e4961c12a6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:17.087155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-88533d13-be69-4f31-b713-e4961c12a6c4 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:17.088466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-88533d13-be69-4f31-b713-e4961c12a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:17.169734 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 513/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:17 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:17.192729 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47edb0-ad8a-493c-8eef-dea3ad6e78f5 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:17.295765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:17.310568 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47edb0-ad8a-493c-8eef-dea3ad6e78f5 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:17.349839 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5021b7-8822-428a-98d9-bdb12f684538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:17.350969 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5021b7-8822-428a-98d9-bdb12f684538 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:17.351563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5021b7-8822-428a-98d9-bdb12f684538 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:17.352033 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5021b7-8822-428a-98d9-bdb12f684538 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:17.353440 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5021b7-8822-428a-98d9-bdb12f684538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:17.355140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5021b7-8822-428a-98d9-bdb12f684538 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:17.357506 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 497/1010] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:17.363560 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88533d13-be69-4f31-b713-e4961c12a6c4 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:17.419912 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:17.469202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4b9c-fec2-4878-be1d-d5cd092fe8dc None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:17.469567 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4b9c-fec2-4878-be1d-d5cd092fe8dc None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:17.470085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4b9c-fec2-4878-be1d-d5cd092fe8dc None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:17.471335 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 498/1011] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:17.477209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81a929-d511-493e-911d-45c56d5a99e4 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:17.477498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81a929-d511-493e-911d-45c56d5a99e4 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:17.477775 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81a929-d511-493e-911d-45c56d5a99e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:17.478872 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88533d13-be69-4f31-b713-e4961c12a6c4 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:17.497292 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-da81a929-d511-493e-911d-45c56d5a99e4 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:17.498586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-da81a929-d511-493e-911d-45c56d5a99e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:17.519694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c523a-5adf-4e1e-8f11-37c84943d986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:17.520342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c523a-5adf-4e1e-8f11-37c84943d986 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:17.520623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c523a-5adf-4e1e-8f11-37c84943d986 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:17.520890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c523a-5adf-4e1e-8f11-37c84943d986 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:17.521608 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c523a-5adf-4e1e-8f11-37c84943d986 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:17.523114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c523a-5adf-4e1e-8f11-37c84943d986 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:17.552442 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 514/1012] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:15:17 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:17.588579 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 499/1013] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:17 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:17.611954 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c523a-5adf-4e1e-8f11-37c84943d986 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:17.751005 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:17.754958 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c523a-5adf-4e1e-8f11-37c84943d986 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:17.793307 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:17.793965 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:17.794283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:17.794559 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:17.795200 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:17.796597 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:17.807458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:17.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:17.809381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 515/1014] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:17.814989 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81a929-d511-493e-911d-45c56d5a99e4 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:17.914081 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:17.930977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81a929-d511-493e-911d-45c56d5a99e4 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:17.973629 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:17.974442 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:17.974796 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:17.975142 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:17.975918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:17.976293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:17.977772 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:17.988534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 500/1015] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:15:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:17.989679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:18.104198 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9bc0-5d9b-41ab-b9ae-e1bc2b3bb8d6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:18.142036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f1c46-8306-4d76-a374-58f1e05bf847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:18.142840 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f1c46-8306-4d76-a374-58f1e05bf847 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:18.143222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f1c46-8306-4d76-a374-58f1e05bf847 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:18.143619 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f1c46-8306-4d76-a374-58f1e05bf847 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:18.144403 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f1c46-8306-4d76-a374-58f1e05bf847 None admin] RBAC: Adding query filter params (type=compute) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:18.144821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f1c46-8306-4d76-a374-58f1e05bf847 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:18.146393 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f1c46-8306-4d76-a374-58f1e05bf847 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:18.161581 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 516/1016] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:15:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:18.163014 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:18.270651 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9bc38-ec49-4bdc-9558-a2a65bdf0ef3 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:18.309673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c15ec-09a8-459a-8037-03b28735798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:18.310498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c15ec-09a8-459a-8037-03b28735798a None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:18.310578 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c15ec-09a8-459a-8037-03b28735798a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:18.310851 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c15ec-09a8-459a-8037-03b28735798a None admin] PATH_INFO: `/v3/endpoints` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:18.311656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c15ec-09a8-459a-8037-03b28735798a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:18.313211 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c15ec-09a8-459a-8037-03b28735798a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:18.338436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 501/1017] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:15:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:18.860703 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e018-74e9-4ddb-92c7-184ecfddea9f None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:18.861240 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e018-74e9-4ddb-92c7-184ecfddea9f None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:18.861240 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e018-74e9-4ddb-92c7-184ecfddea9f None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:18.862696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 517/1018] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:18.868522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca78a6-5347-43d8-b9b6-4bfacc7dfc4f None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:18.868823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca78a6-5347-43d8-b9b6-4bfacc7dfc4f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:18.869081 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca78a6-5347-43d8-b9b6-4bfacc7dfc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:18.890168 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-88ca78a6-5347-43d8-b9b6-4bfacc7dfc4f None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:18.891507 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-88ca78a6-5347-43d8-b9b6-4bfacc7dfc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:18.893986 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:18.982244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 502/1019] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:18 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:19.004526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67e018-74e9-4ddb-92c7-184ecfddea9f None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:19.035725 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:19.126889 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67e018-74e9-4ddb-92c7-184ecfddea9f None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:19.162936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4715a1-8553-4e9c-95d7-da9a40cbe2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:19.163577 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4715a1-8553-4e9c-95d7-da9a40cbe2c9 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:19.163861 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4715a1-8553-4e9c-95d7-da9a40cbe2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:19.164134 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4715a1-8553-4e9c-95d7-da9a40cbe2c9 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:19.164930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4715a1-8553-4e9c-95d7-da9a40cbe2c9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:19.166363 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4715a1-8553-4e9c-95d7-da9a40cbe2c9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:19.185671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 518/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:15:19 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:19.674009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3ccd-558f-4e7f-b6d2-dd3c02ed60cd None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:19.674666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3ccd-558f-4e7f-b6d2-dd3c02ed60cd None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:19.674896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3ccd-558f-4e7f-b6d2-dd3c02ed60cd None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:19.676735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 503/1021] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:19.684433 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a63a8-b0ce-4218-ad87-28a5f16335e1 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:19.684786 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a63a8-b0ce-4218-ad87-28a5f16335e1 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:19.685325 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a63a8-b0ce-4218-ad87-28a5f16335e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:19.706172 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-fd3a63a8-b0ce-4218-ad87-28a5f16335e1 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:19.707474 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-fd3a63a8-b0ce-4218-ad87-28a5f16335e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:19.786810 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 519/1022] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:19.810467 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d3ccd-558f-4e7f-b6d2-dd3c02ed60cd None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:19.924221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d3ccd-558f-4e7f-b6d2-dd3c02ed60cd None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:19.960662 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5fdd8-fbb3-4379-bce7-6574b0f2868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:19.961324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5fdd8-fbb3-4379-bce7-6574b0f2868e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:19.961608 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5fdd8-fbb3-4379-bce7-6574b0f2868e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:19.961885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5fdd8-fbb3-4379-bce7-6574b0f2868e None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:19.963088 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5fdd8-fbb3-4379-bce7-6574b0f2868e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:19.964569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5fdd8-fbb3-4379-bce7-6574b0f2868e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:19.971109 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 504/1023] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:19.972684 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a63a8-b0ce-4218-ad87-28a5f16335e1 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:19.989704 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:20.081948 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a63a8-b0ce-4218-ad87-28a5f16335e1 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:20.116425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ad96f-4aad-4f32-87ae-b69b4b331ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:20.117052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ad96f-4aad-4f32-87ae-b69b4b331ddd None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:20.117332 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ad96f-4aad-4f32-87ae-b69b4b331ddd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:20.117602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ad96f-4aad-4f32-87ae-b69b4b331ddd None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:20.118327 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ad96f-4aad-4f32-87ae-b69b4b331ddd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:20.119768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ad96f-4aad-4f32-87ae-b69b4b331ddd None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:20.146320 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 520/1024] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:15:19 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:20.511880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f799adc-95a1-4428-a2da-49e8c2a1d853 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:20.512127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f799adc-95a1-4428-a2da-49e8c2a1d853 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:20.512400 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f799adc-95a1-4428-a2da-49e8c2a1d853 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:20.513974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 505/1025] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:20.519236 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc91d6b-e35e-4f3a-8530-a638668b5592 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:20.519607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc91d6b-e35e-4f3a-8530-a638668b5592 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:20.519976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc91d6b-e35e-4f3a-8530-a638668b5592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:20.540153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-abc91d6b-e35e-4f3a-8530-a638668b5592 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:20.541371 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-abc91d6b-e35e-4f3a-8530-a638668b5592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:20.610935 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:20.618154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 521/1026] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:20.639895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f799adc-95a1-4428-a2da-49e8c2a1d853 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:20.642944 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:20.773547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f799adc-95a1-4428-a2da-49e8c2a1d853 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:20.812978 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf048d-7cd1-4c0b-91a5-bf5bb1ce751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:20.813634 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf048d-7cd1-4c0b-91a5-bf5bb1ce751a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:20.813922 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf048d-7cd1-4c0b-91a5-bf5bb1ce751a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:20.814218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf048d-7cd1-4c0b-91a5-bf5bb1ce751a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:20.826385 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf048d-7cd1-4c0b-91a5-bf5bb1ce751a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-b9bf048d-7cd1-4c0b-91a5-bf5bb1ce751a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:20.831836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:20.832825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:20.833682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 506/1027] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:15:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:20.839205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc91d6b-e35e-4f3a-8530-a638668b5592 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:20.952666 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc91d6b-e35e-4f3a-8530-a638668b5592 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.013326 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:21.014265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.014528 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.014971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.016426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:21.016980 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:21.019639 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:21.034931 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 522/1028] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:15:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:21.044137 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf048d-7cd1-4c0b-91a5-bf5bb1ce751a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.124081 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:21.176178 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf048d-7cd1-4c0b-91a5-bf5bb1ce751a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.213283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fc168-0292-4368-a573-6afe956a9cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:21.213975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fc168-0292-4368-a573-6afe956a9cc0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.214291 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fc168-0292-4368-a573-6afe956a9cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.214568 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fc168-0292-4368-a573-6afe956a9cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.225807 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fc168-0292-4368-a573-6afe956a9cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:21.227317 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fc168-0292-4368-a573-6afe956a9cc0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-2f4fc168-0292-4368-a573-6afe956a9cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:21.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:21.244299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:21.250375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:21.250375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:21.250375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:21.250375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:21.250375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 507/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:21.251533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7ce4-1c48-410c-9124-293e497053f2 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.269212 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7ce4-1c48-410c-9124-293e497053f2 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.269488 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7ce4-1c48-410c-9124-293e497053f2 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.270889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 508/1030] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:15:21.275960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e4948-98c8-454e-98e1-af4ef0cf5457 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.276240 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e4948-98c8-454e-98e1-af4ef0cf5457 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.276513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e4948-98c8-454e-98e1-af4ef0cf5457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.295522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-848e4948-98c8-454e-98e1-af4ef0cf5457 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:21.296630 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-848e4948-98c8-454e-98e1-af4ef0cf5457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:21.328332 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:21.370310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f4e31-bad9-46e2-b24a-d12b24ab6041 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.372184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 509/1031] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:21.396107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e4948-98c8-454e-98e1-af4ef0cf5457 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.413879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:21.414541 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.414820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.415123 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.415796 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:21.416086 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:21.417474 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:21.431545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 523/1032] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:21.437296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.450657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:21.518533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e4948-98c8-454e-98e1-af4ef0cf5457 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.556849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27db62-0a01-4983-a5ef-c575b3f39823 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.560475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:21.560475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.560764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.560987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] PATH_INFO: `/v3/services/image` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.561627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:21.563146 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:21.573433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:21.574557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:21.575585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 510/1033] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:21.580987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.587824 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:21.601873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:21.602577 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.602864 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.603141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.612048 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:21.613971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:15:21.627616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:21.628814 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:21.629729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 524/1034] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:21.635375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.707002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2a25-e882-408a-99c9-f795e0784b32 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.719939 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:21.758275 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:21.758949 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.759236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.759517 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.760215 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] RBAC: Adding query filter params (name=image) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:21.760531 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:21.761924 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:21.767612 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 511/1035] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:21.773392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.778850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb1b20-2755-4c8a-a635-4610a7a9b5a0 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.816877 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:21.817432 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.817919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.818032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.818835 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:21.819145 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:21.821638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:21.847002 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 525/1036] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:21.849849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.879359 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6339b57-5f93-4fb8-83bd-7e0f989860fb None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.896969 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:21.917025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:21.917750 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:21.918038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:21.918349 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:21.919048 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] RBAC: Adding query filter params (type=image) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:21.919339 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:21.920962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:21.930635 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 512/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:21.932360 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:21.962803 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3dbbf-54af-4272-a056-42bcf70f8cdc None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:21.975409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:22.002207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30a32-f0ae-42e5-93d8-ef75ff60f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.002857 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30a32-f0ae-42e5-93d8-ef75ff60f076 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.003141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30a32-f0ae-42e5-93d8-ef75ff60f076 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.003422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30a32-f0ae-42e5-93d8-ef75ff60f076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.049446 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1485ff7-20bb-412e-b227-3fe0ff9c30a8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.076404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e30a32-f0ae-42e5-93d8-ef75ff60f076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.077789 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e30a32-f0ae-42e5-93d8-ef75ff60f076 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.085881 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.086546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.086834 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.087109 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.087864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] RBAC: Adding query filter params (interface=public, service_id=57591a00c8f549e2a473deca9e4619ff, region_id=RegionOne) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:22.088182 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.089576 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.094764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 513/1038] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/endpoints?service_id=57591a00c8f549e2a473deca9e4619ff&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:22.100816 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.121596 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:22.194252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 526/1039] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:21 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:15:22.195668 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30a32-f0ae-42e5-93d8-ef75ff60f076 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.219279 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:22.222872 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1995e1-c9f8-4400-ba85-7767b8894147 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.259763 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be37d-ef2a-4753-9f02-b472a891b5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.260403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be37d-ef2a-4753-9f02-b472a891b5fb None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.260688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be37d-ef2a-4753-9f02-b472a891b5fb None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.260966 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be37d-ef2a-4753-9f02-b472a891b5fb None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.261636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595be37d-ef2a-4753-9f02-b472a891b5fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.263069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595be37d-ef2a-4753-9f02-b472a891b5fb None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.270151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 514/1040] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:15:22.296390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae042b-e5b2-4ed6-90cf-1257047cfd19 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.296744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae042b-e5b2-4ed6-90cf-1257047cfd19 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.297043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae042b-e5b2-4ed6-90cf-1257047cfd19 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.298693 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 515/1041] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:15:22.304375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a6f6-5b55-4fba-82fd-8f49d4244c23 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.304670 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a6f6-5b55-4fba-82fd-8f49d4244c23 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.304956 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a6f6-5b55-4fba-82fd-8f49d4244c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.323770 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30a32-f0ae-42e5-93d8-ef75ff60f076 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.324132 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-5c08a6f6-5b55-4fba-82fd-8f49d4244c23 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:22.325386 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-5c08a6f6-5b55-4fba-82fd-8f49d4244c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:22.365938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.366562 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.366850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.367125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.376677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:22.379133 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.381129 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:22.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:22.400296 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:22.401118 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:22.401118 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:22.401118 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:22.401118 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:22.405432 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 527/1042] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:22.406510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.416611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 516/1043] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:22.440240 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a6f6-5b55-4fba-82fd-8f49d4244c23 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.516373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ee090-01bc-4936-85e0-ec612bf81f8e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.538703 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:22.553231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a6f6-5b55-4fba-82fd-8f49d4244c23 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.557459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.558080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.558409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.558687 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.559401 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:22.559712 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.561111 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.576287 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 528/1044] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:22.582196 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.591109 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c51873-8cee-49a8-9f56-df24f49d7659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.591777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51873-8cee-49a8-9f56-df24f49d7659 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.592059 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51873-8cee-49a8-9f56-df24f49d7659 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.592332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51873-8cee-49a8-9f56-df24f49d7659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.603466 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c51873-8cee-49a8-9f56-df24f49d7659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-e3c51873-8cee-49a8-9f56-df24f49d7659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:22.608569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:22.609848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:22.610738 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 517/1045] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:22.617268 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c51873-8cee-49a8-9f56-df24f49d7659 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.648735 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:22.707666 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9348f8-2674-428e-a984-a4013f1eac57 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.734193 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c51873-8cee-49a8-9f56-df24f49d7659 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.749314 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.750075 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.750668 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.750852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.758213 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.759701 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:15:22.772009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:22.772886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:22.773717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 529/1046] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:22.779074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.779357 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.779998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.780277 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.780547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.781312 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:22.781615 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.783036 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.791711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 518/1047] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:22.797371 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.816103 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:22.888514 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2c369-a0cf-4307-a521-59f8d47f72df None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.902218 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab68e-a8e9-4410-9fa1-e4ed416151a2 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:22.903225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:22.927630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.928773 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.929339 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.929818 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.930841 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:22.931476 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.933261 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.936786 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60928b1-e10a-4a75-93e6-d284666b290f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:22.937499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60928b1-e10a-4a75-93e6-d284666b290f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:22.937826 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60928b1-e10a-4a75-93e6-d284666b290f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:22.938127 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60928b1-e10a-4a75-93e6-d284666b290f None admin] PATH_INFO: `/v3/users/7c8b326d329a46df9195f263aba419d7` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:22.948423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 530/1048] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:22.953946 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60928b1-e10a-4a75-93e6-d284666b290f None admin] RBAC: Authorizing `identity:get_user(user_id=7c8b326d329a46df9195f263aba419d7)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:22.955482 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60928b1-e10a-4a75-93e6-d284666b290f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:22.955956 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:22.961820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 519/1049] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/users/7c8b326d329a46df9195f263aba419d7 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:22.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60928b1-e10a-4a75-93e6-d284666b290f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:23.053259 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:23.071821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b5d3-e1f1-4c27-92da-f968f6fe2c1b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:23.083681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60928b1-e10a-4a75-93e6-d284666b290f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:23.108563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:23.109455 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:23.109740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:23.110021 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:23.110866 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] RBAC: Adding query filter params (role.id=8c3b00e9cab2495aaeef5f03675c34e0, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=45cd9a93c342491889ca8260bc4510c6) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:23.111180 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:23.112633 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:23.118590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 531/1050] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/role_assignments?user.id=45cd9a93c342491889ca8260bc4510c6&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=8c3b00e9cab2495aaeef5f03675c34e0 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:23.122060 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc150f8-37a6-43d0-8b32-b3b11c3a7b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:23.122741 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc150f8-37a6-43d0-8b32-b3b11c3a7b33 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:23.123024 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc150f8-37a6-43d0-8b32-b3b11c3a7b33 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:23.123299 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc150f8-37a6-43d0-8b32-b3b11c3a7b33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:23.200316 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc150f8-37a6-43d0-8b32-b3b11c3a7b33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:23.201732 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc150f8-37a6-43d0-8b32-b3b11c3a7b33 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:23.285134 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:23.313042 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 520/1051] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:22 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:23.319177 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:23.356487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:23.428080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429426fb-eea6-4437-b1c7-aa8db0b1bab0 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:23.464514 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bd09c-1d78-4d1f-a584-b6b4e2cd3272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:23.465085 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd09c-1d78-4d1f-a584-b6b4e2cd3272 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:23.465367 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd09c-1d78-4d1f-a584-b6b4e2cd3272 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:23.465638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd09c-1d78-4d1f-a584-b6b4e2cd3272 None admin] PATH_INFO: `/v3/projects/46a7c89c251f4b0f9821da91cdb62326` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:23.466843 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bd09c-1d78-4d1f-a584-b6b4e2cd3272 None admin] RBAC: Authorizing `identity:get_project(project_id=46a7c89c251f4b0f9821da91cdb62326)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:23.468276 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bd09c-1d78-4d1f-a584-b6b4e2cd3272 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:23.470253 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 532/1052] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:23 2022] GET /identity/v3/projects/46a7c89c251f4b0f9821da91cdb62326 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:23.476376 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc150f8-37a6-43d0-8b32-b3b11c3a7b33 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:23.597502 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc150f8-37a6-43d0-8b32-b3b11c3a7b33 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:23.632518 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51707725-6726-44de-a123-89c448082600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:23.633140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51707725-6726-44de-a123-89c448082600 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:23.633393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51707725-6726-44de-a123-89c448082600 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:23.633692 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51707725-6726-44de-a123-89c448082600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:23.634526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51707725-6726-44de-a123-89c448082600 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=46a7c89c251f4b0f9821da91cdb62326, user.id=7c8b326d329a46df9195f263aba419d7) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:23.634843 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51707725-6726-44de-a123-89c448082600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:23.636302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51707725-6726-44de-a123-89c448082600 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:23.642753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 521/1053] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:23 2022] GET /identity/v3/role_assignments?user.id=7c8b326d329a46df9195f263aba419d7&scope.project.id=46a7c89c251f4b0f9821da91cdb62326&role.id=40504e80dbb845aeb07ca258977ecacb => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:24.331685 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63877d1-7a29-466a-aba1-d6080a6f9844 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:24.332264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63877d1-7a29-466a-aba1-d6080a6f9844 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:24.332264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63877d1-7a29-466a-aba1-d6080a6f9844 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:24.333935 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 533/1054] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:24.339597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7508f8-45d9-4874-9427-d87aa0dd044a None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:24.340076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7508f8-45d9-4874-9427-d87aa0dd044a None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:24.340167 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7508f8-45d9-4874-9427-d87aa0dd044a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:24.358424 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-2d7508f8-45d9-4874-9427-d87aa0dd044a None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:24.359677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-2d7508f8-45d9-4874-9427-d87aa0dd044a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:24.391597 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:24.431682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 522/1055] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:24.454299 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63877d1-7a29-466a-aba1-d6080a6f9844 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:24.560379 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63877d1-7a29-466a-aba1-d6080a6f9844 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:24.562855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:24.597759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:24.598352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:24.598633 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:24.598903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] PATH_INFO: `/v3/services/image` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:24.599532 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:24.600930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:24.610711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:24.611722 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:24.618297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 534/1056] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:15:24 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:24.619180 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7508f8-45d9-4874-9427-d87aa0dd044a None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:24.731537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7508f8-45d9-4874-9427-d87aa0dd044a None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:24.774223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:24.774872 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:24.775157 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:24.775443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:24.776140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] RBAC: Adding query filter params (name=image) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:24.776448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:24.777852 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:24.783640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 523/1057] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:24.789394 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:24.896575 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc400-8ee9-482d-86e0-1a0c98cfdc8f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:24.931651 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce073e9e-a870-445b-9a98-e44e664f3181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:24.932290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce073e9e-a870-445b-9a98-e44e664f3181 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:24.932575 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce073e9e-a870-445b-9a98-e44e664f3181 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:24.932850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce073e9e-a870-445b-9a98-e44e664f3181 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:24.933545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce073e9e-a870-445b-9a98-e44e664f3181 None admin] RBAC: Adding query filter params (type=image) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:24.933847 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce073e9e-a870-445b-9a98-e44e664f3181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:24.935281 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce073e9e-a870-445b-9a98-e44e664f3181 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:24.940880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 535/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:24.947173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:24.980551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:25.057740 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600a708-cd3e-45a8-93e7-b9d13653fb90 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:25.094095 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d354325-781c-41b4-8c43-67b2808c6b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:25.094755 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d354325-781c-41b4-8c43-67b2808c6b5a None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.095039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d354325-781c-41b4-8c43-67b2808c6b5a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.095317 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d354325-781c-41b4-8c43-67b2808c6b5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.096048 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d354325-781c-41b4-8c43-67b2808c6b5a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:25.097448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d354325-781c-41b4-8c43-67b2808c6b5a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:25.126338 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 524/1059] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:15:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:25.173718 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91549115-cf48-4915-809b-e4056a189bc7 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.174036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91549115-cf48-4915-809b-e4056a189bc7 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.174280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91549115-cf48-4915-809b-e4056a189bc7 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.175744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 536/1060] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:25.182056 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa16648-1d14-411a-ac08-8ff157accd54 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.182437 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa16648-1d14-411a-ac08-8ff157accd54 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.182618 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa16648-1d14-411a-ac08-8ff157accd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.203429 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-9fa16648-1d14-411a-ac08-8ff157accd54 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:25.204735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-9fa16648-1d14-411a-ac08-8ff157accd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:25.289983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 525/1061] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:25.312088 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91549115-cf48-4915-809b-e4056a189bc7 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:25.351644 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:25.428972 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91549115-cf48-4915-809b-e4056a189bc7 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:25.464125 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff17d4-9e14-4883-a63e-2a8c1bc6e9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:25.464790 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff17d4-9e14-4883-a63e-2a8c1bc6e9bb None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.465071 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff17d4-9e14-4883-a63e-2a8c1bc6e9bb None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.465343 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff17d4-9e14-4883-a63e-2a8c1bc6e9bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.476676 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff17d4-9e14-4883-a63e-2a8c1bc6e9bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-eaff17d4-9e14-4883-a63e-2a8c1bc6e9bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:25.481677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:25.482583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:25.483388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 537/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:15:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:25.490536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa16648-1d14-411a-ac08-8ff157accd54 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:25.506824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:25.599819 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa16648-1d14-411a-ac08-8ff157accd54 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:25.636725 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c24b00-968d-48fc-a842-61e731819758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:25.637368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24b00-968d-48fc-a842-61e731819758 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.637644 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24b00-968d-48fc-a842-61e731819758 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.637923 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24b00-968d-48fc-a842-61e731819758 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.638687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c24b00-968d-48fc-a842-61e731819758 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:25.638995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c24b00-968d-48fc-a842-61e731819758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:25.640423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c24b00-968d-48fc-a842-61e731819758 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:25.649074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 526/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:15:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:25.654953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff17d4-9e14-4883-a63e-2a8c1bc6e9bb None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:25.668221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6026fad-40d8-4f74-8753-3873cb5d6699 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.668221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6026fad-40d8-4f74-8753-3873cb5d6699 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.668221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6026fad-40d8-4f74-8753-3873cb5d6699 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.671697 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 527/1064] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:15:25.676468 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f56f5-7dc2-4fd6-b90d-1bca4a8650a2 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.676760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f56f5-7dc2-4fd6-b90d-1bca4a8650a2 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.677071 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f56f5-7dc2-4fd6-b90d-1bca4a8650a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.696753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d16f56f5-7dc2-4fd6-b90d-1bca4a8650a2 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:25.697887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d16f56f5-7dc2-4fd6-b90d-1bca4a8650a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:25.757642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:25.767900 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff17d4-9e14-4883-a63e-2a8c1bc6e9bb None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:25.772549 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 528/1065] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:25.795476 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f56f5-7dc2-4fd6-b90d-1bca4a8650a2 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:25.804237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:25.804850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:25.805305 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.805585 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.805904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.817975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:25.819501 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:25.834955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:25.835890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:25.836759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:25.836759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:25.836759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:25.836759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:25.841131 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 538/1066] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:25.841902 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:25.909852 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f56f5-7dc2-4fd6-b90d-1bca4a8650a2 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:25.948959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e91ef-e1a7-4a18-96dd-9a868322a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:25.949534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e91ef-e1a7-4a18-96dd-9a868322a199 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:25.949820 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e91ef-e1a7-4a18-96dd-9a868322a199 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:25.950207 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e91ef-e1a7-4a18-96dd-9a868322a199 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:25.964156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e91ef-e1a7-4a18-96dd-9a868322a199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:25.968893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2ba7f-525f-4642-900a-2928e8e5e2c6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-527e91ef-e1a7-4a18-96dd-9a868322a199 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:25.969009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:25.970861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:25.972302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 529/1067] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:15:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:25.976520 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e91ef-e1a7-4a18-96dd-9a868322a199 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.009956 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.010609 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.010897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.011174 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.011881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:26.012184 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.013585 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.026732 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 539/1068] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:26.032522 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.036344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:26.050577 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:26.090916 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e91ef-e1a7-4a18-96dd-9a868322a199 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:26.129673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.130336 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.130622 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.130897 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.131727 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:26.132040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.133458 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.143690 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 530/1069] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:15:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:26.149423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.150019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc16f7-5708-401c-b2c0-6cf89a6f6dee None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:26.190724 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.191391 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.191671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.191946 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.200507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.201951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:15:26.214468 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:26.215599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:26.216495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 540/1070] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:15:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:26.221737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.271318 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f70b50-bd11-42a0-8bc9-b6247ad86326 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:26.296932 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:26.308954 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:26.318778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e3b23-0f84-4eba-930b-d4b9d16b05ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.319430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3b23-0f84-4eba-930b-d4b9d16b05ad None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.319975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3b23-0f84-4eba-930b-d4b9d16b05ad None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.319975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3b23-0f84-4eba-930b-d4b9d16b05ad None admin] PATH_INFO: `/v3/users/7c8b326d329a46df9195f263aba419d7` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.321539 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e3b23-0f84-4eba-930b-d4b9d16b05ad None admin] RBAC: Authorizing `identity:get_user(user_id=7c8b326d329a46df9195f263aba419d7)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.323023 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e3b23-0f84-4eba-930b-d4b9d16b05ad None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.329997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 531/1071] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:26 2022] GET /identity/v3/users/7c8b326d329a46df9195f263aba419d7 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:26.331097 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e3b23-0f84-4eba-930b-d4b9d16b05ad None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.345976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9419ab7-e6b1-4ac3-897a-f2ca45d084fb None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:26.390206 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.391136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.391496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.391783 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.392772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:26.393388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.394884 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.414365 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 541/1072] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:15:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:26.420623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.450024 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e3b23-0f84-4eba-930b-d4b9d16b05ad None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:26.486890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87def4f7-6e8c-4179-859e-499d2fe42171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.487822 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def4f7-6e8c-4179-859e-499d2fe42171 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.488259 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def4f7-6e8c-4179-859e-499d2fe42171 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.488699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def4f7-6e8c-4179-859e-499d2fe42171 None admin] PATH_INFO: `/v3/projects/46a7c89c251f4b0f9821da91cdb62326` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.490046 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87def4f7-6e8c-4179-859e-499d2fe42171 None admin] RBAC: Authorizing `identity:get_project(project_id=46a7c89c251f4b0f9821da91cdb62326)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.491690 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87def4f7-6e8c-4179-859e-499d2fe42171 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.494070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 532/1073] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:26 2022] GET /identity/v3/projects/46a7c89c251f4b0f9821da91cdb62326 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:26.500790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87def4f7-6e8c-4179-859e-499d2fe42171 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.543986 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3704f5-fc34-4e78-90c5-c12a78ba536a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:26.572982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:26.576751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:26.588064 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.588729 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.589736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.589736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.602395 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.603886 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:26.621178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:26.622689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:26.623878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:26.623878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:26.623878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:26.623878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:26.623878 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 542/1074] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:26.628275 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87def4f7-6e8c-4179-859e-499d2fe42171 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:26.629293 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.670179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b707c7-2e03-4357-aa14-a4a956bf468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.670179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b707c7-2e03-4357-aa14-a4a956bf468a None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.670179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b707c7-2e03-4357-aa14-a4a956bf468a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.670179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b707c7-2e03-4357-aa14-a4a956bf468a None admin] PATH_INFO: `/v3/projects/46a7c89c251f4b0f9821da91cdb62326/users/7c8b326d329a46df9195f263aba419d7/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.670821 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b707c7-2e03-4357-aa14-a4a956bf468a None admin] RBAC: Authorizing `identity:create_grant(project_id=46a7c89c251f4b0f9821da91cdb62326, user_id=7c8b326d329a46df9195f263aba419d7, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.671601 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b707c7-2e03-4357-aa14-a4a956bf468a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.702138 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 533/1075] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:15:26 2022] PUT /identity/v3/projects/46a7c89c251f4b0f9821da91cdb62326/users/7c8b326d329a46df9195f263aba419d7/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:15:26.786061 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6ca68-1767-4265-88b7-7af946c658d2 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:26.831301 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:26.831979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:26.832267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:26.832553 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:26.833274 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:26.833617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:26.855420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 543/1076] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:26.857106 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b707c7-2e03-4357-aa14-a4a956bf468a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:26.977818 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b707c7-2e03-4357-aa14-a4a956bf468a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:27.002248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:27.015343 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:27.016301 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:27.016745 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:27.017170 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:27.024563 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:27.026270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:15:27.038605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:27.039554 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:27.040394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:15:27.040394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:15:27.040394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:15:27.040394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:15:27.040394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:15:27.040394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:15:27.040394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:27.040394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:27.040933 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 534/1077] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:27.046415 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:27.082408 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:27.153540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f4d6e-8602-49ae-a7d1-ac45daab209f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:27.189188 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a05003-212b-495c-8afe-9f7b10ee9966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:27.189830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a05003-212b-495c-8afe-9f7b10ee9966 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:27.190100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a05003-212b-495c-8afe-9f7b10ee9966 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:27.190409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a05003-212b-495c-8afe-9f7b10ee9966 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:27.191167 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a05003-212b-495c-8afe-9f7b10ee9966 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:27.191478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a05003-212b-495c-8afe-9f7b10ee9966 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:27.192969 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a05003-212b-495c-8afe-9f7b10ee9966 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:27.205876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 544/1078] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:27.211979 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:27.222754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdfead-85cc-46ef-9002-aa648bf315f2 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:27.223061 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdfead-85cc-46ef-9002-aa648bf315f2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:27.223337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdfead-85cc-46ef-9002-aa648bf315f2 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:27.224778 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 545/1079] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:27.230203 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2bc06-4fc1-4f78-b737-e8230aa649ac None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:27.230495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2bc06-4fc1-4f78-b737-e8230aa649ac None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:27.230768 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2bc06-4fc1-4f78-b737-e8230aa649ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:27.249713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-08d2bc06-4fc1-4f78-b737-e8230aa649ac None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:27.251063 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-08d2bc06-4fc1-4f78-b737-e8230aa649ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:27.319689 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392f143-1c40-483e-bd20-8f40f6fe4c16 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:27.324199 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 546/1080] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:27.346399 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2bc06-4fc1-4f78-b737-e8230aa649ac None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:27.355940 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d136d6b-6296-418b-9ceb-1de7cd430c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:27.356742 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136d6b-6296-418b-9ceb-1de7cd430c15 None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:27.357067 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136d6b-6296-418b-9ceb-1de7cd430c15 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:27.357421 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136d6b-6296-418b-9ceb-1de7cd430c15 None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/45cd9a93c342491889ca8260bc4510c6/roles/8c3b00e9cab2495aaeef5f03675c34e0` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:27.359860 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d136d6b-6296-418b-9ceb-1de7cd430c15 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=45cd9a93c342491889ca8260bc4510c6, role_id=8c3b00e9cab2495aaeef5f03675c34e0)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:27.361566 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d136d6b-6296-418b-9ceb-1de7cd430c15 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:27.371432 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:27.386500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 535/1081] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:15:27 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/45cd9a93c342491889ca8260bc4510c6/roles/8c3b00e9cab2495aaeef5f03675c34e0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:15:27.457472 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2bc06-4fc1-4f78-b737-e8230aa649ac None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:27.496040 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec488212-b682-423b-9175-6b99094bdac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:27.496711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec488212-b682-423b-9175-6b99094bdac1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:27.496994 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec488212-b682-423b-9175-6b99094bdac1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:27.497267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec488212-b682-423b-9175-6b99094bdac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:27.576310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec488212-b682-423b-9175-6b99094bdac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:27.577731 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec488212-b682-423b-9175-6b99094bdac1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:27.602217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:27.683032 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 547/1082] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:15:27.689000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d136d6b-6296-418b-9ceb-1de7cd430c15 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:27.744864 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:27.800085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d136d6b-6296-418b-9ceb-1de7cd430c15 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:27.836090 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15291919-3281-48fa-9c0f-a70008cecadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:27.837040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291919-3281-48fa-9c0f-a70008cecadc None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:27.837040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291919-3281-48fa-9c0f-a70008cecadc None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:27.838477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291919-3281-48fa-9c0f-a70008cecadc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:27.860500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15291919-3281-48fa-9c0f-a70008cecadc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:27.861980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15291919-3281-48fa-9c0f-a70008cecadc None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-15291919-3281-48fa-9c0f-a70008cecadc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:27.877886 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:27.879122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:27.879983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:27.879983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:27.879983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:27.879983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:27.879983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 536/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:27.887052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec488212-b682-423b-9175-6b99094bdac1 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:27.993744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec488212-b682-423b-9175-6b99094bdac1 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:28.017565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:28.029865 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7436a-dfd6-4f2c-9fc8-da630902a5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:28.030529 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7436a-dfd6-4f2c-9fc8-da630902a5f7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:28.030809 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7436a-dfd6-4f2c-9fc8-da630902a5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:28.031139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7436a-dfd6-4f2c-9fc8-da630902a5f7 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:28.031839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7436a-dfd6-4f2c-9fc8-da630902a5f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:28.032150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7436a-dfd6-4f2c-9fc8-da630902a5f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:28.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7436a-dfd6-4f2c-9fc8-da630902a5f7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:28.050221 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 548/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:28.749572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0991-e6df-403c-a863-e19f21acf3f2 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:28.750280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0991-e6df-403c-a863-e19f21acf3f2 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:28.750280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0991-e6df-403c-a863-e19f21acf3f2 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:28.751547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 537/1085] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:28.757038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897212e3-bee6-434e-ac2e-8d5a1613e475 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:28.757460 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897212e3-bee6-434e-ac2e-8d5a1613e475 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:28.757620 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897212e3-bee6-434e-ac2e-8d5a1613e475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:28.777038 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-897212e3-bee6-434e-ac2e-8d5a1613e475 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:28.778410 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-897212e3-bee6-434e-ac2e-8d5a1613e475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:28.851833 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 549/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:28.875474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a0991-e6df-403c-a863-e19f21acf3f2 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:28.980893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:28.988963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a0991-e6df-403c-a863-e19f21acf3f2 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:29.025430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d36d6c-212f-479b-b0ae-d74cbc534462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:29.026080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d36d6c-212f-479b-b0ae-d74cbc534462 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.026393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d36d6c-212f-479b-b0ae-d74cbc534462 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.026655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d36d6c-212f-479b-b0ae-d74cbc534462 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.038210 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d36d6c-212f-479b-b0ae-d74cbc534462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-f7d36d6c-212f-479b-b0ae-d74cbc534462 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:29.043171 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:29.044078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:29.045091 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 538/1087] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:29.050842 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897212e3-bee6-434e-ac2e-8d5a1613e475 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:29.154333 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897212e3-bee6-434e-ac2e-8d5a1613e475 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:29.169364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:29.191744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:29.192367 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.192646 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.192917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.193638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:29.193940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:29.195340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:29.204300 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 550/1088] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:29.210455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d36d6c-212f-479b-b0ae-d74cbc534462 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:29.315877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d36d6c-212f-479b-b0ae-d74cbc534462 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:29.355496 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8856db-0b5c-4e15-8f99-a59b31f82930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:29.356164 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8856db-0b5c-4e15-8f99-a59b31f82930 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.356488 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8856db-0b5c-4e15-8f99-a59b31f82930 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.356687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8856db-0b5c-4e15-8f99-a59b31f82930 None admin] PATH_INFO: `/v3/users/7c8b326d329a46df9195f263aba419d7` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.357960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8856db-0b5c-4e15-8f99-a59b31f82930 None admin] RBAC: Authorizing `identity:get_user(user_id=7c8b326d329a46df9195f263aba419d7)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:29.359696 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8856db-0b5c-4e15-8f99-a59b31f82930 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:29.361418 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 539/1089] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:29 2022] GET /identity/v3/users/7c8b326d329a46df9195f263aba419d7 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:29.367773 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:29.407690 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac79e3-e13c-4076-baca-dd4552b3b486 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.407977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac79e3-e13c-4076-baca-dd4552b3b486 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.408253 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac79e3-e13c-4076-baca-dd4552b3b486 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.409678 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 540/1090] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:29.415545 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d5146-40a1-4b98-82b8-af6fd5642f8a None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.415839 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d5146-40a1-4b98-82b8-af6fd5642f8a None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.416095 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d5146-40a1-4b98-82b8-af6fd5642f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.434940 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-f17d5146-40a1-4b98-82b8-af6fd5642f8a None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:29.436188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f17d5146-40a1-4b98-82b8-af6fd5642f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:29.456139 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:29.481265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d566d-61b3-42da-af23-038bc6b8e397 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:29.512531 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 541/1091] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:15:29.518610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb7e85-a989-421a-b19a-2f231fad6694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:29.519245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb7e85-a989-421a-b19a-2f231fad6694 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.519521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb7e85-a989-421a-b19a-2f231fad6694 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.519792 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb7e85-a989-421a-b19a-2f231fad6694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.536053 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d5146-40a1-4b98-82b8-af6fd5642f8a None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:29.570623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:29.604413 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb7e85-a989-421a-b19a-2f231fad6694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:29.605914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb7e85-a989-421a-b19a-2f231fad6694 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:29.659190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d5146-40a1-4b98-82b8-af6fd5642f8a None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:29.705800 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeecef-7a3e-4cc1-875a-a888334e833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:29.706503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeecef-7a3e-4cc1-875a-a888334e833e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.706783 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeecef-7a3e-4cc1-875a-a888334e833e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.707055 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeecef-7a3e-4cc1-875a-a888334e833e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.721442 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeecef-7a3e-4cc1-875a-a888334e833e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:29.727594 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 551/1092] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-7cfeecef-7a3e-4cc1-875a-a888334e833e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:29.731261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:29.732602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:15:29.733445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:29.734512 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 542/1093] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:15:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:29.736089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb7e85-a989-421a-b19a-2f231fad6694 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:29.740672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeecef-7a3e-4cc1-875a-a888334e833e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:29.758889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:29.823277 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:29.852260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb7e85-a989-421a-b19a-2f231fad6694 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:29.852770 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeecef-7a3e-4cc1-875a-a888334e833e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:29.889600 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c14a25-2bd5-4391-a9a1-814363ad7bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:29.889893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:29.890302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c14a25-2bd5-4391-a9a1-814363ad7bd6 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.890415 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:29.890664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c14a25-2bd5-4391-a9a1-814363ad7bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.890756 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:29.890901 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c14a25-2bd5-4391-a9a1-814363ad7bd6 None admin] PATH_INFO: `/v3/projects/46a7c89c251f4b0f9821da91cdb62326` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.891001 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:29.891642 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:29.891947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:29.892129 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c14a25-2bd5-4391-a9a1-814363ad7bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=46a7c89c251f4b0f9821da91cdb62326)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:29.894160 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:29.894288 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c14a25-2bd5-4391-a9a1-814363ad7bd6 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:29.896500 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 552/1094] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:29 2022] GET /identity/v3/projects/46a7c89c251f4b0f9821da91cdb62326 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:29.901728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c14a25-2bd5-4391-a9a1-814363ad7bd6 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:29.907682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 543/1095] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:15:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:29.908856 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:29.999125 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:30.025953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c14a25-2bd5-4391-a9a1-814363ad7bd6 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:30.026958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4edccb-5a5f-49d5-bd88-83e2789a44a1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:30.069683 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:30.070374 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:30.070651 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:30.071289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:30.072183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=46a7c89c251f4b0f9821da91cdb62326, user.id=7c8b326d329a46df9195f263aba419d7) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:30.072488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:30.074675 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:30.074815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:30.074904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:30.075069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:30.075711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:30.081949 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 553/1096] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:29 2022] GET /identity/v3/role_assignments?user.id=7c8b326d329a46df9195f263aba419d7&scope.project.id=46a7c89c251f4b0f9821da91cdb62326&role.id=40504e80dbb845aeb07ca258977ecacb => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:30.088883 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:30.090440 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:30.106995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:30.108087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:30.108913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:30.108913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:30.108913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:30.108913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:30.108913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 544/1097] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:30.115258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:30.128970 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:30.243128 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e512e6-5897-4f1a-9fb5-82efff3e4784 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:30.283041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:30.283747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:30.284020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:30.284302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:30.285036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:30.285331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:30.286833 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:30.300251 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 554/1098] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:30.307703 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:30.428778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff5d5b-88bb-4e1d-bb2b-bb902e0a2e37 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:30.462757 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:30.474210 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:30.474868 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:30.475154 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:30.475433 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:30.483933 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:30.485504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:15:30.500402 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:30.501381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:30.502352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 545/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:15:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:15:30.508234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:30.587602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:30.626568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d31288-6221-479e-9944-c8fbe2a68a66 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:30.664508 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:30.665147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:30.665429 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:30.665701 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:30.666604 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:30.666960 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:30.668477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:30.690686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 555/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:15:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:30.696844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:30.816603 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b149b3b-1d6f-4ef7-994f-61ee64a6c6bf None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:30.862810 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b25c0-afb4-4986-b8e4-7f739c832eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:30.863449 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b25c0-afb4-4986-b8e4-7f739c832eea None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:30.863730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b25c0-afb4-4986-b8e4-7f739c832eea None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:30.864005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b25c0-afb4-4986-b8e4-7f739c832eea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:30.924022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:30.943083 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b25c0-afb4-4986-b8e4-7f739c832eea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:30.945613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b25c0-afb4-4986-b8e4-7f739c832eea None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:31.054555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 546/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:31.061053 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:31.186584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f494aa-4b9f-4170-8979-fb4ce305c03c None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:31.234889 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:31.247340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:31.248016 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:31.248311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:31.248591 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:31.260457 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:31.262003 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:15:31.278618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:31.279858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:31.280937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:15:31.280937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:15:31.280937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:15:31.280937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:31.282294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 556/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:31.286468 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b25c0-afb4-4986-b8e4-7f739c832eea None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:31.388553 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:31.402755 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b25c0-afb4-4986-b8e4-7f739c832eea None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:31.445232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:31.446273 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:31.446681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:31.446954 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:31.447663 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:31.447993 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:31.449557 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:31.463654 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 547/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:31.469087 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:31.585836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bae7e7-34bd-4589-85d3-a24fb240ca84 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:31.623250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:31.623904 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:31.624202 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:31.624448 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:31.632114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:31.633575 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:15:31.646291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:31.647364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:31.648412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 557/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:31.654076 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:31.771255 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ac89-ddee-4a6b-b027-980ac6f04898 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:31.813259 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8eb6b-31c3-4c68-ab46-638b0210a22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:31.813888 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8eb6b-31c3-4c68-ab46-638b0210a22e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:31.814230 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8eb6b-31c3-4c68-ab46-638b0210a22e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:31.814501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8eb6b-31c3-4c68-ab46-638b0210a22e None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:31.815357 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8eb6b-31c3-4c68-ab46-638b0210a22e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:31.815655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8eb6b-31c3-4c68-ab46-638b0210a22e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:31.817258 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8eb6b-31c3-4c68-ab46-638b0210a22e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:31.831687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 548/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:31.839185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:31.918831 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:31.987686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44507411-ce2c-49f1-adf3-22e4c3d600e7 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:32.022692 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085bbc9-051d-41d6-9f4f-1fbd0254ca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:32.023326 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085bbc9-051d-41d6-9f4f-1fbd0254ca13 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:32.023604 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085bbc9-051d-41d6-9f4f-1fbd0254ca13 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:32.023876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085bbc9-051d-41d6-9f4f-1fbd0254ca13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:32.024678 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085bbc9-051d-41d6-9f4f-1fbd0254ca13 None admin] RBAC: Adding query filter params (role.id=8c3b00e9cab2495aaeef5f03675c34e0, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=45cd9a93c342491889ca8260bc4510c6) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:32.025015 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085bbc9-051d-41d6-9f4f-1fbd0254ca13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:32.026502 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085bbc9-051d-41d6-9f4f-1fbd0254ca13 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:32.033452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 558/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:31 2022] GET /identity/v3/role_assignments?user.id=45cd9a93c342491889ca8260bc4510c6&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=8c3b00e9cab2495aaeef5f03675c34e0 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:32.226308 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc194bf7-117a-4dec-8fc4-2648842fa7ac None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:32.226598 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc194bf7-117a-4dec-8fc4-2648842fa7ac None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:32.226902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc194bf7-117a-4dec-8fc4-2648842fa7ac None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:32.228276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 549/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:32.234637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31269-59b2-493b-91fa-4e1483a34058 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:32.235183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31269-59b2-493b-91fa-4e1483a34058 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:32.235280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31269-59b2-493b-91fa-4e1483a34058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:32.256223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-61a31269-59b2-493b-91fa-4e1483a34058 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:32.257495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-61a31269-59b2-493b-91fa-4e1483a34058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:32.336342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 559/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:32.361450 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc194bf7-117a-4dec-8fc4-2648842fa7ac None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:32.378525 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:32.496079 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc194bf7-117a-4dec-8fc4-2648842fa7ac None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:32.539144 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8795eb-52fd-4ce3-9638-7f54e8c2c098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:32.540101 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8795eb-52fd-4ce3-9638-7f54e8c2c098 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:32.540520 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8795eb-52fd-4ce3-9638-7f54e8c2c098 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:32.540924 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8795eb-52fd-4ce3-9638-7f54e8c2c098 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:32.542938 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8795eb-52fd-4ce3-9638-7f54e8c2c098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:32.544547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8795eb-52fd-4ce3-9638-7f54e8c2c098 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:32.546800 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 550/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:32.553870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a31269-59b2-493b-91fa-4e1483a34058 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:32.602414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:32.723603 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a31269-59b2-493b-91fa-4e1483a34058 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:32.769690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae391a50-436d-40eb-9e9f-85598fd94cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:32.770301 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae391a50-436d-40eb-9e9f-85598fd94cf5 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:32.770580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae391a50-436d-40eb-9e9f-85598fd94cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:32.770849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae391a50-436d-40eb-9e9f-85598fd94cf5 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:32.771847 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae391a50-436d-40eb-9e9f-85598fd94cf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:32.773000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae391a50-436d-40eb-9e9f-85598fd94cf5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:32.803594 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 560/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:15:32 2022] POST /identity/v3/users => generated 286 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:34.893785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e29df-8b38-4c31-9d16-ffbed87090f6 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:34.894445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e29df-8b38-4c31-9d16-ffbed87090f6 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:34.894445 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e29df-8b38-4c31-9d16-ffbed87090f6 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:34.895748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 551/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:34.901606 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a4d0-6649-4da4-910e-149d870bd901 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:34.902095 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a4d0-6649-4da4-910e-149d870bd901 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:34.902248 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a4d0-6649-4da4-910e-149d870bd901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:34.923039 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-5c36a4d0-6649-4da4-910e-149d870bd901 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:34.924181 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-5c36a4d0-6649-4da4-910e-149d870bd901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:35.002259 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 561/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:35.026477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e29df-8b38-4c31-9d16-ffbed87090f6 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:35.113373 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:35.141485 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e29df-8b38-4c31-9d16-ffbed87090f6 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:35.180699 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03d0f6-af4f-4edb-bb30-8e5e57c2d99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:35.181267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03d0f6-af4f-4edb-bb30-8e5e57c2d99d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:35.181531 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03d0f6-af4f-4edb-bb30-8e5e57c2d99d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:35.181800 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03d0f6-af4f-4edb-bb30-8e5e57c2d99d None admin] PATH_INFO: `/v3/roles/0f002225dde34ef1a2adda2f583b8520` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:35.183500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03d0f6-af4f-4edb-bb30-8e5e57c2d99d None admin] RBAC: Authorizing `identity:get_role(role_id=0f002225dde34ef1a2adda2f583b8520)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:35.184917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03d0f6-af4f-4edb-bb30-8e5e57c2d99d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:35.186408 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 552/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:35 2022] GET /identity/v3/roles/0f002225dde34ef1a2adda2f583b8520 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:35.194411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36a4d0-6649-4da4-910e-149d870bd901 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:35.214912 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:35.308978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36a4d0-6649-4da4-910e-149d870bd901 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:35.352750 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199f333-1cc4-49f6-b596-988ece11e4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:35.353445 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199f333-1cc4-49f6-b596-988ece11e4d3 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:35.353726 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199f333-1cc4-49f6-b596-988ece11e4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:35.353998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199f333-1cc4-49f6-b596-988ece11e4d3 None admin] PATH_INFO: `/v3/users/d4d5cfabc82f40f285941e8abc75994a` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:35.373836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199f333-1cc4-49f6-b596-988ece11e4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=d4d5cfabc82f40f285941e8abc75994a)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:35.375419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199f333-1cc4-49f6-b596-988ece11e4d3 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:35.377265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 562/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:35 2022] GET /identity/v3/users/d4d5cfabc82f40f285941e8abc75994a => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:35.384726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03d0f6-af4f-4edb-bb30-8e5e57c2d99d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:35.503726 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03d0f6-af4f-4edb-bb30-8e5e57c2d99d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:35.539793 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75ca91-54e0-4388-8057-9204ab4e2378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:35.540421 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ca91-54e0-4388-8057-9204ab4e2378 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:35.540710 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ca91-54e0-4388-8057-9204ab4e2378 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:35.540986 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ca91-54e0-4388-8057-9204ab4e2378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:35.555891 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:35.618761 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75ca91-54e0-4388-8057-9204ab4e2378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:35.620328 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75ca91-54e0-4388-8057-9204ab4e2378 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:35.745838 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 553/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:35.752258 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199f333-1cc4-49f6-b596-988ece11e4d3 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:35.869661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:35.899170 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199f333-1cc4-49f6-b596-988ece11e4d3 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:35.939989 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c360a-ef04-4ddb-889d-03647058e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:35.940839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c360a-ef04-4ddb-889d-03647058e881 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:35.941006 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c360a-ef04-4ddb-889d-03647058e881 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:35.941286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c360a-ef04-4ddb-889d-03647058e881 None admin] PATH_INFO: `/v3/projects/46a7c89c251f4b0f9821da91cdb62326` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:35.942470 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c360a-ef04-4ddb-889d-03647058e881 None admin] RBAC: Authorizing `identity:get_project(project_id=46a7c89c251f4b0f9821da91cdb62326)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:35.943893 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c360a-ef04-4ddb-889d-03647058e881 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:35.945648 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 563/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:35 2022] GET /identity/v3/projects/46a7c89c251f4b0f9821da91cdb62326 => generated 311 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:35.954176 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75ca91-54e0-4388-8057-9204ab4e2378 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:36.029677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:36.097448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75ca91-54e0-4388-8057-9204ab4e2378 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:36.138862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93be619-aadb-489e-8024-5918980e21c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:36.139580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93be619-aadb-489e-8024-5918980e21c9 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:36.139912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93be619-aadb-489e-8024-5918980e21c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:36.140206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93be619-aadb-489e-8024-5918980e21c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:36.141293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93be619-aadb-489e-8024-5918980e21c9 None admin] RBAC: Adding query filter params (role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=46a7c89c251f4b0f9821da91cdb62326, user.id=d4d5cfabc82f40f285941e8abc75994a) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:36.141628 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93be619-aadb-489e-8024-5918980e21c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:36.143322 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93be619-aadb-489e-8024-5918980e21c9 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:36.149961 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 554/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:35 2022] GET /identity/v3/role_assignments?user.id=d4d5cfabc82f40f285941e8abc75994a&scope.project.id=46a7c89c251f4b0f9821da91cdb62326&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:15:38.257270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eac74-bec6-4fd1-8578-9ec57293e12e None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:38.257859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eac74-bec6-4fd1-8578-9ec57293e12e None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:38.257859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eac74-bec6-4fd1-8578-9ec57293e12e None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:38.259234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 564/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:38.266586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbe5fe-28d7-4958-ad98-8801055d9a42 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:38.267071 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbe5fe-28d7-4958-ad98-8801055d9a42 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:38.267178 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbe5fe-28d7-4958-ad98-8801055d9a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:38.291403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-f0dbe5fe-28d7-4958-ad98-8801055d9a42 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:38.293069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f0dbe5fe-28d7-4958-ad98-8801055d9a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:38.380668 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 555/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:38.404270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eac74-bec6-4fd1-8578-9ec57293e12e None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:38.530289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eac74-bec6-4fd1-8578-9ec57293e12e None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:38.572369 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e86b5-5471-4125-8665-5ae00e5b16bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:38.573047 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e86b5-5471-4125-8665-5ae00e5b16bd None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:38.573325 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e86b5-5471-4125-8665-5ae00e5b16bd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:38.573599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e86b5-5471-4125-8665-5ae00e5b16bd None admin] PATH_INFO: `/v3/roles/0f002225dde34ef1a2adda2f583b8520` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:38.575344 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e86b5-5471-4125-8665-5ae00e5b16bd None admin] RBAC: Authorizing `identity:get_role(role_id=0f002225dde34ef1a2adda2f583b8520)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:38.576782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e86b5-5471-4125-8665-5ae00e5b16bd None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:38.578307 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 565/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:38 2022] GET /identity/v3/roles/0f002225dde34ef1a2adda2f583b8520 => generated 223 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:38.584933 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbe5fe-28d7-4958-ad98-8801055d9a42 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:38.635409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:38.714511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbe5fe-28d7-4958-ad98-8801055d9a42 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:38.757929 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f60dfe-184d-4dd1-bcb5-c53211f5377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:38.758714 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f60dfe-184d-4dd1-bcb5-c53211f5377b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:38.758997 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f60dfe-184d-4dd1-bcb5-c53211f5377b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:38.759284 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f60dfe-184d-4dd1-bcb5-c53211f5377b None admin] PATH_INFO: `/v3/users/d4d5cfabc82f40f285941e8abc75994a` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:38.760614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f60dfe-184d-4dd1-bcb5-c53211f5377b None admin] RBAC: Authorizing `identity:get_user(user_id=d4d5cfabc82f40f285941e8abc75994a)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:38.762203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f60dfe-184d-4dd1-bcb5-c53211f5377b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:38.763987 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 556/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:38 2022] GET /identity/v3/users/d4d5cfabc82f40f285941e8abc75994a => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:38.771269 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e86b5-5471-4125-8665-5ae00e5b16bd None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:38.795085 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:38.912709 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e86b5-5471-4125-8665-5ae00e5b16bd None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:38.951558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca674e75-675d-40d2-96e5-39d43510973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:38.952230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca674e75-675d-40d2-96e5-39d43510973c None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:38.952585 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca674e75-675d-40d2-96e5-39d43510973c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:38.952918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca674e75-675d-40d2-96e5-39d43510973c None admin] PATH_INFO: `/v3/projects/46a7c89c251f4b0f9821da91cdb62326` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:38.954170 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca674e75-675d-40d2-96e5-39d43510973c None admin] RBAC: Authorizing `identity:get_project(project_id=46a7c89c251f4b0f9821da91cdb62326)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:38.955657 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca674e75-675d-40d2-96e5-39d43510973c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:38.957456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 566/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:38 2022] GET /identity/v3/projects/46a7c89c251f4b0f9821da91cdb62326 => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:38.963904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f60dfe-184d-4dd1-bcb5-c53211f5377b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:39.091232 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f60dfe-184d-4dd1-bcb5-c53211f5377b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:39.105344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:39.145666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba3740-5edd-4ee9-9527-61fd808087ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:39.146705 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba3740-5edd-4ee9-9527-61fd808087ff None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:39.147080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba3740-5edd-4ee9-9527-61fd808087ff None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:39.147518 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba3740-5edd-4ee9-9527-61fd808087ff None admin] PATH_INFO: `/v3/projects/46a7c89c251f4b0f9821da91cdb62326/users/d4d5cfabc82f40f285941e8abc75994a/roles/0f002225dde34ef1a2adda2f583b8520` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:39.150177 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba3740-5edd-4ee9-9527-61fd808087ff None admin] RBAC: Authorizing `identity:create_grant(project_id=46a7c89c251f4b0f9821da91cdb62326, user_id=d4d5cfabc82f40f285941e8abc75994a, role_id=0f002225dde34ef1a2adda2f583b8520)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:39.152614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba3740-5edd-4ee9-9527-61fd808087ff None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:39.171345 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 557/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:15:38 2022] PUT /identity/v3/projects/46a7c89c251f4b0f9821da91cdb62326/users/d4d5cfabc82f40f285941e8abc75994a/roles/0f002225dde34ef1a2adda2f583b8520 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:15:41.299609 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c7fa5-ad52-4493-af11-1699f6311af4 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:41.300192 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c7fa5-ad52-4493-af11-1699f6311af4 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:41.300192 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c7fa5-ad52-4493-af11-1699f6311af4 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:41.301616 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 567/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:41.307361 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a862d3b-032e-415c-9e00-f15207720972 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:41.307852 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a862d3b-032e-415c-9e00-f15207720972 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:41.308006 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a862d3b-032e-415c-9e00-f15207720972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:41.327352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-5a862d3b-032e-415c-9e00-f15207720972 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:41.328655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-5a862d3b-032e-415c-9e00-f15207720972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:41.398197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 558/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:41.421708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c7fa5-ad52-4493-af11-1699f6311af4 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:41.515672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:41.542822 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c7fa5-ad52-4493-af11-1699f6311af4 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:41.599217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32929b52-858b-4e6f-85da-127c152ef55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:41.600197 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929b52-858b-4e6f-85da-127c152ef55e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:41.600672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929b52-858b-4e6f-85da-127c152ef55e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:41.601139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929b52-858b-4e6f-85da-127c152ef55e None admin] PATH_INFO: `/v3/roles/0f002225dde34ef1a2adda2f583b8520` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:41.603671 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32929b52-858b-4e6f-85da-127c152ef55e None admin] RBAC: Authorizing `identity:get_role(role_id=0f002225dde34ef1a2adda2f583b8520)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:41.606144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32929b52-858b-4e6f-85da-127c152ef55e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:41.608644 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 568/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:41 2022] GET /identity/v3/roles/0f002225dde34ef1a2adda2f583b8520 => generated 223 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:41.616540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a862d3b-032e-415c-9e00-f15207720972 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:41.753626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:41.760922 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a862d3b-032e-415c-9e00-f15207720972 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:41.799887 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444b744-240e-4992-a6d9-2390af6a4080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:41.800627 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444b744-240e-4992-a6d9-2390af6a4080 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:41.800981 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444b744-240e-4992-a6d9-2390af6a4080 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:41.801267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444b744-240e-4992-a6d9-2390af6a4080 None admin] PATH_INFO: `/v3/users/d4d5cfabc82f40f285941e8abc75994a` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:41.802713 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444b744-240e-4992-a6d9-2390af6a4080 None admin] RBAC: Authorizing `identity:get_user(user_id=d4d5cfabc82f40f285941e8abc75994a)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:41.804328 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444b744-240e-4992-a6d9-2390af6a4080 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:41.805999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 559/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:41 2022] GET /identity/v3/users/d4d5cfabc82f40f285941e8abc75994a => generated 286 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:41.812584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32929b52-858b-4e6f-85da-127c152ef55e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:41.953535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32929b52-858b-4e6f-85da-127c152ef55e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:41.996798 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae2374-a8f7-4b37-9475-8a4bb2583e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:41.997543 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae2374-a8f7-4b37-9475-8a4bb2583e93 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:41.997877 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae2374-a8f7-4b37-9475-8a4bb2583e93 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:41.998312 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae2374-a8f7-4b37-9475-8a4bb2583e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:42.021602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:42.081057 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae2374-a8f7-4b37-9475-8a4bb2583e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:42.082584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae2374-a8f7-4b37-9475-8a4bb2583e93 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:42.194951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 569/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:42.204040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444b744-240e-4992-a6d9-2390af6a4080 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:42.339788 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444b744-240e-4992-a6d9-2390af6a4080 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:42.383586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d1951-1de0-45ab-9f72-ed9ca6961939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:42.384309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1951-1de0-45ab-9f72-ed9ca6961939 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:42.384594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1951-1de0-45ab-9f72-ed9ca6961939 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:42.384878 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1951-1de0-45ab-9f72-ed9ca6961939 None admin] PATH_INFO: `/v3/projects/46a7c89c251f4b0f9821da91cdb62326` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:42.386151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d1951-1de0-45ab-9f72-ed9ca6961939 None admin] RBAC: Authorizing `identity:get_project(project_id=46a7c89c251f4b0f9821da91cdb62326)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:42.387976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d1951-1de0-45ab-9f72-ed9ca6961939 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:42.389717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 560/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:42 2022] GET /identity/v3/projects/46a7c89c251f4b0f9821da91cdb62326 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:42.397148 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae2374-a8f7-4b37-9475-8a4bb2583e93 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:42.457765 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:42.520426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae2374-a8f7-4b37-9475-8a4bb2583e93 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:42.561630 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af9d59-6ce4-4d48-8b3d-8f0588b07bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:42.562565 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9d59-6ce4-4d48-8b3d-8f0588b07bd9 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:42.562912 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9d59-6ce4-4d48-8b3d-8f0588b07bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:42.563264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9d59-6ce4-4d48-8b3d-8f0588b07bd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:42.564381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af9d59-6ce4-4d48-8b3d-8f0588b07bd9 None admin] RBAC: Adding query filter params (role.id=0f002225dde34ef1a2adda2f583b8520, scope.project.id=46a7c89c251f4b0f9821da91cdb62326, user.id=d4d5cfabc82f40f285941e8abc75994a) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:42.564800 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af9d59-6ce4-4d48-8b3d-8f0588b07bd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:42.566766 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af9d59-6ce4-4d48-8b3d-8f0588b07bd9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:42.574852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 570/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:42 2022] GET /identity/v3/role_assignments?user.id=d4d5cfabc82f40f285941e8abc75994a&scope.project.id=46a7c89c251f4b0f9821da91cdb62326&role.id=0f002225dde34ef1a2adda2f583b8520 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:44.714030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22829422-651c-406f-88bf-5f47d134ec60 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:44.714867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22829422-651c-406f-88bf-5f47d134ec60 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:44.714867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22829422-651c-406f-88bf-5f47d134ec60 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:44.715985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 561/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:44.721989 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd699a4-926f-413a-8a6b-2e2d0f0f4b0c None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:44.722582 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd699a4-926f-413a-8a6b-2e2d0f0f4b0c None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:44.722839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd699a4-926f-413a-8a6b-2e2d0f0f4b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:44.745138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-1bd699a4-926f-413a-8a6b-2e2d0f0f4b0c None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:44.746411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-1bd699a4-926f-413a-8a6b-2e2d0f0f4b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:44.818481 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 571/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:44.842373 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22829422-651c-406f-88bf-5f47d134ec60 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:44.877595 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:44.953487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22829422-651c-406f-88bf-5f47d134ec60 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:44.991457 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91570f-1d31-4c44-83b3-770213f507e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:44.992175 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91570f-1d31-4c44-83b3-770213f507e5 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:44.992472 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91570f-1d31-4c44-83b3-770213f507e5 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:44.992744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91570f-1d31-4c44-83b3-770213f507e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:44.993944 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91570f-1d31-4c44-83b3-770213f507e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:44.995434 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91570f-1d31-4c44-83b3-770213f507e5 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:44.997118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 562/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:45.003330 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd699a4-926f-413a-8a6b-2e2d0f0f4b0c None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:45.049138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:45.113054 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd699a4-926f-413a-8a6b-2e2d0f0f4b0c None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:45.149382 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068c36f-9605-48f5-a6e1-1ef8bd6153df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:45.150008 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c36f-9605-48f5-a6e1-1ef8bd6153df None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:45.150341 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c36f-9605-48f5-a6e1-1ef8bd6153df None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:45.150672 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c36f-9605-48f5-a6e1-1ef8bd6153df None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:45.151440 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068c36f-9605-48f5-a6e1-1ef8bd6153df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:45.152957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068c36f-9605-48f5-a6e1-1ef8bd6153df None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:45.186147 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 572/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:15:45 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:47.233178 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2c8e-d855-4e91-a7bf-375ce2338009 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:47.233831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2c8e-d855-4e91-a7bf-375ce2338009 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:47.233831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2c8e-d855-4e91-a7bf-375ce2338009 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:47.235162 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 563/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:47.240955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b86a4-b16a-4121-94bc-6938bd1910bb None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:47.241434 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b86a4-b16a-4121-94bc-6938bd1910bb None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:47.241540 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b86a4-b16a-4121-94bc-6938bd1910bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:47.261256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-d87b86a4-b16a-4121-94bc-6938bd1910bb None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:47.262522 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-d87b86a4-b16a-4121-94bc-6938bd1910bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:47.331755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 573/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:47.356366 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea2c8e-d855-4e91-a7bf-375ce2338009 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:47.465511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:47.470613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea2c8e-d855-4e91-a7bf-375ce2338009 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:47.508694 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ae1ce-1b6b-4d5f-95e6-2f6730312a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:47.509260 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ae1ce-1b6b-4d5f-95e6-2f6730312a10 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:47.509547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ae1ce-1b6b-4d5f-95e6-2f6730312a10 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:47.509824 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ae1ce-1b6b-4d5f-95e6-2f6730312a10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:47.511089 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ae1ce-1b6b-4d5f-95e6-2f6730312a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:47.512555 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ae1ce-1b6b-4d5f-95e6-2f6730312a10 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:47.514224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 564/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:15:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:47.520479 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b86a4-b16a-4121-94bc-6938bd1910bb None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:47.547856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:47.631060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b86a4-b16a-4121-94bc-6938bd1910bb None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:47.667997 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bab5a0-7b8f-407e-bedf-7d6786d96881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:47.668634 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab5a0-7b8f-407e-bedf-7d6786d96881 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:47.668957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab5a0-7b8f-407e-bedf-7d6786d96881 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:47.669223 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab5a0-7b8f-407e-bedf-7d6786d96881 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:47.669923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bab5a0-7b8f-407e-bedf-7d6786d96881 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:47.671712 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bab5a0-7b8f-407e-bedf-7d6786d96881 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:47.694926 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 574/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:15:47 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:15:49.778332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0615eaa-f709-4a01-bb0d-ba81c6dfec82 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:49.778913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0615eaa-f709-4a01-bb0d-ba81c6dfec82 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:49.778913 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0615eaa-f709-4a01-bb0d-ba81c6dfec82 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:49.780238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 565/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:49.785695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201cdb1-7754-4d6d-adbf-936ea18ec583 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:49.786139 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201cdb1-7754-4d6d-adbf-936ea18ec583 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:49.786244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201cdb1-7754-4d6d-adbf-936ea18ec583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:49.805286 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-0201cdb1-7754-4d6d-adbf-936ea18ec583 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:49.806520 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-0201cdb1-7754-4d6d-adbf-936ea18ec583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:49.874096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 575/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:49.897633 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0615eaa-f709-4a01-bb0d-ba81c6dfec82 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:50.014642 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0615eaa-f709-4a01-bb0d-ba81c6dfec82 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:50.051947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e997c6-4ee5-478e-8159-778cc76d11a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:50.052593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e997c6-4ee5-478e-8159-778cc76d11a1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:50.052881 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e997c6-4ee5-478e-8159-778cc76d11a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:50.053157 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e997c6-4ee5-478e-8159-778cc76d11a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:50.064359 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e997c6-4ee5-478e-8159-778cc76d11a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-c5e997c6-4ee5-478e-8159-778cc76d11a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:50.069409 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:50.070548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:50.071778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 566/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:15:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:50.077364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201cdb1-7754-4d6d-adbf-936ea18ec583 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:50.090204 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:50.182252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201cdb1-7754-4d6d-adbf-936ea18ec583 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:50.217853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:50.218535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:50.218811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:50.219083 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:50.219802 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:50.220097 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:50.221473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:50.229921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 576/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:50.236191 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e997c6-4ee5-478e-8159-778cc76d11a1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:50.267844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:50.340404 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e997c6-4ee5-478e-8159-778cc76d11a1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:50.376302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e86261-ff29-478c-9ae0-397de4e93f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:50.376950 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86261-ff29-478c-9ae0-397de4e93f57 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:50.377228 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86261-ff29-478c-9ae0-397de4e93f57 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:50.377500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86261-ff29-478c-9ae0-397de4e93f57 None admin] PATH_INFO: `/v3/users/0c4a3527837349faa44d57e8998b3973` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:50.391564 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e86261-ff29-478c-9ae0-397de4e93f57 None admin] RBAC: Authorizing `identity:get_user(user_id=0c4a3527837349faa44d57e8998b3973)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:50.393035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e86261-ff29-478c-9ae0-397de4e93f57 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:50.394760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 567/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:50 2022] GET /identity/v3/users/0c4a3527837349faa44d57e8998b3973 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:50.401029 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:50.477312 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:50.510849 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eebe7-f1db-482a-b414-731af08874ab None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:50.551378 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35541a12-6991-42d5-bc58-e4cb215a4e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:50.551911 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541a12-6991-42d5-bc58-e4cb215a4e3b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:50.552192 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541a12-6991-42d5-bc58-e4cb215a4e3b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:50.552471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541a12-6991-42d5-bc58-e4cb215a4e3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:50.623272 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35541a12-6991-42d5-bc58-e4cb215a4e3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:50.624806 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35541a12-6991-42d5-bc58-e4cb215a4e3b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:50.713817 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:50.754544 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 577/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:50.760532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e86261-ff29-478c-9ae0-397de4e93f57 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:50.856746 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:50.877050 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e86261-ff29-478c-9ae0-397de4e93f57 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:50.913249 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9485ea5-cb1d-4fd8-ae4c-f535eec47150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:50.913900 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485ea5-cb1d-4fd8-ae4c-f535eec47150 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:50.914195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485ea5-cb1d-4fd8-ae4c-f535eec47150 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:50.914487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485ea5-cb1d-4fd8-ae4c-f535eec47150 None admin] PATH_INFO: `/v3/projects/33c7f15384544e0397c1907d354ee1ba` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:50.915717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9485ea5-cb1d-4fd8-ae4c-f535eec47150 None admin] RBAC: Authorizing `identity:get_project(project_id=33c7f15384544e0397c1907d354ee1ba)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:50.917166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9485ea5-cb1d-4fd8-ae4c-f535eec47150 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:50.918904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 568/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:50 2022] GET /identity/v3/projects/33c7f15384544e0397c1907d354ee1ba => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:50.925799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35541a12-6991-42d5-bc58-e4cb215a4e3b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:51.031868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35541a12-6991-42d5-bc58-e4cb215a4e3b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:51.067462 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd35d73-34e7-4e1e-9dc6-9610bcd63a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:51.068083 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd35d73-34e7-4e1e-9dc6-9610bcd63a44 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:51.068372 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd35d73-34e7-4e1e-9dc6-9610bcd63a44 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:51.068642 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd35d73-34e7-4e1e-9dc6-9610bcd63a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:51.069441 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd35d73-34e7-4e1e-9dc6-9610bcd63a44 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=33c7f15384544e0397c1907d354ee1ba, user.id=0c4a3527837349faa44d57e8998b3973) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:51.069750 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd35d73-34e7-4e1e-9dc6-9610bcd63a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:51.071605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd35d73-34e7-4e1e-9dc6-9610bcd63a44 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:51.077426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 578/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:50 2022] GET /identity/v3/role_assignments?user.id=0c4a3527837349faa44d57e8998b3973&scope.project.id=33c7f15384544e0397c1907d354ee1ba&role.id=40504e80dbb845aeb07ca258977ecacb => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:53.097266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304aa4-b07f-46c1-a7f1-3e50229c2830 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:53.097829 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304aa4-b07f-46c1-a7f1-3e50229c2830 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:53.097829 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304aa4-b07f-46c1-a7f1-3e50229c2830 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:53.099163 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 569/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:53.107121 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731072-5e54-44b3-bafd-5cfac2a3a955 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:53.107623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731072-5e54-44b3-bafd-5cfac2a3a955 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:53.107979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731072-5e54-44b3-bafd-5cfac2a3a955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:53.133456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-f5731072-5e54-44b3-bafd-5cfac2a3a955 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:53.134982 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-f5731072-5e54-44b3-bafd-5cfac2a3a955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:53.173058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:53.220414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 579/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:53.241861 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304aa4-b07f-46c1-a7f1-3e50229c2830 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:53.356056 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304aa4-b07f-46c1-a7f1-3e50229c2830 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:53.393833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cdce3-2b66-40a9-afe4-efe58441fe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:53.394515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cdce3-2b66-40a9-afe4-efe58441fe25 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:53.394803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cdce3-2b66-40a9-afe4-efe58441fe25 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:53.395078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cdce3-2b66-40a9-afe4-efe58441fe25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:53.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cdce3-2b66-40a9-afe4-efe58441fe25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-c35cdce3-2b66-40a9-afe4-efe58441fe25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:53.411173 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:53.412309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:15:53.413332 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 570/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:15:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:53.420260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5731072-5e54-44b3-bafd-5cfac2a3a955 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:53.541158 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5731072-5e54-44b3-bafd-5cfac2a3a955 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:53.577959 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:53.578624 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:53.578911 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:53.579168 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:53.579887 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:53.580205 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:53.581597 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:53.591049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 580/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:15:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:53.597850 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cdce3-2b66-40a9-afe4-efe58441fe25 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:53.616287 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:53.714625 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cdce3-2b66-40a9-afe4-efe58441fe25 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:53.754569 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b1ffb-115e-48b4-a199-081cb26a1ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:53.755210 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b1ffb-115e-48b4-a199-081cb26a1ec8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:53.755505 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b1ffb-115e-48b4-a199-081cb26a1ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:53.755805 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b1ffb-115e-48b4-a199-081cb26a1ec8 None admin] PATH_INFO: `/v3/users/0c4a3527837349faa44d57e8998b3973` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:53.757078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b1ffb-115e-48b4-a199-081cb26a1ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=0c4a3527837349faa44d57e8998b3973)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:53.758707 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b1ffb-115e-48b4-a199-081cb26a1ec8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:53.760375 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 571/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:53 2022] GET /identity/v3/users/0c4a3527837349faa44d57e8998b3973 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:53.766527 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:53.798364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:53.887534 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256dce2-9ce0-4ca6-aacf-967c11355b92 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:53.923918 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb20d-57d7-476c-8ae7-6a9e8a182b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:53.924592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb20d-57d7-476c-8ae7-6a9e8a182b94 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:53.924875 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb20d-57d7-476c-8ae7-6a9e8a182b94 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:53.925153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb20d-57d7-476c-8ae7-6a9e8a182b94 None admin] PATH_INFO: `/v3/projects/33c7f15384544e0397c1907d354ee1ba` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:53.926404 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb20d-57d7-476c-8ae7-6a9e8a182b94 None admin] RBAC: Authorizing `identity:get_project(project_id=33c7f15384544e0397c1907d354ee1ba)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:53.927852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb20d-57d7-476c-8ae7-6a9e8a182b94 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:53.929560 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 581/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:53 2022] GET /identity/v3/projects/33c7f15384544e0397c1907d354ee1ba => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:53.936078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b1ffb-115e-48b4-a199-081cb26a1ec8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:54.024889 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:54.044583 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b1ffb-115e-48b4-a199-081cb26a1ec8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:54.082184 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcded51c-9071-4d76-a295-1ceb6dc63e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:54.082877 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded51c-9071-4d76-a295-1ceb6dc63e82 None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:54.083158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded51c-9071-4d76-a295-1ceb6dc63e82 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:54.083436 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded51c-9071-4d76-a295-1ceb6dc63e82 None admin] PATH_INFO: `/v3/projects/33c7f15384544e0397c1907d354ee1ba/users/0c4a3527837349faa44d57e8998b3973/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:54.085466 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcded51c-9071-4d76-a295-1ceb6dc63e82 None admin] RBAC: Authorizing `identity:create_grant(project_id=33c7f15384544e0397c1907d354ee1ba, user_id=0c4a3527837349faa44d57e8998b3973, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:54.086953 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcded51c-9071-4d76-a295-1ceb6dc63e82 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:54.103657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 572/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:15:53 2022] PUT /identity/v3/projects/33c7f15384544e0397c1907d354ee1ba/users/0c4a3527837349faa44d57e8998b3973/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:15:56.144825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04a22a-3f92-47c0-b915-cbe6a0b7a73b None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:56.145422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04a22a-3f92-47c0-b915-cbe6a0b7a73b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:56.145422 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04a22a-3f92-47c0-b915-cbe6a0b7a73b None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:56.146756 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 582/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:56.152370 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6de4a-bea8-40bd-b8de-e69edd136565 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:56.152876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6de4a-bea8-40bd-b8de-e69edd136565 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:56.152876 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6de4a-bea8-40bd-b8de-e69edd136565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:56.172506 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-bdb6de4a-bea8-40bd-b8de-e69edd136565 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:56.173746 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-bdb6de4a-bea8-40bd-b8de-e69edd136565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:56.248747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 573/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:56.272723 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04a22a-3f92-47c0-b915-cbe6a0b7a73b None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:56.384514 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:56.393757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04a22a-3f92-47c0-b915-cbe6a0b7a73b None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:56.431117 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe467ee9-f23c-44f7-a189-e096c316f293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:56.431689 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467ee9-f23c-44f7-a189-e096c316f293 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:56.431971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467ee9-f23c-44f7-a189-e096c316f293 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:56.432250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467ee9-f23c-44f7-a189-e096c316f293 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:56.446170 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe467ee9-f23c-44f7-a189-e096c316f293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-fe467ee9-f23c-44f7-a189-e096c316f293 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:15:56.451183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:15:56.452207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:15:56.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 583/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:15:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:15:56.459040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6de4a-bea8-40bd-b8de-e69edd136565 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:56.535941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:56.581202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6de4a-bea8-40bd-b8de-e69edd136565 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:56.617514 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:56.618165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:56.618455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:56.618731 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:56.619490 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:56.619802 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:56.621206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:56.629762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 574/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:15:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:56.635978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe467ee9-f23c-44f7-a189-e096c316f293 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:56.763377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe467ee9-f23c-44f7-a189-e096c316f293 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:56.799614 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2d2d3-3c40-4dd3-9cca-e3d3ca4a571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:56.800271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2d2d3-3c40-4dd3-9cca-e3d3ca4a571c None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:56.800556 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2d2d3-3c40-4dd3-9cca-e3d3ca4a571c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:56.800833 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2d2d3-3c40-4dd3-9cca-e3d3ca4a571c None admin] PATH_INFO: `/v3/users/0c4a3527837349faa44d57e8998b3973` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:56.802015 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2d2d3-3c40-4dd3-9cca-e3d3ca4a571c None admin] RBAC: Authorizing `identity:get_user(user_id=0c4a3527837349faa44d57e8998b3973)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:56.803507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2d2d3-3c40-4dd3-9cca-e3d3ca4a571c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:56.805164 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 584/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:15:56 2022] GET /identity/v3/users/0c4a3527837349faa44d57e8998b3973 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:56.811414 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:56.921043 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e6d5d-2ed7-483c-9779-b0c15a5576a4 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:56.958859 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25602413-4cac-4eea-872e-0ace28d8e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:56.959511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25602413-4cac-4eea-872e-0ace28d8e83d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:56.959794 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25602413-4cac-4eea-872e-0ace28d8e83d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:56.960072 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25602413-4cac-4eea-872e-0ace28d8e83d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:56.963370 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:57.036907 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25602413-4cac-4eea-872e-0ace28d8e83d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:57.038319 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25602413-4cac-4eea-872e-0ace28d8e83d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:57.143295 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 575/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:15:57.149584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2d2d3-3c40-4dd3-9cca-e3d3ca4a571c None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:57.176325 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:57.254879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2d2d3-3c40-4dd3-9cca-e3d3ca4a571c None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:57.288496 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93757f2-e0b8-4f01-b790-dd0548a0ec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:57.289073 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93757f2-e0b8-4f01-b790-dd0548a0ec81 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:57.289361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93757f2-e0b8-4f01-b790-dd0548a0ec81 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:57.289645 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93757f2-e0b8-4f01-b790-dd0548a0ec81 None admin] PATH_INFO: `/v3/projects/33c7f15384544e0397c1907d354ee1ba` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:57.290936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93757f2-e0b8-4f01-b790-dd0548a0ec81 None admin] RBAC: Authorizing `identity:get_project(project_id=33c7f15384544e0397c1907d354ee1ba)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:57.292406 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93757f2-e0b8-4f01-b790-dd0548a0ec81 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:57.294151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 585/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:15:57 2022] GET /identity/v3/projects/33c7f15384544e0397c1907d354ee1ba => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:57.300790 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25602413-4cac-4eea-872e-0ace28d8e83d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:57.336499 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:57.409951 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25602413-4cac-4eea-872e-0ace28d8e83d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:57.448237 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552671c1-2105-4761-9621-5304e9a2f576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:57.448237 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552671c1-2105-4761-9621-5304e9a2f576 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:57.448573 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552671c1-2105-4761-9621-5304e9a2f576 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:57.448668 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552671c1-2105-4761-9621-5304e9a2f576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:57.449432 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552671c1-2105-4761-9621-5304e9a2f576 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=33c7f15384544e0397c1907d354ee1ba, user.id=0c4a3527837349faa44d57e8998b3973) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:15:57.449744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552671c1-2105-4761-9621-5304e9a2f576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:57.451247 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552671c1-2105-4761-9621-5304e9a2f576 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:57.457686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 576/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:15:57 2022] GET /identity/v3/role_assignments?user.id=0c4a3527837349faa44d57e8998b3973&scope.project.id=33c7f15384544e0397c1907d354ee1ba&role.id=40504e80dbb845aeb07ca258977ecacb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:15:59.519634 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1265f8e-2d49-43e0-b66c-eb0066076477 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:59.520254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1265f8e-2d49-43e0-b66c-eb0066076477 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:59.520254 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1265f8e-2d49-43e0-b66c-eb0066076477 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:59.521531 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 586/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:15:59.527025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c03501-6a90-4c4b-98ab-280d631087b2 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:59.527772 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c03501-6a90-4c4b-98ab-280d631087b2 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:59.527772 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c03501-6a90-4c4b-98ab-280d631087b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:59.548191 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-13c03501-6a90-4c4b-98ab-280d631087b2 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:15:59.549457 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-13c03501-6a90-4c4b-98ab-280d631087b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:15:59.628684 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 577/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:15:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:15:59.651318 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1265f8e-2d49-43e0-b66c-eb0066076477 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:15:59.758830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:59.771521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1265f8e-2d49-43e0-b66c-eb0066076477 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:15:59.812282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d39f-cc9f-41a7-884f-a4dedcbf152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:15:59.812927 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d39f-cc9f-41a7-884f-a4dedcbf152a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:15:59.813211 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d39f-cc9f-41a7-884f-a4dedcbf152a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:15:59.813484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d39f-cc9f-41a7-884f-a4dedcbf152a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:15:59.887702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3d39f-cc9f-41a7-884f-a4dedcbf152a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:15:59.889261 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3d39f-cc9f-41a7-884f-a4dedcbf152a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:15:59.986561 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:15:59.997919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 587/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:15:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:16:00.005302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c03501-6a90-4c4b-98ab-280d631087b2 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:00.035009 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:00.111438 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c03501-6a90-4c4b-98ab-280d631087b2 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:00.155847 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10050dbb-3ea9-41f2-9433-76b4432a7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:00.156495 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050dbb-3ea9-41f2-9433-76b4432a7529 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:00.156778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050dbb-3ea9-41f2-9433-76b4432a7529 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:00.157054 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050dbb-3ea9-41f2-9433-76b4432a7529 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:00.165696 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10050dbb-3ea9-41f2-9433-76b4432a7529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:00.167258 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10050dbb-3ea9-41f2-9433-76b4432a7529 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-10050dbb-3ea9-41f2-9433-76b4432a7529 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:16:00.181764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:16:00.183020 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:16:00.183853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 578/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:16:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:16:00.190611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d39f-cc9f-41a7-884f-a4dedcbf152a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:00.299392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d39f-cc9f-41a7-884f-a4dedcbf152a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:00.340309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5106652-d4e5-4085-b9c0-e559d3f00b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:00.341276 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5106652-d4e5-4085-b9c0-e559d3f00b8c None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:00.341737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5106652-d4e5-4085-b9c0-e559d3f00b8c None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:00.342237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5106652-d4e5-4085-b9c0-e559d3f00b8c None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:00.343392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5106652-d4e5-4085-b9c0-e559d3f00b8c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:16:00.343696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5106652-d4e5-4085-b9c0-e559d3f00b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:00.345090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5106652-d4e5-4085-b9c0-e559d3f00b8c None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:00.353551 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 588/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:16:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:02.354917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85a67e-c21f-4192-83e2-c4d74f35d70b None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:02.355510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85a67e-c21f-4192-83e2-c4d74f35d70b None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:02.355510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85a67e-c21f-4192-83e2-c4d74f35d70b None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:02.356831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 579/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:16:02.362334 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e5422-53fa-4ba4-a5ad-9c7cd370b2fb None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:02.362782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e5422-53fa-4ba4-a5ad-9c7cd370b2fb None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:02.362880 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e5422-53fa-4ba4-a5ad-9c7cd370b2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:02.381494 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-9e9e5422-53fa-4ba4-a5ad-9c7cd370b2fb None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:16:02.382716 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-9e9e5422-53fa-4ba4-a5ad-9c7cd370b2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:16:02.435321 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:02.453503 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 589/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:16:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:16:02.476519 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85a67e-c21f-4192-83e2-c4d74f35d70b None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:02.565965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:02.580403 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85a67e-c21f-4192-83e2-c4d74f35d70b None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:02.616010 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9a7f5-18d7-4d72-8eb6-bdf90e9d3cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:02.616558 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9a7f5-18d7-4d72-8eb6-bdf90e9d3cef None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:02.616836 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9a7f5-18d7-4d72-8eb6-bdf90e9d3cef None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:02.617107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9a7f5-18d7-4d72-8eb6-bdf90e9d3cef None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:02.617838 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9a7f5-18d7-4d72-8eb6-bdf90e9d3cef None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:02.619298 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9a7f5-18d7-4d72-8eb6-bdf90e9d3cef None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:02.638493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 580/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jul 28 08:16:02 2022] POST /identity/v3/domains => generated 267 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:16:04.664388 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca236d7-282c-46f0-bf0c-214df28054ec None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:04.665262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca236d7-282c-46f0-bf0c-214df28054ec None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:04.665262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca236d7-282c-46f0-bf0c-214df28054ec None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:04.667424 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 590/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:16:04.676100 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea438a4-41e2-4377-ac79-191b7dc02bc9 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:04.676516 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea438a4-41e2-4377-ac79-191b7dc02bc9 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:04.676674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea438a4-41e2-4377-ac79-191b7dc02bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:04.700108 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-bea438a4-41e2-4377-ac79-191b7dc02bc9 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:16:04.701334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-bea438a4-41e2-4377-ac79-191b7dc02bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:16:04.784266 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 581/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:16:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:16:04.810157 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca236d7-282c-46f0-bf0c-214df28054ec None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:04.919232 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca236d7-282c-46f0-bf0c-214df28054ec None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:04.960264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234e022-b1b0-4d88-a60e-59ade5ab60ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:04.960932 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234e022-b1b0-4d88-a60e-59ade5ab60ff None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:04.961188 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234e022-b1b0-4d88-a60e-59ade5ab60ff None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:04.961489 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234e022-b1b0-4d88-a60e-59ade5ab60ff None admin] PATH_INFO: `/v3/domains/7b289e050b5b44c2b1a1a14703474145` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:04.974143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234e022-b1b0-4d88-a60e-59ade5ab60ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=7b289e050b5b44c2b1a1a14703474145)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:04.975650 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234e022-b1b0-4d88-a60e-59ade5ab60ff None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:04.977418 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 591/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:16:04 2022] GET /identity/v3/domains/7b289e050b5b44c2b1a1a14703474145 => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:04.983831 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea438a4-41e2-4377-ac79-191b7dc02bc9 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:05.071584 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:05.096976 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea438a4-41e2-4377-ac79-191b7dc02bc9 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:05.141065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ec37f-b86f-4c2d-aa80-9cd05b5fe585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:05.141696 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ec37f-b86f-4c2d-aa80-9cd05b5fe585 None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:05.141984 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ec37f-b86f-4c2d-aa80-9cd05b5fe585 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:05.142293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ec37f-b86f-4c2d-aa80-9cd05b5fe585 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:05.142985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ec37f-b86f-4c2d-aa80-9cd05b5fe585 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:05.144452 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ec37f-b86f-4c2d-aa80-9cd05b5fe585 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:05.173533 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 582/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:16:04 2022] POST /identity/v3/projects => generated 361 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:16:07.266897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2024dbb-5c9b-4670-849d-3dc5ac4dc382 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:07.267636 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2024dbb-5c9b-4670-849d-3dc5ac4dc382 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:07.267636 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2024dbb-5c9b-4670-849d-3dc5ac4dc382 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:07.268885 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 592/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:16:07.275975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c166c-91ab-4dcb-8b49-874a6e78ce42 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:07.276463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c166c-91ab-4dcb-8b49-874a6e78ce42 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:07.276580 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c166c-91ab-4dcb-8b49-874a6e78ce42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:07.295510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-9c5c166c-91ab-4dcb-8b49-874a6e78ce42 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:16:07.296769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-9c5c166c-91ab-4dcb-8b49-874a6e78ce42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:16:07.367475 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 583/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:16:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:16:07.392242 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2024dbb-5c9b-4670-849d-3dc5ac4dc382 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:07.443962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:07.501343 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2024dbb-5c9b-4670-849d-3dc5ac4dc382 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:07.537973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fdf52-9130-43b3-885e-1257298f6fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:07.538547 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fdf52-9130-43b3-885e-1257298f6fc8 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:07.538826 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fdf52-9130-43b3-885e-1257298f6fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:07.539096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fdf52-9130-43b3-885e-1257298f6fc8 None admin] PATH_INFO: `/v3/domains/7b289e050b5b44c2b1a1a14703474145` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:07.540316 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fdf52-9130-43b3-885e-1257298f6fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7b289e050b5b44c2b1a1a14703474145)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:07.541761 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fdf52-9130-43b3-885e-1257298f6fc8 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:07.543425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 593/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:16:07 2022] GET /identity/v3/domains/7b289e050b5b44c2b1a1a14703474145 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:07.549677 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c166c-91ab-4dcb-8b49-874a6e78ce42 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:07.618167 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:07.663011 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c166c-91ab-4dcb-8b49-874a6e78ce42 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:07.700599 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c5cc1-fe74-4d7f-acfd-96f1302addab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:07.701263 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c5cc1-fe74-4d7f-acfd-96f1302addab None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:07.701546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c5cc1-fe74-4d7f-acfd-96f1302addab None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:07.701798 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c5cc1-fe74-4d7f-acfd-96f1302addab None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:07.702556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c5cc1-fe74-4d7f-acfd-96f1302addab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:07.704034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c5cc1-fe74-4d7f-acfd-96f1302addab None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:07.728611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 584/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:16:07 2022] POST /identity/v3/users => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:16:09.766067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353e085-5041-4311-9bab-afa93e6f2e8d None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:09.766675 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353e085-5041-4311-9bab-afa93e6f2e8d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:09.766675 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353e085-5041-4311-9bab-afa93e6f2e8d None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:09.768048 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 594/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:16:09.773438 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489f8e-b5c1-4bae-b750-3a4439dbac2e None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:09.773909 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489f8e-b5c1-4bae-b750-3a4439dbac2e None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:09.774000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489f8e-b5c1-4bae-b750-3a4439dbac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:09.793566 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-00489f8e-b5c1-4bae-b750-3a4439dbac2e None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:16:09.794816 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-00489f8e-b5c1-4bae-b750-3a4439dbac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:16:09.868904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 585/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:16:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:16:09.892778 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353e085-5041-4311-9bab-afa93e6f2e8d None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:10.018608 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353e085-5041-4311-9bab-afa93e6f2e8d None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:10.033167 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:10.056420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15eea7-7f6d-448d-b739-b07c31c3db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:10.057080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15eea7-7f6d-448d-b739-b07c31c3db5b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:10.057362 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15eea7-7f6d-448d-b739-b07c31c3db5b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:10.057623 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15eea7-7f6d-448d-b739-b07c31c3db5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:10.069006 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15eea7-7f6d-448d-b739-b07c31c3db5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-bb15eea7-7f6d-448d-b739-b07c31c3db5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:16:10.074392 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:16:10.075558 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:16:10.076506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 595/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:16:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:16:10.081943 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00489f8e-b5c1-4bae-b750-3a4439dbac2e None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:10.131584 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:10.186384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00489f8e-b5c1-4bae-b750-3a4439dbac2e None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:10.221578 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:10.222231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:10.222507 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:10.222778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:10.223479 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:16:10.223778 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:10.225162 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:10.233137 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 586/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:10.239243 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15eea7-7f6d-448d-b739-b07c31c3db5b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:10.345843 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15eea7-7f6d-448d-b739-b07c31c3db5b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:10.381123 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12eac5-1cd7-4961-b1e4-50259a2710ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:10.381763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12eac5-1cd7-4961-b1e4-50259a2710ec None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:10.382044 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12eac5-1cd7-4961-b1e4-50259a2710ec None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:10.382354 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12eac5-1cd7-4961-b1e4-50259a2710ec None admin] PATH_INFO: `/v3/users/5120260c67de4828bdc14a50719006db` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:10.396143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12eac5-1cd7-4961-b1e4-50259a2710ec None admin] RBAC: Authorizing `identity:get_user(user_id=5120260c67de4828bdc14a50719006db)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:10.397604 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12eac5-1cd7-4961-b1e4-50259a2710ec None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:10.399326 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 596/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:16:10 2022] GET /identity/v3/users/5120260c67de4828bdc14a50719006db => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:10.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:10.509804 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f048357-dd57-4e95-abd2-8bb596761b42 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:10.518906 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:10.546592 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19a5b4-d754-4a48-87e3-7ecc74239842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:10.547224 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a5b4-d754-4a48-87e3-7ecc74239842 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:10.547501 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a5b4-d754-4a48-87e3-7ecc74239842 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:10.547773 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a5b4-d754-4a48-87e3-7ecc74239842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:10.618086 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19a5b4-d754-4a48-87e3-7ecc74239842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:10.619517 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19a5b4-d754-4a48-87e3-7ecc74239842 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:10.720204 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 587/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:16:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:16:10.727708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12eac5-1cd7-4961-b1e4-50259a2710ec None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:10.762767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:10.836871 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12eac5-1cd7-4961-b1e4-50259a2710ec None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:10.873908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5796cf4-91fd-404b-8823-da14afdc83d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:10.874598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5796cf4-91fd-404b-8823-da14afdc83d7 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:10.874882 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5796cf4-91fd-404b-8823-da14afdc83d7 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:10.875160 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5796cf4-91fd-404b-8823-da14afdc83d7 None admin] PATH_INFO: `/v3/projects/01ca3dec3ea944f8a6ae2b9faf2bf79d` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:10.876460 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5796cf4-91fd-404b-8823-da14afdc83d7 None admin] RBAC: Authorizing `identity:get_project(project_id=01ca3dec3ea944f8a6ae2b9faf2bf79d)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:10.877964 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5796cf4-91fd-404b-8823-da14afdc83d7 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:10.879748 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 597/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:16:10 2022] GET /identity/v3/projects/01ca3dec3ea944f8a6ae2b9faf2bf79d => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:10.886382 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19a5b4-d754-4a48-87e3-7ecc74239842 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:10.897107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:10.994206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19a5b4-d754-4a48-87e3-7ecc74239842 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:11.036288 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26559fb0-9fc2-4037-82a8-da05c88943a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:11.036920 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26559fb0-9fc2-4037-82a8-da05c88943a3 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:11.037196 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26559fb0-9fc2-4037-82a8-da05c88943a3 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:11.037465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26559fb0-9fc2-4037-82a8-da05c88943a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:11.038302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26559fb0-9fc2-4037-82a8-da05c88943a3 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=01ca3dec3ea944f8a6ae2b9faf2bf79d, user.id=5120260c67de4828bdc14a50719006db) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:16:11.038609 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26559fb0-9fc2-4037-82a8-da05c88943a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:11.040063 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26559fb0-9fc2-4037-82a8-da05c88943a3 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:11.046265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 588/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:16:10 2022] GET /identity/v3/role_assignments?user.id=5120260c67de4828bdc14a50719006db&scope.project.id=01ca3dec3ea944f8a6ae2b9faf2bf79d&role.id=40504e80dbb845aeb07ca258977ecacb => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:16:13.032329 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70541210-a056-48c3-9685-b7604be66517 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:13.032889 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70541210-a056-48c3-9685-b7604be66517 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:13.032889 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70541210-a056-48c3-9685-b7604be66517 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:13.034151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 598/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:16:13.039753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f4ee0-e790-468c-b5fe-0c5faaa26f68 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:13.040185 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f4ee0-e790-468c-b5fe-0c5faaa26f68 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:13.040443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f4ee0-e790-468c-b5fe-0c5faaa26f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:13.059942 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-046f4ee0-e790-468c-b5fe-0c5faaa26f68 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:16:13.061278 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-046f4ee0-e790-468c-b5fe-0c5faaa26f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:16:13.133452 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 589/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:16:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:16:13.156237 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70541210-a056-48c3-9685-b7604be66517 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:13.273070 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:13.278861 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70541210-a056-48c3-9685-b7604be66517 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:13.325838 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658a5ab-e878-47f8-979b-7157a81d65fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:13.326612 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658a5ab-e878-47f8-979b-7157a81d65fd None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:13.327001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658a5ab-e878-47f8-979b-7157a81d65fd None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:13.327252 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658a5ab-e878-47f8-979b-7157a81d65fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:13.338928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658a5ab-e878-47f8-979b-7157a81d65fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-e658a5ab-e878-47f8-979b-7157a81d65fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:16:13.343971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:16:13.345007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:16:13.346007 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 599/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:16:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:16:13.353357 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f4ee0-e790-468c-b5fe-0c5faaa26f68 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:13.355657 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:13.475974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f4ee0-e790-468c-b5fe-0c5faaa26f68 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:13.513459 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:13.514096 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:13.514511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:13.514792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:13.515524 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:16:13.515830 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:13.517234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:13.526358 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 590/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:13.533736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658a5ab-e878-47f8-979b-7157a81d65fd None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:13.643046 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658a5ab-e878-47f8-979b-7157a81d65fd None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:13.680312 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dadde1-51c2-4dbe-8919-b92ed737407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:13.680976 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dadde1-51c2-4dbe-8919-b92ed737407b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:13.681260 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dadde1-51c2-4dbe-8919-b92ed737407b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:13.681538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dadde1-51c2-4dbe-8919-b92ed737407b None admin] PATH_INFO: `/v3/users/5120260c67de4828bdc14a50719006db` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:13.682760 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dadde1-51c2-4dbe-8919-b92ed737407b None admin] RBAC: Authorizing `identity:get_user(user_id=5120260c67de4828bdc14a50719006db)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:13.684238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dadde1-51c2-4dbe-8919-b92ed737407b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:13.685936 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 600/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:16:13 2022] GET /identity/v3/users/5120260c67de4828bdc14a50719006db => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:13.692327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:13.758528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:13.797174 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ee6e3-fbee-4e64-b8f9-43e22136fe36 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:13.833236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ef3dc-be68-4bc7-8fd2-06e83427f729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:13.833882 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ef3dc-be68-4bc7-8fd2-06e83427f729 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:13.834181 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ef3dc-be68-4bc7-8fd2-06e83427f729 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:13.834478 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ef3dc-be68-4bc7-8fd2-06e83427f729 None admin] PATH_INFO: `/v3/projects/01ca3dec3ea944f8a6ae2b9faf2bf79d` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:13.835747 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ef3dc-be68-4bc7-8fd2-06e83427f729 None admin] RBAC: Authorizing `identity:get_project(project_id=01ca3dec3ea944f8a6ae2b9faf2bf79d)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:13.837221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ef3dc-be68-4bc7-8fd2-06e83427f729 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:13.838937 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 591/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:16:13 2022] GET /identity/v3/projects/01ca3dec3ea944f8a6ae2b9faf2bf79d => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:13.845448 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dadde1-51c2-4dbe-8919-b92ed737407b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:13.876187 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:13.952421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dadde1-51c2-4dbe-8919-b92ed737407b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:13.992058 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb697-a90f-4a1a-83e5-7f3afd42198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:13.992717 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb697-a90f-4a1a-83e5-7f3afd42198d None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:13.992999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb697-a90f-4a1a-83e5-7f3afd42198d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:13.993273 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb697-a90f-4a1a-83e5-7f3afd42198d None admin] PATH_INFO: `/v3/projects/01ca3dec3ea944f8a6ae2b9faf2bf79d/users/5120260c67de4828bdc14a50719006db/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:13.995351 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb697-a90f-4a1a-83e5-7f3afd42198d None admin] RBAC: Authorizing `identity:create_grant(project_id=01ca3dec3ea944f8a6ae2b9faf2bf79d, user_id=5120260c67de4828bdc14a50719006db, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:13.996827 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb697-a90f-4a1a-83e5-7f3afd42198d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:14.012733 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 601/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:16:13 2022] PUT /identity/v3/projects/01ca3dec3ea944f8a6ae2b9faf2bf79d/users/5120260c67de4828bdc14a50719006db/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:16:15.988215 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62a6f6-2777-431d-8b15-6e5c4fcaf5a2 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:15.988748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62a6f6-2777-431d-8b15-6e5c4fcaf5a2 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:15.988748 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62a6f6-2777-431d-8b15-6e5c4fcaf5a2 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:15.990030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 592/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:16:15.995601 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004dec1-5b3c-44d5-9d89-e59f326fdab2 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:15.996052 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004dec1-5b3c-44d5-9d89-e59f326fdab2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:15.996143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004dec1-5b3c-44d5-9d89-e59f326fdab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:16.015644 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-1004dec1-5b3c-44d5-9d89-e59f326fdab2 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:16:16.016855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-1004dec1-5b3c-44d5-9d89-e59f326fdab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:16:16.093097 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 602/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:16:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:16:16.116775 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62a6f6-2777-431d-8b15-6e5c4fcaf5a2 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:16.236320 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62a6f6-2777-431d-8b15-6e5c4fcaf5a2 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:16.268828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:16.275323 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb133e1-20ad-4ae0-9dce-ce22ab7a3d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:16.275855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb133e1-20ad-4ae0-9dce-ce22ab7a3d1e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:16.276121 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb133e1-20ad-4ae0-9dce-ce22ab7a3d1e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:16.276397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb133e1-20ad-4ae0-9dce-ce22ab7a3d1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:16.287225 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb133e1-20ad-4ae0-9dce-ce22ab7a3d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-abb133e1-20ad-4ae0-9dce-ce22ab7a3d1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:16:16.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:16:16.293354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:16:16.294192 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 593/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:16:16.300190 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004dec1-5b3c-44d5-9d89-e59f326fdab2 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:16.321301 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:16.407096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004dec1-5b3c-44d5-9d89-e59f326fdab2 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:16.442384 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:16.443008 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:16.443292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:16.443566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:16.444290 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:16:16.444598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:16.446381 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:16.454728 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 603/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:16.460895 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb133e1-20ad-4ae0-9dce-ce22ab7a3d1e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:16.561418 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb133e1-20ad-4ae0-9dce-ce22ab7a3d1e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:16.599260 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dc444-1488-4f03-b957-989c1eb6df6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:16.599901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dc444-1488-4f03-b957-989c1eb6df6f None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:16.600179 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dc444-1488-4f03-b957-989c1eb6df6f None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:16.600449 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dc444-1488-4f03-b957-989c1eb6df6f None admin] PATH_INFO: `/v3/users/5120260c67de4828bdc14a50719006db` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:16.601681 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6dc444-1488-4f03-b957-989c1eb6df6f None admin] RBAC: Authorizing `identity:get_user(user_id=5120260c67de4828bdc14a50719006db)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:16.603174 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6dc444-1488-4f03-b957-989c1eb6df6f None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:16.604796 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 594/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:16:16 2022] GET /identity/v3/users/5120260c67de4828bdc14a50719006db => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:16.611156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:16.698485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:16.718951 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299a57d-50c7-4e3e-ab99-6ba74e8c6115 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:16.753854 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d1a90-ce73-49fe-ab2b-f6e8d759325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:16.754523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d1a90-ce73-49fe-ab2b-f6e8d759325f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:16.754800 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d1a90-ce73-49fe-ab2b-f6e8d759325f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:16.755067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d1a90-ce73-49fe-ab2b-f6e8d759325f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:16.825679 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d1a90-ce73-49fe-ab2b-f6e8d759325f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:16.827112 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d1a90-ce73-49fe-ab2b-f6e8d759325f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:16.908855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:16.929420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 604/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:16:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:16:16.935744 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dc444-1488-4f03-b957-989c1eb6df6f None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:17.000982 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:17.043443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dc444-1488-4f03-b957-989c1eb6df6f None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:17.079833 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e680c-5372-4c1e-b726-55f32559b317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:17.080432 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e680c-5372-4c1e-b726-55f32559b317 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:17.080683 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e680c-5372-4c1e-b726-55f32559b317 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:17.080954 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e680c-5372-4c1e-b726-55f32559b317 None admin] PATH_INFO: `/v3/projects/01ca3dec3ea944f8a6ae2b9faf2bf79d` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:17.082185 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e680c-5372-4c1e-b726-55f32559b317 None admin] RBAC: Authorizing `identity:get_project(project_id=01ca3dec3ea944f8a6ae2b9faf2bf79d)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:17.083667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e680c-5372-4c1e-b726-55f32559b317 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:17.085347 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 595/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:16:16 2022] GET /identity/v3/projects/01ca3dec3ea944f8a6ae2b9faf2bf79d => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:17.091839 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d1a90-ce73-49fe-ab2b-f6e8d759325f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:16:17.195867 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d1a90-ce73-49fe-ab2b-f6e8d759325f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:16:17.234580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd628e-2da5-4b2e-97ac-f72f5097f369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:16:17.235212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd628e-2da5-4b2e-97ac-f72f5097f369 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:16:17.235497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd628e-2da5-4b2e-97ac-f72f5097f369 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:16:17.235860 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd628e-2da5-4b2e-97ac-f72f5097f369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:16:17.236669 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd628e-2da5-4b2e-97ac-f72f5097f369 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=01ca3dec3ea944f8a6ae2b9faf2bf79d, user.id=5120260c67de4828bdc14a50719006db) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:16:17.237049 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd628e-2da5-4b2e-97ac-f72f5097f369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:16:17.238477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd628e-2da5-4b2e-97ac-f72f5097f369 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:16:17.244385 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 605/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:16:17 2022] GET /identity/v3/role_assignments?user.id=5120260c67de4828bdc14a50719006db&scope.project.id=01ca3dec3ea944f8a6ae2b9faf2bf79d&role.id=40504e80dbb845aeb07ca258977ecacb => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:16:27.074980 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=36 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:16:27.241713 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=78 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:13.121432 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d7a5f-b2a4-462b-a802-faa1bec4cbf0 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:13.122072 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d7a5f-b2a4-462b-a802-faa1bec4cbf0 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:13.122072 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d7a5f-b2a4-462b-a802-faa1bec4cbf0 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:13.123417 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 596/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:13.129432 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716777a-87df-454b-862f-19c8e75952dc None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:13.129953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716777a-87df-454b-862f-19c8e75952dc None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:13.129953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716777a-87df-454b-862f-19c8e75952dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:13.152319 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-b716777a-87df-454b-862f-19c8e75952dc None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:13.153828 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-b716777a-87df-454b-862f-19c8e75952dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:13.245612 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 606/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:13.270924 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d7a5f-b2a4-462b-a802-faa1bec4cbf0 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:13.395749 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d7a5f-b2a4-462b-a802-faa1bec4cbf0 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:13.437030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:13.437872 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:13.437985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:13.438559 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:13.454631 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:13.456803 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:13.474974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:13.476140 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:13.482587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:13.482587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:13.482587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:13.482587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:13.482587 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 597/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:13.484484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716777a-87df-454b-862f-19c8e75952dc None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:13.619440 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716777a-87df-454b-862f-19c8e75952dc None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:13.658006 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:13.674533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fdeb1-5d91-495b-9088-ae6e14b384a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:13.675244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fdeb1-5d91-495b-9088-ae6e14b384a0 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:13.675535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fdeb1-5d91-495b-9088-ae6e14b384a0 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:13.675822 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fdeb1-5d91-495b-9088-ae6e14b384a0 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:13.676593 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fdeb1-5d91-495b-9088-ae6e14b384a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:13.676903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fdeb1-5d91-495b-9088-ae6e14b384a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:13.678383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fdeb1-5d91-495b-9088-ae6e14b384a0 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:13.696584 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 607/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:13.698324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:13.774816 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:13.851123 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeaac21-2b45-4799-955d-bb55dbaa82fc None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:13.898855 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06f580-9817-4f0a-b581-4de91bb1766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:13.899500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06f580-9817-4f0a-b581-4de91bb1766b None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:13.899787 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06f580-9817-4f0a-b581-4de91bb1766b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:13.900118 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06f580-9817-4f0a-b581-4de91bb1766b None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:13.901452 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06f580-9817-4f0a-b581-4de91bb1766b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:13.902972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06f580-9817-4f0a-b581-4de91bb1766b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:13.937302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 598/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:17:13 2022] POST /identity/v3/users => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:17:16.248777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b828b-bee7-4bae-bfef-e6d598c79c8d None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:16.249686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b828b-bee7-4bae-bfef-e6d598c79c8d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:16.249686 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b828b-bee7-4bae-bfef-e6d598c79c8d None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:16.251100 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 608/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:16.260918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546ebf-20bb-4a28-97e6-0c43d2f1239e None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:16.260918 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546ebf-20bb-4a28-97e6-0c43d2f1239e None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:16.261734 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546ebf-20bb-4a28-97e6-0c43d2f1239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:16.283914 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-35546ebf-20bb-4a28-97e6-0c43d2f1239e None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:16.285374 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-35546ebf-20bb-4a28-97e6-0c43d2f1239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:16.381283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 599/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:16.406727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b828b-bee7-4bae-bfef-e6d598c79c8d None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:16.575311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b828b-bee7-4bae-bfef-e6d598c79c8d None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:16.626034 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dceee-2aca-4df9-bc2d-053e98f2f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:16.629897 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dceee-2aca-4df9-bc2d-053e98f2f69f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:16.630250 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dceee-2aca-4df9-bc2d-053e98f2f69f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:16.630490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dceee-2aca-4df9-bc2d-053e98f2f69f None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:16.644444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dceee-2aca-4df9-bc2d-053e98f2f69f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-831dceee-2aca-4df9-bc2d-053e98f2f69f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:17:16.650447 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:16.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:16.653280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 609/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:17:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:16.662223 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35546ebf-20bb-4a28-97e6-0c43d2f1239e None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:16.739680 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:16.836222 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35546ebf-20bb-4a28-97e6-0c43d2f1239e None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:16.882915 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:16.883624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:16.884069 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:16.884369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:16.885087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:16.885397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:16.886860 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:16.901594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 600/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:17:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:16.909536 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dceee-2aca-4df9-bc2d-053e98f2f69f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:16.984419 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:17.057608 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dceee-2aca-4df9-bc2d-053e98f2f69f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:17.100966 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:17.101654 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:17.102060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:17.102430 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:17.115303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:17.117001 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:17.133303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:17.134563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:17.135446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:17.135446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:17.135446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:17.135446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:17.135446 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 610/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:17.140851 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:17.272921 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:17.276611 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca026-4d1e-464d-aadf-4c470028bcd0 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:17.321655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:17.322461 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:17.322812 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:17.323195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:17.324078 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:17.324465 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:17.326035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:17.345397 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 601/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:17.352031 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:17.482792 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45694c0-16cf-48dc-a53a-7549bae920fc None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:17.485151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:17.536229 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:17.537167 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:17.537580 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:17.537942 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] PATH_INFO: `/v3/users/placement` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:17.548693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:17.550696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:17:17.569538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:17.571154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:17.572569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 611/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:17.582849 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:17.756164 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9158fc1-f5ce-4121-a03d-3557079ebcd7 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:17.805941 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:17.806809 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:17.807126 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:17.807408 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:17.808314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:17.808623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:17.810236 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:17.832706 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 602/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:17:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:17.839680 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:17.999060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9718989-de2a-4ba8-a9b8-0a65f556909a None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:18.045819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea106e4-b45c-464d-b0f7-69374fa7f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:18.046869 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea106e4-b45c-464d-b0f7-69374fa7f92f None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:18.047009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea106e4-b45c-464d-b0f7-69374fa7f92f None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:18.047261 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea106e4-b45c-464d-b0f7-69374fa7f92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:18.098955 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:18.164490 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea106e4-b45c-464d-b0f7-69374fa7f92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:18.167151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea106e4-b45c-464d-b0f7-69374fa7f92f None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:18.335348 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 612/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:17:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:17:18.337064 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:18.372080 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:18.491863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597e31d-9501-4064-aa9b-6d3d97d3f484 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:18.539774 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:18.540460 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:18.540736 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:18.541022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:18.556791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:18.558695 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:18.576034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:18.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:18.578330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:18.578330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:18.578330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:18.578330 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:18.579539 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 603/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:18.586244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea106e4-b45c-464d-b0f7-69374fa7f92f None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:18.680264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:18.730454 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea106e4-b45c-464d-b0f7-69374fa7f92f None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:18.770194 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:18.771198 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:18.771444 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:18.771701 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:18.772506 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:18.772796 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:18.774274 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:18.789759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 613/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:18.798242 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:18.895914 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:18.940700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ad1ea-1b8e-475a-a21f-2ee59f536949 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:19.001654 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:19.003161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:19.003632 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:19.004102 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:19.013901 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:19.015495 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:17:19.029075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:19.032532 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:19.033985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:17:19.033985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:17:19.033985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:17:19.033985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:17:19.033985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:17:19.033985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:17:19.033985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:19.033985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:19.036276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 604/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:19.037414 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:19.179908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a47574-29e8-463e-98e8-b12db43904af None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:19.211475 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:19.228572 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14231d-42a3-4868-a407-052091067d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:19.230141 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14231d-42a3-4868-a407-052091067d8d None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:19.230267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14231d-42a3-4868-a407-052091067d8d None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:19.230815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14231d-42a3-4868-a407-052091067d8d None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:19.232263 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14231d-42a3-4868-a407-052091067d8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:19.232757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14231d-42a3-4868-a407-052091067d8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:19.234843 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14231d-42a3-4868-a407-052091067d8d None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:19.249006 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 614/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:19.256126 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:19.385328 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510724ee-1c9c-4e21-ace7-ee6db9dd9dd4 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:19.428525 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:19.440333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e0df4-aca6-49fa-bbe6-433211bcad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:19.440333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e0df4-aca6-49fa-bbe6-433211bcad98 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:19.440333 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e0df4-aca6-49fa-bbe6-433211bcad98 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:19.440860 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e0df4-aca6-49fa-bbe6-433211bcad98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:19.442007 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e0df4-aca6-49fa-bbe6-433211bcad98 None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=89b69b21ab61476c86b189d941949cc1) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:19.442596 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e0df4-aca6-49fa-bbe6-433211bcad98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:19.445602 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e0df4-aca6-49fa-bbe6-433211bcad98 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:19.455048 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 605/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:17:19 2022] GET /identity/v3/role_assignments?user.id=89b69b21ab61476c86b189d941949cc1&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:21.672096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc68737-07b7-49f2-b263-b02f727cc6ab None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:21.672950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc68737-07b7-49f2-b263-b02f727cc6ab None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:21.672950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc68737-07b7-49f2-b263-b02f727cc6ab None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:21.674993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 615/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:17:21.680449 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4625fe-8a35-48c6-84ec-e4738a7a98cf None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:21.681283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4625fe-8a35-48c6-84ec-e4738a7a98cf None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:21.681283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4625fe-8a35-48c6-84ec-e4738a7a98cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:21.702828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-1d4625fe-8a35-48c6-84ec-e4738a7a98cf None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:21.704071 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-1d4625fe-8a35-48c6-84ec-e4738a7a98cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:21.798172 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 606/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:17:21.822441 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc68737-07b7-49f2-b263-b02f727cc6ab None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:21.961218 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc68737-07b7-49f2-b263-b02f727cc6ab None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:22.009622 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d9d20-404a-419d-81ad-4f0649781d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:22.010475 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d9d20-404a-419d-81ad-4f0649781d96 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:22.010609 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d9d20-404a-419d-81ad-4f0649781d96 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:22.010868 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d9d20-404a-419d-81ad-4f0649781d96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:22.023538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d9d20-404a-419d-81ad-4f0649781d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-8d0d9d20-404a-419d-81ad-4f0649781d96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:17:22.028740 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:22.030836 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:22.032755 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 616/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:17:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:22.039108 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4625fe-8a35-48c6-84ec-e4738a7a98cf None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:22.182437 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4625fe-8a35-48c6-84ec-e4738a7a98cf None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:22.221174 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:22.237329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:22.238344 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:22.238787 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:22.239215 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:22.240158 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:22.240594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:22.242313 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:22.252847 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 607/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:17:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:17:22.260811 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d9d20-404a-419d-81ad-4f0649781d96 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:22.310274 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:22.380937 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d9d20-404a-419d-81ad-4f0649781d96 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:22.419770 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:22.420605 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:22.420877 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:22.421010 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:22.434442 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:22.436322 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:22.458517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:22.460107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:22.461225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:22.461225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:22.461225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:22.461225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:22.461225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 617/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:22.469045 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:22.629122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8adff-6d6c-43e2-a9f6-b4b1854737c1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:22.678947 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:22.679788 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:22.680159 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:22.680537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:22.681659 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:22.682059 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:22.684197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:22.701451 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 608/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:17:22.707879 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:22.851324 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:22.866502 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59444aed-5ac1-40f4-af2b-e69875722977 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:22.924356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:22.924917 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:22.925195 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:22.925466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] PATH_INFO: `/v3/users/placement` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:22.935350 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:22.938424 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:17:22.958067 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:22.959297 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:22.966303 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 618/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:22.968267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:23.020560 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:23.091696 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebe124-53e6-4cb8-bb34-34452d97a80d None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:23.159354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:23.160679 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:23.160899 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:23.161473 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:23.163079 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:23.163573 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:23.166056 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:23.211965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 609/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:17:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:23.219677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:23.384800 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de40ad-19e7-481d-bb0b-a970c9770346 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:23.434270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:23.435451 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:23.435970 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:23.436443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:23.453214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:23.455244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:23.479357 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:23.481337 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:23.483234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:23.483234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:23.483234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:23.483234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:23.483234 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 619/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:23.491310 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:23.647953 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec464b55-c157-4756-bf5b-bf86c29b5b32 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:23.650879 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:23.701786 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:23.702842 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:23.703322 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:23.703781 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:23.705040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:23.705559 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:23.708267 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:23.734350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 610/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:23.739053 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:23.757308 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:23.894853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82f86e-82ad-41cb-b4e9-7a0c3e0da061 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:23.951161 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:23.952188 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:23.952497 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:23.952825 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:23.962827 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:23.964631 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:17:23.984119 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:23.986020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:23.988590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 620/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:23.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:24.122949 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0b0ae-c34a-4c04-ac38-4667a3b001a6 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:24.161261 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42305b02-42a6-4b70-8bd7-c224d087f132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:24.162029 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305b02-42a6-4b70-8bd7-c224d087f132 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:24.162365 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305b02-42a6-4b70-8bd7-c224d087f132 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:24.162665 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305b02-42a6-4b70-8bd7-c224d087f132 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:24.163500 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42305b02-42a6-4b70-8bd7-c224d087f132 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:24.163811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42305b02-42a6-4b70-8bd7-c224d087f132 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:24.165646 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42305b02-42a6-4b70-8bd7-c224d087f132 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:24.180464 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 611/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:17:24.187908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:24.313570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:24.360270 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18da1fd-f519-420a-bad0-b9b27a7e1f44 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:24.409291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa91f2-6366-4b25-86d8-96cb556f1217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:24.410068 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa91f2-6366-4b25-86d8-96cb556f1217 None admin] REQUEST_METHOD: `PUT` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:24.410473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa91f2-6366-4b25-86d8-96cb556f1217 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:24.410846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa91f2-6366-4b25-86d8-96cb556f1217 None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/89b69b21ab61476c86b189d941949cc1/roles/55d67b43cde54213983e27412c2a314c` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:24.429530 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa91f2-6366-4b25-86d8-96cb556f1217 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=89b69b21ab61476c86b189d941949cc1, role_id=55d67b43cde54213983e27412c2a314c)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:24.431426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa91f2-6366-4b25-86d8-96cb556f1217 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:24.448523 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 621/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:17:24 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/89b69b21ab61476c86b189d941949cc1/roles/55d67b43cde54213983e27412c2a314c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:17:26.998999 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56ac6-e509-44e1-8526-61a32b5d5a63 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:26.999552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56ac6-e509-44e1-8526-61a32b5d5a63 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:26.999552 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56ac6-e509-44e1-8526-61a32b5d5a63 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:27.000862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 612/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:27.007874 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ee70-1b5b-4f52-a2d1-864bac6f7e1b None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:27.008443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ee70-1b5b-4f52-a2d1-864bac6f7e1b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:27.008824 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ee70-1b5b-4f52-a2d1-864bac6f7e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:27.034020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-0362ee70-1b5b-4f52-a2d1-864bac6f7e1b None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:27.035507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-0362ee70-1b5b-4f52-a2d1-864bac6f7e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:27.125592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 622/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:27.149280 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b56ac6-e509-44e1-8526-61a32b5d5a63 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:27.179479 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:27.282714 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b56ac6-e509-44e1-8526-61a32b5d5a63 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:27.322651 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbc8ae-25b6-4125-8108-6a47be5e6735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:27.323329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbc8ae-25b6-4125-8108-6a47be5e6735 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:27.323707 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbc8ae-25b6-4125-8108-6a47be5e6735 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:27.323982 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbc8ae-25b6-4125-8108-6a47be5e6735 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:27.337585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbc8ae-25b6-4125-8108-6a47be5e6735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-98bbc8ae-25b6-4125-8108-6a47be5e6735 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:17:27.342896 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:27.344114 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:27.345155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 613/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:17:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:27.353819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362ee70-1b5b-4f52-a2d1-864bac6f7e1b None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:27.413311 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:27.482598 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362ee70-1b5b-4f52-a2d1-864bac6f7e1b None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:27.523152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:27.523794 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:27.524076 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:27.524354 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:27.525132 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] RBAC: Adding query filter params (name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:27.525405 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:27.526930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:27.537792 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 623/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:17:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:27.544510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbc8ae-25b6-4125-8108-6a47be5e6735 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:27.651912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:27.677613 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbc8ae-25b6-4125-8108-6a47be5e6735 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:27.724276 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:27.725480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:27.725959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:27.726508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:27.747241 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:27.751502 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:27.774508 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:27.775666 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:27.781739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:27.781739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:27.781739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:27.781739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:27.781739 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 614/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:27.782638 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:27.935169 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c9fb7-d484-4321-b76c-e8937ec6a3fa None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:27.949903 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:27.988610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:27.989271 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:27.989566 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:27.989822 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:27.990588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:27.990898 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:27.992328 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:28.009402 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 624/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:17:28.015733 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:28.150036 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420797da-6190-49cb-9a63-e318a79807a1 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:28.186711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:28.187456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:28.187842 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:28.188217 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] PATH_INFO: `/v3/users/placement` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:28.191650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:28.199789 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:28.201972 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:17:28.219068 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:28.220281 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:28.221334 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 615/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:28.227136 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:28.388461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448f10d-826f-4803-9c27-aa1e210a95bb None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:28.442408 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:28.443411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:28.443855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:28.444397 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:28.446082 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:28.446521 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:28.448517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:28.476441 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 625/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:17:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:28.483561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:28.624545 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4a7fe-8006-492f-889d-0659c0b3b269 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:28.682859 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc4da0-5942-42a0-8346-618e4540e91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:28.683506 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc4da0-5942-42a0-8346-618e4540e91e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:28.683788 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc4da0-5942-42a0-8346-618e4540e91e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:28.684066 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc4da0-5942-42a0-8346-618e4540e91e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:28.809012 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:28.822002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc4da0-5942-42a0-8346-618e4540e91e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:28.824636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc4da0-5942-42a0-8346-618e4540e91e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:28.974165 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 616/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:17:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:17:28.981029 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:29.015941 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:29.112097 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62577796-4312-4064-9142-6525ba6e8d21 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:29.159294 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:29.160005 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:29.160282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:29.160563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:29.172244 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:29.173758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:29.192116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:29.193153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:29.193993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:29.193993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:29.193993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:29.193993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:29.199041 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 626/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:29.200995 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc4da0-5942-42a0-8346-618e4540e91e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:29.312378 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:29.319507 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc4da0-5942-42a0-8346-618e4540e91e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:29.359674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:29.360324 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:29.360620 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:29.360885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:29.361605 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:29.361893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:29.363407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:29.378150 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 617/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:29.383238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:29.475304 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:29.506409 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eeb90-23ea-4e2f-8ac5-f8bf9c3f5850 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:29.548797 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:29.549423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:29.549705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:29.549998 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:29.558120 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:29.559795 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:17:29.573488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:29.574757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:29.575661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 627/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:29.581065 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:29.762390 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941b0a6-f0a5-4262-98f1-5b332a63e16b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:29.812104 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d690da5-18fc-4a7b-b72b-45d4203256b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:29.812753 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d690da5-18fc-4a7b-b72b-45d4203256b2 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:29.813070 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d690da5-18fc-4a7b-b72b-45d4203256b2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:29.813350 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d690da5-18fc-4a7b-b72b-45d4203256b2 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:29.814151 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d690da5-18fc-4a7b-b72b-45d4203256b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:29.814462 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d690da5-18fc-4a7b-b72b-45d4203256b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:29.816056 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d690da5-18fc-4a7b-b72b-45d4203256b2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:29.834709 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 618/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:17:29.841859 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:30.011570 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076859c-ab0f-4563-9eeb-d72d6a325571 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:30.053898 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7bf61-34ff-455e-8924-c27a30115ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:30.054627 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7bf61-34ff-455e-8924-c27a30115ebc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:30.054901 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7bf61-34ff-455e-8924-c27a30115ebc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:30.055120 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7bf61-34ff-455e-8924-c27a30115ebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:30.056649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7bf61-34ff-455e-8924-c27a30115ebc None admin] RBAC: Adding query filter params (role.id=55d67b43cde54213983e27412c2a314c, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=89b69b21ab61476c86b189d941949cc1) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:30.056649 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7bf61-34ff-455e-8924-c27a30115ebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:30.057908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7bf61-34ff-455e-8924-c27a30115ebc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:30.082153 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 628/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:17:29 2022] GET /identity/v3/role_assignments?user.id=89b69b21ab61476c86b189d941949cc1&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=55d67b43cde54213983e27412c2a314c => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:32.306293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbaa49-125d-4774-a899-a85bafb63925 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:32.306893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbaa49-125d-4774-a899-a85bafb63925 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:32.306893 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbaa49-125d-4774-a899-a85bafb63925 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:32.308688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 619/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:32.315156 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8a67e-a35e-4dc3-b264-3331158ee7a0 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:32.315742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8a67e-a35e-4dc3-b264-3331158ee7a0 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:32.315742 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8a67e-a35e-4dc3-b264-3331158ee7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:32.325289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:32.340910 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-1ba8a67e-a35e-4dc3-b264-3331158ee7a0 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:32.342314 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-1ba8a67e-a35e-4dc3-b264-3331158ee7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:32.426934 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 629/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:32.453722 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbaa49-125d-4774-a899-a85bafb63925 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:32.474766 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:32.596216 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbaa49-125d-4774-a899-a85bafb63925 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:32.660385 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42254-2e62-4fc0-a067-4aabfc976c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:32.661079 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42254-2e62-4fc0-a067-4aabfc976c02 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:32.661380 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42254-2e62-4fc0-a067-4aabfc976c02 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:32.661736 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42254-2e62-4fc0-a067-4aabfc976c02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:32.677848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad42254-2e62-4fc0-a067-4aabfc976c02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-1ad42254-2e62-4fc0-a067-4aabfc976c02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:17:32.683503 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:32.685198 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:32.686529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 620/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:32.692074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8a67e-a35e-4dc3-b264-3331158ee7a0 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:32.835784 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8a67e-a35e-4dc3-b264-3331158ee7a0 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:32.872456 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:32.879245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:32.879961 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:32.880253 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:32.880611 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:32.881461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:32.881870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:32.884114 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:32.895926 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 630/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:17:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:32.902757 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42254-2e62-4fc0-a067-4aabfc976c02 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:33.008314 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:33.041730 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42254-2e62-4fc0-a067-4aabfc976c02 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:33.087852 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:33.088547 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:33.088832 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:33.089155 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:33.105649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:33.107313 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:33.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:33.127393 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:33.128372 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:33.128372 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:33.128372 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:33.128372 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:33.128372 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 621/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:33.133738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:33.299883 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f927c-ade7-4edc-911f-842d921486e4 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:33.346045 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4059955-9697-4423-9378-953478096638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:33.347018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4059955-9697-4423-9378-953478096638 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:33.347018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4059955-9697-4423-9378-953478096638 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:33.347240 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4059955-9697-4423-9378-953478096638 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:33.347938 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4059955-9697-4423-9378-953478096638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:33.348238 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4059955-9697-4423-9378-953478096638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:33.349640 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4059955-9697-4423-9378-953478096638 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:33.364026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 631/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:33.372116 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:33.509152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e1fb4-3c54-4f1f-a084-881e4ef2c0fa None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:33.542474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:33.556004 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:33.556674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:33.556959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:33.557303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] PATH_INFO: `/v3/users/placement` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:33.566213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:33.567688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:17:33.580799 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:33.582858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:33.584867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 622/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:33.590599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4059955-9697-4423-9378-953478096638 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:33.656074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:33.719009 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4059955-9697-4423-9378-953478096638 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:33.762870 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:33.763458 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:33.763777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:33.764127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:33.765088 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:33.765538 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:33.767113 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:33.789169 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 632/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:17:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:17:33.795808 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:33.926528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e53c8-6e6a-4a80-967e-efe506ae8d78 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:33.970837 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffbc1b-ff4d-4b88-8ab1-7e4a8404c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:33.971480 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffbc1b-ff4d-4b88-8ab1-7e4a8404c356 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:33.971765 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffbc1b-ff4d-4b88-8ab1-7e4a8404c356 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:33.972074 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffbc1b-ff4d-4b88-8ab1-7e4a8404c356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:34.041647 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:34.056107 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffbc1b-ff4d-4b88-8ab1-7e4a8404c356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:34.057720 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffbc1b-ff4d-4b88-8ab1-7e4a8404c356 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:34.192648 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 623/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:17:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:17:34.199408 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:34.347747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e6b7e-2294-4983-bab1-94114a44612b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:34.367746 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:34.396736 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:34.397665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:34.398084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:34.398661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:34.412767 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:34.414485 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:34.432774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:34.434754 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:34.436463 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:34.436463 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:34.436463 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:34.436463 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:34.436463 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 633/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:34.440551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffbc1b-ff4d-4b88-8ab1-7e4a8404c356 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:34.578546 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:34.600380 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffbc1b-ff4d-4b88-8ab1-7e4a8404c356 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:34.658898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:34.659904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:34.660352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:34.660819 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:34.662209 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:34.662733 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:34.664718 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:34.690154 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 624/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:34.691807 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:34.838526 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961f463-fb9e-40a3-93ae-7fe804f12a03 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:34.887643 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:34.889179 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:34.889594 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:34.890075 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:34.901211 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:34.902745 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:17:34.918486 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:34.919738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:34.920655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 634/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:34.927438 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:35.054318 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1fd7b-cca7-4d27-82b4-7e1f5002521a None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:35.099674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a105cef-4469-426c-9065-a5b4ab5c6d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:35.100316 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a105cef-4469-426c-9065-a5b4ab5c6d08 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:35.100592 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a105cef-4469-426c-9065-a5b4ab5c6d08 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:35.100863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a105cef-4469-426c-9065-a5b4ab5c6d08 None admin] PATH_INFO: `/v3/projects` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:35.101626 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a105cef-4469-426c-9065-a5b4ab5c6d08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:35.101963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a105cef-4469-426c-9065-a5b4ab5c6d08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:35.103443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a105cef-4469-426c-9065-a5b4ab5c6d08 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:35.122885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 625/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:35.130507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:35.168469 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:35.268086 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cce921-206a-4e25-bef3-812a0e6e4013 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:35.312800 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3b79f-18e0-4599-970a-2e0832ec439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:35.313618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3b79f-18e0-4599-970a-2e0832ec439a None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:35.313935 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3b79f-18e0-4599-970a-2e0832ec439a None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:35.314391 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3b79f-18e0-4599-970a-2e0832ec439a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:35.315743 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3b79f-18e0-4599-970a-2e0832ec439a None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=89b69b21ab61476c86b189d941949cc1) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:35.316342 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3b79f-18e0-4599-970a-2e0832ec439a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:35.318191 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3b79f-18e0-4599-970a-2e0832ec439a None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:35.326682 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 635/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:17:35 2022] GET /identity/v3/role_assignments?user.id=89b69b21ab61476c86b189d941949cc1&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=40504e80dbb845aeb07ca258977ecacb => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:17:37.536757 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e7a2-ada9-4d28-aa7a-c4668d23f6b8 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:37.537561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e7a2-ada9-4d28-aa7a-c4668d23f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:37.537561 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e7a2-ada9-4d28-aa7a-c4668d23f6b8 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:37.539650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 626/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:37.545962 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096cf69-898b-4a24-ac4b-23d812e703c7 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:37.546830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096cf69-898b-4a24-ac4b-23d812e703c7 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:37.546830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096cf69-898b-4a24-ac4b-23d812e703c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:37.566397 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-0096cf69-898b-4a24-ac4b-23d812e703c7 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:37.567637 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-0096cf69-898b-4a24-ac4b-23d812e703c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:37.666196 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 636/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:37.699711 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8e7a2-ada9-4d28-aa7a-c4668d23f6b8 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:37.713526 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:37.847697 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8e7a2-ada9-4d28-aa7a-c4668d23f6b8 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:37.895931 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cdfff-6324-4140-9a7e-071e2bdab78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:37.896645 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cdfff-6324-4140-9a7e-071e2bdab78e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:37.896974 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cdfff-6324-4140-9a7e-071e2bdab78e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:37.897473 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cdfff-6324-4140-9a7e-071e2bdab78e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:37.911469 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cdfff-6324-4140-9a7e-071e2bdab78e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-b18cdfff-6324-4140-9a7e-071e2bdab78e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:17:37.917591 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:37.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:37.919792 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 627/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:37.925735 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096cf69-898b-4a24-ac4b-23d812e703c7 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:37.950964 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:38.058776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096cf69-898b-4a24-ac4b-23d812e703c7 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:38.104251 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:38.104774 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:38.105036 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:38.105322 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] PATH_INFO: `/v3/roles` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:38.106074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:38.106488 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:38.107965 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:38.118214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 637/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:38.125041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cdfff-6324-4140-9a7e-071e2bdab78e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:38.207166 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:38.249161 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cdfff-6324-4140-9a7e-071e2bdab78e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:38.288551 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:38.289298 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:38.289575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:38.289853 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:38.303507 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:38.305460 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:38.324293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:38.325384 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:38.326456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:38.326456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:38.326456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:38.326456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:38.326456 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 628/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:38.333134 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:38.454914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:38.480320 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d21fe-f2a7-4457-881e-a25b311f56d2 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:38.529282 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:38.529971 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:38.530292 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:38.530572 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:38.531563 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:38.531989 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:38.533793 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:38.552723 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 638/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:38.560163 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:38.687005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9443db-2906-4874-9f18-f02ac1a80823 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:38.712695 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:38.736691 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:38.737840 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:38.738398 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:38.738919 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] PATH_INFO: `/v3/users/placement` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:38.752917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:38.755197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:17:38.777687 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:38.779195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:38.780297 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 629/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:38.785737 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:38.922022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f78043-0137-4a74-b120-64a7641a5873 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:38.967675 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:38.968245 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:38.968597 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:38.968974 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] PATH_INFO: `/v3/users` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:38.970074 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:38.970574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:38.972186 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:38.984331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:39.000150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 639/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:17:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:39.006885 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:39.152873 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f841b7-b16b-4714-87b9-a789e2f38c21 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:39.207777 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:39.208585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:39.208956 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:39.209392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:39.222925 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:39.224331 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:39.240811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:39.242188 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:39.243041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:39.243041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:39.243041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:39.243041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:39.243041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 630/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:39.248676 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:39.376574 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acee9f-8efd-4df7-a533-ea8133942e30 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:39.420492 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:39.421516 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:39.422000 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:39.422504 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:39.423712 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:39.424222 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:39.426666 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:39.446478 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 640/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:39.453806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:39.497005 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:39.601202 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a0891-6ca8-445c-bf71-6999bd1ffb86 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:39.641522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:39.642512 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:39.642948 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:39.643368 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:39.654513 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:39.656636 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:17:39.676244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:39.677153 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:39.678030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:17:39.678030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:17:39.678030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:17:39.678030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:17:39.678030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:17:39.678030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:17:39.678030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:39.678030 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:39.678670 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 631/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:17:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:39.684696 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:39.766413 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:39.849674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a18efc-9580-49c5-90f0-67e9fc94f18e None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:39.889852 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c5ed8-4004-4776-92b1-862618990f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:39.890533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5ed8-4004-4776-92b1-862618990f58 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:39.890820 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5ed8-4004-4776-92b1-862618990f58 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:39.891096 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5ed8-4004-4776-92b1-862618990f58 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:39.891881 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c5ed8-4004-4776-92b1-862618990f58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:39.892184 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c5ed8-4004-4776-92b1-862618990f58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:39.894053 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c5ed8-4004-4776-92b1-862618990f58 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:39.912776 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 641/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:17:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:39.919200 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:40.023247 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:40.033497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb0661-bfd4-415a-be5c-a5435ac41be8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:40.071103 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee076d-f0ba-4239-80bf-7b5652333fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:40.071797 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee076d-f0ba-4239-80bf-7b5652333fc2 None admin] REQUEST_METHOD: `PUT` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:40.072084 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee076d-f0ba-4239-80bf-7b5652333fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:40.072362 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee076d-f0ba-4239-80bf-7b5652333fc2 None admin] PATH_INFO: `/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/89b69b21ab61476c86b189d941949cc1/roles/40504e80dbb845aeb07ca258977ecacb` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:40.074455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee076d-f0ba-4239-80bf-7b5652333fc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec7a252ce6472d9f5c1bacc1d725c3, user_id=89b69b21ab61476c86b189d941949cc1, role_id=40504e80dbb845aeb07ca258977ecacb)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:40.075955 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee076d-f0ba-4239-80bf-7b5652333fc2 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:40.091832 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 632/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:17:39 2022] PUT /identity/v3/projects/a8ec7a252ce6472d9f5c1bacc1d725c3/users/89b69b21ab61476c86b189d941949cc1/roles/40504e80dbb845aeb07ca258977ecacb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:17:42.165853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304481c-3325-44f6-99c7-69f8324b7002 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:42.166480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304481c-3325-44f6-99c7-69f8324b7002 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:42.166480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304481c-3325-44f6-99c7-69f8324b7002 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:42.167827 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 642/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:42.173614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9fc84-59ec-4d27-aa66-60d4e2926aa2 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:42.174082 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9fc84-59ec-4d27-aa66-60d4e2926aa2 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:42.174283 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9fc84-59ec-4d27-aa66-60d4e2926aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:42.193825 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-8aa9fc84-59ec-4d27-aa66-60d4e2926aa2 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:42.195448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-8aa9fc84-59ec-4d27-aa66-60d4e2926aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:42.273213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 633/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:42.297373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304481c-3325-44f6-99c7-69f8324b7002 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:42.418065 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304481c-3325-44f6-99c7-69f8324b7002 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:42.439991 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:42.459377 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5f05a-55f9-4260-b2b8-309b2f35fe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:42.460055 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5f05a-55f9-4260-b2b8-309b2f35fe36 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:42.460338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5f05a-55f9-4260-b2b8-309b2f35fe36 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:42.460612 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5f05a-55f9-4260-b2b8-309b2f35fe36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:42.471944 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5f05a-55f9-4260-b2b8-309b2f35fe36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-eea5f05a-55f9-4260-b2b8-309b2f35fe36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:17:42.476973 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:42.478618 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:42.480425 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 643/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:17:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:42.485998 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9fc84-59ec-4d27-aa66-60d4e2926aa2 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:42.573842 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:42.593440 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9fc84-59ec-4d27-aa66-60d4e2926aa2 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:42.630293 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:42.630927 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:42.631211 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:42.631487 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] PATH_INFO: `/v3/roles` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:42.632221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] RBAC: Adding query filter params (name=admin) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:42.632529 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:42.633923 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:42.642991 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 634/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:17:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:42.649194 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5f05a-55f9-4260-b2b8-309b2f35fe36 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:42.782240 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5f05a-55f9-4260-b2b8-309b2f35fe36 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:42.834739 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:42.835427 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:42.835708 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:42.835978 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:42.848871 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:42.850426 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:42.868914 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:42.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:42.871533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:42.871533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:42.871533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:42.871533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:42.871533 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 644/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:42.879059 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:43.002676 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510ed0d-0162-4b0b-97e1-c7140126353b None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:43.042801 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:43.043449 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:43.044233 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:43.044510 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] PATH_INFO: `/v3/domains` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:43.045245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:43.045559 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:43.047040 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:43.061116 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 635/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:43.067289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:43.106321 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:43.182256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9356325-7886-4389-a4cb-2adb9c1b2bc5 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:43.218546 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:43.219207 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:43.219487 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:43.219759 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] PATH_INFO: `/v3/users/placement` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:43.229031 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:43.230626 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:17:43.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:43.243763 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:43.244705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 645/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:43.251028 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:43.256129 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:43.362394 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70180757-8353-47b9-a523-b0a00f79af5e None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:43.398967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:43.399619 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:43.399904 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:43.400156 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] PATH_INFO: `/v3/users` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:43.400989 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:43.401302 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:43.402827 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:43.421497 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 636/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:17:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:43.427805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:43.528744 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:43.540019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b38a9e-7060-4c2d-a0ab-750ef948180b None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:43.579060 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ce1fa-8122-41fc-a77d-ea34bd4a6f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:43.579750 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ce1fa-8122-41fc-a77d-ea34bd4a6f46 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:43.580035 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ce1fa-8122-41fc-a77d-ea34bd4a6f46 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:43.580369 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ce1fa-8122-41fc-a77d-ea34bd4a6f46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:43.660661 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ce1fa-8122-41fc-a77d-ea34bd4a6f46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:43.662184 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ce1fa-8122-41fc-a77d-ea34bd4a6f46 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:43.785470 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:43.798383 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 646/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:17:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:17:43.804669 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:43.869719 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:43.911922 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f3337-799d-4bd5-b632-768e6b5e4ba8 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:43.949528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:43.950331 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:43.950662 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:43.950982 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:43.969203 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:43.972143 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:17:44.001420 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:44.002593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:44.003624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:17:44.003624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:17:44.003624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:17:44.003624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:44.003624 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 637/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:44.009819 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ce1fa-8122-41fc-a77d-ea34bd4a6f46 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:44.123408 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ce1fa-8122-41fc-a77d-ea34bd4a6f46 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:44.160606 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:44.161373 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:44.161757 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:44.162155 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] PATH_INFO: `/v3/domains` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:44.162950 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:44.163349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:44.164846 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:44.177078 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 647/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:44.183238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:44.289988 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c3e5d-a1fd-4275-bfc0-83bcfe65c9cd None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:44.304160 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:44.326443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:44.328369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:44.328369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:44.328369 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:44.334690 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:44.336175 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:17:44.348700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:44.349735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: ERROR keystone.server.flask.application Jul 28 08:17:44.350957 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 638/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:17:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:17:44.356230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:44.434573 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:44.486667 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8529e-09de-4b72-ad8c-31b56b433ffc None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:44.525356 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de84f73-8caf-4b84-8308-3d01811dfac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:44.525975 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84f73-8caf-4b84-8308-3d01811dfac2 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:44.526287 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84f73-8caf-4b84-8308-3d01811dfac2 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:44.526564 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84f73-8caf-4b84-8308-3d01811dfac2 None admin] PATH_INFO: `/v3/projects` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:44.527310 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de84f73-8caf-4b84-8308-3d01811dfac2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:44.527617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de84f73-8caf-4b84-8308-3d01811dfac2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:44.529107 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de84f73-8caf-4b84-8308-3d01811dfac2 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:44.541154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 648/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:17:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:44.547594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:44.650785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243590ed-70db-46d0-8ba8-45e3b5f29738 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:44.685776 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7690ca0-df03-4096-819a-878499593195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:44.686361 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7690ca0-df03-4096-819a-878499593195 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:44.686640 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7690ca0-df03-4096-819a-878499593195 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:44.686907 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7690ca0-df03-4096-819a-878499593195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:44.687696 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7690ca0-df03-4096-819a-878499593195 None admin] RBAC: Adding query filter params (role.id=40504e80dbb845aeb07ca258977ecacb, scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3, user.id=89b69b21ab61476c86b189d941949cc1) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:44.688000 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7690ca0-df03-4096-819a-878499593195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:44.689444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7690ca0-df03-4096-819a-878499593195 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:44.695649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 639/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:17:44 2022] GET /identity/v3/role_assignments?user.id=89b69b21ab61476c86b189d941949cc1&scope.project.id=a8ec7a252ce6472d9f5c1bacc1d725c3&role.id=40504e80dbb845aeb07ca258977ecacb => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:46.684888 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0ec4-855d-4efe-afef-7fed99149ac1 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:46.685751 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0ec4-855d-4efe-afef-7fed99149ac1 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:46.685751 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0ec4-855d-4efe-afef-7fed99149ac1 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:46.688013 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 649/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:46.697152 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf926c-79f1-4d65-9882-69f4501fad39 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:46.697880 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf926c-79f1-4d65-9882-69f4501fad39 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:46.698038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf926c-79f1-4d65-9882-69f4501fad39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:46.723844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-0caf926c-79f1-4d65-9882-69f4501fad39 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:46.725075 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-0caf926c-79f1-4d65-9882-69f4501fad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:46.758890 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:46.805838 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 640/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:46.829805 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb0ec4-855d-4efe-afef-7fed99149ac1 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:46.944986 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb0ec4-855d-4efe-afef-7fed99149ac1 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:46.964037 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:47.006151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:47.007075 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:47.007491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:47.007890 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] PATH_INFO: `/v3/services/placement` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:47.008916 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:47.011183 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:47.021507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:47.022693 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:47.023694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 650/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:17:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:47.031462 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf926c-79f1-4d65-9882-69f4501fad39 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:47.138575 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf926c-79f1-4d65-9882-69f4501fad39 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:47.173751 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf7aaf-9cee-4de2-aca0-a4be284fe565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:47.174400 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7aaf-9cee-4de2-aca0-a4be284fe565 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:47.174675 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7aaf-9cee-4de2-aca0-a4be284fe565 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:47.174944 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7aaf-9cee-4de2-aca0-a4be284fe565 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:47.175618 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf7aaf-9cee-4de2-aca0-a4be284fe565 None admin] RBAC: Adding query filter params (name=placement) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:47.175916 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf7aaf-9cee-4de2-aca0-a4be284fe565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:47.177306 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf7aaf-9cee-4de2-aca0-a4be284fe565 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:47.182588 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 641/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:17:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:47.189484 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:47.301986 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1572401-71ed-49be-a8e8-3f59db90ab50 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:47.338727 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b8eec-cb19-4a1d-b9a4-1dbbe06544c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:47.339349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b8eec-cb19-4a1d-b9a4-1dbbe06544c9 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:47.339632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b8eec-cb19-4a1d-b9a4-1dbbe06544c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:47.339908 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b8eec-cb19-4a1d-b9a4-1dbbe06544c9 None admin] PATH_INFO: `/v3/services` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:47.340617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b8eec-cb19-4a1d-b9a4-1dbbe06544c9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:47.340928 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b8eec-cb19-4a1d-b9a4-1dbbe06544c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:47.342340 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b8eec-cb19-4a1d-b9a4-1dbbe06544c9 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:47.347190 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 651/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:17:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:49.319842 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbfd60-3388-4e07-81fa-03c8000b3bac None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:49.320865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbfd60-3388-4e07-81fa-03c8000b3bac None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:49.320865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbfd60-3388-4e07-81fa-03c8000b3bac None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:49.322138 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 642/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:49.327602 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be842ff5-524f-45c8-9b27-9ca3ba562fcc None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:49.328443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be842ff5-524f-45c8-9b27-9ca3ba562fcc None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:49.328443 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be842ff5-524f-45c8-9b27-9ca3ba562fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:49.347328 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-be842ff5-524f-45c8-9b27-9ca3ba562fcc None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:49.348560 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-be842ff5-524f-45c8-9b27-9ca3ba562fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:49.380200 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:49.420411 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 652/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:49.443011 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbfd60-3388-4e07-81fa-03c8000b3bac None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:49.485977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:49.559929 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbfd60-3388-4e07-81fa-03c8000b3bac None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:49.596841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea337df-e756-4b42-9458-49529733397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:49.597354 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea337df-e756-4b42-9458-49529733397c None admin] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:49.597637 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea337df-e756-4b42-9458-49529733397c None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:49.597914 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea337df-e756-4b42-9458-49529733397c None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:49.598656 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea337df-e756-4b42-9458-49529733397c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:49.600077 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea337df-e756-4b42-9458-49529733397c None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:49.617125 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 643/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:17:49 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:17:51.612289 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2b4d5-2e57-499d-a52e-ef7b3567ab53 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:51.612873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2b4d5-2e57-499d-a52e-ef7b3567ab53 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:51.612873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2b4d5-2e57-499d-a52e-ef7b3567ab53 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:51.614833 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 653/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:51.624418 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d8bf-0c39-46e3-a0c8-f3b675a4e0fa None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:51.625244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d8bf-0c39-46e3-a0c8-f3b675a4e0fa None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:51.625411 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d8bf-0c39-46e3-a0c8-f3b675a4e0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:51.649932 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-ec73d8bf-0c39-46e3-a0c8-f3b675a4e0fa None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:51.651195 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-ec73d8bf-0c39-46e3-a0c8-f3b675a4e0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:51.729935 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 644/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:17:51.754702 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2b4d5-2e57-499d-a52e-ef7b3567ab53 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:51.865152 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2b4d5-2e57-499d-a52e-ef7b3567ab53 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:51.901480 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:51.902148 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:51.902438 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:51.902787 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] PATH_INFO: `/v3/services/placement` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:51.903434 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:51.904821 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:51.914782 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:51.916106 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:51.917239 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 654/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:17:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:51.923022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73d8bf-0c39-46e3-a0c8-f3b675a4e0fa None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:51.942270 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:52.033537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73d8bf-0c39-46e3-a0c8-f3b675a4e0fa None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:52.069305 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:52.069931 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:52.070589 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:52.070860 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:52.071545 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] RBAC: Adding query filter params (name=placement) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:52.071844 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:52.073240 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:52.078762 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 645/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:17:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:52.085044 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:52.119329 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:52.194815 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3d87-dfad-4406-9bba-de5835876849 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:52.234305 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2761e79-f32a-42bd-b348-29dd2be7dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:52.234913 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2761e79-f32a-42bd-b348-29dd2be7dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:52.235194 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2761e79-f32a-42bd-b348-29dd2be7dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:52.235471 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2761e79-f32a-42bd-b348-29dd2be7dbe1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:52.236267 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2761e79-f32a-42bd-b348-29dd2be7dbe1 None admin] RBAC: Adding query filter params (interface=public, service_id=d6e31bb218634c3e930993e3470a7d5b, region_id=RegionOne) {{(pid=144042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:52.236576 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2761e79-f32a-42bd-b348-29dd2be7dbe1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:52.237952 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2761e79-f32a-42bd-b348-29dd2be7dbe1 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:52.244442 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 655/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:17:52 2022] GET /identity/v3/endpoints?service_id=d6e31bb218634c3e930993e3470a7d5b&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:52.250959 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:52.341719 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:52.362550 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54d603-9ba2-4d61-98ab-f85318979587 None admin] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:52.398806 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dcece-5472-4478-8258-6f1f7b4dabba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:52.399440 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dcece-5472-4478-8258-6f1f7b4dabba None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:52.399727 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dcece-5472-4478-8258-6f1f7b4dabba None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:52.400003 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dcece-5472-4478-8258-6f1f7b4dabba None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:52.400655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dcece-5472-4478-8258-6f1f7b4dabba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:52.402061 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dcece-5472-4478-8258-6f1f7b4dabba None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:52.409275 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 646/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:17:52 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:17:54.460423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146a7f1-5e73-41ed-9437-700722f93bf6 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:54.461129 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146a7f1-5e73-41ed-9437-700722f93bf6 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:54.461129 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146a7f1-5e73-41ed-9437-700722f93bf6 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:54.462437 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 656/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:17:54.468912 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cd5d5-72be-4322-bdb5-0f2546abff79 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:54.469540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cd5d5-72be-4322-bdb5-0f2546abff79 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:54.469540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cd5d5-72be-4322-bdb5-0f2546abff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:54.490516 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-448cd5d5-72be-4322-bdb5-0f2546abff79 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:17:54.491776 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-448cd5d5-72be-4322-bdb5-0f2546abff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:17:54.579593 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 647/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:17:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:17:54.603214 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146a7f1-5e73-41ed-9437-700722f93bf6 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:54.709477 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:54.718080 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146a7f1-5e73-41ed-9437-700722f93bf6 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:54.757786 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:54.758500 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:54.758752 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:54.759029 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] PATH_INFO: `/v3/services/placement` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:54.759662 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:54.761093 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:17:54.771185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:17:54.772439 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application with Lock( Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: ERROR keystone.server.flask.application Jul 28 08:17:54.773375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 657/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:17:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:17:54.780142 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cd5d5-72be-4322-bdb5-0f2546abff79 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:54.872331 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:17:54.897769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cd5d5-72be-4322-bdb5-0f2546abff79 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:54.940859 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9034f23-0ab0-4d78-8a91-8981865f7c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:54.941483 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034f23-0ab0-4d78-8a91-8981865f7c78 None admin] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:54.941764 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034f23-0ab0-4d78-8a91-8981865f7c78 None admin] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:54.942041 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034f23-0ab0-4d78-8a91-8981865f7c78 None admin] PATH_INFO: `/v3/services` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:54.942830 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9034f23-0ab0-4d78-8a91-8981865f7c78 None admin] RBAC: Adding query filter params (name=placement) {{(pid=144043) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:17:54.943135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9034f23-0ab0-4d78-8a91-8981865f7c78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:54.944537 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9034f23-0ab0-4d78-8a91-8981865f7c78 None admin] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:54.950245 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 648/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:17:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:17:54.956610 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:17:55.078022 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76def0-9a62-44e6-be06-28b89e50f959 None admin] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:17:55.119331 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953b312-f5c5-432a-a743-2a69bbc68649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab47d79a05944b2eb59466f830650026', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:17:55.119996 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b312-f5c5-432a-a743-2a69bbc68649 None admin] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:17:55.120283 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b312-f5c5-432a-a743-2a69bbc68649 None admin] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:17:55.120560 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b312-f5c5-432a-a743-2a69bbc68649 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:17:55.121291 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953b312-f5c5-432a-a743-2a69bbc68649 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:17:55.122790 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953b312-f5c5-432a-a743-2a69bbc68649 None admin] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:17:55.147089 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 658/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:17:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:18:04.947674 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=30 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:18:05.140993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:00.883491 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a98304-7a61-4491-bac8-14ec8b1cd160 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:00.884109 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a98304-7a61-4491-bac8-14ec8b1cd160 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:00.884109 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a98304-7a61-4491-bac8-14ec8b1cd160 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:00.885455 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 649/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:00.890924 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e5cd2-e0e3-46ea-a24d-2fae730e364b None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:00.891399 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e5cd2-e0e3-46ea-a24d-2fae730e364b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:00.891494 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e5cd2-e0e3-46ea-a24d-2fae730e364b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:01.065387 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-180e5cd2-e0e3-46ea-a24d-2fae730e364b None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:01.066655 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-180e5cd2-e0e3-46ea-a24d-2fae730e364b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:01.305143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 659/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:01.373477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0534-dacd-4d15-aa26-28e4267db4ca None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:01.373761 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0534-dacd-4d15-aa26-28e4267db4ca None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:01.374039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0534-dacd-4d15-aa26-28e4267db4ca None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:01.375702 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 650/1309] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 08:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:01.384090 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f7559-5ab3-48c1-96e7-e6aca48eae88 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:01.384595 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f7559-5ab3-48c1-96e7-e6aca48eae88 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:01.385026 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f7559-5ab3-48c1-96e7-e6aca48eae88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:01.542932 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-520f7559-5ab3-48c1-96e7-e6aca48eae88 None None] MFA Rules not processed for user `05afd25e30764af0819436cc91dd85c5`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:01.549777 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-520f7559-5ab3-48c1-96e7-e6aca48eae88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:01.573887 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:01.640705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 660/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 08:19:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:19:01.650307 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc0534-dacd-4d15-aa26-28e4267db4ca None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:01.652506 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f7dc0534-dacd-4d15-aa26-28e4267db4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:01.873821 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc0534-dacd-4d15-aa26-28e4267db4ca None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:01.987983 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bbea8-14e7-467a-92f9-890436cae7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:01.988650 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bbea8-14e7-467a-92f9-890436cae7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:01.988990 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bbea8-14e7-467a-92f9-890436cae7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:01.989190 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bbea8-14e7-467a-92f9-890436cae7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:01.991038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-0f1bbea8-14e7-467a-92f9-890436cae7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:02.055406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:02.168811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bbea8-14e7-467a-92f9-890436cae7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:02.170852 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bbea8-14e7-467a-92f9-890436cae7e6 service neutron] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:02.193514 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 651/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:04.680873 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937cc73-e2df-4669-af34-5ebd298f0534 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:04.681274 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937cc73-e2df-4669-af34-5ebd298f0534 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:04.681461 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937cc73-e2df-4669-af34-5ebd298f0534 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:04.686824 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 661/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:04.688239 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209de5-df3e-456c-9749-56b15f476578 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:04.688684 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209de5-df3e-456c-9749-56b15f476578 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:04.688785 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209de5-df3e-456c-9749-56b15f476578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:04.772858 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-7e209de5-df3e-456c-9749-56b15f476578 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:04.774238 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-7e209de5-df3e-456c-9749-56b15f476578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:04.877462 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 652/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:19:04.916384 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937cc73-e2df-4669-af34-5ebd298f0534 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:04.918246 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-8937cc73-e2df-4669-af34-5ebd298f0534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:05.038364 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937cc73-e2df-4669-af34-5ebd298f0534 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:05.116459 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:05.119930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66a7f-ad7b-4b4e-af9b-aae2613a6ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:05.120979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66a7f-ad7b-4b4e-af9b-aae2613a6ffe service neutron] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:05.121738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66a7f-ad7b-4b4e-af9b-aae2613a6ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:05.122266 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66a7f-ad7b-4b4e-af9b-aae2613a6ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:05.124695 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-92b66a7f-ad7b-4b4e-af9b-aae2613a6ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:05.268143 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66a7f-ad7b-4b4e-af9b-aae2613a6ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:05.269641 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66a7f-ad7b-4b4e-af9b-aae2613a6ffe service neutron] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:05.295320 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 662/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:07.653027 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8071cb-ba5d-4ee2-a89e-8be969efd3de None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:07.653589 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8071cb-ba5d-4ee2-a89e-8be969efd3de None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:07.653589 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8071cb-ba5d-4ee2-a89e-8be969efd3de None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:07.655035 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 653/1315] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:07.660494 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbc851-ea33-4ed8-9795-4fe84ba8bd9d None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:07.660953 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbc851-ea33-4ed8-9795-4fe84ba8bd9d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:07.661044 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbc851-ea33-4ed8-9795-4fe84ba8bd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:07.729515 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-ccdbc851-ea33-4ed8-9795-4fe84ba8bd9d None None] MFA Rules not processed for user `cee2a726271640868cff7cbf256bac2d`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:07.730765 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-ccdbc851-ea33-4ed8-9795-4fe84ba8bd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:07.788130 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:07.838694 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 663/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:07.879515 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8071cb-ba5d-4ee2-a89e-8be969efd3de None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:07.881085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-1c8071cb-ba5d-4ee2-a89e-8be969efd3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:07.891254 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:07.990383 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8071cb-ba5d-4ee2-a89e-8be969efd3de None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:08.065025 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ddb25-df66-4a0b-85e0-afecfc05b30c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:08.065662 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ddb25-df66-4a0b-85e0-afecfc05b30c service neutron] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:08.065938 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ddb25-df66-4a0b-85e0-afecfc05b30c service neutron] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:08.066235 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ddb25-df66-4a0b-85e0-afecfc05b30c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:08.067898 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-227ddb25-df66-4a0b-85e0-afecfc05b30c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:08.145340 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:08.195352 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ddb25-df66-4a0b-85e0-afecfc05b30c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:08.196811 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ddb25-df66-4a0b-85e0-afecfc05b30c service neutron] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:08.217849 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 654/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:10.838029 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f79b6e-8027-4772-97d0-786496af3f7e None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:10.838738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f79b6e-8027-4772-97d0-786496af3f7e None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:10.838738 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f79b6e-8027-4772-97d0-786496af3f7e None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:10.840075 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 664/1318] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:10.845700 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0743a2-d93e-4f46-92c6-1f0621427f43 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:10.846183 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0743a2-d93e-4f46-92c6-1f0621427f43 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:10.846303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0743a2-d93e-4f46-92c6-1f0621427f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:10.915717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-ae0743a2-d93e-4f46-92c6-1f0621427f43 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:10.916977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-ae0743a2-d93e-4f46-92c6-1f0621427f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:11.020741 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 655/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:11.074452 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872c2eb-4eb4-4cbe-bc81-541aec1efdd1 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:11.074690 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872c2eb-4eb4-4cbe-bc81-541aec1efdd1 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:11.074988 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872c2eb-4eb4-4cbe-bc81-541aec1efdd1 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:11.076375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 665/1320] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 08:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:11.082511 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb10c1-4529-4cab-9690-dc7888e692b5 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:11.082804 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb10c1-4529-4cab-9690-dc7888e692b5 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:11.083083 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb10c1-4529-4cab-9690-dc7888e692b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:11.110392 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:11.170709 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-cccb10c1-4529-4cab-9690-dc7888e692b5 None None] MFA Rules not processed for user `05afd25e30764af0819436cc91dd85c5`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:11.171945 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-cccb10c1-4529-4cab-9690-dc7888e692b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:11.253199 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 656/1321] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 08:19:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:11.262837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872c2eb-4eb4-4cbe-bc81-541aec1efdd1 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:11.264466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-f872c2eb-4eb4-4cbe-bc81-541aec1efdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:11.372818 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872c2eb-4eb4-4cbe-bc81-541aec1efdd1 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:11.448116 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ee96f-3f8a-4ce6-9820-63f96a787082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:11.448747 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee96f-3f8a-4ce6-9820-63f96a787082 service neutron] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:11.449031 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee96f-3f8a-4ce6-9820-63f96a787082 service neutron] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:11.449301 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee96f-3f8a-4ce6-9820-63f96a787082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:11.450999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-e90ee96f-3f8a-4ce6-9820-63f96a787082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:11.459020 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:11.569302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ee96f-3f8a-4ce6-9820-63f96a787082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:11.571185 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ee96f-3f8a-4ce6-9820-63f96a787082 service neutron] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:11.589338 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 666/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:14.746423 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e53b7-0d5d-4252-a8a7-70916e626d53 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:14.747006 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e53b7-0d5d-4252-a8a7-70916e626d53 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:14.747006 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e53b7-0d5d-4252-a8a7-70916e626d53 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:14.748443 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 657/1323] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:14.754495 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462cf4d-19e8-4cae-bb52-0c03b5a470d6 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:14.754993 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462cf4d-19e8-4cae-bb52-0c03b5a470d6 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:14.755102 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462cf4d-19e8-4cae-bb52-0c03b5a470d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:14.824607 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-2462cf4d-19e8-4cae-bb52-0c03b5a470d6 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:14.825838 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-2462cf4d-19e8-4cae-bb52-0c03b5a470d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:14.889715 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:14.933847 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 667/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:19:14.971303 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e53b7-0d5d-4252-a8a7-70916e626d53 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:14.972908 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-c36e53b7-0d5d-4252-a8a7-70916e626d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:15.096842 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e53b7-0d5d-4252-a8a7-70916e626d53 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:15.112894 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:15.188758 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a314b4-57bb-40f4-998a-dec2be9f62df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:15.189309 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a314b4-57bb-40f4-998a-dec2be9f62df service neutron] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:15.189594 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a314b4-57bb-40f4-998a-dec2be9f62df service neutron] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:15.189867 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a314b4-57bb-40f4-998a-dec2be9f62df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:15.191717 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-09a314b4-57bb-40f4-998a-dec2be9f62df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:15.328259 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a314b4-57bb-40f4-998a-dec2be9f62df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:15.329746 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a314b4-57bb-40f4-998a-dec2be9f62df service neutron] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:15.349219 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 658/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:17.413778 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685ba2-4cff-410a-9938-c23c5e3353ec None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:17.414401 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685ba2-4cff-410a-9938-c23c5e3353ec None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:17.414401 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685ba2-4cff-410a-9938-c23c5e3353ec None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:17.415725 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 668/1326] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:17.421327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c09be4-d82c-456e-a9ad-54cf589a65da None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:17.421776 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c09be4-d82c-456e-a9ad-54cf589a65da None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:17.421863 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c09be4-d82c-456e-a9ad-54cf589a65da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:17.476201 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:17.498448 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-01c09be4-d82c-456e-a9ad-54cf589a65da None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:17.499708 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-01c09be4-d82c-456e-a9ad-54cf589a65da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:17.623381 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 659/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:17.663766 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a685ba2-4cff-410a-9938-c23c5e3353ec None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:17.665906 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-8a685ba2-4cff-410a-9938-c23c5e3353ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:17.802632 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a685ba2-4cff-410a-9938-c23c5e3353ec None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:17.890423 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda899cd-b708-4522-ae28-20aeb9bf5d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:17.890979 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda899cd-b708-4522-ae28-20aeb9bf5d33 service neutron] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:17.891264 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda899cd-b708-4522-ae28-20aeb9bf5d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:17.891537 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda899cd-b708-4522-ae28-20aeb9bf5d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:17.893225 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-bda899cd-b708-4522-ae28-20aeb9bf5d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:17.911781 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:18.009684 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda899cd-b708-4522-ae28-20aeb9bf5d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:18.011302 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda899cd-b708-4522-ae28-20aeb9bf5d33 service neutron] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:18.029999 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 669/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 08:19:20.102244 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1db1-be5f-4dc9-a79e-395bf16fd4cb None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:20.102862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1db1-be5f-4dc9-a79e-395bf16fd4cb None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:20.102862 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1db1-be5f-4dc9-a79e-395bf16fd4cb None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:20.104248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 660/1329] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:20.109865 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e8514-f31d-4e57-aa8b-deefd10c3d75 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:20.110349 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e8514-f31d-4e57-aa8b-deefd10c3d75 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:20.110440 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e8514-f31d-4e57-aa8b-deefd10c3d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:20.177247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-1a8e8514-f31d-4e57-aa8b-deefd10c3d75 None None] MFA Rules not processed for user `cee2a726271640868cff7cbf256bac2d`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:20.178491 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-1a8e8514-f31d-4e57-aa8b-deefd10c3d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:20.246535 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:20.282227 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 670/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:20 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:20.320841 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1db1-be5f-4dc9-a79e-395bf16fd4cb None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:20.323019 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-e30d1db1-be5f-4dc9-a79e-395bf16fd4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:20.432413 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1db1-be5f-4dc9-a79e-395bf16fd4cb None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:20.453430 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:20.512486 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde7b7c-2e72-4b7e-9923-bf7bbf123a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:20.513120 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7b7c-2e72-4b7e-9923-bf7bbf123a3a service neutron] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:20.513405 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7b7c-2e72-4b7e-9923-bf7bbf123a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:20.513682 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7b7c-2e72-4b7e-9923-bf7bbf123a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:20.515387 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-8fde7b7c-2e72-4b7e-9923-bf7bbf123a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:20.632478 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde7b7c-2e72-4b7e-9923-bf7bbf123a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:20.633958 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde7b7c-2e72-4b7e-9923-bf7bbf123a3a service neutron] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:20.655840 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 661/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:20 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:22.936230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814e423-7d3d-472c-89a4-b29658331ecc None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:22.936837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814e423-7d3d-472c-89a4-b29658331ecc None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:22.936837 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814e423-7d3d-472c-89a4-b29658331ecc None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:22.938088 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 671/1332] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:22.943612 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569b38-d57a-4506-a117-e673824dbb20 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:22.944046 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569b38-d57a-4506-a117-e673824dbb20 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:22.944135 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569b38-d57a-4506-a117-e673824dbb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:23.009585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:23.013558 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d3569b38-d57a-4506-a117-e673824dbb20 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:23.015514 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d3569b38-d57a-4506-a117-e673824dbb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:23.116327 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 662/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:23.152517 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814e423-7d3d-472c-89a4-b29658331ecc None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:23.154084 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-a814e423-7d3d-472c-89a4-b29658331ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:23.263042 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814e423-7d3d-472c-89a4-b29658331ecc None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:23.341193 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041680a3-3015-4909-a19f-4c99b187593d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:23.341830 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041680a3-3015-4909-a19f-4c99b187593d service neutron] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:23.342138 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041680a3-3015-4909-a19f-4c99b187593d service neutron] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:23.342405 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041680a3-3015-4909-a19f-4c99b187593d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:23.344109 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-041680a3-3015-4909-a19f-4c99b187593d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:23.373079 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:23.468151 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041680a3-3015-4909-a19f-4c99b187593d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:23.469617 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041680a3-3015-4909-a19f-4c99b187593d service neutron] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:23.488818 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 672/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:25.539463 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da0b25-58eb-4182-88ca-19e225307cc8 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:25.540038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da0b25-58eb-4182-88ca-19e225307cc8 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:25.540038 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da0b25-58eb-4182-88ca-19e225307cc8 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:25.541498 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 663/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:25.546957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a1e1e-1e65-4eab-ae50-57aa7636218e None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:25.547420 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a1e1e-1e65-4eab-ae50-57aa7636218e None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:25.547511 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a1e1e-1e65-4eab-ae50-57aa7636218e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:25.615698 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-725a1e1e-1e65-4eab-ae50-57aa7636218e None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:25.616919 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-725a1e1e-1e65-4eab-ae50-57aa7636218e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:25.694923 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:25.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 673/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:25.759477 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da0b25-58eb-4182-88ca-19e225307cc8 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:25.761095 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-77da0b25-58eb-4182-88ca-19e225307cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:25.878454 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da0b25-58eb-4182-88ca-19e225307cc8 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:25.909095 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:25.959059 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdb674-7a97-4ed8-b088-faf10f83630b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:25.959688 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdb674-7a97-4ed8-b088-faf10f83630b service neutron] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:25.959965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdb674-7a97-4ed8-b088-faf10f83630b service neutron] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:25.960234 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdb674-7a97-4ed8-b088-faf10f83630b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:25.961911 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-dbfdb674-7a97-4ed8-b088-faf10f83630b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:26.088801 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdb674-7a97-4ed8-b088-faf10f83630b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:26.090347 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdb674-7a97-4ed8-b088-faf10f83630b service neutron] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:26.110888 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 664/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:28.604361 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f334-c194-4364-8d5b-73d222ebeef2 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:28.605011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f334-c194-4364-8d5b-73d222ebeef2 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:28.605011 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f334-c194-4364-8d5b-73d222ebeef2 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:28.606359 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 674/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:28.612147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5099f94-a128-4abf-b9b7-19a90af8244f None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:28.612686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5099f94-a128-4abf-b9b7-19a90af8244f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:28.612686 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5099f94-a128-4abf-b9b7-19a90af8244f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:28.685521 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d5099f94-a128-4abf-b9b7-19a90af8244f None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:28.686823 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d5099f94-a128-4abf-b9b7-19a90af8244f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:28.696353 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:28.792694 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 665/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:28.829711 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21f334-c194-4364-8d5b-73d222ebeef2 None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:28.831314 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-0d21f334-c194-4364-8d5b-73d222ebeef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:28.955018 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21f334-c194-4364-8d5b-73d222ebeef2 None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:29.036665 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab9dd7-c1c1-4a0c-a7cc-e8383115aa12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:29.037307 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab9dd7-c1c1-4a0c-a7cc-e8383115aa12 service neutron] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:29.037590 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab9dd7-c1c1-4a0c-a7cc-e8383115aa12 service neutron] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:29.037863 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab9dd7-c1c1-4a0c-a7cc-e8383115aa12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:29.039592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-5eab9dd7-c1c1-4a0c-a7cc-e8383115aa12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:29.076524 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:29.154790 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab9dd7-c1c1-4a0c-a7cc-e8383115aa12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:29.156262 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab9dd7-c1c1-4a0c-a7cc-e8383115aa12 service neutron] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:29.175475 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 675/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:33.190310 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286809e2-6ec8-486c-a4da-233921c2350b None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:33.190917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286809e2-6ec8-486c-a4da-233921c2350b None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:33.190917 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286809e2-6ec8-486c-a4da-233921c2350b None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:33.192265 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 666/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:33.198703 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879d41-338c-4b73-b980-5337fa16fbcf None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:33.199257 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879d41-338c-4b73-b980-5337fa16fbcf None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:33.199257 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879d41-338c-4b73-b980-5337fa16fbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:33.274568 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-3d879d41-338c-4b73-b980-5337fa16fbcf None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:33.275799 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-3d879d41-338c-4b73-b980-5337fa16fbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:33.383850 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:33.397772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 676/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:33.436985 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286809e2-6ec8-486c-a4da-233921c2350b None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:33.438601 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-286809e2-6ec8-486c-a4da-233921c2350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:33.554848 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286809e2-6ec8-486c-a4da-233921c2350b None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:33.596673 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:33.635939 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090dc19-3036-4921-b4aa-24dec856f3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:33.636571 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090dc19-3036-4921-b4aa-24dec856f3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:33.636847 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090dc19-3036-4921-b4aa-24dec856f3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:33.637115 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090dc19-3036-4921-b4aa-24dec856f3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:33.638873 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-a090dc19-3036-4921-b4aa-24dec856f3b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:33.766545 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090dc19-3036-4921-b4aa-24dec856f3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:33.768365 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090dc19-3036-4921-b4aa-24dec856f3b5 service neutron] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:33.791649 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 667/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:36.601466 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b39d-2abe-43cb-a8ac-0ab493dad2cc None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:36.602064 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b39d-2abe-43cb-a8ac-0ab493dad2cc None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:36.602064 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b39d-2abe-43cb-a8ac-0ab493dad2cc None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:36.603522 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 677/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:36.608968 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c6d76-cd71-464d-80b3-9bc97f308703 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:36.609437 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c6d76-cd71-464d-80b3-9bc97f308703 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:36.609536 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c6d76-cd71-464d-80b3-9bc97f308703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:36.678468 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-3a6c6d76-cd71-464d-80b3-9bc97f308703 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:36.679714 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-3a6c6d76-cd71-464d-80b3-9bc97f308703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:36.699474 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:36.789522 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 668/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:36.826920 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b39d-2abe-43cb-a8ac-0ab493dad2cc None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:36.828481 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-4bf2b39d-2abe-43cb-a8ac-0ab493dad2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:36.937280 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b39d-2abe-43cb-a8ac-0ab493dad2cc None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:37.014013 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947d183-3906-4278-bb00-f43136952d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:37.014674 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947d183-3906-4278-bb00-f43136952d10 service neutron] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:37.014957 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947d183-3906-4278-bb00-f43136952d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:37.015233 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947d183-3906-4278-bb00-f43136952d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:37.016855 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-d947d183-3906-4278-bb00-f43136952d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:37.069985 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:37.137412 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947d183-3906-4278-bb00-f43136952d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:37.138929 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947d183-3906-4278-bb00-f43136952d10 service neutron] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:37.159473 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 678/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:41.527122 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e724b-39b4-46a1-98bc-e328c1bdba9f None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:41.527715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e724b-39b4-46a1-98bc-e328c1bdba9f None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:41.527715 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e724b-39b4-46a1-98bc-e328c1bdba9f None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:41.529092 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 669/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:41.534829 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ce5d5-e6c0-42e3-af7c-58ea1c5738c9 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:41.535352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ce5d5-e6c0-42e3-af7c-58ea1c5738c9 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:41.535352 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ce5d5-e6c0-42e3-af7c-58ea1c5738c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:41.622921 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-698ce5d5-e6c0-42e3-af7c-58ea1c5738c9 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:41.624470 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-698ce5d5-e6c0-42e3-af7c-58ea1c5738c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:41.744545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:41.750154 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 679/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:41.787213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e724b-39b4-46a1-98bc-e328c1bdba9f None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:41.788816 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-113e724b-39b4-46a1-98bc-e328c1bdba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:41.912693 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e724b-39b4-46a1-98bc-e328c1bdba9f None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:41.965374 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:41.992585 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df7801-01cb-414b-b664-cc1d18071c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05afd25e30764af0819436cc91dd85c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:41.993221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df7801-01cb-414b-b664-cc1d18071c2f service neutron] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:41.993540 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df7801-01cb-414b-b664-cc1d18071c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:41.993815 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df7801-01cb-414b-b664-cc1d18071c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:41.995504 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-c5df7801-01cb-414b-b664-cc1d18071c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:42.106768 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df7801-01cb-414b-b664-cc1d18071c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:42.108231 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df7801-01cb-414b-b664-cc1d18071c2f service neutron] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:42.126769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 670/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:47.082688 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a02f4f-2cb8-47be-8e59-4d9b6892edce None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:47.083226 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a02f4f-2cb8-47be-8e59-4d9b6892edce None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:47.083226 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a02f4f-2cb8-47be-8e59-4d9b6892edce None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:47.084588 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 680/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:47.090298 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3774fcd-0428-4c6d-96da-c2360cd0f009 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:47.090760 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3774fcd-0428-4c6d-96da-c2360cd0f009 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:47.090899 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3774fcd-0428-4c6d-96da-c2360cd0f009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:47.270977 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-d3774fcd-0428-4c6d-96da-c2360cd0f009 None None] MFA Rules not processed for user `89b69b21ab61476c86b189d941949cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:47.272556 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-d3774fcd-0428-4c6d-96da-c2360cd0f009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:47.338586 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:47.419865 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 671/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:19:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:51.524029 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e10449-9090-4c8c-85a1-b2e7c85f2282 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:51.524592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e10449-9090-4c8c-85a1-b2e7c85f2282 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:51.524592 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e10449-9090-4c8c-85a1-b2e7c85f2282 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:51.526256 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 681/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:51.531406 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b55760-ed65-4a1e-be4f-2787eca0bd40 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:51.531903 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b55760-ed65-4a1e-be4f-2787eca0bd40 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:51.532206 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b55760-ed65-4a1e-be4f-2787eca0bd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:51.619002 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-a5b55760-ed65-4a1e-be4f-2787eca0bd40 None None] MFA Rules not processed for user `89b69b21ab61476c86b189d941949cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:51.620407 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-a5b55760-ed65-4a1e-be4f-2787eca0bd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:51.700452 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:51.752329 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 672/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:19:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:53.049421 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6a21b-982c-4d7c-b579-735bd11139d6 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:53.049940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6a21b-982c-4d7c-b579-735bd11139d6 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:53.049940 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6a21b-982c-4d7c-b579-735bd11139d6 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:53.051309 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 682/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:53.057168 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feead351-e3a6-4e42-b2cf-80351ab849d9 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:53.057667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feead351-e3a6-4e42-b2cf-80351ab849d9 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:53.057759 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feead351-e3a6-4e42-b2cf-80351ab849d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:53.138672 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-feead351-e3a6-4e42-b2cf-80351ab849d9 None None] MFA Rules not processed for user `89b69b21ab61476c86b189d941949cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:53.139967 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-feead351-e3a6-4e42-b2cf-80351ab849d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:53.251579 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 673/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:19:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:19:56.678008 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2deba3-c709-410c-a37c-6b51fa9a3b64 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:56.678586 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2deba3-c709-410c-a37c-6b51fa9a3b64 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:56.678758 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2deba3-c709-410c-a37c-6b51fa9a3b64 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:56.680500 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 683/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:56.685960 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f04d6-cf2d-4b21-b1ce-e8de8974dd03 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:56.686902 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f04d6-cf2d-4b21-b1ce-e8de8974dd03 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:56.687415 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f04d6-cf2d-4b21-b1ce-e8de8974dd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:56.732182 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:56.772965 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-f13f04d6-cf2d-4b21-b1ce-e8de8974dd03 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:56.775779 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f13f04d6-cf2d-4b21-b1ce-e8de8974dd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:56.892614 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 674/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:56.948898 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8ffb8-24d2-4266-87cc-d73f6352946b None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:56.949313 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8ffb8-24d2-4266-87cc-d73f6352946b None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:56.949692 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8ffb8-24d2-4266-87cc-d73f6352946b None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:56.951219 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 684/1358] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 08:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:56.961530 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88807e8d-2645-4690-a421-ead83ad25c58 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:56.962022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88807e8d-2645-4690-a421-ead83ad25c58 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:56.962221 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88807e8d-2645-4690-a421-ead83ad25c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:57.052197 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-88807e8d-2645-4690-a421-ead83ad25c58 None None] MFA Rules not processed for user `45cd9a93c342491889ca8260bc4510c6`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:57.053623 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-88807e8d-2645-4690-a421-ead83ad25c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:57.107061 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:57.196539 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 675/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 08:19:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:19:57.206507 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8ffb8-24d2-4266-87cc-d73f6352946b None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:19:57.209144 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-22a8ffb8-24d2-4266-87cc-d73f6352946b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:57.367856 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8ffb8-24d2-4266-87cc-d73f6352946b None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:19:57.454659 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:19:57.504510 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa193b-f241-4546-b394-c89a661617b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45cd9a93c342491889ca8260bc4510c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:19:57.505150 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa193b-f241-4546-b394-c89a661617b2 service nova] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:57.505434 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa193b-f241-4546-b394-c89a661617b2 service nova] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:57.505705 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa193b-f241-4546-b394-c89a661617b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:57.507583 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-a5aa193b-f241-4546-b394-c89a661617b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:57.612171 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa193b-f241-4546-b394-c89a661617b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:19:57.614516 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa193b-f241-4546-b394-c89a661617b2 service nova] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:19:57.645087 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 685/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:19:59.867022 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394d264-8e8a-4721-a4ea-12e8c039b3c6 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:59.867478 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394d264-8e8a-4721-a4ea-12e8c039b3c6 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:59.867571 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394d264-8e8a-4721-a4ea-12e8c039b3c6 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:59.874609 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab8854-5502-4e24-8630-4083435eaa6c None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:19:59.875042 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab8854-5502-4e24-8630-4083435eaa6c None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:19:59.875212 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab8854-5502-4e24-8630-4083435eaa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:19:59.878528 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 676/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:19:59 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:19:59.950479 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-5eab8854-5502-4e24-8630-4083435eaa6c None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:19:59.951734 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-5eab8854-5502-4e24-8630-4083435eaa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:19:59.994178 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:00.064818 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 686/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:20:00.113213 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394d264-8e8a-4721-a4ea-12e8c039b3c6 None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:20:00.114869 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f394d264-8e8a-4721-a4ea-12e8c039b3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:00.287049 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394d264-8e8a-4721-a4ea-12e8c039b3c6 None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:20:00.297377 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:00.437370 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aac3f-3247-408b-b276-917631421b81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45cd9a93c342491889ca8260bc4510c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:20:00.438096 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aac3f-3247-408b-b276-917631421b81 service nova] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:00.438493 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aac3f-3247-408b-b276-917631421b81 service nova] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:00.438828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aac3f-3247-408b-b276-917631421b81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:00.440534 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-8a7aac3f-3247-408b-b276-917631421b81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:00.541428 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aac3f-3247-408b-b276-917631421b81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:20:00.543119 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aac3f-3247-408b-b276-917631421b81 service nova] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:20:00.564828 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 677/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:20:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:20:02.931929 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbcdda-760f-458d-bc0c-597b281d96c7 None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:02.932519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbcdda-760f-458d-bc0c-597b281d96c7 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:02.932519 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbcdda-760f-458d-bc0c-597b281d96c7 None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:02.934042 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 687/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:20:02.940469 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4bfb-8f92-4b9e-b073-710853db7360 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:02.941425 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4bfb-8f92-4b9e-b073-710853db7360 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:02.942012 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4bfb-8f92-4b9e-b073-710853db7360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:02.959758 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:03.021667 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-eccf4bfb-8f92-4b9e-b073-710853db7360 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:20:03.023034 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-eccf4bfb-8f92-4b9e-b073-710853db7360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:03.153735 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 678/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:20:03.201265 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f98f7a-b65a-4844-9ac0-2f4dd85fc83e None None] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:03.201599 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f98f7a-b65a-4844-9ac0-2f4dd85fc83e None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:03.201832 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f98f7a-b65a-4844-9ac0-2f4dd85fc83e None None] PATH_INFO: `/` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:03.203479 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 688/1366] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 08:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:20:03.209444 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4052354-0b01-4e98-a3d2-76b2209269b4 None None] REQUEST_METHOD: `POST` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:03.210147 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4052354-0b01-4e98-a3d2-76b2209269b4 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:03.210625 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4052354-0b01-4e98-a3d2-76b2209269b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:03.305045 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.auth.core [None req-f4052354-0b01-4e98-a3d2-76b2209269b4 None None] MFA Rules not processed for user `45cd9a93c342491889ca8260bc4510c6`. Rule list: `[]` (Enabled: `True`). {{(pid=144043) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:20:03.306750 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-f4052354-0b01-4e98-a3d2-76b2209269b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:03.348962 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:03.452039 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 679/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 08:20:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:20:03.461545 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f98f7a-b65a-4844-9ac0-2f4dd85fc83e None None] Authenticating user token {{(pid=144042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:20:03.463217 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-58f98f7a-b65a-4844-9ac0-2f4dd85fc83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:03.660513 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f98f7a-b65a-4844-9ac0-2f4dd85fc83e None None] Validating token access rules against request {{(pid=144042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:20:03.690677 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:03.834408 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfbd0e-d723-4494-b55d-bdf587d0b6aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45cd9a93c342491889ca8260bc4510c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:20:03.835165 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfbd0e-d723-4494-b55d-bdf587d0b6aa service nova] REQUEST_METHOD: `GET` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:03.835514 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfbd0e-d723-4494-b55d-bdf587d0b6aa service nova] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:03.835858 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfbd0e-d723-4494-b55d-bdf587d0b6aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:03.837876 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-dcbfbd0e-d723-4494-b55d-bdf587d0b6aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:03.952853 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfbd0e-d723-4494-b55d-bdf587d0b6aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:20:03.954573 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfbd0e-d723-4494-b55d-bdf587d0b6aa service nova] RBAC: Authorization granted {{(pid=144042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:20:03.978235 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 689/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 08:20:04.485769 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2f1e-b5a2-4a3b-8cbd-a260b394deb7 None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:04.486376 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2f1e-b5a2-4a3b-8cbd-a260b394deb7 None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:04.486376 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2f1e-b5a2-4a3b-8cbd-a260b394deb7 None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:04.487963 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 680/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:20:04.493567 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae18-2731-4bf6-b51f-a081fe6b1184 None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:04.493899 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae18-2731-4bf6-b51f-a081fe6b1184 None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:04.494230 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae18-2731-4bf6-b51f-a081fe6b1184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:04.534772 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:04.573664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-9127ae18-2731-4bf6-b51f-a081fe6b1184 None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:20:04.574956 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-9127ae18-2731-4bf6-b51f-a081fe6b1184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:04.698166 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 690/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:20:06.108172 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a07fe-17ff-48fe-a825-a6b93a9220fe None None] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:06.108767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a07fe-17ff-48fe-a825-a6b93a9220fe None None] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:06.108767 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a07fe-17ff-48fe-a825-a6b93a9220fe None None] PATH_INFO: `/` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:06.110073 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 681/1371] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:20:06.115569 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3067c-4753-4843-b23a-27c061cbe76d None None] REQUEST_METHOD: `POST` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:06.116019 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3067c-4753-4843-b23a-27c061cbe76d None None] SCRIPT_NAME: `/identity` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:06.116127 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3067c-4753-4843-b23a-27c061cbe76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:06.188247 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.auth.core [None req-5db3067c-4753-4843-b23a-27c061cbe76d None None] MFA Rules not processed for user `ab47d79a05944b2eb59466f830650026`. Rule list: `[]` (Enabled: `True`). {{(pid=144042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:20:06.190066 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG keystone.common.fernet_utils [None req-5db3067c-4753-4843-b23a-27c061cbe76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:06.251792 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=100 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:06.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: [pid: 144042|app: 0|req: 691/1372] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:20:06.353520 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a07fe-17ff-48fe-a825-a6b93a9220fe None None] Authenticating user token {{(pid=144043) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:20:06.355087 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-155a07fe-17ff-48fe-a825-a6b93a9220fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:06.634248 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:06.644655 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a07fe-17ff-48fe-a825-a6b93a9220fe None None] Validating token access rules against request {{(pid=144043) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:20:06.896975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0e4c5-c842-4d40-8cde-c5c909d80693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45cd9a93c342491889ca8260bc4510c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8ec7a252ce6472d9f5c1bacc1d725c3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144043) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:20:06.896975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0e4c5-c842-4d40-8cde-c5c909d80693 service nova] REQUEST_METHOD: `GET` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:20:06.896975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0e4c5-c842-4d40-8cde-c5c909d80693 service nova] SCRIPT_NAME: `/identity` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:20:06.896975 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0e4c5-c842-4d40-8cde-c5c909d80693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=144043) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:20:06.916085 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.fernet_utils [None req-07b0e4c5-c842-4d40-8cde-c5c909d80693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144043) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:20:06.971572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0e4c5-c842-4d40-8cde-c5c909d80693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:20:06.971572 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0e4c5-c842-4d40-8cde-c5c909d80693 service nova] RBAC: Authorization granted {{(pid=144043) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:20:06.981473 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: DEBUG dbcounter [-] [144043] Writing DB stats keystone:SELECT=100 {{(pid=144043) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:06.984449 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: [pid: 144043|app: 0|req: 682/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:20:16.301930 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: DEBUG dbcounter [-] [144042] Writing DB stats keystone:SELECT=19 {{(pid=144042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:20:27.269902 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: Thu Jul 28 08:20:27 2022 - graceful shutdown triggered... Jul 28 08:20:27.270427 dv-cinder-851351-1-iscsi devstack@keystone.service[144043]: Gracefully killing worker 2 (pid: 144043)... Jul 28 08:20:27.270191 dv-cinder-851351-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Jul 28 08:20:27.273664 dv-cinder-851351-1-iscsi devstack@keystone.service[144042]: Gracefully killing worker 1 (pid: 144042)... Jul 28 08:20:28.271110 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: worker 1 buried after 1 seconds Jul 28 08:20:28.271110 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: worker 2 buried after 1 seconds Jul 28 08:20:28.271110 dv-cinder-851351-1-iscsi devstack@keystone.service[144028]: goodbye to uWSGI. Jul 28 08:20:28.272866 dv-cinder-851351-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Jul 28 08:20:28.273426 dv-cinder-851351-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. Jul 28 08:25:17.594574 dv-cinder-851351-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 08:25:17.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 08:25:17 2022] *** Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: nodename: dv-cinder-851351-1-iscsi Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: machine: x86_64 Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: clock source: unix Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: pcre jit disabled Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: detected number of CPU cores: 4 Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: current working directory: / Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: detected binary path: /usr/bin/uwsgi-core Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: your processes number limit is 47680 Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: your memory page size is 4096 bytes Jul 28 08:25:17.607862 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: detected max file descriptor number: 2048 Jul 28 08:25:17.608847 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: lock engine: pthread robust mutexes Jul 28 08:25:17.608847 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: thunder lock: enabled Jul 28 08:25:17.608847 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 08:25:17.608847 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 08:25:17.648222 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: Python main interpreter initialized at 0x55aff25cb9c0 Jul 28 08:25:17.648222 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: python threads support enabled Jul 28 08:25:17.648222 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: your server socket listen backlog is limited to 100 connections Jul 28 08:25:17.648222 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: your mercy for graceful operations on workers is 90 seconds Jul 28 08:25:17.648841 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 08:25:17.648841 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: *** Operational MODE: preforking *** Jul 28 08:25:17.649042 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: *** uWSGI is running in multiple interpreter mode *** Jul 28 08:25:17.649042 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: spawned uWSGI master process (pid: 207757) Jul 28 08:25:17.649164 dv-cinder-851351-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 08:25:17.654139 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: spawned uWSGI worker 1 (pid: 207769, cores: 1) Jul 28 08:25:17.654139 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: spawned uWSGI worker 2 (pid: 207770, cores: 1) Jul 28 08:25:17.654139 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 08:25:23.341665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:25:23.341665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: from cryptography.utils import int_from_bytes Jul 28 08:25:23.343001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:25:23.343001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: from cryptography.utils import int_from_bytes Jul 28 08:25:23.343144 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:25:23.343144 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: from cryptography.utils import int_from_bytes Jul 28 08:25:23.343144 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:25:23.343144 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: from cryptography.utils import int_from_bytes Jul 28 08:25:23.394153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:25:23.394498 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:25:23.394498 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] command line args: [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:25:23.394844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:25:23.394993 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ================================================================================ {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:25:23.395203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] admin_token = **** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.395526 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] config_dir = [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.395704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.395894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] config_source = [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.396103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] debug = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.396284 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.397733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.397733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] insecure_debug = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.397733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.397733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.397733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] list_limit = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.397733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] log_config_append = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.399120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.399336 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:25:23.399490 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] log_dir = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.399729 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:25:23.399900 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] log_file = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.400055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] command line args: [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:25:23.400205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.400350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:25:23.400500 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.400668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ================================================================================ {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:25:23.400800 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.400935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] admin_token = **** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.401105 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.401204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] config_dir = [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.401328 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.401434 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.401558 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.401660 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] config_source = [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.401660 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] debug = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.401863 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402092 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402237 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402500 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] insecure_debug = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402681 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.402930 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] list_limit = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403089 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403161 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] log_config_append = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] notification_format = cadf {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403331 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403331 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] log_dir = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] log_file = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.403862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.405161 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.405497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] publish_errors = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.405599 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.405686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.405769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.405878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.406044 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.406214 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.406364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.406500 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.406624 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.406751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.406880 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] standard_threads = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] strict_password_check = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407513 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407605 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] use_eventlog = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407805 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] use_journal = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.407982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.408082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] use_json = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.408082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] use_stderr = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.408309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] notification_format = cadf {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.408441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] use_syslog = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.408564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.408688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] watch_log_file = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.408688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.408908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.409033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.409162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] publish_errors = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.409278 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.409406 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.409535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.409671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.409671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.409906 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.409906 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.410175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.410175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.410427 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.410561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] standard_threads = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.410684 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.410806 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] strict_password_check = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.410929 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.411057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.411243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.411371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] use_eventlog = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.411463 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.411548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] use_journal = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.411548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] use_json = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.411700 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.411797 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] use_stderr = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.411883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.411969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] use_syslog = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.412063 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.412196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] watch_log_file = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:25:23.412319 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.412449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.412571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.412694 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.412809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.412939 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.413061 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.413187 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.413321 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.413456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.413596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.413728 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.413853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.413989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.414154 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.414286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.414422 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.414537 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.414659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.414659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.414900 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415357 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415453 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415723 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415813 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.415907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416010 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416196 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416281 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416374 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416739 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.416921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417345 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417434 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417620 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417620 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.417962 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418059 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418250 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418430 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418514 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418784 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.418944 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419409 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419500 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419687 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.419959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420148 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420148 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420314 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420404 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420502 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420677 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420863 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.420963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421334 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421334 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421774 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.421954 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422043 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422421 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422733 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422902 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.422987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423139 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423359 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423434 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.enabled = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423517 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423588 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.423922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.424057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.424329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.424477 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.424585 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.424679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.424759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.424837 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.425079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.425224 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.425329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.425418 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.425491 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.425682 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.425836 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.425969 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.426068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.426280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.426429 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.426533 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.426616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.426866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427020 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427474 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427742 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427889 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.427988 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.428089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.428175 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.428330 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.428534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.428715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.428907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.429091 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.429283 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.429450 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.429630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.429796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.429970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.430172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.enabled = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.430350 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.430518 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.430686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.430787 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.430889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.431053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.431219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] auth.external = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.431389 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.431545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] auth.mapped = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.431545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.431881 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432209 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432314 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] auth.password = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432403 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432495 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] auth.token = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432744 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432828 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] catalog.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.432990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.433090 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.433183 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.433276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.433363 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.433513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.433673 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.433841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.434003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.434206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.434362 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] credential.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.434532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.434696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] credential.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.434869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.435033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.435209 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.435381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.435534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.435685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.435855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.436024 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.436176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.436338 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.436488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.436488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.436770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.436946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.437102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.437266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.437421 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.437598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.437761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.437927 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.438131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.438304 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.438467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.438624 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] auth.external = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.438787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.438964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] auth.mapped = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.439127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.439127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.439457 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.439612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] federation.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.439772 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.439942 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.440106 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] auth.password = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.440281 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] federation.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.440445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] auth.token = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.440607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.440769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.440935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.441095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] catalog.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.441325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.441471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.441471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.441627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.441720 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.441824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.441965 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.442166 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.442322 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.442494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.442666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] credential.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.442822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.442984 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] credential.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.443586 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.443750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.443902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.444067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.444224 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.444397 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.444563 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.444729 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.444882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445721 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.445829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.446149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.446149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.446585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.446744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.446900 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.447060 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.447225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.447393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.447581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.447745 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.447908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448078 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448192 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] federation.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448287 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448397 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448486 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] federation.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448754 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.448855 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.449029 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.449182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.449332 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.449486 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.449649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.449831 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450310 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450399 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450653 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450783 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.450933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.451076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.451232 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.451386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.451545 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.451680 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.451847 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.452012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.452165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.452330 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.452504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.452671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.452850 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.453009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.453164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.453320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.453493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.453653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.453807 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.453952 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.454048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.454183 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.password = **** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.454294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.454446 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.454611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.454776 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.454931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.455098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.455271 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.455436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.455605 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.455883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.455989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.456065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.456162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.456265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.456360 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.456475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.456638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.456815 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.456987 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.457152 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.457306 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.457472 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.457635 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.457635 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.458124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.458267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.458375 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.458481 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.458481 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.458707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.458889 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.459050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.459221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.459389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.459554 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.459726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.459945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.460142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.460308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.460472 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.460636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.460808 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.460937 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.461030 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.461183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.461346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.461501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.461671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.461831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.461979 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.462176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.password = **** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.462343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.462503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.462683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.462843 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.463006 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.463170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.463324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.463478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.463652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.463813 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.463976 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.464139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.464292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.464460 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.464631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.464791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.464951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.465113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.465283 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.465442 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.465627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.465780 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.465933 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.466082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.466362 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.466515 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.466515 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.466793 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.466934 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.467073 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.467209 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.467365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.467535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.467690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.467842 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.468003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] policy.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.468158 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.468309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.468449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.468544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.468657 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.468821 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.468974 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.469125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] receipt.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.469298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.469452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.469604 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.469767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.469930 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.470094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.470299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.470461 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.470622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.470774 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.470925 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.471087 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] resource.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.471250 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.471406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.471568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.471734 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] resource.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.471734 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.472025 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.472199 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.472364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.472364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.472665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.472665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] revoke.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.472967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.472967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.473257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.473420 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.473584 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.473728 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.473836 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] role.cache_time = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.473932 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.474027 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] role.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.474142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.474270 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] role.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.474393 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.474570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] role.list_limit = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.474722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.474890 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.475045 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] policy.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.475215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.475378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.475685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.475852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.476007 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.476147 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.476304 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.476459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] receipt.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.476627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.476792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.476944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.477100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.477257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.477413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.477584 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.477748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.477900 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.478054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.478249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.478404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] resource.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.478547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.478666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.478831 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.478998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] resource.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.478998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.479302 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.479471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.479636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.479787 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.479945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.480096 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] revoke.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.480203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.480321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.480479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.480643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.480800 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.480953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] role.cache_time = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.481108 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.481280 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] role.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.481437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.481571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] role.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.481663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.481757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] role.list_limit = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.481915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.482051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.482278 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.482437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.482593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.482746 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.482910 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.483080 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.483246 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.483410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.483571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.483735 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.483906 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.484021 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.484108 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.484253 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.484418 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.484586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.484762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] token.cache_time = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.484916 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.485082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] token.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.485237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.485376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.485476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.485579 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] token.provider = fernet {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.485668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.485785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.485785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.486126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.486302 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.486473 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.486632 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.486788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.486921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.487074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.487227 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.487395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.487566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] trust.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.487731 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.487898 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.488053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.488230 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.488407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.488573 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.488735 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.488891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.489031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.489128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.489319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.489495 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.489685 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.489853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.490017 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.490269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.490449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.490449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.490614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.connection = **** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.490695 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.490783 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.490902 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.491056 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.491211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.491366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.491540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] token.cache_time = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.491698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.491887 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] token.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.492102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.492265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.492423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.492596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] token.provider = fernet {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.492748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.492908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.493066 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.493223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.493391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.493575 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.493733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.493890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.494060 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.494298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.494471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.494639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.494639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] trust.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.494943 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.495108 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.495254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.495419 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.495593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.495759 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.495922 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.496089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.496089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.496345 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.496441 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.496587 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.496741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.496894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=207769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.connection = **** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.497051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.499070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.499070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:25:23.499070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=207770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:25:23.512888 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.513179 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.513478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.513609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.513715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.513931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.523261 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.523383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.554049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.554379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.554535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.554677 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.609458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.609729 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.661994 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.662340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.664509 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.664821 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.730261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.730542 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.730701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.730701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.731016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.731213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.731213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.731503 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.731609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.731713 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.731883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.732157 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.755811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.756098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.756258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.756407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.756544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.756688 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.756813 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.756933 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.757064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.757187 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.757305 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.757432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.757551 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.757829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.769087 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.769591 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.782429 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.782730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.782885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.782885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.783185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.783349 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.783349 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.783628 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.783801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.783967 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.784076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=207770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.784166 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.784371 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.784613 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=207769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:25:23.801266 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.801506 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.801506 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:25:23.801736 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:25:23.801830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.802036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:25:23.802353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.802462 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.802631 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.802811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:25:23.802981 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.803132 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.803301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.803434 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.803764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.804086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.813036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.813322 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.813601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.813899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.814307 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.814671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.815003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.815343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.821483 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.823007 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.826549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.826838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.827121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.827413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.827699 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.827964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.828236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.828512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.828676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.828828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.829009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.829196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.829354 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.829521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.829744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.830088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.830436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.830764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.831107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.831441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.831799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.832131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.853222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.854793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.855072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.855962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.857640 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.857945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.859864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.860097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.860344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.862702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.862996 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.863262 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.869917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.870371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.872380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.872885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.875801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.876110 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.876379 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.876663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.876944 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.877233 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.877506 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.877798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.877916 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.878247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.878534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.878822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.879100 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.879376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.879658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.880002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.894352 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.894672 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.896453 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.896745 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.898721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.900788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.901962 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.902295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.903995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.904292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.906799 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.908782 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.910029 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.912088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.913375 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.913679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.913970 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.914289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.914570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.914868 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.915369 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.915666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.915950 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.916259 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.916539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.916822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.931411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.931742 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.931975 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.932218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.933184 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.933487 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.933744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.934011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.939291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.940873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.941087 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.941240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.942642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.942896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.944560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.944907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.945145 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.945407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.945672 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.946085 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.946370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.946664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.946957 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.947249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.957364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.957646 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.957957 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.958270 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.958555 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.958835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.959127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.959418 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.959755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.960128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.960569 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.960891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.961210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.961545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.975262 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.977682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.978323 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.980778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.981325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.983811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.984110 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.985747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.986032 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.986432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.986664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.988184 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.988481 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.988798 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.993101 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.994771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.995448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.997068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:23.997735 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.998025 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.998343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:23.998626 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.000057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.000337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.000626 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.000905 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.006953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:24.007243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.007536 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.009247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:24.009541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.009819 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.014549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=207770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:24.014824 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.015108 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.015388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.015676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.015961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.016238 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.016513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.016913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.017197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.017479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.017750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.018032 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.018303 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=207770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.018583 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=207769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:25:24.018982 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.019381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.019771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.020181 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.020650 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.021053 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.021440 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.021974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.022406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.022812 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.023209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.023596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.023980 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=207769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:25:24.052569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55aff25cb9c0 pid: 207770 (default app) Jul 28 08:25:24.056243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1da6f-f533-4013-973c-e183ac5fcd49 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:24.056529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1da6f-f533-4013-973c-e183ac5fcd49 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:24.056810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1da6f-f533-4013-973c-e183ac5fcd49 None None] PATH_INFO: `/v3` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:24.058211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55aff25cb9c0 pid: 207769 (default app) Jul 28 08:25:24.061324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jul 28 08:25:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:31.826863 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd3097-dfc7-412d-88f0-4af408fd5b05 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:31.827475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd3097-dfc7-412d-88f0-4af408fd5b05 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:31.827475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd3097-dfc7-412d-88f0-4af408fd5b05 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:31.831916 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:31.838102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:31.838665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:31.838665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:31.915349 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: INFO dbcounter [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] Registered counter for database keystone Jul 28 08:25:31.938200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG oslo_db.sqlalchemy.engines [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=207770) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:25:31.946120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writer thread running {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:25:32.127888 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG passlib.handlers.bcrypt [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=207770) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:25:32.128316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG passlib.handlers.bcrypt [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=207770) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:25:32.183770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:32.185469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:32.333311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:25:32.357356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd3097-dfc7-412d-88f0-4af408fd5b05 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:32.427596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: INFO dbcounter [None req-f0fd3097-dfc7-412d-88f0-4af408fd5b05 None None] Registered counter for database keystone Jul 28 08:25:32.447940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG oslo_db.sqlalchemy.engines [None req-f0fd3097-dfc7-412d-88f0-4af408fd5b05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=207769) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:25:32.456186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writer thread running {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:25:32.583085 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd3097-dfc7-412d-88f0-4af408fd5b05 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:32.587829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d4c2f-9138-4e85-9988-b7650c1595fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52cdb50da43d48649726866000557d76', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:32.588469 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d4c2f-9138-4e85-9988-b7650c1595fc admin admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:32.588778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d4c2f-9138-4e85-9988-b7650c1595fc admin admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:32.589058 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d4c2f-9138-4e85-9988-b7650c1595fc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:32.598258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d4c2f-9138-4e85-9988-b7650c1595fc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:32.650709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: WARNING keystone.common.rbac_enforcer.enforcer [None req-420d4c2f-9138-4e85-9988-b7650c1595fc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:25:32.651051 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d4c2f-9138-4e85-9988-b7650c1595fc admin admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:32.661921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:25:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:25:32.668399 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:32.684064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8996d8b-1828-4d3c-a408-361e4b107cce None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:32.689640 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063cb3d-875b-44dc-b3df-2b7a1e967eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52cdb50da43d48649726866000557d76', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:32.690852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cb3d-875b-44dc-b3df-2b7a1e967eca admin admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:32.691141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cb3d-875b-44dc-b3df-2b7a1e967eca admin admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:32.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cb3d-875b-44dc-b3df-2b7a1e967eca admin admin] PATH_INFO: `/v3/projects/52cdb50da43d48649726866000557d76` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:32.693550 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063cb3d-875b-44dc-b3df-2b7a1e967eca admin admin] RBAC: Authorizing `identity:get_project(project_id=52cdb50da43d48649726866000557d76)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:32.744391 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7063cb3d-875b-44dc-b3df-2b7a1e967eca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:25:32.744792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063cb3d-875b-44dc-b3df-2b7a1e967eca admin admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:32.749756 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jul 28 08:25:32 2022] GET /identity/v3/projects/52cdb50da43d48649726866000557d76 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:34.759502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c3a4c-513b-47db-9a97-348fd23a7e2d None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:34.760101 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c3a4c-513b-47db-9a97-348fd23a7e2d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:34.760101 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c3a4c-513b-47db-9a97-348fd23a7e2d None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:34.764296 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:34.770678 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dd34d-5a50-48f9-99e2-41b5953fec4c None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:34.771166 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dd34d-5a50-48f9-99e2-41b5953fec4c None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:34.771166 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dd34d-5a50-48f9-99e2-41b5953fec4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:34.793410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-ac5dd34d-5a50-48f9-99e2-41b5953fec4c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:34.795049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-ac5dd34d-5a50-48f9-99e2-41b5953fec4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:34.804930 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:25:34.827645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c3a4c-513b-47db-9a97-348fd23a7e2d None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:34.842287 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c3a4c-513b-47db-9a97-348fd23a7e2d None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:34.847381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf76a4-d1ab-4f40-8b95-eda93629ece2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52cdb50da43d48649726866000557d76', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:34.848045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf76a4-d1ab-4f40-8b95-eda93629ece2 admin admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:34.848335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf76a4-d1ab-4f40-8b95-eda93629ece2 admin admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:34.848614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf76a4-d1ab-4f40-8b95-eda93629ece2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:34.857045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf76a4-d1ab-4f40-8b95-eda93629ece2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:34.858874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf76a4-d1ab-4f40-8b95-eda93629ece2 admin admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:34.868386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:25:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:25:34.874377 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dd34d-5a50-48f9-99e2-41b5953fec4c None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:34.884491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dd34d-5a50-48f9-99e2-41b5953fec4c None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:34.890127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3e07b-5742-4551-89ab-bf0703d18ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52cdb50da43d48649726866000557d76', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:34.890797 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3e07b-5742-4551-89ab-bf0703d18ccb admin admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:34.891083 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3e07b-5742-4551-89ab-bf0703d18ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:34.891364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3e07b-5742-4551-89ab-bf0703d18ccb admin admin] PATH_INFO: `/v3/users/5a8bf4bf753c41b2a872093710161103` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:34.892928 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3e07b-5742-4551-89ab-bf0703d18ccb admin admin] RBAC: Authorizing `identity:get_user(user_id=5a8bf4bf753c41b2a872093710161103)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:34.894852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3e07b-5742-4551-89ab-bf0703d18ccb admin admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:34.896479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jul 28 08:25:34 2022] GET /identity/v3/users/5a8bf4bf753c41b2a872093710161103 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:36.958153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f401c-3ce6-4d66-926a-aa665eabe401 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:36.958727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f401c-3ce6-4d66-926a-aa665eabe401 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:36.958727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f401c-3ce6-4d66-926a-aa665eabe401 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:36.960198 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:25:36.966304 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe59ff-ab79-4235-81b5-43abd60effad None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:36.966833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe59ff-ab79-4235-81b5-43abd60effad None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:36.966944 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe59ff-ab79-4235-81b5-43abd60effad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:36.990392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-1ebe59ff-ab79-4235-81b5-43abd60effad None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:36.991639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-1ebe59ff-ab79-4235-81b5-43abd60effad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:37.021737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e850f-86ac-4ec5-a7f3-e35036018ed7 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.022727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e850f-86ac-4ec5-a7f3-e35036018ed7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.023014 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e850f-86ac-4ec5-a7f3-e35036018ed7 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.024372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:37.029438 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.029720 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.030037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.073375 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG passlib.handlers.bcrypt [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=207769) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:25:37.073940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG passlib.handlers.bcrypt [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=207769) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:25:37.081291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:37.103811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe59ff-ab79-4235-81b5-43abd60effad None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:37.119413 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:37.121073 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:37.128502 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:37.215341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe59ff-ab79-4235-81b5-43abd60effad None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:37.225240 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:25:37.247020 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:37.251242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f271b2c-5142-4632-af21-9ae71501b202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:37.251909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f271b2c-5142-4632-af21-9ae71501b202 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.252192 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f271b2c-5142-4632-af21-9ae71501b202 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.252467 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f271b2c-5142-4632-af21-9ae71501b202 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.264199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f271b2c-5142-4632-af21-9ae71501b202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-1f271b2c-5142-4632-af21-9ae71501b202 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:37.273240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:37.274363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:25:37.275371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:25:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:25:37.280525 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f271b2c-5142-4632-af21-9ae71501b202 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:37.352025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61929a3f-388d-4fb8-9fdd-b77b87c03c0e None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:37.354566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:37.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:37.386146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f271b2c-5142-4632-af21-9ae71501b202 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:37.396072 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d518a87-7c2d-4612-be7f-a23c5e804394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:37.396967 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d518a87-7c2d-4612-be7f-a23c5e804394 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.397256 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d518a87-7c2d-4612-be7f-a23c5e804394 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.397519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d518a87-7c2d-4612-be7f-a23c5e804394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.419972 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:37.420598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.420877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.421151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.421876 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:37.422208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:37.423623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:37.436707 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:25:37.442569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:37.468119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d518a87-7c2d-4612-be7f-a23c5e804394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:37.469636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d518a87-7c2d-4612-be7f-a23c5e804394 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:37.587758 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:25:37.593854 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d518a87-7c2d-4612-be7f-a23c5e804394 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:37.597772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:37.660823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad52ce-ce17-4e57-a727-9644882f3d45 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:37.696116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f334b4a-1359-4ee7-8c99-21114f127b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:37.696744 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f334b4a-1359-4ee7-8c99-21114f127b00 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.697024 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f334b4a-1359-4ee7-8c99-21114f127b00 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.697301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f334b4a-1359-4ee7-8c99-21114f127b00 None admin] PATH_INFO: `/v3/users/5a8bf4bf753c41b2a872093710161103` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.698672 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f334b4a-1359-4ee7-8c99-21114f127b00 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8bf4bf753c41b2a872093710161103)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:37.700175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f334b4a-1359-4ee7-8c99-21114f127b00 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:37.701911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:37 2022] GET /identity/v3/users/5a8bf4bf753c41b2a872093710161103 => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:25:37.702611 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d518a87-7c2d-4612-be7f-a23c5e804394 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:37.707518 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f334b4a-1359-4ee7-8c99-21114f127b00 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:37.723158 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:37.743586 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:37.744478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.744762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.745037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.763800 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:37.765302 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:25:37.784172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:37.785232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:37.786460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:25:37.786460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:25:37.786460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:25:37.786460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:25:37.786460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:25:37.791529 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:37.814029 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f334b4a-1359-4ee7-8c99-21114f127b00 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:37.848162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb9e8a-12c9-49f0-b100-20cd2cd6e37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:37.848849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb9e8a-12c9-49f0-b100-20cd2cd6e37a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.849131 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb9e8a-12c9-49f0-b100-20cd2cd6e37a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.849398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb9e8a-12c9-49f0-b100-20cd2cd6e37a None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.849708 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:37.850609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb9e8a-12c9-49f0-b100-20cd2cd6e37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:37.852251 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb9e8a-12c9-49f0-b100-20cd2cd6e37a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:37.853953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:37.860367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb9e8a-12c9-49f0-b100-20cd2cd6e37a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:37.900959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf660a-67e5-4db1-9d01-eca3f3feab7b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:37.936801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c40579-55e9-4819-a06b-3cc0be1893be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:37.937430 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40579-55e9-4819-a06b-3cc0be1893be None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:37.937717 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40579-55e9-4819-a06b-3cc0be1893be None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:37.937995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40579-55e9-4819-a06b-3cc0be1893be None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:37.938756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c40579-55e9-4819-a06b-3cc0be1893be None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:37.939083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c40579-55e9-4819-a06b-3cc0be1893be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:37.940508 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c40579-55e9-4819-a06b-3cc0be1893be None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:37.942841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:37.959309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:25:37.963465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb9e8a-12c9-49f0-b100-20cd2cd6e37a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:38.003782 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e766-4c5b-4bf4-931e-95cfa6a77e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:38.004471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e766-4c5b-4bf4-931e-95cfa6a77e5e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:38.004703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e766-4c5b-4bf4-931e-95cfa6a77e5e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:38.004941 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e766-4c5b-4bf4-931e-95cfa6a77e5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:38.005743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4e766-4c5b-4bf4-931e-95cfa6a77e5e None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.domain.id=default, user.id=5a8bf4bf753c41b2a872093710161103) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:38.006055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4e766-4c5b-4bf4-931e-95cfa6a77e5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:38.007597 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4e766-4c5b-4bf4-931e-95cfa6a77e5e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:38.015742 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:25:37 2022] GET /identity/v3/role_assignments?user.id=5a8bf4bf753c41b2a872093710161103&scope.domain.id=default&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:40.383124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c79f-6b08-4e4b-bd4f-cc997f5da484 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.384206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c79f-6b08-4e4b-bd4f-cc997f5da484 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.384729 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c79f-6b08-4e4b-bd4f-cc997f5da484 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.386573 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:40.392293 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3646-6bfa-4102-89f5-f4356f8c961d None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.392770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3646-6bfa-4102-89f5-f4356f8c961d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.392863 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3646-6bfa-4102-89f5-f4356f8c961d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.423496 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-228a3646-6bfa-4102-89f5-f4356f8c961d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:40.424730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-228a3646-6bfa-4102-89f5-f4356f8c961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:40.486527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3226f6-f6a7-4711-af0b-6fa19226dad8 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.487202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3226f6-f6a7-4711-af0b-6fa19226dad8 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.487694 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3226f6-f6a7-4711-af0b-6fa19226dad8 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.489582 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:40.495330 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4db60-f772-4829-9f65-6f037ed2f3d3 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.495938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4db60-f772-4829-9f65-6f037ed2f3d3 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.496450 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4db60-f772-4829-9f65-6f037ed2f3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.538559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-39e4db60-f772-4829-9f65-6f037ed2f3d3 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:40.539812 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-39e4db60-f772-4829-9f65-6f037ed2f3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:40.551504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:40.595991 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53bd84-b508-4a37-8644-ba8fb28ef52d None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.596364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53bd84-b508-4a37-8644-ba8fb28ef52d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.596549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53bd84-b508-4a37-8644-ba8fb28ef52d None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.598095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 13/25] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:25:40.599221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53bd84-b508-4a37-8644-ba8fb28ef52d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:40.702969 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 13/26] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:25:40.706443 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbd21-3094-4dfb-9a95-32fec07ef719 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.706913 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbd21-3094-4dfb-9a95-32fec07ef719 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.708169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbd21-3094-4dfb-9a95-32fec07ef719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.738955 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-7bbbbd21-3094-4dfb-9a95-32fec07ef719 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:40.740870 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-7bbbbd21-3094-4dfb-9a95-32fec07ef719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:40.742080 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:40.774027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53bd84-b508-4a37-8644-ba8fb28ef52d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:40.796115 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:40.834306 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 14/27] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:40.835956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717213-c648-47c5-9512-bae68ab53ce5 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.836459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717213-c648-47c5-9512-bae68ab53ce5 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.836948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717213-c648-47c5-9512-bae68ab53ce5 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.837824 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca970e-578a-43bf-b3b1-ae3f4c74327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:40.838650 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 15/28] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:40.838792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca970e-578a-43bf-b3b1-ae3f4c74327d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.839141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca970e-578a-43bf-b3b1-ae3f4c74327d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.839637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717213-c648-47c5-9512-bae68ab53ce5 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:40.839762 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca970e-578a-43bf-b3b1-ae3f4c74327d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.864154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca970e-578a-43bf-b3b1-ae3f4c74327d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-60ca970e-578a-43bf-b3b1-ae3f4c74327d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:40.871622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:40.872920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:40.874049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:25:40.876387 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 14/29] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:25:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:25:40.878761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84fb5e-4596-4482-bbad-b96576940b6a None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:40.879241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84fb5e-4596-4482-bbad-b96576940b6a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:40.879762 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84fb5e-4596-4482-bbad-b96576940b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:40.900169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-7c84fb5e-4596-4482-bbad-b96576940b6a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:40.901550 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-7c84fb5e-4596-4482-bbad-b96576940b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:40.989874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717213-c648-47c5-9512-bae68ab53ce5 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:40.994409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 15/30] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:40.995487 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84fb5e-4596-4482-bbad-b96576940b6a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:41.026870 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:41.027929 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.028412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.028917 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.040528 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:41.042367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:25:41.059631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:41.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:41.061633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:25:41.061633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:25:41.061633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:25:41.061633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:25:41.063767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 16/31] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:41.065011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:41.095337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:41.143455 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84fb5e-4596-4482-bbad-b96576940b6a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:41.187680 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b754600-3d30-4fd3-a509-1969935bdb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:41.188338 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b754600-3d30-4fd3-a509-1969935bdb68 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.188683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b754600-3d30-4fd3-a509-1969935bdb68 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.188964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b754600-3d30-4fd3-a509-1969935bdb68 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.189851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b754600-3d30-4fd3-a509-1969935bdb68 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:41.191465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b754600-3d30-4fd3-a509-1969935bdb68 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:41.211824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:41.287762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95187-b414-4275-a0ca-4b1b9f8e16a5 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:41.324335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:41.325348 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.325854 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.326389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.327318 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:41.327826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:41.329398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:41.337098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 16/32] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:25:40 2022] POST /identity/v3/roles => generated 219 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:25:41.338473 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b754600-3d30-4fd3-a509-1969935bdb68 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:41.344198 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 17/33] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:41.350016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:41.451267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:41.462780 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd44816-43a4-4732-823f-b3737c3ec56d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:41.501519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:41.502645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.503167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.503654 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.504612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:41.505140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:41.506877 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:41.526132 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 18/34] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:41.527305 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:41.572433 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b754600-3d30-4fd3-a509-1969935bdb68 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:41.630864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:41.650621 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874af82d-2711-41a5-9161-ab13e3dbb44a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:41.663514 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305bcf-b7e5-4b63-b466-72caa43630d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:41.664276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305bcf-b7e5-4b63-b466-72caa43630d6 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.664609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305bcf-b7e5-4b63-b466-72caa43630d6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.664891 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305bcf-b7e5-4b63-b466-72caa43630d6 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.665638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305bcf-b7e5-4b63-b466-72caa43630d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:41.668427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305bcf-b7e5-4b63-b466-72caa43630d6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:41.688411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 17/35] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:25:41 2022] POST /identity/v3/roles => generated 225 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:25:41.690025 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305bcf-b7e5-4b63-b466-72caa43630d6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:41.695251 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6de56-49de-4e30-904f-727bff462bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:41.696423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6de56-49de-4e30-904f-727bff462bfd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.696919 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6de56-49de-4e30-904f-727bff462bfd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.697403 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6de56-49de-4e30-904f-727bff462bfd None admin] PATH_INFO: `/v3/users/5a8bf4bf753c41b2a872093710161103` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.700064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6de56-49de-4e30-904f-727bff462bfd None admin] RBAC: Authorizing `identity:get_user(user_id=5a8bf4bf753c41b2a872093710161103)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:41.702051 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6de56-49de-4e30-904f-727bff462bfd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:41.704677 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 19/36] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:41 2022] GET /identity/v3/users/5a8bf4bf753c41b2a872093710161103 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:41.711712 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6de56-49de-4e30-904f-727bff462bfd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:41.751521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:41.851056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305bcf-b7e5-4b63-b466-72caa43630d6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:41.863396 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6de56-49de-4e30-904f-727bff462bfd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:41.895274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84c350-ab26-4792-97a0-68e81779d495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:41.896220 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84c350-ab26-4792-97a0-68e81779d495 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.896681 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84c350-ab26-4792-97a0-68e81779d495 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.897117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84c350-ab26-4792-97a0-68e81779d495 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.898009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84c350-ab26-4792-97a0-68e81779d495 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:41.899791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84c350-ab26-4792-97a0-68e81779d495 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:41.905228 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623e170-b130-4e0d-a1fd-2dadbdbfd3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:41.906032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623e170-b130-4e0d-a1fd-2dadbdbfd3de None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.906389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623e170-b130-4e0d-a1fd-2dadbdbfd3de None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.906670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623e170-b130-4e0d-a1fd-2dadbdbfd3de None admin] PATH_INFO: `/v3/domains/default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.908189 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623e170-b130-4e0d-a1fd-2dadbdbfd3de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:41.909756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623e170-b130-4e0d-a1fd-2dadbdbfd3de None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:41.911710 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 20/37] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:25:41.913128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc298-c706-4f7f-a45c-d5f205175f66 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.913612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc298-c706-4f7f-a45c-d5f205175f66 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.914137 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc298-c706-4f7f-a45c-d5f205175f66 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.915813 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 21/38] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:41.919377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdc298-c706-4f7f-a45c-d5f205175f66 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:41.940173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 18/39] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:25:41 2022] POST /identity/v3/projects => generated 301 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:25:41.941689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce325-be16-4f80-a29e-014ce5fcd031 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:41.942373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce325-be16-4f80-a29e-014ce5fcd031 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:41.942818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce325-be16-4f80-a29e-014ce5fcd031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:41.975362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-1bfce325-be16-4f80-a29e-014ce5fcd031 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:41.977013 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-1bfce325-be16-4f80-a29e-014ce5fcd031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:42.023419 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:42.057285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:42.078476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 19/40] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:42.086934 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdc298-c706-4f7f-a45c-d5f205175f66 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:42.104728 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfce325-be16-4f80-a29e-014ce5fcd031 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:42.135943 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707166c-ead8-42b0-8820-d0e6f60de182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:42.136662 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707166c-ead8-42b0-8820-d0e6f60de182 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:42.136944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707166c-ead8-42b0-8820-d0e6f60de182 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:42.137221 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707166c-ead8-42b0-8820-d0e6f60de182 None admin] PATH_INFO: `/v3/domains/default/users/5a8bf4bf753c41b2a872093710161103/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:42.143653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707166c-ead8-42b0-8820-d0e6f60de182 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5a8bf4bf753c41b2a872093710161103, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:42.145316 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707166c-ead8-42b0-8820-d0e6f60de182 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:42.243742 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfce325-be16-4f80-a29e-014ce5fcd031 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:42.279036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e5b19-ae81-452c-b1e6-0ea72c4ed5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:42.279667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5b19-ae81-452c-b1e6-0ea72c4ed5b2 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:42.279947 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5b19-ae81-452c-b1e6-0ea72c4ed5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:42.280220 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5b19-ae81-452c-b1e6-0ea72c4ed5b2 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:42.280955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e5b19-ae81-452c-b1e6-0ea72c4ed5b2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:42.282400 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e5b19-ae81-452c-b1e6-0ea72c4ed5b2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:42.286064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 22/41] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jul 28 08:25:41 2022] PUT /identity/v3/domains/default/users/5a8bf4bf753c41b2a872093710161103/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:25:42.287965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b4a0-ed9d-4ecb-9c7a-207dca9e4c09 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:42.288263 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b4a0-ed9d-4ecb-9c7a-207dca9e4c09 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:42.288546 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b4a0-ed9d-4ecb-9c7a-207dca9e4c09 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:42.293134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 23/42] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:42.300033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e4c10-d181-4d7b-9150-5bfccca8ca98 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:42.300392 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e4c10-d181-4d7b-9150-5bfccca8ca98 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:42.300652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e4c10-d181-4d7b-9150-5bfccca8ca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:42.306720 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 20/43] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:25:42 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:25:42.330703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-261e4c10-d181-4d7b-9150-5bfccca8ca98 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:42.332477 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-261e4c10-d181-4d7b-9150-5bfccca8ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:42.438151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 24/44] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:42.454294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e5b19-ae81-452c-b1e6-0ea72c4ed5b2 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:42.490831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:42.575950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e5b19-ae81-452c-b1e6-0ea72c4ed5b2 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:42.611079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52eb58-d7a6-4ba1-8120-2223bf779160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:42.611757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52eb58-d7a6-4ba1-8120-2223bf779160 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:42.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52eb58-d7a6-4ba1-8120-2223bf779160 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:42.612327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52eb58-d7a6-4ba1-8120-2223bf779160 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:42.613533 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52eb58-d7a6-4ba1-8120-2223bf779160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:42.615123 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52eb58-d7a6-4ba1-8120-2223bf779160 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:42.616762 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 21/45] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:42.622849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e4c10-d181-4d7b-9150-5bfccca8ca98 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:42.714560 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:42.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e4c10-d181-4d7b-9150-5bfccca8ca98 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:42.782604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c65ff-ccff-4b8e-a70b-519ed25f7f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:42.783283 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c65ff-ccff-4b8e-a70b-519ed25f7f10 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:42.783573 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c65ff-ccff-4b8e-a70b-519ed25f7f10 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:42.783854 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c65ff-ccff-4b8e-a70b-519ed25f7f10 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:42.784574 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c65ff-ccff-4b8e-a70b-519ed25f7f10 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:42.786248 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c65ff-ccff-4b8e-a70b-519ed25f7f10 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:42.819908 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 25/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:25:42 2022] POST /identity/v3/projects => generated 312 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:25:44.325583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7505-85fe-40de-9151-b023c2185100 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:44.326218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7505-85fe-40de-9151-b023c2185100 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:44.326218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7505-85fe-40de-9151-b023c2185100 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:44.327547 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 22/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:44.333183 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911330d-9787-4f93-a87f-574029d669a6 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:44.333643 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911330d-9787-4f93-a87f-574029d669a6 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:44.333749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911330d-9787-4f93-a87f-574029d669a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:44.354444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-0911330d-9787-4f93-a87f-574029d669a6 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:44.355713 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-0911330d-9787-4f93-a87f-574029d669a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:44.431790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 26/48] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:44.455194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e7505-85fe-40de-9151-b023c2185100 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:44.558431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:44.567354 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e7505-85fe-40de-9151-b023c2185100 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:44.605114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39336e-5448-4214-a2d1-8afed922a59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:44.605680 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39336e-5448-4214-a2d1-8afed922a59d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:44.605972 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39336e-5448-4214-a2d1-8afed922a59d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:44.606285 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39336e-5448-4214-a2d1-8afed922a59d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:44.617434 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39336e-5448-4214-a2d1-8afed922a59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-7d39336e-5448-4214-a2d1-8afed922a59d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:44.622548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:44.623796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:25:44.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 23/49] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:25:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:44.632033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911330d-9787-4f93-a87f-574029d669a6 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:44.693657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:44.739802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911330d-9787-4f93-a87f-574029d669a6 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:44.776827 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ddfe1-2c65-4630-9dad-6d768213ae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:44.777485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddfe1-2c65-4630-9dad-6d768213ae21 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:44.777778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddfe1-2c65-4630-9dad-6d768213ae21 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:44.778055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddfe1-2c65-4630-9dad-6d768213ae21 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:44.778848 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ddfe1-2c65-4630-9dad-6d768213ae21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:44.779156 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ddfe1-2c65-4630-9dad-6d768213ae21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:44.780595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ddfe1-2c65-4630-9dad-6d768213ae21 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:44.789929 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 27/50] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:44.795710 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39336e-5448-4214-a2d1-8afed922a59d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:44.848766 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22dd9e-dcdb-4b8f-ba55-0571375e9f30 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:44.849178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22dd9e-dcdb-4b8f-ba55-0571375e9f30 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:44.849595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22dd9e-dcdb-4b8f-ba55-0571375e9f30 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:44.851108 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 28/51] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:44.856615 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a59f-094e-40d5-aa08-d8e584a44945 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:44.856841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a59f-094e-40d5-aa08-d8e584a44945 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:44.857119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a59f-094e-40d5-aa08-d8e584a44945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:44.876577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-5367a59f-094e-40d5-aa08-d8e584a44945 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:44.877811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-5367a59f-094e-40d5-aa08-d8e584a44945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:44.904545 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39336e-5448-4214-a2d1-8afed922a59d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:44.939766 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56171f-8f1e-4c6a-8343-4684d416620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:44.940426 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56171f-8f1e-4c6a-8343-4684d416620b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:44.940714 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56171f-8f1e-4c6a-8343-4684d416620b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:44.941020 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56171f-8f1e-4c6a-8343-4684d416620b None admin] PATH_INFO: `/v3/users/5a8bf4bf753c41b2a872093710161103` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:44.942370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56171f-8f1e-4c6a-8343-4684d416620b None admin] RBAC: Authorizing `identity:get_user(user_id=5a8bf4bf753c41b2a872093710161103)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:44.944231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56171f-8f1e-4c6a-8343-4684d416620b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:44.945883 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 24/52] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:44 2022] GET /identity/v3/users/5a8bf4bf753c41b2a872093710161103 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:44.951771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56171f-8f1e-4c6a-8343-4684d416620b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:44.954140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 29/53] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:44.977351 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367a59f-094e-40d5-aa08-d8e584a44945 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:44.978936 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:45.033090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:45.059486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56171f-8f1e-4c6a-8343-4684d416620b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:45.086932 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367a59f-094e-40d5-aa08-d8e584a44945 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:45.094753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f4019-1b35-4d42-b718-0c9b4a4ff2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:45.095427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f4019-1b35-4d42-b718-0c9b4a4ff2d0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:45.095719 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f4019-1b35-4d42-b718-0c9b4a4ff2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:45.096003 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f4019-1b35-4d42-b718-0c9b4a4ff2d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:45.097249 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f4019-1b35-4d42-b718-0c9b4a4ff2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:45.098821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f4019-1b35-4d42-b718-0c9b4a4ff2d0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:45.100471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 25/54] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:25:45.107492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f4019-1b35-4d42-b718-0c9b4a4ff2d0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:45.123126 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73067ab-5fb3-4e76-9e8d-39f96abe6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:45.123790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73067ab-5fb3-4e76-9e8d-39f96abe6c3d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:45.124083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73067ab-5fb3-4e76-9e8d-39f96abe6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:45.124365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73067ab-5fb3-4e76-9e8d-39f96abe6c3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:45.125605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73067ab-5fb3-4e76-9e8d-39f96abe6c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:45.127131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73067ab-5fb3-4e76-9e8d-39f96abe6c3d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:45.128779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 30/55] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:25:45.134774 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73067ab-5fb3-4e76-9e8d-39f96abe6c3d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:45.201592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:45.212593 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f4019-1b35-4d42-b718-0c9b4a4ff2d0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:45.242814 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73067ab-5fb3-4e76-9e8d-39f96abe6c3d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:45.250316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d6074-2b2b-4e01-838a-021f9d92821f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:45.250963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6074-2b2b-4e01-838a-021f9d92821f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:45.251250 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6074-2b2b-4e01-838a-021f9d92821f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:45.251540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6074-2b2b-4e01-838a-021f9d92821f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:45.252340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d6074-2b2b-4e01-838a-021f9d92821f None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.domain.id=default, user.id=5a8bf4bf753c41b2a872093710161103) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:45.252665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d6074-2b2b-4e01-838a-021f9d92821f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:45.254156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d6074-2b2b-4e01-838a-021f9d92821f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:45.260666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 26/56] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:25:45 2022] GET /identity/v3/role_assignments?user.id=5a8bf4bf753c41b2a872093710161103&scope.domain.id=default&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:45.261141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:45.286855 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190d781-7700-44af-bb2a-15d602ebfd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:45.287525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190d781-7700-44af-bb2a-15d602ebfd35 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:45.287818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190d781-7700-44af-bb2a-15d602ebfd35 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:45.288102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190d781-7700-44af-bb2a-15d602ebfd35 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:45.288838 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190d781-7700-44af-bb2a-15d602ebfd35 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:45.290441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190d781-7700-44af-bb2a-15d602ebfd35 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:45.330128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 31/57] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:25:45 2022] POST /identity/v3/projects => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:25:47.381716 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176b1a2-389c-4c22-a668-a268c872b630 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:47.382390 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176b1a2-389c-4c22-a668-a268c872b630 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:47.382390 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176b1a2-389c-4c22-a668-a268c872b630 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:47.383692 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 27/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:47.389150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00adca3-cb35-45e9-b0dc-c0fafafc7968 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:47.389607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00adca3-cb35-45e9-b0dc-c0fafafc7968 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:47.389711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00adca3-cb35-45e9-b0dc-c0fafafc7968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:47.409481 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-b00adca3-cb35-45e9-b0dc-c0fafafc7968 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:47.410772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-b00adca3-cb35-45e9-b0dc-c0fafafc7968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:47.486906 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 32/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:47.510871 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176b1a2-389c-4c22-a668-a268c872b630 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:47.619256 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176b1a2-389c-4c22-a668-a268c872b630 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:47.654373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a82ed5-8846-4c2a-8e81-358b12595655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:47.654950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82ed5-8846-4c2a-8e81-358b12595655 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:47.655234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82ed5-8846-4c2a-8e81-358b12595655 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:47.655569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82ed5-8846-4c2a-8e81-358b12595655 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:47.656742 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a82ed5-8846-4c2a-8e81-358b12595655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:47.658257 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a82ed5-8846-4c2a-8e81-358b12595655 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:47.659894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 28/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:47.666084 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00adca3-cb35-45e9-b0dc-c0fafafc7968 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:47.775036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:47.778767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00adca3-cb35-45e9-b0dc-c0fafafc7968 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:47.816664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8fe83-812c-4835-b072-f8edc6457915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:47.817409 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8fe83-812c-4835-b072-f8edc6457915 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:47.817700 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8fe83-812c-4835-b072-f8edc6457915 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:47.817978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8fe83-812c-4835-b072-f8edc6457915 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:47.819087 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8fe83-812c-4835-b072-f8edc6457915 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:47.820572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8fe83-812c-4835-b072-f8edc6457915 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:47.850305 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 33/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:25:47 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:25:51.118210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41763ee9-b3a1-4caa-bc97-921d0e58d64a None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.118789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41763ee9-b3a1-4caa-bc97-921d0e58d64a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.118789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41763ee9-b3a1-4caa-bc97-921d0e58d64a None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.120175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 29/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:51.132838 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaea9db-c8af-4fba-9b68-a95f70601432 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.134302 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaea9db-c8af-4fba-9b68-a95f70601432 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.135031 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaea9db-c8af-4fba-9b68-a95f70601432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.163254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-acaea9db-c8af-4fba-9b68-a95f70601432 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:51.164857 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-acaea9db-c8af-4fba-9b68-a95f70601432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:51.256638 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 34/63] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:51.274467 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4b369-e52f-4eb1-96f8-605b7214bd46 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.275101 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4b369-e52f-4eb1-96f8-605b7214bd46 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.275588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4b369-e52f-4eb1-96f8-605b7214bd46 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.277188 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 30/64] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:51.282913 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705922f-eb3b-486d-a95f-ff56307c21eb None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.283209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705922f-eb3b-486d-a95f-ff56307c21eb None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.283493 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705922f-eb3b-486d-a95f-ff56307c21eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.286337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4b369-e52f-4eb1-96f8-605b7214bd46 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:51.304003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-e705922f-eb3b-486d-a95f-ff56307c21eb None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:51.310129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-e705922f-eb3b-486d-a95f-ff56307c21eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:51.344899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:51.387750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 35/65] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:51.389572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576143d-ff15-42a9-8da1-aa8a455f5e0c None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.389871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576143d-ff15-42a9-8da1-aa8a455f5e0c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.390177 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576143d-ff15-42a9-8da1-aa8a455f5e0c None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.392150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 36/66] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:51.398343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f70e92-84ab-4ea1-ae88-4d42bf0e20de None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.398883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f70e92-84ab-4ea1-ae88-4d42bf0e20de None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.399380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f70e92-84ab-4ea1-ae88-4d42bf0e20de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.423650 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-d8f70e92-84ab-4ea1-ae88-4d42bf0e20de None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:51.425207 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-d8f70e92-84ab-4ea1-ae88-4d42bf0e20de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:51.438035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:51.469381 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4b369-e52f-4eb1-96f8-605b7214bd46 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:51.512616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478cd96-1b0c-4f73-a36a-4bb53ece8372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:51.513649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cd96-1b0c-4f73-a36a-4bb53ece8372 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.514156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cd96-1b0c-4f73-a36a-4bb53ece8372 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.514638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cd96-1b0c-4f73-a36a-4bb53ece8372 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.515377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 37/67] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:25:51.516482 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f70e92-84ab-4ea1-ae88-4d42bf0e20de None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:51.532303 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478cd96-1b0c-4f73-a36a-4bb53ece8372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-c478cd96-1b0c-4f73-a36a-4bb53ece8372 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:51.537879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:51.539086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:25:51.540079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:25:51.542618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 31/68] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:25:51.544503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16109198-1970-421e-a4dd-fa5082940e8e None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.545011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16109198-1970-421e-a4dd-fa5082940e8e None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.545522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16109198-1970-421e-a4dd-fa5082940e8e None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.547317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 32/69] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:51.549124 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16109198-1970-421e-a4dd-fa5082940e8e None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:51.637068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f70e92-84ab-4ea1-ae88-4d42bf0e20de None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:51.643255 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:51.663815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16109198-1970-421e-a4dd-fa5082940e8e None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:51.686657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab19e7-8846-4c52-bf1d-8e208162d1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:51.687461 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab19e7-8846-4c52-bf1d-8e208162d1b4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.687757 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab19e7-8846-4c52-bf1d-8e208162d1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.688037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab19e7-8846-4c52-bf1d-8e208162d1b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.701517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab19e7-8846-4c52-bf1d-8e208162d1b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:51.704289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:51.704997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.705290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.705575 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.706395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] RBAC: Adding query filter params (name=member) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:51.706712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:51.708177 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-f5ab19e7-8846-4c52-bf1d-8e208162d1b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:51.708669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:51.709635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 38/70] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:25:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:51.711279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5416c-c6a2-4fcf-991c-06e9863c16c7 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.712107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5416c-c6a2-4fcf-991c-06e9863c16c7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.712107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5416c-c6a2-4fcf-991c-06e9863c16c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.728656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 33/71] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:25:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:25:51.728656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:51.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:51.732836 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-89e5416c-c6a2-4fcf-991c-06e9863c16c7 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:51.734162 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-89e5416c-c6a2-4fcf-991c-06e9863c16c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:51.805789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 39/72] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:51.806702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5416c-c6a2-4fcf-991c-06e9863c16c7 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:51.839577 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eea4d0-07c3-48cd-a1d9-cc86943f1a99 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:51.877074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf87f-67c0-4881-9ebd-611e4cffb648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:51.877759 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf87f-67c0-4881-9ebd-611e4cffb648 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.878073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf87f-67c0-4881-9ebd-611e4cffb648 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.878367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf87f-67c0-4881-9ebd-611e4cffb648 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.889402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbf87f-67c0-4881-9ebd-611e4cffb648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-a3bbf87f-67c0-4881-9ebd-611e4cffb648 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:51.895712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:51.896747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:25:51.897830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 34/73] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:25:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:51.898986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf87f-67c0-4881-9ebd-611e4cffb648 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:51.903082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:51.928156 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5416c-c6a2-4fcf-991c-06e9863c16c7 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:51.964436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:51.965080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:51.965371 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:51.965655 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:51.966407 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:51.966711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:51.968167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:51.970766 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:51.977522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 40/74] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:51.978384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.008189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf87f-67c0-4881-9ebd-611e4cffb648 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.043866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ca6db-aa76-4444-9b7c-933c12b82c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.044535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ca6db-aa76-4444-9b7c-933c12b82c9a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.044822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ca6db-aa76-4444-9b7c-933c12b82c9a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.045098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ca6db-aa76-4444-9b7c-933c12b82c9a None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.059582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ca6db-aa76-4444-9b7c-933c12b82c9a None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.061097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ca6db-aa76-4444-9b7c-933c12b82c9a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.062924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 35/75] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:51 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:52.064221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a834-0221-48c9-b315-4e9a5663394a None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.064510 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a834-0221-48c9-b315-4e9a5663394a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.064785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a834-0221-48c9-b315-4e9a5663394a None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.066435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 36/76] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:52.067020 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a834-0221-48c9-b315-4e9a5663394a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.091644 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66207c7d-2a72-489c-86df-57a32815fe42 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.127577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719393bc-3052-4ec0-b6fa-0e4b590ae746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.128233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393bc-3052-4ec0-b6fa-0e4b590ae746 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.128516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393bc-3052-4ec0-b6fa-0e4b590ae746 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.128796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393bc-3052-4ec0-b6fa-0e4b590ae746 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.135064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:52.139753 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719393bc-3052-4ec0-b6fa-0e4b590ae746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-719393bc-3052-4ec0-b6fa-0e4b590ae746 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:52.145437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:52.146668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 41/77] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:52.147597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719393bc-3052-4ec0-b6fa-0e4b590ae746 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.179224 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a834-0221-48c9-b315-4e9a5663394a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.209443 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:52.213641 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.214323 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.214648 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.214923 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.215655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:52.215967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.217367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.225923 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 37/78] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:52.226786 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.254455 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719393bc-3052-4ec0-b6fa-0e4b590ae746 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.287487 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255a935-16c4-4b93-a994-7a71ca79412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.288146 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255a935-16c4-4b93-a994-7a71ca79412a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.288427 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255a935-16c4-4b93-a994-7a71ca79412a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.288951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255a935-16c4-4b93-a994-7a71ca79412a None admin] PATH_INFO: `/v3/users/5a8bf4bf753c41b2a872093710161103` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.289935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255a935-16c4-4b93-a994-7a71ca79412a None admin] RBAC: Authorizing `identity:get_user(user_id=5a8bf4bf753c41b2a872093710161103)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.291437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255a935-16c4-4b93-a994-7a71ca79412a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.293088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 42/79] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/users/5a8bf4bf753c41b2a872093710161103 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:52.298768 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79467ec4-cd6a-4d12-9a59-a89767137178 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.299052 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79467ec4-cd6a-4d12-9a59-a89767137178 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.299348 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79467ec4-cd6a-4d12-9a59-a89767137178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.317326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-79467ec4-cd6a-4d12-9a59-a89767137178 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:52.318947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-79467ec4-cd6a-4d12-9a59-a89767137178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:52.331634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe6214-db77-4027-84b4-02cdb37f0e00 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.355066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:52.366467 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac87070-d23b-407f-9324-920f3d2e6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.367119 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac87070-d23b-407f-9324-920f3d2e6f59 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.367408 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac87070-d23b-407f-9324-920f3d2e6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.367687 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac87070-d23b-407f-9324-920f3d2e6f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.387675 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 43/80] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:52.393131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79467ec4-cd6a-4d12-9a59-a89767137178 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.437454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac87070-d23b-407f-9324-920f3d2e6f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.438925 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac87070-d23b-407f-9324-920f3d2e6f59 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.461279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:52.510258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79467ec4-cd6a-4d12-9a59-a89767137178 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.546786 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 38/81] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:25:52.547657 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac87070-d23b-407f-9324-920f3d2e6f59 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.548600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.549461 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.549963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.550450 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.551487 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:52.551785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.553214 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.561999 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 44/82] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:52.562844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.574167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:52.649247 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac87070-d23b-407f-9324-920f3d2e6f59 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.667955 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c290-c136-4db9-bc05-a9880adf4ea4 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.685407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9517375-00b6-432d-9370-32a380559a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.686272 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9517375-00b6-432d-9370-32a380559a70 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.686692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:52.686841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9517375-00b6-432d-9370-32a380559a70 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.687144 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9517375-00b6-432d-9370-32a380559a70 None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.688789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9517375-00b6-432d-9370-32a380559a70 None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.690724 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9517375-00b6-432d-9370-32a380559a70 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.698355 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 39/83] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:52.698355 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9517375-00b6-432d-9370-32a380559a70 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.710689 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1addc6e-8061-43fa-9ec9-6968bc28c7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.711340 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1addc6e-8061-43fa-9ec9-6968bc28c7ef None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.711619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1addc6e-8061-43fa-9ec9-6968bc28c7ef None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.711914 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1addc6e-8061-43fa-9ec9-6968bc28c7ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.783334 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1addc6e-8061-43fa-9ec9-6968bc28c7ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.784824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1addc6e-8061-43fa-9ec9-6968bc28c7ef None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.793222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:52.802656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9517375-00b6-432d-9370-32a380559a70 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.837468 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc673c2-7f85-4071-b7a6-a1424f827dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.838159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc673c2-7f85-4071-b7a6-a1424f827dc4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.838445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc673c2-7f85-4071-b7a6-a1424f827dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.838722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc673c2-7f85-4071-b7a6-a1424f827dc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.839892 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc673c2-7f85-4071-b7a6-a1424f827dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.841402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc673c2-7f85-4071-b7a6-a1424f827dc4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.843062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 40/84] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:52.843830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc673c2-7f85-4071-b7a6-a1424f827dc4 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.893135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 45/85] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:25:52.893135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1addc6e-8061-43fa-9ec9-6968bc28c7ef None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:52.912991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:52.954140 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc673c2-7f85-4071-b7a6-a1424f827dc4 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:52.990378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6f88e-8ad2-4084-a99e-b58c54251661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:52.991039 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f88e-8ad2-4084-a99e-b58c54251661 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:52.991314 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f88e-8ad2-4084-a99e-b58c54251661 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:52.991607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f88e-8ad2-4084-a99e-b58c54251661 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:52.992822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6f88e-8ad2-4084-a99e-b58c54251661 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:52.994318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6f88e-8ad2-4084-a99e-b58c54251661 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:52.996018 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 41/86] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:52.996792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6f88e-8ad2-4084-a99e-b58c54251661 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:53.006996 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1addc6e-8061-43fa-9ec9-6968bc28c7ef None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:53.026493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:53.042156 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509514f0-5499-4730-8c72-75402a66d3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:53.042823 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509514f0-5499-4730-8c72-75402a66d3b2 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:53.043118 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509514f0-5499-4730-8c72-75402a66d3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:53.043387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509514f0-5499-4730-8c72-75402a66d3b2 None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:53.044591 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509514f0-5499-4730-8c72-75402a66d3b2 None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:53.046068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509514f0-5499-4730-8c72-75402a66d3b2 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:53.047766 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 46/87] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:53.053188 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509514f0-5499-4730-8c72-75402a66d3b2 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:53.113314 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6f88e-8ad2-4084-a99e-b58c54251661 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:53.149067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654ff1c-a273-46ce-99d5-8f1ec8229300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:53.149797 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654ff1c-a273-46ce-99d5-8f1ec8229300 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:53.149995 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654ff1c-a273-46ce-99d5-8f1ec8229300 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:53.150328 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654ff1c-a273-46ce-99d5-8f1ec8229300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:53.158197 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:53.177776 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509514f0-5499-4730-8c72-75402a66d3b2 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:53.213799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d0fb6-42e7-4ef3-ba5a-ded9140ce497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:53.214398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d0fb6-42e7-4ef3-ba5a-ded9140ce497 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:53.214678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d0fb6-42e7-4ef3-ba5a-ded9140ce497 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:53.214946 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d0fb6-42e7-4ef3-ba5a-ded9140ce497 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:53.215660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d0fb6-42e7-4ef3-ba5a-ded9140ce497 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:53.217134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d0fb6-42e7-4ef3-ba5a-ded9140ce497 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:53.222714 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654ff1c-a273-46ce-99d5-8f1ec8229300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:53.224281 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654ff1c-a273-46ce-99d5-8f1ec8229300 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:53.240527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 47/88] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:25:53 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:25:53.243541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d0fb6-42e7-4ef3-ba5a-ded9140ce497 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:53.255264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:53.343766 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 42/89] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:25:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:25:53.344666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654ff1c-a273-46ce-99d5-8f1ec8229300 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:53.386570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d0fb6-42e7-4ef3-ba5a-ded9140ce497 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:53.439154 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2ffa2-93c3-4aee-b2e7-a92b92b91037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:53.440527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2ffa2-93c3-4aee-b2e7-a92b92b91037 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:53.441005 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2ffa2-93c3-4aee-b2e7-a92b92b91037 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:53.441513 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2ffa2-93c3-4aee-b2e7-a92b92b91037 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:53.444165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2ffa2-93c3-4aee-b2e7-a92b92b91037 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:53.446704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2ffa2-93c3-4aee-b2e7-a92b92b91037 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:53.450136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 48/90] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:53 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:53.455214 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2ffa2-93c3-4aee-b2e7-a92b92b91037 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:53.469185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:53.471740 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654ff1c-a273-46ce-99d5-8f1ec8229300 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:53.508296 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:53.527703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:53.528630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:53.529229 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:53.529662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:53.530985 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] RBAC: Adding query filter params (role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=278f44bb741f4075b58db40b154bb64d, user.id=169f2ee6fbe84afe8edc653401e01357) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:53.531486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:53.533164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:53.548351 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 43/91] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:25:53 2022] GET /identity/v3/role_assignments?user.id=169f2ee6fbe84afe8edc653401e01357&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:53.549066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:53.616274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2ffa2-93c3-4aee-b2e7-a92b92b91037 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:53.664200 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26805f2-ecde-49d1-9c50-805903735a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:53.664999 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26805f2-ecde-49d1-9c50-805903735a76 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:53.665292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26805f2-ecde-49d1-9c50-805903735a76 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:53.665573 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26805f2-ecde-49d1-9c50-805903735a76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:53.712690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d134f9-c081-4024-9a3c-c8c326e17f12 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:53.781492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26805f2-ecde-49d1-9c50-805903735a76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:53.783731 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26805f2-ecde-49d1-9c50-805903735a76 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:53.790891 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb28ed-0824-47be-a396-e27df25d8d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:53.791569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb28ed-0824-47be-a396-e27df25d8d66 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:53.791856 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb28ed-0824-47be-a396-e27df25d8d66 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:53.792137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb28ed-0824-47be-a396-e27df25d8d66 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:53.793552 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb28ed-0824-47be-a396-e27df25d8d66 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:53.795149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb28ed-0824-47be-a396-e27df25d8d66 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:53.796917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 44/92] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:53 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:53.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb28ed-0824-47be-a396-e27df25d8d66 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:53.798406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:53.903656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:53.932697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 49/93] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:25:53.934380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26805f2-ecde-49d1-9c50-805903735a76 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:54.022872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb28ed-0824-47be-a396-e27df25d8d66 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:54.076471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:54.077028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:54.077312 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:54.077591 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:54.078429 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=278f44bb741f4075b58db40b154bb64d, user.id=5a8bf4bf753c41b2a872093710161103) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:54.078748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:54.080219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:54.096456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 45/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:25:53 2022] GET /identity/v3/role_assignments?user.id=5a8bf4bf753c41b2a872093710161103&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:54.097128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:54.146204 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26805f2-ecde-49d1-9c50-805903735a76 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:54.172649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:54.232606 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:54.233577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:54.234026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:54.234485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:54.235614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] RBAC: Adding query filter params (role.id=d883529ef5284c35b27b759c57353383, scope.project.id=278f44bb741f4075b58db40b154bb64d, user.id=169f2ee6fbe84afe8edc653401e01357) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:54.236182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:54.237948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:54.249769 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 50/95] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:25:53 2022] GET /identity/v3/role_assignments?user.id=169f2ee6fbe84afe8edc653401e01357&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=d883529ef5284c35b27b759c57353383 => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:54.308118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da463115-7898-4bd7-ae6f-0b8597e16d69 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:54.348085 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:54.362444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e577bec-5ebe-4d01-bcf9-62a900be486b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:54.363303 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e577bec-5ebe-4d01-bcf9-62a900be486b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:54.363792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e577bec-5ebe-4d01-bcf9-62a900be486b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:54.364162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e577bec-5ebe-4d01-bcf9-62a900be486b None admin] PATH_INFO: `/v3/projects/b9948655e8d643ab9deb3cc45ac57743` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:54.365684 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e577bec-5ebe-4d01-bcf9-62a900be486b None admin] RBAC: Authorizing `identity:get_project(project_id=b9948655e8d643ab9deb3cc45ac57743)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:54.367407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e577bec-5ebe-4d01-bcf9-62a900be486b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:54.369501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 46/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:54 2022] GET /identity/v3/projects/b9948655e8d643ab9deb3cc45ac57743 => generated 312 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:54.375911 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:54.524945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5c2e6-c1ff-4a7f-8e05-82012e77a3f9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:54.565891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a188897-0fca-42d5-86aa-35b9119e4052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:54.566999 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188897-0fca-42d5-86aa-35b9119e4052 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:54.567287 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188897-0fca-42d5-86aa-35b9119e4052 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:54.567566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188897-0fca-42d5-86aa-35b9119e4052 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:54.568758 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a188897-0fca-42d5-86aa-35b9119e4052 None admin] RBAC: Adding query filter params (role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=b9948655e8d643ab9deb3cc45ac57743, user.id=169f2ee6fbe84afe8edc653401e01357) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:54.569068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a188897-0fca-42d5-86aa-35b9119e4052 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:54.570641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a188897-0fca-42d5-86aa-35b9119e4052 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:54.577451 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 51/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:25:54 2022] GET /identity/v3/role_assignments?user.id=169f2ee6fbe84afe8edc653401e01357&scope.project.id=b9948655e8d643ab9deb3cc45ac57743&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:56.145277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394aa4-3fa9-433c-907f-6073db85dafd None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.146489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394aa4-3fa9-433c-907f-6073db85dafd None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.146994 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394aa4-3fa9-433c-907f-6073db85dafd None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.148877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 47/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:56.154468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e8cd-5d1c-4cbc-8749-dd65c9499ccc None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.155315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e8cd-5d1c-4cbc-8749-dd65c9499ccc None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.155816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e8cd-5d1c-4cbc-8749-dd65c9499ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.183210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-9fa3e8cd-5d1c-4cbc-8749-dd65c9499ccc None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:56.184483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-9fa3e8cd-5d1c-4cbc-8749-dd65c9499ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:56.206242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22bbad-a8aa-4f58-8c54-32fbef877d11 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.206896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22bbad-a8aa-4f58-8c54-32fbef877d11 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.207452 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22bbad-a8aa-4f58-8c54-32fbef877d11 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.209050 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 48/99] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:56.217302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c4644-ddf0-4102-923c-9528b8ba8ffc None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.217591 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c4644-ddf0-4102-923c-9528b8ba8ffc None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.217873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c4644-ddf0-4102-923c-9528b8ba8ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.231356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:56.252977 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-4b6c4644-ddf0-4102-923c-9528b8ba8ffc None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:56.254397 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-4b6c4644-ddf0-4102-923c-9528b8ba8ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:56.298400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 52/100] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:56.322706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e8cd-5d1c-4cbc-8749-dd65c9499ccc None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:56.372095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 49/101] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:25:56.373516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e89d6-f5ec-4227-a155-69411b8a179f None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.373814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e89d6-f5ec-4227-a155-69411b8a179f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.374094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e89d6-f5ec-4227-a155-69411b8a179f None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.375678 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 50/102] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:56.381305 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ab43f-62cb-4ee4-95ee-aff7e1bbca8b None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.381597 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ab43f-62cb-4ee4-95ee-aff7e1bbca8b None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.381886 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ab43f-62cb-4ee4-95ee-aff7e1bbca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.405421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-ac9ab43f-62cb-4ee4-95ee-aff7e1bbca8b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:56.407198 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-ac9ab43f-62cb-4ee4-95ee-aff7e1bbca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:56.469667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e8cd-5d1c-4cbc-8749-dd65c9499ccc None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:56.483084 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 51/103] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:25:56.484043 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ab43f-62cb-4ee4-95ee-aff7e1bbca8b None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:56.509380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda69eef-3d6d-4375-bca6-91a128e07bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:56.510062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda69eef-3d6d-4375-bca6-91a128e07bc7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.510406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda69eef-3d6d-4375-bca6-91a128e07bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.510691 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda69eef-3d6d-4375-bca6-91a128e07bc7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.523498 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda69eef-3d6d-4375-bca6-91a128e07bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-fda69eef-3d6d-4375-bca6-91a128e07bc7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:56.529093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:56.530568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:25:56.531664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 53/104] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:25:56.540028 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda69eef-3d6d-4375-bca6-91a128e07bc7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:56.599659 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:56.610504 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:56.684591 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ab43f-62cb-4ee4-95ee-aff7e1bbca8b None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:56.742189 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda69eef-3d6d-4375-bca6-91a128e07bc7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:56.778497 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383f7cf-2077-4adb-89e4-f57954f00d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:56.779813 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383f7cf-2077-4adb-89e4-f57954f00d1a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.781118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383f7cf-2077-4adb-89e4-f57954f00d1a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.782331 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383f7cf-2077-4adb-89e4-f57954f00d1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.796349 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:56.797393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.798560 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.798856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.799639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:56.800148 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:56.801979 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:56.806189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383f7cf-2077-4adb-89e4-f57954f00d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-e383f7cf-2077-4adb-89e4-f57954f00d1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:56.809392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:56.810666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:25:56.811815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:25:56.813415 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 52/105] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:56.814955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383f7cf-2077-4adb-89e4-f57954f00d1a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:56.828392 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 54/106] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:25:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:25:56.829619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:56.930168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383f7cf-2077-4adb-89e4-f57954f00d1a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:56.946173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:56.971207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817b2a-686d-475c-9d2f-33279c57a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:56.972245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817b2a-686d-475c-9d2f-33279c57a9eb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:56.972688 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817b2a-686d-475c-9d2f-33279c57a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:56.973244 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817b2a-686d-475c-9d2f-33279c57a9eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:56.974810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41817b2a-686d-475c-9d2f-33279c57a9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:56.976745 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41817b2a-686d-475c-9d2f-33279c57a9eb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:56.978906 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 53/107] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:56.979752 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817b2a-686d-475c-9d2f-33279c57a9eb None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:57.064206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:57.092557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817b2a-686d-475c-9d2f-33279c57a9eb None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:57.096804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daf20c-d0b7-4608-93f9-359a07420a8b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:57.128533 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696de2-5a53-45f2-8e73-6a6681fc454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:57.129410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696de2-5a53-45f2-8e73-6a6681fc454a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.129813 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696de2-5a53-45f2-8e73-6a6681fc454a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.130220 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696de2-5a53-45f2-8e73-6a6681fc454a None admin] PATH_INFO: `/v3/users/c977eb24d2534cd3bbb7ec7d6744ae2f` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.145784 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696de2-5a53-45f2-8e73-6a6681fc454a None admin] RBAC: Authorizing `identity:get_user(user_id=c977eb24d2534cd3bbb7ec7d6744ae2f)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:57.152953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696de2-5a53-45f2-8e73-6a6681fc454a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:57.152953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 54/108] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:56 2022] GET /identity/v3/users/c977eb24d2534cd3bbb7ec7d6744ae2f => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:57.152953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696de2-5a53-45f2-8e73-6a6681fc454a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:57.175444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:57.177124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.177582 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.178025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.179721 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:57.180401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:57.182416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:57.195325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 55/109] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:25:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:57.197176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:57.198015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:57.269035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696de2-5a53-45f2-8e73-6a6681fc454a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:57.320218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6183b8f-c38b-408a-b108-66fd2bb34b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:57.320872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183b8f-c38b-408a-b108-66fd2bb34b5b None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.321159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183b8f-c38b-408a-b108-66fd2bb34b5b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.321439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183b8f-c38b-408a-b108-66fd2bb34b5b None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.322176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6183b8f-c38b-408a-b108-66fd2bb34b5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:57.323675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6183b8f-c38b-408a-b108-66fd2bb34b5b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:57.343335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c9f2c-005d-4d11-a00d-daa781d4ad58 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:57.362825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 55/110] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:25:57 2022] POST /identity/v3/projects => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:25:57.362825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ec1c-32df-40a6-976f-a44e4b6f8dec None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.362825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ec1c-32df-40a6-976f-a44e4b6f8dec None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.362825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ec1c-32df-40a6-976f-a44e4b6f8dec None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.362825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 56/111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:57.363954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9ec1c-32df-40a6-976f-a44e4b6f8dec None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:57.406302 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b9111-1abc-403b-8a62-dfcd0fcbe14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:57.406965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b9111-1abc-403b-8a62-dfcd0fcbe14e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.407258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b9111-1abc-403b-8a62-dfcd0fcbe14e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.407646 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b9111-1abc-403b-8a62-dfcd0fcbe14e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.436055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:57.485611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:57.500992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9ec1c-32df-40a6-976f-a44e4b6f8dec None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:57.506583 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b9111-1abc-403b-8a62-dfcd0fcbe14e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:57.508197 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b9111-1abc-403b-8a62-dfcd0fcbe14e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:57.552073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab051f-c7bf-4b02-834e-285e2f4096f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:57.552852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab051f-c7bf-4b02-834e-285e2f4096f3 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.554167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab051f-c7bf-4b02-834e-285e2f4096f3 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.554463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab051f-c7bf-4b02-834e-285e2f4096f3 None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.555967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab051f-c7bf-4b02-834e-285e2f4096f3 None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:57.557578 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab051f-c7bf-4b02-834e-285e2f4096f3 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:57.560568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 57/112] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:57 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:57.562141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f310-495a-4e7f-893b-486b062f6fb7 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.562524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f310-495a-4e7f-893b-486b062f6fb7 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.562807 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f310-495a-4e7f-893b-486b062f6fb7 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.566193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 58/113] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:57.568095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8d7f-5bc8-45cb-ba3e-05281d4bccf8 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.569001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8d7f-5bc8-45cb-ba3e-05281d4bccf8 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.569513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8d7f-5bc8-45cb-ba3e-05281d4bccf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.595845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-f0bd8d7f-5bc8-45cb-ba3e-05281d4bccf8 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:57.597105 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-f0bd8d7f-5bc8-45cb-ba3e-05281d4bccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:57.663352 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 56/114] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:25:57.664239 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b9111-1abc-403b-8a62-dfcd0fcbe14e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:57.682494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 59/115] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:57.686139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4d963-564a-4fd6-8242-def28783652d None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.686139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4d963-564a-4fd6-8242-def28783652d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.686139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4d963-564a-4fd6-8242-def28783652d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.703162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-48f4d963-564a-4fd6-8242-def28783652d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:57.704375 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-48f4d963-564a-4fd6-8242-def28783652d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:57.745529 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:57.789726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 60/116] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:57.790945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4d963-564a-4fd6-8242-def28783652d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:57.798626 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b9111-1abc-403b-8a62-dfcd0fcbe14e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:57.811313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:57.840027 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1a848-c588-4213-bdf9-6d4830165282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:57.841254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1a848-c588-4213-bdf9-6d4830165282 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.841844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1a848-c588-4213-bdf9-6d4830165282 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.842225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1a848-c588-4213-bdf9-6d4830165282 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.843687 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f1a848-c588-4213-bdf9-6d4830165282 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:57.845293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f1a848-c588-4213-bdf9-6d4830165282 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:57.850730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 57/117] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:57 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:57.850730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1a848-c588-4213-bdf9-6d4830165282 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:57.906039 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4d963-564a-4fd6-8242-def28783652d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:57.941813 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66683539-76fa-4cc3-947c-57c76e29e805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:57.943585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66683539-76fa-4cc3-947c-57c76e29e805 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.943957 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66683539-76fa-4cc3-947c-57c76e29e805 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.944294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66683539-76fa-4cc3-947c-57c76e29e805 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.945664 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66683539-76fa-4cc3-947c-57c76e29e805 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:57.947226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66683539-76fa-4cc3-947c-57c76e29e805 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:57.955750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 61/118] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:57 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:57.955750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dc5c8-46c4-4eda-8340-5e2120819037 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:57.955750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dc5c8-46c4-4eda-8340-5e2120819037 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:57.955750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dc5c8-46c4-4eda-8340-5e2120819037 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:57.955750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 62/119] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:57.955750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dc5c8-46c4-4eda-8340-5e2120819037 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:57.980353 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1a848-c588-4213-bdf9-6d4830165282 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.002135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:58.017227 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffacac3-382b-4cba-9b8d-75550207953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.017900 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacac3-382b-4cba-9b8d-75550207953e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.018219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacac3-382b-4cba-9b8d-75550207953e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.018501 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacac3-382b-4cba-9b8d-75550207953e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.029199 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffacac3-382b-4cba-9b8d-75550207953e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-dffacac3-382b-4cba-9b8d-75550207953e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:58.034308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:58.035571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:25:58.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:25:58.040549 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 58/120] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:25:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:58.041380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffacac3-382b-4cba-9b8d-75550207953e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.057989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:58.076432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dc5c8-46c4-4eda-8340-5e2120819037 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.113666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff8422-d918-4261-9c13-01a887f84f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.114464 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff8422-d918-4261-9c13-01a887f84f2f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.114749 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff8422-d918-4261-9c13-01a887f84f2f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.115032 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff8422-d918-4261-9c13-01a887f84f2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.126044 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff8422-d918-4261-9c13-01a887f84f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-0aff8422-d918-4261-9c13-01a887f84f2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:58.131153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:58.132194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:25:58.133205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 63/121] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:25:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:58.142616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270d554-537d-4755-9512-dbd7f34dc68b None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.142902 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270d554-537d-4755-9512-dbd7f34dc68b None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.143183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270d554-537d-4755-9512-dbd7f34dc68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.147441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffacac3-382b-4cba-9b8d-75550207953e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.161491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-d270d554-537d-4755-9512-dbd7f34dc68b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:58.162760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-d270d554-537d-4755-9512-dbd7f34dc68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:58.187112 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db537b0-cbc0-436c-a9f8-0563d8407f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.188123 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db537b0-cbc0-436c-a9f8-0563d8407f9e None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.188413 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db537b0-cbc0-436c-a9f8-0563d8407f9e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.189124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db537b0-cbc0-436c-a9f8-0563d8407f9e None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d/users/169f2ee6fbe84afe8edc653401e01357/roles/cc0fab8f90a44acc90de9af43e2bd7df` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.191186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db537b0-cbc0-436c-a9f8-0563d8407f9e None admin] RBAC: Authorizing `identity:create_grant(project_id=278f44bb741f4075b58db40b154bb64d, user_id=169f2ee6fbe84afe8edc653401e01357, role_id=cc0fab8f90a44acc90de9af43e2bd7df)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:58.192757 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db537b0-cbc0-436c-a9f8-0563d8407f9e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:58.208103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 59/122] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:25:58 2022] PUT /identity/v3/projects/278f44bb741f4075b58db40b154bb64d/users/169f2ee6fbe84afe8edc653401e01357/roles/cc0fab8f90a44acc90de9af43e2bd7df => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:25:58.219141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db537b0-cbc0-436c-a9f8-0563d8407f9e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.246024 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 64/123] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:25:58.247780 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270d554-537d-4755-9512-dbd7f34dc68b None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.276118 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:58.353293 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:58.355833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db537b0-cbc0-436c-a9f8-0563d8407f9e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.382467 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270d554-537d-4755-9512-dbd7f34dc68b None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.400313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.401462 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.402071 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.402599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.403621 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] RBAC: Adding query filter params (role.id=e4367af82d094e189d2679fe6b2c6f9f, scope.project.id=278f44bb741f4075b58db40b154bb64d, user.id=c977eb24d2534cd3bbb7ec7d6744ae2f) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:58.404169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:58.405872 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:58.412416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 60/124] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:25:58 2022] GET /identity/v3/role_assignments?user.id=c977eb24d2534cd3bbb7ec7d6744ae2f&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=e4367af82d094e189d2679fe6b2c6f9f => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:58.413546 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.426126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.426816 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.427102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.427377 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.428132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:58.428513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:58.429998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:58.444788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 65/125] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:25:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:58.444788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.577736 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe059f5-739e-4f5c-a737-ab4097f8dc5e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.581695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:58.600218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34278-ea57-4477-adaa-25c5aed1b3e6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.643373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae9d0d-d251-4cb2-b1eb-9c4e71f52c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.645143 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae9d0d-d251-4cb2-b1eb-9c4e71f52c43 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.645775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae9d0d-d251-4cb2-b1eb-9c4e71f52c43 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.646360 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae9d0d-d251-4cb2-b1eb-9c4e71f52c43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.656627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.657291 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.657578 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.657806 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:58.657926 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.658789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:58.658946 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:58.660963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:58.665938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae9d0d-d251-4cb2-b1eb-9c4e71f52c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:25:58.671730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 61/126] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:58.672411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-93ae9d0d-d251-4cb2-b1eb-9c4e71f52c43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:25:58.673827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:25:58.674849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:25:58.675760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:25:58.678861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 66/127] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:25:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:25:58.680381 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae9d0d-d251-4cb2-b1eb-9c4e71f52c43 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.795356 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae9d0d-d251-4cb2-b1eb-9c4e71f52c43 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.832540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef619f40-f07e-4961-bce6-cf79b5d05e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.833583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef619f40-f07e-4961-bce6-cf79b5d05e0c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.834068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef619f40-f07e-4961-bce6-cf79b5d05e0c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.834647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef619f40-f07e-4961-bce6-cf79b5d05e0c None admin] PATH_INFO: `/v3/users/5a8bf4bf753c41b2a872093710161103` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.836193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef619f40-f07e-4961-bce6-cf79b5d05e0c None admin] RBAC: Authorizing `identity:get_user(user_id=5a8bf4bf753c41b2a872093710161103)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:58.838003 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef619f40-f07e-4961-bce6-cf79b5d05e0c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:58.839885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 67/128] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:58 2022] GET /identity/v3/users/5a8bf4bf753c41b2a872093710161103 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:58.840862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef619f40-f07e-4961-bce6-cf79b5d05e0c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.877265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48416a93-0edc-432e-8794-c6c2975b055c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:58.912649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f7b19-6833-4e31-aff9-01389b8099a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:58.913327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f7b19-6833-4e31-aff9-01389b8099a1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:58.913614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f7b19-6833-4e31-aff9-01389b8099a1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:58.913890 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f7b19-6833-4e31-aff9-01389b8099a1 None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:58.919468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f7b19-6833-4e31-aff9-01389b8099a1 None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:58.921019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f7b19-6833-4e31-aff9-01389b8099a1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:58.923455 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 62/129] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:58 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:58.924056 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f7b19-6833-4e31-aff9-01389b8099a1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:58.970590 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:59.002185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:59.036705 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef619f40-f07e-4961-bce6-cf79b5d05e0c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:59.040692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f7b19-6833-4e31-aff9-01389b8099a1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:59.079433 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb7d74-aad2-485a-9bad-eaa0c507fc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:59.080116 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7d74-aad2-485a-9bad-eaa0c507fc8e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.080410 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7d74-aad2-485a-9bad-eaa0c507fc8e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.080688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7d74-aad2-485a-9bad-eaa0c507fc8e None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.081894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb7d74-aad2-485a-9bad-eaa0c507fc8e None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:59.083776 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb7d74-aad2-485a-9bad-eaa0c507fc8e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:59.085445 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 63/130] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:58 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:59.086284 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb7d74-aad2-485a-9bad-eaa0c507fc8e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:59.119761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:59.120861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.121387 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.121931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.123079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] RBAC: Adding query filter params (name=member) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:25:59.123675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:59.125392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:59.140827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 68/131] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:25:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:59.142227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:59.191823 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb7d74-aad2-485a-9bad-eaa0c507fc8e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:59.193956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:59.228318 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f0754-058a-48e5-ae3d-e375af3e9046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:59.228908 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f0754-058a-48e5-ae3d-e375af3e9046 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.229194 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f0754-058a-48e5-ae3d-e375af3e9046 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.229467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f0754-058a-48e5-ae3d-e375af3e9046 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.230684 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f0754-058a-48e5-ae3d-e375af3e9046 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:59.232155 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f0754-058a-48e5-ae3d-e375af3e9046 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:59.233871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 64/132] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:59 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:59.234685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f0754-058a-48e5-ae3d-e375af3e9046 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:59.341112 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f0754-058a-48e5-ae3d-e375af3e9046 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:59.382617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432da917-3fdd-4f40-80e2-a03b71e34d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:59.383302 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432da917-3fdd-4f40-80e2-a03b71e34d41 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.383590 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432da917-3fdd-4f40-80e2-a03b71e34d41 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.383866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432da917-3fdd-4f40-80e2-a03b71e34d41 None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.385229 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432da917-3fdd-4f40-80e2-a03b71e34d41 None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:59.386739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432da917-3fdd-4f40-80e2-a03b71e34d41 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:59.388409 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 65/133] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:25:59 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:59.389164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432da917-3fdd-4f40-80e2-a03b71e34d41 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:59.396810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d571f-54f6-4d05-9320-345ed3f0304c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:59.442546 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:59.457986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec9102-f4d0-411e-8eeb-165917ebdf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:59.459614 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec9102-f4d0-411e-8eeb-165917ebdf2f None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.459901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec9102-f4d0-411e-8eeb-165917ebdf2f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.460179 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec9102-f4d0-411e-8eeb-165917ebdf2f None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d/users/5a8bf4bf753c41b2a872093710161103/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.463589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec9102-f4d0-411e-8eeb-165917ebdf2f None admin] RBAC: Authorizing `identity:create_grant(project_id=278f44bb741f4075b58db40b154bb64d, user_id=5a8bf4bf753c41b2a872093710161103, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:59.465085 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec9102-f4d0-411e-8eeb-165917ebdf2f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:59.478141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:59.495001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 69/134] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:25:59 2022] PUT /identity/v3/projects/278f44bb741f4075b58db40b154bb64d/users/5a8bf4bf753c41b2a872093710161103/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:25:59.495895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec9102-f4d0-411e-8eeb-165917ebdf2f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:59.635882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432da917-3fdd-4f40-80e2-a03b71e34d41 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:59.708966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5c5d5-a9e8-4497-b0bd-40306c7693d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:59.710023 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c5d5-a9e8-4497-b0bd-40306c7693d3 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.710354 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c5d5-a9e8-4497-b0bd-40306c7693d3 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.710647 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c5d5-a9e8-4497-b0bd-40306c7693d3 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d/users/169f2ee6fbe84afe8edc653401e01357/roles/d883529ef5284c35b27b759c57353383` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.715521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5c5d5-a9e8-4497-b0bd-40306c7693d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=278f44bb741f4075b58db40b154bb64d, user_id=169f2ee6fbe84afe8edc653401e01357, role_id=d883529ef5284c35b27b759c57353383)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:59.717050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5c5d5-a9e8-4497-b0bd-40306c7693d3 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:59.746592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec9102-f4d0-411e-8eeb-165917ebdf2f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:25:59.748021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 66/135] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:25:59 2022] PUT /identity/v3/projects/278f44bb741f4075b58db40b154bb64d/users/169f2ee6fbe84afe8edc653401e01357/roles/d883529ef5284c35b27b759c57353383 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:25:59.800822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfaed59-8c59-41f2-87c9-afff79a7e6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:25:59.801985 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaed59-8c59-41f2-87c9-afff79a7e6ab None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.802532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaed59-8c59-41f2-87c9-afff79a7e6ab None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.803033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaed59-8c59-41f2-87c9-afff79a7e6ab None admin] PATH_INFO: `/v3/projects/b9948655e8d643ab9deb3cc45ac57743` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.806855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfaed59-8c59-41f2-87c9-afff79a7e6ab None admin] RBAC: Authorizing `identity:get_project(project_id=b9948655e8d643ab9deb3cc45ac57743)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:25:59.808625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfaed59-8c59-41f2-87c9-afff79a7e6ab None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:25:59.810693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 70/136] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:25:59 2022] GET /identity/v3/projects/b9948655e8d643ab9deb3cc45ac57743 => generated 312 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:25:59.811965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134d45-f335-4ffa-b274-48b2c6b37546 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.812300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134d45-f335-4ffa-b274-48b2c6b37546 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.812625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134d45-f335-4ffa-b274-48b2c6b37546 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.814277 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 67/137] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:25:59.820191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628fdf-0c8e-4557-bc95-2717fb5117f9 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:25:59.820381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134d45-f335-4ffa-b274-48b2c6b37546 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:25:59.820910 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628fdf-0c8e-4557-bc95-2717fb5117f9 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:25:59.821417 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628fdf-0c8e-4557-bc95-2717fb5117f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:25:59.843500 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-62628fdf-0c8e-4557-bc95-2717fb5117f9 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:25:59.845217 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-62628fdf-0c8e-4557-bc95-2717fb5117f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:25:59.933701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:25:59.973243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 71/138] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:25:59.998007 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62628fdf-0c8e-4557-bc95-2717fb5117f9 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:00.016001 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:00.020541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134d45-f335-4ffa-b274-48b2c6b37546 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:00.089181 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166e340-cb6c-43cf-bbcb-9ce8753d0f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:00.089883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e340-cb6c-43cf-bbcb-9ce8753d0f6e None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:00.090190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e340-cb6c-43cf-bbcb-9ce8753d0f6e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:00.090472 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e340-cb6c-43cf-bbcb-9ce8753d0f6e None admin] PATH_INFO: `/v3/projects/b9948655e8d643ab9deb3cc45ac57743/users/169f2ee6fbe84afe8edc653401e01357/roles/cc0fab8f90a44acc90de9af43e2bd7df` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:00.093643 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166e340-cb6c-43cf-bbcb-9ce8753d0f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=b9948655e8d643ab9deb3cc45ac57743, user_id=169f2ee6fbe84afe8edc653401e01357, role_id=cc0fab8f90a44acc90de9af43e2bd7df)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:00.095622 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166e340-cb6c-43cf-bbcb-9ce8753d0f6e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:00.119173 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 68/139] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:25:59 2022] PUT /identity/v3/projects/b9948655e8d643ab9deb3cc45ac57743/users/169f2ee6fbe84afe8edc653401e01357/roles/cc0fab8f90a44acc90de9af43e2bd7df => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:26:00.164215 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62628fdf-0c8e-4557-bc95-2717fb5117f9 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:00.227120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bce2e1-e4b4-4651-b253-df5395084717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:00.228070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bce2e1-e4b4-4651-b253-df5395084717 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:00.228570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bce2e1-e4b4-4651-b253-df5395084717 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:00.229056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bce2e1-e4b4-4651-b253-df5395084717 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:00.231478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bce2e1-e4b4-4651-b253-df5395084717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:00.234394 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bce2e1-e4b4-4651-b253-df5395084717 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:00.237524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 72/140] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:25:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:00.244174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166e340-cb6c-43cf-bbcb-9ce8753d0f6e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:00.435507 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166e340-cb6c-43cf-bbcb-9ce8753d0f6e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:00.474542 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0f5c9-f6e2-4507-bee2-5191e7da1e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:00.475198 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f5c9-f6e2-4507-bee2-5191e7da1e99 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:00.475480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f5c9-f6e2-4507-bee2-5191e7da1e99 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:00.475755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f5c9-f6e2-4507-bee2-5191e7da1e99 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:00.476461 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0f5c9-f6e2-4507-bee2-5191e7da1e99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:00.477999 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0f5c9-f6e2-4507-bee2-5191e7da1e99 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:00.503790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 69/141] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:00 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:00.892149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce306605-bd25-4b70-89e9-9115977f7e74 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:00.892837 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce306605-bd25-4b70-89e9-9115977f7e74 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:00.892837 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce306605-bd25-4b70-89e9-9115977f7e74 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:00.894314 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 73/142] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:00.899939 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfecac-42e9-4b48-916b-5c1c03653867 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:00.900242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfecac-42e9-4b48-916b-5c1c03653867 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:00.900523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfecac-42e9-4b48-916b-5c1c03653867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:00.920937 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-f4dfecac-42e9-4b48-916b-5c1c03653867 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:00.922200 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-f4dfecac-42e9-4b48-916b-5c1c03653867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:00.933828 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:01.017110 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 70/143] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:01.052683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce306605-bd25-4b70-89e9-9115977f7e74 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:01.125814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:01.190926 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce306605-bd25-4b70-89e9-9115977f7e74 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:01.239402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977e4ad-3a39-4673-91bc-da6cd31827e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:01.240090 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977e4ad-3a39-4673-91bc-da6cd31827e9 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:01.240379 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977e4ad-3a39-4673-91bc-da6cd31827e9 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:01.240653 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977e4ad-3a39-4673-91bc-da6cd31827e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:01.263406 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977e4ad-3a39-4673-91bc-da6cd31827e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-5977e4ad-3a39-4673-91bc-da6cd31827e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:01.268689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:01.269857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:01.270901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 74/144] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:01.278157 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfecac-42e9-4b48-916b-5c1c03653867 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:01.326133 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d183a8c-d561-4e8e-acd8-a79212c28758 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:01.326522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d183a8c-d561-4e8e-acd8-a79212c28758 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:01.326706 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d183a8c-d561-4e8e-acd8-a79212c28758 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:01.328259 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 75/145] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:01.336310 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9b8f3-02bd-4672-a293-ad199636e402 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:01.336310 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9b8f3-02bd-4672-a293-ad199636e402 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:01.336310 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9b8f3-02bd-4672-a293-ad199636e402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:01.359048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-22f9b8f3-02bd-4672-a293-ad199636e402 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:01.361044 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-22f9b8f3-02bd-4672-a293-ad199636e402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:01.474277 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfecac-42e9-4b48-916b-5c1c03653867 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:01.475488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 76/146] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:01.523497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:01.523811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:01.524102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:01.524384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:01.525135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:01.525448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:01.526894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:01.536851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 71/147] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:01.542294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9b8f3-02bd-4672-a293-ad199636e402 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:01.550756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:01.563210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:01.660632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:01.684926 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb5a3d-6c20-4619-bfb6-083ff11452a6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:01.738136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675c49a-4535-4991-83f2-6813ca913595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:01.738827 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675c49a-4535-4991-83f2-6813ca913595 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:01.739106 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675c49a-4535-4991-83f2-6813ca913595 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:01.739381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675c49a-4535-4991-83f2-6813ca913595 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:01.757405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675c49a-4535-4991-83f2-6813ca913595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-8675c49a-4535-4991-83f2-6813ca913595 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:01.765987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:01.767102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:01.768096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:01.772905 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 72/148] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:01.781595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675c49a-4535-4991-83f2-6813ca913595 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:01.799409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9b8f3-02bd-4672-a293-ad199636e402 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:01.874498 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed12ab-0809-4d94-a541-876c6ba2ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:01.874498 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed12ab-0809-4d94-a541-876c6ba2ece2 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:01.874498 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed12ab-0809-4d94-a541-876c6ba2ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:01.882130 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed12ab-0809-4d94-a541-876c6ba2ece2 None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:01.882130 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed12ab-0809-4d94-a541-876c6ba2ece2 None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:01.882130 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed12ab-0809-4d94-a541-876c6ba2ece2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:01.883361 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 77/149] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:01 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:01.893260 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed12ab-0809-4d94-a541-876c6ba2ece2 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:02.025184 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:02.080978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675c49a-4535-4991-83f2-6813ca913595 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:02.134596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:02.135253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:02.135536 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:02.135810 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:02.136558 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] RBAC: Adding query filter params (name=reader) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:02.136866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:02.139642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:02.155670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 73/150] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:02.169915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:02.247071 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed12ab-0809-4d94-a541-876c6ba2ece2 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:02.286595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:02.340650 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b02ecd-6a9c-4b63-8079-dd8e541434e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:02.341588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b02ecd-6a9c-4b63-8079-dd8e541434e4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:02.342034 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b02ecd-6a9c-4b63-8079-dd8e541434e4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:02.342516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b02ecd-6a9c-4b63-8079-dd8e541434e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:02.401694 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b8cda-6de5-4256-927b-e48f8ad9806b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:02.462916 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:02.479861 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf293b-de5d-4e9c-9495-3605ce3360f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:02.480934 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf293b-de5d-4e9c-9495-3605ce3360f1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:02.481437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf293b-de5d-4e9c-9495-3605ce3360f1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:02.481945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf293b-de5d-4e9c-9495-3605ce3360f1 None admin] PATH_INFO: `/v3/users/c977eb24d2534cd3bbb7ec7d6744ae2f` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:02.483423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf293b-de5d-4e9c-9495-3605ce3360f1 None admin] RBAC: Authorizing `identity:get_user(user_id=c977eb24d2534cd3bbb7ec7d6744ae2f)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:02.485133 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf293b-de5d-4e9c-9495-3605ce3360f1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:02.487069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 74/151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:02 2022] GET /identity/v3/users/c977eb24d2534cd3bbb7ec7d6744ae2f => generated 289 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:02.494702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf293b-de5d-4e9c-9495-3605ce3360f1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:02.534141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b02ecd-6a9c-4b63-8079-dd8e541434e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:02.534141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b02ecd-6a9c-4b63-8079-dd8e541434e4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:02.677105 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf293b-de5d-4e9c-9495-3605ce3360f1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:02.738701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1b21e-4056-4027-a348-9d619cba998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:02.739383 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1b21e-4056-4027-a348-9d619cba998b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:02.739667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1b21e-4056-4027-a348-9d619cba998b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:02.739946 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1b21e-4056-4027-a348-9d619cba998b None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:02.741623 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1b21e-4056-4027-a348-9d619cba998b None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:02.743134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1b21e-4056-4027-a348-9d619cba998b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:02.747935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 75/152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:02 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:02.754606 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1b21e-4056-4027-a348-9d619cba998b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:02.780378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 78/153] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 887 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:26:02.787051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b02ecd-6a9c-4b63-8079-dd8e541434e4 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:02.809259 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:02.867259 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:02.936239 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1b21e-4056-4027-a348-9d619cba998b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:02.984902 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b02ecd-6a9c-4b63-8079-dd8e541434e4 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:03.034905 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7ae28-2666-48ec-b69c-a5aff3ba82ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:03.035688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7ae28-2666-48ec-b69c-a5aff3ba82ec None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.035977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7ae28-2666-48ec-b69c-a5aff3ba82ec None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.036256 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7ae28-2666-48ec-b69c-a5aff3ba82ec None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d/users/c977eb24d2534cd3bbb7ec7d6744ae2f/roles/e4367af82d094e189d2679fe6b2c6f9f` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.038440 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7ae28-2666-48ec-b69c-a5aff3ba82ec None admin] RBAC: Authorizing `identity:create_grant(project_id=278f44bb741f4075b58db40b154bb64d, user_id=c977eb24d2534cd3bbb7ec7d6744ae2f, role_id=e4367af82d094e189d2679fe6b2c6f9f)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:03.039949 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7ae28-2666-48ec-b69c-a5aff3ba82ec None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:03.056208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43d8fd-bc3e-49be-9749-939325f3e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:03.056893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43d8fd-bc3e-49be-9749-939325f3e2a2 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.057181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43d8fd-bc3e-49be-9749-939325f3e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.057459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43d8fd-bc3e-49be-9749-939325f3e2a2 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.060484 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43d8fd-bc3e-49be-9749-939325f3e2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:03.062196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43d8fd-bc3e-49be-9749-939325f3e2a2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:03.064494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 79/154] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:02 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:03.072011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43d8fd-bc3e-49be-9749-939325f3e2a2 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:03.073632 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 76/155] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:02 2022] PUT /identity/v3/projects/278f44bb741f4075b58db40b154bb64d/users/c977eb24d2534cd3bbb7ec7d6744ae2f/roles/e4367af82d094e189d2679fe6b2c6f9f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:26:03.194778 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:03.219217 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43d8fd-bc3e-49be-9749-939325f3e2a2 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:03.260441 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec7872-435d-4692-95a1-8a328bf01262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:03.261091 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec7872-435d-4692-95a1-8a328bf01262 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.261438 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec7872-435d-4692-95a1-8a328bf01262 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.261724 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec7872-435d-4692-95a1-8a328bf01262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.262586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec7872-435d-4692-95a1-8a328bf01262 None admin] RBAC: Adding query filter params (role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=278f44bb741f4075b58db40b154bb64d, user.id=169f2ee6fbe84afe8edc653401e01357) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:03.262895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec7872-435d-4692-95a1-8a328bf01262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:03.264444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec7872-435d-4692-95a1-8a328bf01262 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:03.278280 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 80/156] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:03 2022] GET /identity/v3/role_assignments?user.id=169f2ee6fbe84afe8edc653401e01357&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:03.396603 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52759b9b-6d65-403c-85d0-00734a0da8d5 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.396947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52759b9b-6d65-403c-85d0-00734a0da8d5 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.397168 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52759b9b-6d65-403c-85d0-00734a0da8d5 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.398626 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 77/157] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:03.404452 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac79e5-5b45-4907-a374-8e46a6359bb1 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.404776 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac79e5-5b45-4907-a374-8e46a6359bb1 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.405012 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac79e5-5b45-4907-a374-8e46a6359bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.435211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-8bac79e5-5b45-4907-a374-8e46a6359bb1 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:03.436474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-8bac79e5-5b45-4907-a374-8e46a6359bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:03.532483 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 81/158] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:03.567517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52759b9b-6d65-403c-85d0-00734a0da8d5 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:03.619681 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec16365-7b55-4676-96c8-f0262f35b5b8 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.620035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec16365-7b55-4676-96c8-f0262f35b5b8 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.620240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec16365-7b55-4676-96c8-f0262f35b5b8 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.621668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 82/159] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:03.653710 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335250ce-4638-46ac-bcf6-7dc684f38a63 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.654120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335250ce-4638-46ac-bcf6-7dc684f38a63 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.654313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335250ce-4638-46ac-bcf6-7dc684f38a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.687217 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-335250ce-4638-46ac-bcf6-7dc684f38a63 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:03.688518 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-335250ce-4638-46ac-bcf6-7dc684f38a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:03.730241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52759b9b-6d65-403c-85d0-00734a0da8d5 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:03.761859 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:03.785858 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd8853-6383-468d-ad24-9ce426e1547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:03.786956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8853-6383-468d-ad24-9ce426e1547d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.787453 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8853-6383-468d-ad24-9ce426e1547d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.787961 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8853-6383-468d-ad24-9ce426e1547d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.804827 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd8853-6383-468d-ad24-9ce426e1547d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:03.808513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 83/160] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-bbfd8853-6383-468d-ad24-9ce426e1547d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:03.811095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:03.812103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:03.813017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:03.815518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 78/161] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:03.827275 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335250ce-4638-46ac-bcf6-7dc684f38a63 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:03.831534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:03.854386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6160e0c-441b-454e-86c6-0e1d591449fd None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:03.854688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6160e0c-441b-454e-86c6-0e1d591449fd None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:03.854943 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6160e0c-441b-454e-86c6-0e1d591449fd None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:03.856354 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 79/162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:03.858938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6160e0c-441b-454e-86c6-0e1d591449fd None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:03.980289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335250ce-4638-46ac-bcf6-7dc684f38a63 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:03.990343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6160e0c-441b-454e-86c6-0e1d591449fd None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:04.035359 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba530d34-5749-4da0-b517-93e0d27c3cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:04.036124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530d34-5749-4da0-b517-93e0d27c3cd5 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:04.036351 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530d34-5749-4da0-b517-93e0d27c3cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:04.036631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530d34-5749-4da0-b517-93e0d27c3cd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:04.037860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd171cca-b3fe-4aef-b00e-2b6667cd46f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:04.038596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171cca-b3fe-4aef-b00e-2b6667cd46f3 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:04.038875 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171cca-b3fe-4aef-b00e-2b6667cd46f3 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:04.039239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171cca-b3fe-4aef-b00e-2b6667cd46f3 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:04.039997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd171cca-b3fe-4aef-b00e-2b6667cd46f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:04.040309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd171cca-b3fe-4aef-b00e-2b6667cd46f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:04.041764 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd171cca-b3fe-4aef-b00e-2b6667cd46f3 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:04.059219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 84/163] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:04.061587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c5ed-4712-4e91-8416-05b2b2aec311 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:04.061881 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c5ed-4712-4e91-8416-05b2b2aec311 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:04.062190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c5ed-4712-4e91-8416-05b2b2aec311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:04.063233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba530d34-5749-4da0-b517-93e0d27c3cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-ba530d34-5749-4da0-b517-93e0d27c3cd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:04.073048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:04.074142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 80/164] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:26:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:04.076290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba530d34-5749-4da0-b517-93e0d27c3cd5 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:04.101746 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-73a4c5ed-4712-4e91-8416-05b2b2aec311 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:04.103031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-73a4c5ed-4712-4e91-8416-05b2b2aec311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:04.200144 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:04.243185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:04.272697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 85/165] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:04.274156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4c5ed-4712-4e91-8416-05b2b2aec311 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:04.322358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba530d34-5749-4da0-b517-93e0d27c3cd5 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:04.402882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aa45a-9e2e-44e8-94af-d5b2b90e4b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:04.403541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aa45a-9e2e-44e8-94af-d5b2b90e4b47 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:04.403831 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aa45a-9e2e-44e8-94af-d5b2b90e4b47 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:04.404118 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aa45a-9e2e-44e8-94af-d5b2b90e4b47 None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:04.405410 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aa45a-9e2e-44e8-94af-d5b2b90e4b47 None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:04.406986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aa45a-9e2e-44e8-94af-d5b2b90e4b47 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:04.409030 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 81/166] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:04 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:04.409892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aa45a-9e2e-44e8-94af-d5b2b90e4b47 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:04.482505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4c5ed-4712-4e91-8416-05b2b2aec311 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:04.555115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:04.556164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:04.556675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:04.557261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:04.558238 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:04.558763 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:04.560377 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:04.582143 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 86/167] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:04.583535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:04.617898 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aa45a-9e2e-44e8-94af-d5b2b90e4b47 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:04.643248 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:04.683100 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bcdea-6579-4ea0-9abf-0ca100b0a697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:04.683923 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bcdea-6579-4ea0-9abf-0ca100b0a697 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:04.684211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bcdea-6579-4ea0-9abf-0ca100b0a697 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:04.684500 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bcdea-6579-4ea0-9abf-0ca100b0a697 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:04.711230 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bcdea-6579-4ea0-9abf-0ca100b0a697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:04.712239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-1d7bcdea-6579-4ea0-9abf-0ca100b0a697 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:04.717479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:04.718811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 82/168] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:04.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bcdea-6579-4ea0-9abf-0ca100b0a697 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:04.809444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdc8ca-c172-49c2-9c38-15b2b8860c89 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:04.853531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63a64-84f2-42cb-bd77-4fa3f6917401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:04.854097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63a64-84f2-42cb-bd77-4fa3f6917401 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:04.854591 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63a64-84f2-42cb-bd77-4fa3f6917401 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:04.854880 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63a64-84f2-42cb-bd77-4fa3f6917401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:04.958295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc63a64-84f2-42cb-bd77-4fa3f6917401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:04.959821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc63a64-84f2-42cb-bd77-4fa3f6917401 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:05.041058 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bcdea-6579-4ea0-9abf-0ca100b0a697 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:05.078524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:05.090864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1218c15-5858-456a-b9c1-2b8aca075339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:05.091544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1218c15-5858-456a-b9c1-2b8aca075339 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.091826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1218c15-5858-456a-b9c1-2b8aca075339 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.092104 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1218c15-5858-456a-b9c1-2b8aca075339 None admin] PATH_INFO: `/v3/users/5a8bf4bf753c41b2a872093710161103` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.093312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1218c15-5858-456a-b9c1-2b8aca075339 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8bf4bf753c41b2a872093710161103)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:05.094834 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1218c15-5858-456a-b9c1-2b8aca075339 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:05.096488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 83/169] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:04 2022] GET /identity/v3/users/5a8bf4bf753c41b2a872093710161103 => generated 247 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:05.097994 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1218c15-5858-456a-b9c1-2b8aca075339 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:05.162030 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 87/170] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:05.163575 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63a64-84f2-42cb-bd77-4fa3f6917401 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:05.190695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:05.310153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1218c15-5858-456a-b9c1-2b8aca075339 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:05.328383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63a64-84f2-42cb-bd77-4fa3f6917401 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:05.375675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacffb8-247b-4612-9692-58c2a4b4dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:05.376756 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacffb8-247b-4612-9692-58c2a4b4dba2 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.377334 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacffb8-247b-4612-9692-58c2a4b4dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.377822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacffb8-247b-4612-9692-58c2a4b4dba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.387829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:05.388491 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.388773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.389047 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.389786 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:05.390089 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:05.391552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:05.408975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 84/171] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:05.409707 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:05.409825 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:05.457738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacffb8-247b-4612-9692-58c2a4b4dba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:05.459176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacffb8-247b-4612-9692-58c2a4b4dba2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:05.564193 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c155f-c9b9-4b76-8501-30b64577c9ae None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:05.564815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 88/172] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:05.570623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacffb8-247b-4612-9692-58c2a4b4dba2 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:05.571992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:05.610612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cea91e-27b7-4add-a46a-2316c4e99a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:05.611792 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea91e-27b7-4add-a46a-2316c4e99a0b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.612294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea91e-27b7-4add-a46a-2316c4e99a0b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.612806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea91e-27b7-4add-a46a-2316c4e99a0b None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.614586 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cea91e-27b7-4add-a46a-2316c4e99a0b None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:05.616286 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cea91e-27b7-4add-a46a-2316c4e99a0b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:05.618806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 85/173] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:05 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:05.620718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ce64f-c099-468b-a645-9a99827005bc None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.621294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ce64f-c099-468b-a645-9a99827005bc None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.621785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ce64f-c099-468b-a645-9a99827005bc None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.627684 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 86/174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:05.628424 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:05.630991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e5108-f479-4760-810b-728cc2b325e8 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.631607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e5108-f479-4760-810b-728cc2b325e8 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.632071 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e5108-f479-4760-810b-728cc2b325e8 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.634065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 87/175] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:05.635201 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e5108-f479-4760-810b-728cc2b325e8 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:05.671535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacffb8-247b-4612-9692-58c2a4b4dba2 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:05.708023 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be9ff2-34f5-43d4-a30d-e28f80282900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:05.708750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be9ff2-34f5-43d4-a30d-e28f80282900 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.709036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be9ff2-34f5-43d4-a30d-e28f80282900 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.709312 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be9ff2-34f5-43d4-a30d-e28f80282900 None admin] PATH_INFO: `/v3/users/169f2ee6fbe84afe8edc653401e01357` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.710593 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be9ff2-34f5-43d4-a30d-e28f80282900 None admin] RBAC: Authorizing `identity:get_user(user_id=169f2ee6fbe84afe8edc653401e01357)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:05.712069 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be9ff2-34f5-43d4-a30d-e28f80282900 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:05.718199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 89/176] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:05 2022] GET /identity/v3/users/169f2ee6fbe84afe8edc653401e01357 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:05.718199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be9ff2-34f5-43d4-a30d-e28f80282900 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:05.765049 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e5108-f479-4760-810b-728cc2b325e8 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:05.804874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2c123-e624-4dd8-8b2b-ab0a67e1ecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:05.806055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2c123-e624-4dd8-8b2b-ab0a67e1ecb1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.806601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2c123-e624-4dd8-8b2b-ab0a67e1ecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.807095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2c123-e624-4dd8-8b2b-ab0a67e1ecb1 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.808665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2c123-e624-4dd8-8b2b-ab0a67e1ecb1 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:05.810615 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2c123-e624-4dd8-8b2b-ab0a67e1ecb1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:05.812851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 88/177] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:05 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:05.814384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac438d-1d67-4ada-af3a-128284e5ee22 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.814887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac438d-1d67-4ada-af3a-128284e5ee22 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.815405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac438d-1d67-4ada-af3a-128284e5ee22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.833617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be9ff2-34f5-43d4-a30d-e28f80282900 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:05.835856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-cbac438d-1d67-4ada-af3a-128284e5ee22 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:05.837092 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-cbac438d-1d67-4ada-af3a-128284e5ee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:05.862355 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:05.870343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:05.877040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6776f3b-cbbf-49f4-b41e-14215ab7b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:05.878723 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776f3b-cbbf-49f4-b41e-14215ab7b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.878723 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776f3b-cbbf-49f4-b41e-14215ab7b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.878723 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776f3b-cbbf-49f4-b41e-14215ab7b5b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.879186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6776f3b-cbbf-49f4-b41e-14215ab7b5b0 None admin] RBAC: Adding query filter params (role.id=d883529ef5284c35b27b759c57353383, scope.project.id=278f44bb741f4075b58db40b154bb64d, user.id=169f2ee6fbe84afe8edc653401e01357) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:05.879546 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6776f3b-cbbf-49f4-b41e-14215ab7b5b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:05.880982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6776f3b-cbbf-49f4-b41e-14215ab7b5b0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:05.887650 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 90/178] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:05 2022] GET /identity/v3/role_assignments?user.id=169f2ee6fbe84afe8edc653401e01357&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=d883529ef5284c35b27b759c57353383 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:05.890362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27119e-34f1-4eb3-890c-840fae1478ea None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:05.890362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27119e-34f1-4eb3-890c-840fae1478ea None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:05.890362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27119e-34f1-4eb3-890c-840fae1478ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:05.922570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-8d27119e-34f1-4eb3-890c-840fae1478ea None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:05.924746 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-8d27119e-34f1-4eb3-890c-840fae1478ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:05.937417 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 89/179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:05.942354 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac438d-1d67-4ada-af3a-128284e5ee22 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.049662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 91/180] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:06.057103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa514-81e3-4db5-88b0-60c28bdce475 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.057589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa514-81e3-4db5-88b0-60c28bdce475 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.058087 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa514-81e3-4db5-88b0-60c28bdce475 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.065357 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 92/181] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:06 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:06.066753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fa514-81e3-4db5-88b0-60c28bdce475 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.091325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac438d-1d67-4ada-af3a-128284e5ee22 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:06.129921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cc974-0a5f-4dcd-aeae-267e6b6d65be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:06.130636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cc974-0a5f-4dcd-aeae-267e6b6d65be None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.130925 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cc974-0a5f-4dcd-aeae-267e6b6d65be None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.131203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cc974-0a5f-4dcd-aeae-267e6b6d65be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.175322 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:06.213322 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cc974-0a5f-4dcd-aeae-267e6b6d65be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:06.214373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fa514-81e3-4db5-88b0-60c28bdce475 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:06.214840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cc974-0a5f-4dcd-aeae-267e6b6d65be None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:06.236037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:06.251880 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:06.252519 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.252799 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.253072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.253886 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=278f44bb741f4075b58db40b154bb64d, user.id=5a8bf4bf753c41b2a872093710161103) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:06.254260 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:06.255736 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:06.261818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 93/182] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:06 2022] GET /identity/v3/role_assignments?user.id=5a8bf4bf753c41b2a872093710161103&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:06.262671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.352168 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 90/183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:06.352168 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524e131-04c0-4938-aeee-e3daaed2901e None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.352168 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524e131-04c0-4938-aeee-e3daaed2901e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.352168 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524e131-04c0-4938-aeee-e3daaed2901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.371430 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-e524e131-04c0-4938-aeee-e3daaed2901e None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:06.372689 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-e524e131-04c0-4938-aeee-e3daaed2901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:06.414445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985afbb5-474b-460f-95a6-e868ad6ad8f1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:06.452134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:06.456551 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9933fe-678a-4274-bc2c-9a9515f679f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:06.457506 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9933fe-678a-4274-bc2c-9a9515f679f5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.457635 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9933fe-678a-4274-bc2c-9a9515f679f5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.457901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9933fe-678a-4274-bc2c-9a9515f679f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.460218 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 91/184] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:06.461594 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524e131-04c0-4938-aeee-e3daaed2901e None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.471713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9933fe-678a-4274-bc2c-9a9515f679f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-2c9933fe-678a-4274-bc2c-9a9515f679f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:06.478172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:06.479273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 94/185] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:06.487696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f774075-05b5-4edd-835c-35ce4c02c9d7 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.488548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f774075-05b5-4edd-835c-35ce4c02c9d7 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.488548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f774075-05b5-4edd-835c-35ce4c02c9d7 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.488548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 95/186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:06.488548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f774075-05b5-4edd-835c-35ce4c02c9d7 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.544249 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:06.610133 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f774075-05b5-4edd-835c-35ce4c02c9d7 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:06.616766 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524e131-04c0-4938-aeee-e3daaed2901e None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:06.648917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c8bb8-3906-489b-9811-d1eb8b13ef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:06.649582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c8bb8-3906-489b-9811-d1eb8b13ef89 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.649864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c8bb8-3906-489b-9811-d1eb8b13ef89 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.650132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c8bb8-3906-489b-9811-d1eb8b13ef89 None admin] PATH_INFO: `/v3/projects/b9948655e8d643ab9deb3cc45ac57743` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.651415 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c8bb8-3906-489b-9811-d1eb8b13ef89 None admin] RBAC: Authorizing `identity:get_project(project_id=b9948655e8d643ab9deb3cc45ac57743)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:06.652869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c8bb8-3906-489b-9811-d1eb8b13ef89 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:06.654733 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 96/187] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:06 2022] GET /identity/v3/projects/b9948655e8d643ab9deb3cc45ac57743 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:06.655462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c8bb8-3906-489b-9811-d1eb8b13ef89 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.660730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90888487-b1b8-4523-90a1-2c1e4d72713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:06.661433 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888487-b1b8-4523-90a1-2c1e4d72713b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.661718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888487-b1b8-4523-90a1-2c1e4d72713b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.661997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888487-b1b8-4523-90a1-2c1e4d72713b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.674471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90888487-b1b8-4523-90a1-2c1e4d72713b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-90888487-b1b8-4523-90a1-2c1e4d72713b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:06.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:06.680803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:06.681709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 92/188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:26:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:06.682517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90888487-b1b8-4523-90a1-2c1e4d72713b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.747245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:06.769961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c8bb8-3906-489b-9811-d1eb8b13ef89 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:06.784065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:06.795331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90888487-b1b8-4523-90a1-2c1e4d72713b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:06.807085 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522af84f-1bcd-4db1-a77d-f3394cbef5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:06.807754 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af84f-1bcd-4db1-a77d-f3394cbef5af None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.808043 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af84f-1bcd-4db1-a77d-f3394cbef5af None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.808317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af84f-1bcd-4db1-a77d-f3394cbef5af None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.809503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522af84f-1bcd-4db1-a77d-f3394cbef5af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:06.811026 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522af84f-1bcd-4db1-a77d-f3394cbef5af None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:06.812668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 97/189] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:06.813801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff92c9a-9eab-4752-9ea5-8c56e3ee5868 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.814090 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff92c9a-9eab-4752-9ea5-8c56e3ee5868 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.814392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff92c9a-9eab-4752-9ea5-8c56e3ee5868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.832877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-9ff92c9a-9eab-4752-9ea5-8c56e3ee5868 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:06.834132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-9ff92c9a-9eab-4752-9ea5-8c56e3ee5868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:06.837057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:06.837706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:06.837990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:06.838300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:06.839045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:06.839355 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:06.840763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:06.849510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 93/190] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:06.850378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.905603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 98/191] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:06.906541 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff92c9a-9eab-4752-9ea5-8c56e3ee5868 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:06.964827 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca32abd-365e-45a3-b0a3-2f42649b6186 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:06.984806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:07.000557 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.001200 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.001484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.001762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.002598 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] RBAC: Adding query filter params (role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=b9948655e8d643ab9deb3cc45ac57743, user.id=169f2ee6fbe84afe8edc653401e01357) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:07.002918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.004397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.010302 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 94/192] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:06 2022] GET /identity/v3/role_assignments?user.id=169f2ee6fbe84afe8edc653401e01357&scope.project.id=b9948655e8d643ab9deb3cc45ac57743&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:07.011087 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:07.012505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff92c9a-9eab-4752-9ea5-8c56e3ee5868 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.027921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:07.058552 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.059268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.059560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.059835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.060637 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:07.060951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.062434 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.075065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 99/193] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:07.075897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:07.138966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c34298-56a7-4ba1-bf4e-c3ff0aa2c95c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.182099 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5df63-b3d2-48ca-8a08-d49d306adcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.183021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5df63-b3d2-48ca-8a08-d49d306adcc5 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.183021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5df63-b3d2-48ca-8a08-d49d306adcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.183269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5df63-b3d2-48ca-8a08-d49d306adcc5 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.184029 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5df63-b3d2-48ca-8a08-d49d306adcc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.185407 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5df63-b3d2-48ca-8a08-d49d306adcc5 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.207909 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 95/194] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:07 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:07.208775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5df63-b3d2-48ca-8a08-d49d306adcc5 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:07.209416 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f1189-b5a8-4700-81a1-80b6013b5af1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.253958 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d416bb0-1d1c-4c4d-a233-7f20ce367ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.255075 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d416bb0-1d1c-4c4d-a233-7f20ce367ec7 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.255440 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d416bb0-1d1c-4c4d-a233-7f20ce367ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.255764 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d416bb0-1d1c-4c4d-a233-7f20ce367ec7 None admin] PATH_INFO: `/v3/users/740ebdac18f444bb8f90ad4538f791ee` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.271131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:07.273118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d416bb0-1d1c-4c4d-a233-7f20ce367ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=740ebdac18f444bb8f90ad4538f791ee)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.274712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d416bb0-1d1c-4c4d-a233-7f20ce367ec7 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.276970 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 100/195] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:07 2022] GET /identity/v3/users/740ebdac18f444bb8f90ad4538f791ee => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:07.277854 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d416bb0-1d1c-4c4d-a233-7f20ce367ec7 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:07.316289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:07.344566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5df63-b3d2-48ca-8a08-d49d306adcc5 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.383860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658d61-90b6-44c2-b0c0-d6a6d01044fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.384542 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658d61-90b6-44c2-b0c0-d6a6d01044fe None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.384828 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658d61-90b6-44c2-b0c0-d6a6d01044fe None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.385111 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658d61-90b6-44c2-b0c0-d6a6d01044fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.397534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df658d61-90b6-44c2-b0c0-d6a6d01044fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:07.398727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d416bb0-1d1c-4c4d-a233-7f20ce367ec7 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-df658d61-90b6-44c2-b0c0-d6a6d01044fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:07.403141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:07.404329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:07.405456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 96/196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:07.406414 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658d61-90b6-44c2-b0c0-d6a6d01044fe None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:07.448838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d614dd9-ea9e-4dc6-977b-bb6752ba2d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.450287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d614dd9-ea9e-4dc6-977b-bb6752ba2d33 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.452070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d614dd9-ea9e-4dc6-977b-bb6752ba2d33 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.452070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d614dd9-ea9e-4dc6-977b-bb6752ba2d33 None admin] PATH_INFO: `/v3/users/740ebdac18f444bb8f90ad4538f791ee` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.452070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d614dd9-ea9e-4dc6-977b-bb6752ba2d33 None admin] RBAC: Authorizing `identity:get_user(user_id=740ebdac18f444bb8f90ad4538f791ee)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.453576 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d614dd9-ea9e-4dc6-977b-bb6752ba2d33 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.455867 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 101/197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:07 2022] GET /identity/v3/users/740ebdac18f444bb8f90ad4538f791ee => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:07.456542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d614dd9-ea9e-4dc6-977b-bb6752ba2d33 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:07.535669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:07.540178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658d61-90b6-44c2-b0c0-d6a6d01044fe None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.589697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7a63d-80a3-4dc4-8d99-5bfa2902d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.590816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7a63d-80a3-4dc4-8d99-5bfa2902d23d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.591335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7a63d-80a3-4dc4-8d99-5bfa2902d23d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.591851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7a63d-80a3-4dc4-8d99-5bfa2902d23d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.595432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d614dd9-ea9e-4dc6-977b-bb6752ba2d33 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.598347 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:07.634845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.635814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.636412 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.636904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.637933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:07.638590 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.640341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.650446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 102/198] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:07.651756 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:07.667733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7a63d-80a3-4dc4-8d99-5bfa2902d23d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.669258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7a63d-80a3-4dc4-8d99-5bfa2902d23d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.773554 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee640d1-addf-412b-902f-2000a0052a0e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.803457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:07.812842 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 97/199] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:07.813644 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7a63d-80a3-4dc4-8d99-5bfa2902d23d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:07.820028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7735ce-37d7-4dae-b6c6-e2356b1890e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.821053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7735ce-37d7-4dae-b6c6-e2356b1890e1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.821545 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7735ce-37d7-4dae-b6c6-e2356b1890e1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.822048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7735ce-37d7-4dae-b6c6-e2356b1890e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.855273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:07.916968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7735ce-37d7-4dae-b6c6-e2356b1890e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.918906 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7735ce-37d7-4dae-b6c6-e2356b1890e1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.952174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7a63d-80a3-4dc4-8d99-5bfa2902d23d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:07.991329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23491e5-c981-4e55-9ec4-042fe8d64f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:07.992167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23491e5-c981-4e55-9ec4-042fe8d64f41 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:07.992554 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23491e5-c981-4e55-9ec4-042fe8d64f41 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:07.992926 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23491e5-c981-4e55-9ec4-042fe8d64f41 None admin] PATH_INFO: `/v3/users/c977eb24d2534cd3bbb7ec7d6744ae2f` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:07.994384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23491e5-c981-4e55-9ec4-042fe8d64f41 None admin] RBAC: Authorizing `identity:get_user(user_id=c977eb24d2534cd3bbb7ec7d6744ae2f)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:07.996082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23491e5-c981-4e55-9ec4-042fe8d64f41 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:07.997849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 98/200] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:07 2022] GET /identity/v3/users/c977eb24d2534cd3bbb7ec7d6744ae2f => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:07.998939 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23491e5-c981-4e55-9ec4-042fe8d64f41 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:08.044080 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 103/201] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:08.045789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7735ce-37d7-4dae-b6c6-e2356b1890e1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:08.080422 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:08.119046 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23491e5-c981-4e55-9ec4-042fe8d64f41 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:08.129126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:08.159133 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6591be-5cbc-42c2-b3a2-afab9e1fd28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:08.159989 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6591be-5cbc-42c2-b3a2-afab9e1fd28c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:08.160589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6591be-5cbc-42c2-b3a2-afab9e1fd28c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:08.160966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6591be-5cbc-42c2-b3a2-afab9e1fd28c None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:08.162369 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6591be-5cbc-42c2-b3a2-afab9e1fd28c None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:08.164016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6591be-5cbc-42c2-b3a2-afab9e1fd28c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:08.166136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 99/202] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:07 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:08.172053 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6591be-5cbc-42c2-b3a2-afab9e1fd28c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:08.179163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7735ce-37d7-4dae-b6c6-e2356b1890e1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:08.217596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb10ab6-6f21-4694-9c6b-6fd6db58b33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:08.218678 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb10ab6-6f21-4694-9c6b-6fd6db58b33c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:08.219137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb10ab6-6f21-4694-9c6b-6fd6db58b33c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:08.219566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb10ab6-6f21-4694-9c6b-6fd6db58b33c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:08.280693 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6591be-5cbc-42c2-b3a2-afab9e1fd28c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:08.292828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb10ab6-6f21-4694-9c6b-6fd6db58b33c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:08.294976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb10ab6-6f21-4694-9c6b-6fd6db58b33c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:08.317040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4720d-5483-4fa2-ad3f-7f38220b289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:08.317969 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4720d-5483-4fa2-ad3f-7f38220b289d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:08.318269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4720d-5483-4fa2-ad3f-7f38220b289d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:08.319206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4720d-5483-4fa2-ad3f-7f38220b289d None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:08.320616 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4720d-5483-4fa2-ad3f-7f38220b289d None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:08.322288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4720d-5483-4fa2-ad3f-7f38220b289d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:08.324508 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 100/203] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:08 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:08.325676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4720d-5483-4fa2-ad3f-7f38220b289d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:08.330457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:08.358904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:08.406439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 104/204] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:08.407689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb10ab6-6f21-4694-9c6b-6fd6db58b33c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:08.456158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4720d-5483-4fa2-ad3f-7f38220b289d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:08.504427 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:08.505491 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:08.505985 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:08.506508 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:08.507539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] RBAC: Adding query filter params (role.id=d883529ef5284c35b27b759c57353383, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93, user.id=740ebdac18f444bb8f90ad4538f791ee) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:08.508063 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:08.509758 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:08.516172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 101/205] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:08 2022] GET /identity/v3/role_assignments?user.id=740ebdac18f444bb8f90ad4538f791ee&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=d883529ef5284c35b27b759c57353383 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:08.517377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:08.539123 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb10ab6-6f21-4694-9c6b-6fd6db58b33c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:08.606083 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:08.606674 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:08.606954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:08.607226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:08.608038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93, user.id=740ebdac18f444bb8f90ad4538f791ee) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:08.608346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:08.609837 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:08.624168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 105/206] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:08 2022] GET /identity/v3/role_assignments?user.id=740ebdac18f444bb8f90ad4538f791ee&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:08.668008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:08.736866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99966912-63b8-499c-96aa-a5221149f609 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:08.781783 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24968f-c5e5-44de-b1b5-c26f868966ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:08.782511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24968f-c5e5-44de-b1b5-c26f868966ca None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:08.782795 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24968f-c5e5-44de-b1b5-c26f868966ca None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:08.783072 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24968f-c5e5-44de-b1b5-c26f868966ca None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:08.784311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24968f-c5e5-44de-b1b5-c26f868966ca None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:08.785977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24968f-c5e5-44de-b1b5-c26f868966ca None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:08.788222 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 102/207] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:08 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:08.794691 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:08.824981 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:08.926962 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0098bd-24a2-404d-beb4-fa5cac10b56e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:08.966187 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59819229-1cbf-4741-8832-1d3112bbd616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:08.966879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819229-1cbf-4741-8832-1d3112bbd616 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:08.967166 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819229-1cbf-4741-8832-1d3112bbd616 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:08.967440 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819229-1cbf-4741-8832-1d3112bbd616 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:08.968298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59819229-1cbf-4741-8832-1d3112bbd616 None admin] RBAC: Adding query filter params (role.id=e4367af82d094e189d2679fe6b2c6f9f, scope.project.id=278f44bb741f4075b58db40b154bb64d, user.id=c977eb24d2534cd3bbb7ec7d6744ae2f) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:08.968671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59819229-1cbf-4741-8832-1d3112bbd616 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:08.970204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59819229-1cbf-4741-8832-1d3112bbd616 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:08.977119 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 106/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:08 2022] GET /identity/v3/role_assignments?user.id=c977eb24d2534cd3bbb7ec7d6744ae2f&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=e4367af82d094e189d2679fe6b2c6f9f => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:09.604963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac5c6-79e6-4a58-a1ca-6dd5eefa5b41 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:09.605339 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac5c6-79e6-4a58-a1ca-6dd5eefa5b41 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:09.605527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac5c6-79e6-4a58-a1ca-6dd5eefa5b41 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:09.607283 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 103/209] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:09.608472 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2b15f-1cff-4e2d-9d6a-15492e2eb134 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:09.609323 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2b15f-1cff-4e2d-9d6a-15492e2eb134 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:09.609819 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2b15f-1cff-4e2d-9d6a-15492e2eb134 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:09.611498 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 107/210] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:09.616440 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006c2ed-6ded-4d24-a110-7bbf7cf64efa None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:09.616727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006c2ed-6ded-4d24-a110-7bbf7cf64efa None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:09.617007 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006c2ed-6ded-4d24-a110-7bbf7cf64efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:09.617879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6c5ce-b89f-4ce0-a2b7-da37e9b17d56 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:09.618429 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6c5ce-b89f-4ce0-a2b7-da37e9b17d56 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:09.618915 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6c5ce-b89f-4ce0-a2b7-da37e9b17d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:09.640199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-63e6c5ce-b89f-4ce0-a2b7-da37e9b17d56 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:09.641504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-63e6c5ce-b89f-4ce0-a2b7-da37e9b17d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:09.653911 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-2006c2ed-6ded-4d24-a110-7bbf7cf64efa None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:09.655978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-2006c2ed-6ded-4d24-a110-7bbf7cf64efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:09.734211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 108/211] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:09.741140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 104/212] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:09.757339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6c5ce-b89f-4ce0-a2b7-da37e9b17d56 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:09.768948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006c2ed-6ded-4d24-a110-7bbf7cf64efa None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:09.808758 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:09.841652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:09.861328 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6c5ce-b89f-4ce0-a2b7-da37e9b17d56 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:09.881385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006c2ed-6ded-4d24-a110-7bbf7cf64efa None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:09.896708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749f7d-e507-497e-be1f-7c023f428f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:09.897465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749f7d-e507-497e-be1f-7c023f428f48 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:09.897824 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749f7d-e507-497e-be1f-7c023f428f48 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:09.898201 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749f7d-e507-497e-be1f-7c023f428f48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:09.899458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749f7d-e507-497e-be1f-7c023f428f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:09.901006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749f7d-e507-497e-be1f-7c023f428f48 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:09.906150 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 109/213] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:09.908751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749f7d-e507-497e-be1f-7c023f428f48 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:09.935007 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12aebf-28e5-47b4-8869-ca56459bc0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:09.935871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12aebf-28e5-47b4-8869-ca56459bc0d0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:09.936166 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12aebf-28e5-47b4-8869-ca56459bc0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:09.936449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12aebf-28e5-47b4-8869-ca56459bc0d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:09.949116 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12aebf-28e5-47b4-8869-ca56459bc0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-bc12aebf-28e5-47b4-8869-ca56459bc0d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:09.954517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:09.955511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:09.956400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:09.959247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 105/214] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:09.968806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12aebf-28e5-47b4-8869-ca56459bc0d0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:10.020324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:10.038878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749f7d-e507-497e-be1f-7c023f428f48 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:10.076893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba65f5-70fc-4e04-8dde-d017cf0e181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:10.077441 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba65f5-70fc-4e04-8dde-d017cf0e181d None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:10.077724 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba65f5-70fc-4e04-8dde-d017cf0e181d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:10.078001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba65f5-70fc-4e04-8dde-d017cf0e181d None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:10.078350 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12aebf-28e5-47b4-8869-ca56459bc0d0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:10.078753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba65f5-70fc-4e04-8dde-d017cf0e181d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:10.080348 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba65f5-70fc-4e04-8dde-d017cf0e181d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:10.107280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:10.116425 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 110/215] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:09 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:10.119022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:10.119748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:10.120119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:10.120466 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:10.121280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:10.121661 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:10.123257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:10.137284 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 106/216] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:10.142907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba65f5-70fc-4e04-8dde-d017cf0e181d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:10.259146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba65f5-70fc-4e04-8dde-d017cf0e181d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:10.301304 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0664f49-fd3c-41d3-86d0-43af7d077f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:10.301970 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0664f49-fd3c-41d3-86d0-43af7d077f9f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:10.302367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0664f49-fd3c-41d3-86d0-43af7d077f9f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:10.302646 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0664f49-fd3c-41d3-86d0-43af7d077f9f None admin] PATH_INFO: `/v3/users/18e8396b4a734ad6bea128b4117494d1` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:10.315990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:10.319231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0664f49-fd3c-41d3-86d0-43af7d077f9f None admin] RBAC: Authorizing `identity:get_user(user_id=18e8396b4a734ad6bea128b4117494d1)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:10.320980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0664f49-fd3c-41d3-86d0-43af7d077f9f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:10.323613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 111/217] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:10 2022] GET /identity/v3/users/18e8396b4a734ad6bea128b4117494d1 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:10.329485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:10.440011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe7374-47d9-495c-b6dd-0e24ef4258f8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:10.477756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1711161-886c-450b-bcfb-111033711e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:10.478457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711161-886c-450b-bcfb-111033711e6b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:10.478743 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711161-886c-450b-bcfb-111033711e6b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:10.479022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711161-886c-450b-bcfb-111033711e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:10.540094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:10.554222 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1711161-886c-450b-bcfb-111033711e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:10.555717 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1711161-886c-450b-bcfb-111033711e6b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:10.661011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 107/218] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:10.666794 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0664f49-fd3c-41d3-86d0-43af7d077f9f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:10.787267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0664f49-fd3c-41d3-86d0-43af7d077f9f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:10.835416 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec266b-0b62-46fa-90c1-ac5198b8802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:10.836103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec266b-0b62-46fa-90c1-ac5198b8802a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:10.836384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec266b-0b62-46fa-90c1-ac5198b8802a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:10.836658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec266b-0b62-46fa-90c1-ac5198b8802a None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:10.842725 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec266b-0b62-46fa-90c1-ac5198b8802a None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:10.844261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec266b-0b62-46fa-90c1-ac5198b8802a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:10.846424 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 112/219] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:10 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:10.852269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1711161-886c-450b-bcfb-111033711e6b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:10.948035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:10.961017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1711161-886c-450b-bcfb-111033711e6b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:10.983933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30154942-c791-41bf-b971-04fbf38d7b7d None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:10.984215 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30154942-c791-41bf-b971-04fbf38d7b7d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:10.984495 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30154942-c791-41bf-b971-04fbf38d7b7d None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:10.985873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 113/220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:10.992974 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d214a-5dea-44c7-8877-1387ac35207b None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:10.993262 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d214a-5dea-44c7-8877-1387ac35207b None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:10.993531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d214a-5dea-44c7-8877-1387ac35207b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.017968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-8f4d214a-5dea-44c7-8877-1387ac35207b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:11.018381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665fa74-ca1d-4325-ad9d-5408bcefcbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:11.018965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665fa74-ca1d-4325-ad9d-5408bcefcbaf None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:11.019245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665fa74-ca1d-4325-ad9d-5408bcefcbaf None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:11.019520 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665fa74-ca1d-4325-ad9d-5408bcefcbaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.020377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665fa74-ca1d-4325-ad9d-5408bcefcbaf None admin] RBAC: Adding query filter params (role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93, user.id=18e8396b4a734ad6bea128b4117494d1) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:11.020687 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665fa74-ca1d-4325-ad9d-5408bcefcbaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:11.020953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-8f4d214a-5dea-44c7-8877-1387ac35207b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:11.022182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665fa74-ca1d-4325-ad9d-5408bcefcbaf None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:11.030439 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 108/221] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:10 2022] GET /identity/v3/role_assignments?user.id=18e8396b4a734ad6bea128b4117494d1&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:11.031582 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3232b86-14a7-4fb2-a0de-f862d1ad2ef1 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:11.031934 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3232b86-14a7-4fb2-a0de-f862d1ad2ef1 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:11.032216 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3232b86-14a7-4fb2-a0de-f862d1ad2ef1 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.033871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 109/222] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:11.044459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882faf-fc8d-404c-900c-d16b2f6efdab None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:11.046552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882faf-fc8d-404c-900c-d16b2f6efdab None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:11.047277 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882faf-fc8d-404c-900c-d16b2f6efdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.081520 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-71882faf-fc8d-404c-900c-d16b2f6efdab None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:11.083182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-71882faf-fc8d-404c-900c-d16b2f6efdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:11.154509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:11.168115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 114/223] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:11.202602 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 110/224] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:11.207627 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d214a-5dea-44c7-8877-1387ac35207b None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:11.233731 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71882faf-fc8d-404c-900c-d16b2f6efdab None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:11.365229 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d214a-5dea-44c7-8877-1387ac35207b None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:11.379488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:11.408981 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71882faf-fc8d-404c-900c-d16b2f6efdab None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:11.432829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc837f9a-ab0e-4f23-b1c1-dc5ed800bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:11.433500 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc837f9a-ab0e-4f23-b1c1-dc5ed800bcae None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:11.433785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc837f9a-ab0e-4f23-b1c1-dc5ed800bcae None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:11.434053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc837f9a-ab0e-4f23-b1c1-dc5ed800bcae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.462256 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f813e9a-ed79-4940-a65b-3a3aef4252f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:11.463341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f813e9a-ed79-4940-a65b-3a3aef4252f5 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:11.463715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f813e9a-ed79-4940-a65b-3a3aef4252f5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:11.464088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f813e9a-ed79-4940-a65b-3a3aef4252f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.474798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc837f9a-ab0e-4f23-b1c1-dc5ed800bcae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:11.475878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f813e9a-ed79-4940-a65b-3a3aef4252f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-fc837f9a-ab0e-4f23-b1c1-dc5ed800bcae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:11.479931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:11.481205 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:11.482413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 115/225] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-9f813e9a-ed79-4940-a65b-3a3aef4252f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:11.484391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:11.485681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:11.487272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 111/226] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:26:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:11.497828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc837f9a-ab0e-4f23-b1c1-dc5ed800bcae None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:11.498163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f813e9a-ed79-4940-a65b-3a3aef4252f5 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:11.636672 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:11.643478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f813e9a-ed79-4940-a65b-3a3aef4252f5 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:11.706889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:11.707604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:11.707962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:11.708246 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.708996 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:11.709307 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:11.710759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:11.711095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc837f9a-ab0e-4f23-b1c1-dc5ed800bcae None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:11.729648 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 112/227] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:11.735352 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:11.747321 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:11.749480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:11.750198 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:11.750465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:11.750738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.751474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:11.751785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:11.753398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:11.762877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 116/228] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:11.769008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:11.948695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7482387-970f-4355-b026-7f93e43a4203 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:11.990083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a710f4e-29ed-43a0-a790-3ed117f944de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:11.990678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a710f4e-29ed-43a0-a790-3ed117f944de None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:11.990961 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a710f4e-29ed-43a0-a790-3ed117f944de None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:11.991890 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a710f4e-29ed-43a0-a790-3ed117f944de None admin] PATH_INFO: `/v3/users/740ebdac18f444bb8f90ad4538f791ee` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:11.993283 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a710f4e-29ed-43a0-a790-3ed117f944de None admin] RBAC: Authorizing `identity:get_user(user_id=740ebdac18f444bb8f90ad4538f791ee)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:11.994800 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a710f4e-29ed-43a0-a790-3ed117f944de None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:11.996803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 113/229] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:11 2022] GET /identity/v3/users/740ebdac18f444bb8f90ad4538f791ee => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:12.002279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a710f4e-29ed-43a0-a790-3ed117f944de None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:12.014938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ba07-a385-49da-b1ae-6f3d2e930c35 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:12.056233 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:12.088661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6335a0e-f1d1-4ca0-8090-84f3ec9deeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:12.089738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6335a0e-f1d1-4ca0-8090-84f3ec9deeda None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.090269 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6335a0e-f1d1-4ca0-8090-84f3ec9deeda None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.090764 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6335a0e-f1d1-4ca0-8090-84f3ec9deeda None admin] PATH_INFO: `/v3/users/740ebdac18f444bb8f90ad4538f791ee` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:12.093920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6335a0e-f1d1-4ca0-8090-84f3ec9deeda None admin] RBAC: Authorizing `identity:get_user(user_id=740ebdac18f444bb8f90ad4538f791ee)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:12.095671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6335a0e-f1d1-4ca0-8090-84f3ec9deeda None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:12.097783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 117/230] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:11 2022] GET /identity/v3/users/740ebdac18f444bb8f90ad4538f791ee => generated 283 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:12.103676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6335a0e-f1d1-4ca0-8090-84f3ec9deeda None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:12.149532 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:12.202096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a710f4e-29ed-43a0-a790-3ed117f944de None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:12.241653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dcb8a-7bed-4947-a9f8-eefdf885a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:12.243069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dcb8a-7bed-4947-a9f8-eefdf885a77b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.243570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dcb8a-7bed-4947-a9f8-eefdf885a77b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.244057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dcb8a-7bed-4947-a9f8-eefdf885a77b None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:12.245527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dcb8a-7bed-4947-a9f8-eefdf885a77b None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:12.247843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dcb8a-7bed-4947-a9f8-eefdf885a77b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:12.249903 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 114/231] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:12 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:12.256257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dcb8a-7bed-4947-a9f8-eefdf885a77b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:12.320295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6335a0e-f1d1-4ca0-8090-84f3ec9deeda None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:12.385341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad23da-ee90-496f-b97a-5d1cebec631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:12.386058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad23da-ee90-496f-b97a-5d1cebec631d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.386361 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad23da-ee90-496f-b97a-5d1cebec631d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.386682 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad23da-ee90-496f-b97a-5d1cebec631d None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:12.387897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad23da-ee90-496f-b97a-5d1cebec631d None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:12.389377 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad23da-ee90-496f-b97a-5d1cebec631d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:12.391469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 118/232] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:12 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:12.397234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad23da-ee90-496f-b97a-5d1cebec631d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:12.446836 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dcb8a-7bed-4947-a9f8-eefdf885a77b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:12.503184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:12.513918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8c4d3-022d-44f9-b73c-2884515ac087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:12.514652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8c4d3-022d-44f9-b73c-2884515ac087 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.514938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8c4d3-022d-44f9-b73c-2884515ac087 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.515218 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8c4d3-022d-44f9-b73c-2884515ac087 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/users/740ebdac18f444bb8f90ad4538f791ee/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:12.519300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8c4d3-022d-44f9-b73c-2884515ac087 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93, user_id=740ebdac18f444bb8f90ad4538f791ee, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:12.521762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8c4d3-022d-44f9-b73c-2884515ac087 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:12.539262 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 115/233] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:12 2022] PUT /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/users/740ebdac18f444bb8f90ad4538f791ee/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:26:12.540456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:12.604088 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad23da-ee90-496f-b97a-5d1cebec631d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:12.669797 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e3178-8836-417c-8dc8-3d5a88ccbefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:12.670921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3178-8836-417c-8dc8-3d5a88ccbefd None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.671452 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3178-8836-417c-8dc8-3d5a88ccbefd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.671945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3178-8836-417c-8dc8-3d5a88ccbefd None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/users/740ebdac18f444bb8f90ad4538f791ee/roles/d883529ef5284c35b27b759c57353383` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:12.674503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e3178-8836-417c-8dc8-3d5a88ccbefd None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93, user_id=740ebdac18f444bb8f90ad4538f791ee, role_id=d883529ef5284c35b27b759c57353383)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:12.676346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e3178-8836-417c-8dc8-3d5a88ccbefd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:12.710567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 119/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:12 2022] PUT /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/users/740ebdac18f444bb8f90ad4538f791ee/roles/d883529ef5284c35b27b759c57353383 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:26:12.893136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53e9bb-f722-4859-a30d-8ea1b30b7b70 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.893495 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53e9bb-f722-4859-a30d-8ea1b30b7b70 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.893686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53e9bb-f722-4859-a30d-8ea1b30b7b70 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:12.895588 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 116/235] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:12.901132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc103a-c5b8-4970-908c-5aa0de2670cf None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.902082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc103a-c5b8-4970-908c-5aa0de2670cf None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.902599 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc103a-c5b8-4970-908c-5aa0de2670cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:12.926712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-c5bc103a-c5b8-4970-908c-5aa0de2670cf None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:12.928268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-c5bc103a-c5b8-4970-908c-5aa0de2670cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:12.981221 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaea4f-611a-49a3-a501-fb97c831be80 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.981764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaea4f-611a-49a3-a501-fb97c831be80 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.981764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaea4f-611a-49a3-a501-fb97c831be80 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:12.984653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 117/236] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:12.989318 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ce30-8444-417b-87e1-caf196d09374 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:12.989643 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ce30-8444-417b-87e1-caf196d09374 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:12.989876 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ce30-8444-417b-87e1-caf196d09374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:13.008650 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 120/237] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:13.023892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-7043ce30-8444-417b-87e1-caf196d09374 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:13.025519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-7043ce30-8444-417b-87e1-caf196d09374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:13.035944 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc103a-c5b8-4970-908c-5aa0de2670cf None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:13.130051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:13.159606 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 118/238] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:13.196717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc103a-c5b8-4970-908c-5aa0de2670cf None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:13.199198 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043ce30-8444-417b-87e1-caf196d09374 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:13.235435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e077f-e21e-4726-ba5e-f753d9191d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:13.236787 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e077f-e21e-4726-ba5e-f753d9191d65 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:13.237293 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e077f-e21e-4726-ba5e-f753d9191d65 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:13.237763 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e077f-e21e-4726-ba5e-f753d9191d65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:13.239207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e077f-e21e-4726-ba5e-f753d9191d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:13.240927 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e077f-e21e-4726-ba5e-f753d9191d65 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:13.243599 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 121/239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:13.251164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e077f-e21e-4726-ba5e-f753d9191d65 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:13.419273 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043ce30-8444-417b-87e1-caf196d09374 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:13.434516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e077f-e21e-4726-ba5e-f753d9191d65 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:13.449555 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:13.458681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:13.466657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf409651-c2bb-4b55-b2aa-095167ca7fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:13.468362 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409651-c2bb-4b55-b2aa-095167ca7fa2 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:13.468644 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409651-c2bb-4b55-b2aa-095167ca7fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:13.468950 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409651-c2bb-4b55-b2aa-095167ca7fa2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:13.476820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb1ff3-883c-4bc0-9e18-c76fc2b6bd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:13.477460 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1ff3-883c-4bc0-9e18-c76fc2b6bd35 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:13.477740 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1ff3-883c-4bc0-9e18-c76fc2b6bd35 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:13.478016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1ff3-883c-4bc0-9e18-c76fc2b6bd35 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:13.478748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb1ff3-883c-4bc0-9e18-c76fc2b6bd35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:13.480255 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb1ff3-883c-4bc0-9e18-c76fc2b6bd35 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:13.483355 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf409651-c2bb-4b55-b2aa-095167ca7fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-bf409651-c2bb-4b55-b2aa-095167ca7fa2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:13.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:13.491329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:13.492702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:13.495748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 119/240] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:13.497590 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb0f3a-28f2-4089-96ad-7f554e6ac27c None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:13.498358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb0f3a-28f2-4089-96ad-7f554e6ac27c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:13.499048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb0f3a-28f2-4089-96ad-7f554e6ac27c None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:13.500979 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 120/241] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:13.501908 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb0f3a-28f2-4089-96ad-7f554e6ac27c None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:13.511974 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 122/242] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:13 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:26:13.518223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46514259-d58d-4d3b-984c-f6caa0d2124a None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:13.518223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46514259-d58d-4d3b-984c-f6caa0d2124a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:13.518223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46514259-d58d-4d3b-984c-f6caa0d2124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:13.548925 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-46514259-d58d-4d3b-984c-f6caa0d2124a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:13.553831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-46514259-d58d-4d3b-984c-f6caa0d2124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:13.710634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 123/243] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:13.725865 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb0f3a-28f2-4089-96ad-7f554e6ac27c None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:13.744072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46514259-d58d-4d3b-984c-f6caa0d2124a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:13.777325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:13.777973 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:13.778301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:13.778576 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:13.779678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] RBAC: Adding query filter params (name=reader) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:13.779994 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:13.781705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:13.796892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 121/244] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:13.802823 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:13.886385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:13.892161 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:13.897691 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46514259-d58d-4d3b-984c-f6caa0d2124a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:13.953231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a9e3b-89a0-43a9-b50b-527f18f1ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:13.955007 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a9e3b-89a0-43a9-b50b-527f18f1ef00 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:13.955512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a9e3b-89a0-43a9-b50b-527f18f1ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:13.956082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a9e3b-89a0-43a9-b50b-527f18f1ef00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:13.956569 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e51c47-3820-4a68-a738-d72fca97d848 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:13.970702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a9e3b-89a0-43a9-b50b-527f18f1ef00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-770a9e3b-89a0-43a9-b50b-527f18f1ef00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:13.978474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:13.979539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:13.980454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:13.983587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 124/245] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:13.990693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a9e3b-89a0-43a9-b50b-527f18f1ef00 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:14.017297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047ca61-1341-4187-82e5-af2446f81cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:14.018478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047ca61-1341-4187-82e5-af2446f81cd6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:14.019265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047ca61-1341-4187-82e5-af2446f81cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:14.019734 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047ca61-1341-4187-82e5-af2446f81cd6 None admin] PATH_INFO: `/v3/users/2c271af4e6324d4c9608780f154f7cd6` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:14.048074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047ca61-1341-4187-82e5-af2446f81cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=2c271af4e6324d4c9608780f154f7cd6)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:14.049886 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047ca61-1341-4187-82e5-af2446f81cd6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:14.052560 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 122/246] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:13 2022] GET /identity/v3/users/2c271af4e6324d4c9608780f154f7cd6 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:14.059032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047ca61-1341-4187-82e5-af2446f81cd6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:14.158844 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a9e3b-89a0-43a9-b50b-527f18f1ef00 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:14.183433 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047ca61-1341-4187-82e5-af2446f81cd6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:14.196303 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:14.226536 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd557f6d-eda4-42cb-83aa-c6561d7f85e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:14.228025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd557f6d-eda4-42cb-83aa-c6561d7f85e7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:14.228846 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:14.229381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd557f6d-eda4-42cb-83aa-c6561d7f85e7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:14.230229 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:14.230714 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:14.230826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd557f6d-eda4-42cb-83aa-c6561d7f85e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:14.231268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:14.232211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:14.232817 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:14.234476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:14.246764 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 125/247] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:14.259486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:14.297236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:14.362431 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd557f6d-eda4-42cb-83aa-c6561d7f85e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:14.364327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd557f6d-eda4-42cb-83aa-c6561d7f85e7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:14.402696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d270c79-b457-43ec-a660-7a1d0ce666b0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:14.439566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabbd1c-39fb-4879-bf87-a15b5f857148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:14.440334 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabbd1c-39fb-4879-bf87-a15b5f857148 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:14.440703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabbd1c-39fb-4879-bf87-a15b5f857148 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:14.441051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabbd1c-39fb-4879-bf87-a15b5f857148 None admin] PATH_INFO: `/v3/users/18e8396b4a734ad6bea128b4117494d1` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:14.442423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabbd1c-39fb-4879-bf87-a15b5f857148 None admin] RBAC: Authorizing `identity:get_user(user_id=18e8396b4a734ad6bea128b4117494d1)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:14.444376 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabbd1c-39fb-4879-bf87-a15b5f857148 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:14.446521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 126/248] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:14 2022] GET /identity/v3/users/18e8396b4a734ad6bea128b4117494d1 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:14.452198 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabbd1c-39fb-4879-bf87-a15b5f857148 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:14.479877 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 123/249] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:26:14.487320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd557f6d-eda4-42cb-83aa-c6561d7f85e7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:14.508163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:14.607649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:14.612915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd557f6d-eda4-42cb-83aa-c6561d7f85e7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:14.626886 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabbd1c-39fb-4879-bf87-a15b5f857148 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:14.663754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bccd79-2cb5-427f-b5d6-b8250c5f6bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:14.665353 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bccd79-2cb5-427f-b5d6-b8250c5f6bab None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:14.666084 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bccd79-2cb5-427f-b5d6-b8250c5f6bab None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:14.668629 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bccd79-2cb5-427f-b5d6-b8250c5f6bab None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:14.670475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bccd79-2cb5-427f-b5d6-b8250c5f6bab None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:14.674002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bccd79-2cb5-427f-b5d6-b8250c5f6bab None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:14.677050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 124/250] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:14 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:14.690801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bccd79-2cb5-427f-b5d6-b8250c5f6bab None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:14.697618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b775a6-45e4-4599-a27d-adf166515830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:14.698343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b775a6-45e4-4599-a27d-adf166515830 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:14.698628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b775a6-45e4-4599-a27d-adf166515830 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:14.698911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b775a6-45e4-4599-a27d-adf166515830 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:14.702158 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b775a6-45e4-4599-a27d-adf166515830 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:14.702158 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b775a6-45e4-4599-a27d-adf166515830 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:14.703830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 127/251] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:14 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:14.710320 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b775a6-45e4-4599-a27d-adf166515830 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:14.826855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b775a6-45e4-4599-a27d-adf166515830 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:14.863875 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b47de8-7c5b-43cb-8754-68bd2a72d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:14.864884 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b47de8-7c5b-43cb-8754-68bd2a72d26b None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:14.865321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b47de8-7c5b-43cb-8754-68bd2a72d26b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:14.865754 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b47de8-7c5b-43cb-8754-68bd2a72d26b None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/users/18e8396b4a734ad6bea128b4117494d1/roles/cc0fab8f90a44acc90de9af43e2bd7df` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:14.866506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:14.868231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b47de8-7c5b-43cb-8754-68bd2a72d26b None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93, user_id=18e8396b4a734ad6bea128b4117494d1, role_id=cc0fab8f90a44acc90de9af43e2bd7df)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:14.869895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b47de8-7c5b-43cb-8754-68bd2a72d26b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:14.888300 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 128/252] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:14 2022] PUT /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/users/18e8396b4a734ad6bea128b4117494d1/roles/cc0fab8f90a44acc90de9af43e2bd7df => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:14.913893 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bccd79-2cb5-427f-b5d6-b8250c5f6bab None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:14.975068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a9158-65fa-4da8-9f32-28e1277f98d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:14.975759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a9158-65fa-4da8-9f32-28e1277f98d4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:14.976041 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a9158-65fa-4da8-9f32-28e1277f98d4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:14.976312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a9158-65fa-4da8-9f32-28e1277f98d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:14.977122 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a9158-65fa-4da8-9f32-28e1277f98d4 None admin] RBAC: Adding query filter params (role.id=e4367af82d094e189d2679fe6b2c6f9f, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93, user.id=2c271af4e6324d4c9608780f154f7cd6) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:14.977433 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a9158-65fa-4da8-9f32-28e1277f98d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:14.978938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a9158-65fa-4da8-9f32-28e1277f98d4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:14.987313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 125/253] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:14 2022] GET /identity/v3/role_assignments?user.id=2c271af4e6324d4c9608780f154f7cd6&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=e4367af82d094e189d2679fe6b2c6f9f => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:15.253325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb4a12-bbda-4fd2-a950-92dde1948ef6 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:15.253952 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb4a12-bbda-4fd2-a950-92dde1948ef6 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:15.253952 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb4a12-bbda-4fd2-a950-92dde1948ef6 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:15.255648 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 129/254] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:15.262262 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0232919-3a06-47c2-83ee-478ff0ed8863 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:15.262808 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0232919-3a06-47c2-83ee-478ff0ed8863 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:15.262808 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0232919-3a06-47c2-83ee-478ff0ed8863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:15.292063 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-e0232919-3a06-47c2-83ee-478ff0ed8863 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:15.293363 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-e0232919-3a06-47c2-83ee-478ff0ed8863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:15.387867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 126/255] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:15.428610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb4a12-bbda-4fd2-a950-92dde1948ef6 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:15.454871 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:15.586048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb4a12-bbda-4fd2-a950-92dde1948ef6 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:15.635326 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520506b-e49b-484e-907c-f7513993df36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:15.636022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520506b-e49b-484e-907c-f7513993df36 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:15.636307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520506b-e49b-484e-907c-f7513993df36 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:15.636588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520506b-e49b-484e-907c-f7513993df36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:15.657744 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520506b-e49b-484e-907c-f7513993df36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-f520506b-e49b-484e-907c-f7513993df36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:15.663277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:15.664511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:15.665507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 130/256] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:26:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:15.670918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0232919-3a06-47c2-83ee-478ff0ed8863 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:15.794930 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:15.831523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0232919-3a06-47c2-83ee-478ff0ed8863 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:15.880093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:15.880816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:15.881037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e98d-4593-4d2e-871e-8429044daea4 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:15.881161 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:15.881370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:15.881472 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e98d-4593-4d2e-871e-8429044daea4 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:15.881757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e98d-4593-4d2e-871e-8429044daea4 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:15.882130 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:15.882448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:15.883734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 131/257] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:15.883852 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:15.893564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a88c76-d4d4-4734-8f99-fd031878e581 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:15.893916 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a88c76-d4d4-4734-8f99-fd031878e581 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:15.894166 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a88c76-d4d4-4734-8f99-fd031878e581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:15.897798 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 127/258] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:15.903313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:15.927156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-86a88c76-d4d4-4734-8f99-fd031878e581 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:15.928407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-86a88c76-d4d4-4734-8f99-fd031878e581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:16.040862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 132/259] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:16.080096 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a88c76-d4d4-4734-8f99-fd031878e581 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:16.106509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:16.140528 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190b9cf-a0e5-40a9-94a7-623777fb283a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:16.255496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909abc7a-d0c1-48ef-b32a-724aa5384b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:16.256189 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909abc7a-d0c1-48ef-b32a-724aa5384b25 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:16.256483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909abc7a-d0c1-48ef-b32a-724aa5384b25 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:16.256765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909abc7a-d0c1-48ef-b32a-724aa5384b25 None admin] PATH_INFO: `/v3/users/740ebdac18f444bb8f90ad4538f791ee` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:16.262870 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909abc7a-d0c1-48ef-b32a-724aa5384b25 None admin] RBAC: Authorizing `identity:get_user(user_id=740ebdac18f444bb8f90ad4538f791ee)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:16.264501 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909abc7a-d0c1-48ef-b32a-724aa5384b25 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:16.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 128/260] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:15 2022] GET /identity/v3/users/740ebdac18f444bb8f90ad4538f791ee => generated 283 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:16.270965 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a88c76-d4d4-4734-8f99-fd031878e581 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:16.272627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909abc7a-d0c1-48ef-b32a-724aa5384b25 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:16.282685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:16.334959 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570ef24-41d8-4ff3-9073-03c4d28e9453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:16.335556 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570ef24-41d8-4ff3-9073-03c4d28e9453 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:16.335845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570ef24-41d8-4ff3-9073-03c4d28e9453 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:16.336124 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570ef24-41d8-4ff3-9073-03c4d28e9453 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:16.351537 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570ef24-41d8-4ff3-9073-03c4d28e9453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-0570ef24-41d8-4ff3-9073-03c4d28e9453 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:16.356634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:16.357655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:16.358622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:16.366614 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 133/261] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:16.377796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570ef24-41d8-4ff3-9073-03c4d28e9453 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:16.492299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909abc7a-d0c1-48ef-b32a-724aa5384b25 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:16.494474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:16.533142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570ef24-41d8-4ff3-9073-03c4d28e9453 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:16.573474 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd381c0-1075-4a31-8d58-be9847dca217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:16.574151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd381c0-1075-4a31-8d58-be9847dca217 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:16.574441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd381c0-1075-4a31-8d58-be9847dca217 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:16.574719 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd381c0-1075-4a31-8d58-be9847dca217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:16.606938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:16.607510 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:16.607798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:16.608080 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:16.608833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:16.609148 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:16.610654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:16.632289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 134/262] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:16.643252 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:16.685808 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd381c0-1075-4a31-8d58-be9847dca217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:16.687355 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd381c0-1075-4a31-8d58-be9847dca217 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:16.689767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:16.831109 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c92e3-dfcd-4296-b6a9-c59277f91439 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:16.880788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:16.899836 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5bba-2f69-4fe2-be0b-04daf4f365a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:16.900514 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5bba-2f69-4fe2-be0b-04daf4f365a5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:16.900794 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5bba-2f69-4fe2-be0b-04daf4f365a5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:16.901077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5bba-2f69-4fe2-be0b-04daf4f365a5 None admin] PATH_INFO: `/v3/users/740ebdac18f444bb8f90ad4538f791ee` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:16.906853 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5bba-2f69-4fe2-be0b-04daf4f365a5 None admin] RBAC: Authorizing `identity:get_user(user_id=740ebdac18f444bb8f90ad4538f791ee)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:16.908355 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5bba-2f69-4fe2-be0b-04daf4f365a5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:16.910534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 135/263] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:16 2022] GET /identity/v3/users/740ebdac18f444bb8f90ad4538f791ee => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:16.913924 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 129/264] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:26:16.916209 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5bba-2f69-4fe2-be0b-04daf4f365a5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:16.919467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd381c0-1075-4a31-8d58-be9847dca217 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:17.070154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5bba-2f69-4fe2-be0b-04daf4f365a5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:17.117459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9866849-d007-472e-8c28-060a3725f9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:17.118452 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866849-d007-472e-8c28-060a3725f9c1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.118906 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866849-d007-472e-8c28-060a3725f9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.119337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866849-d007-472e-8c28-060a3725f9c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.187779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd381c0-1075-4a31-8d58-be9847dca217 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:17.189789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:17.206565 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:17.208419 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9866849-d007-472e-8c28-060a3725f9c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:17.210697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9866849-d007-472e-8c28-060a3725f9c1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:17.228582 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55385ff1-2425-43bf-876b-04fca9aab519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:17.229656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55385ff1-2425-43bf-876b-04fca9aab519 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.230275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55385ff1-2425-43bf-876b-04fca9aab519 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.231066 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55385ff1-2425-43bf-876b-04fca9aab519 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.233426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55385ff1-2425-43bf-876b-04fca9aab519 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:17.236309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55385ff1-2425-43bf-876b-04fca9aab519 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:17.239661 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 130/265] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:16 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:17.241355 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15efac7-7a62-46a6-8b36-ed8ca9fc0b62 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.241868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15efac7-7a62-46a6-8b36-ed8ca9fc0b62 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.242416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15efac7-7a62-46a6-8b36-ed8ca9fc0b62 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.244279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 131/266] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:17.245826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2564ced-c605-49ff-a4c8-051d8d5a077b None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.246378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2564ced-c605-49ff-a4c8-051d8d5a077b None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.246908 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2564ced-c605-49ff-a4c8-051d8d5a077b None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.248724 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 132/267] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:17.249816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2564ced-c605-49ff-a4c8-051d8d5a077b None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:17.350291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 136/268] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:17.353860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c056e5-eedd-4c26-9a48-1f58fb386de5 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.354175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c056e5-eedd-4c26-9a48-1f58fb386de5 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.354462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c056e5-eedd-4c26-9a48-1f58fb386de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.373295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-b8c056e5-eedd-4c26-9a48-1f58fb386de5 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:17.374798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-b8c056e5-eedd-4c26-9a48-1f58fb386de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:17.382253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2564ced-c605-49ff-a4c8-051d8d5a077b None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:17.435017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59bd8c-478f-4e30-a297-04cd59f4fd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:17.436336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59bd8c-478f-4e30-a297-04cd59f4fd18 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.436713 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59bd8c-478f-4e30-a297-04cd59f4fd18 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.437082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59bd8c-478f-4e30-a297-04cd59f4fd18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.438046 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59bd8c-478f-4e30-a297-04cd59f4fd18 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93, user.id=740ebdac18f444bb8f90ad4538f791ee) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:17.438462 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59bd8c-478f-4e30-a297-04cd59f4fd18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:17.440605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59bd8c-478f-4e30-a297-04cd59f4fd18 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:17.455080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 133/269] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:17 2022] GET /identity/v3/role_assignments?user.id=740ebdac18f444bb8f90ad4538f791ee&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:17.455080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ac533-4046-4c98-8219-1771e9e6f817 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.455080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ac533-4046-4c98-8219-1771e9e6f817 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.455080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ac533-4046-4c98-8219-1771e9e6f817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.469078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 137/270] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:17.470408 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c056e5-eedd-4c26-9a48-1f58fb386de5 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:17.478136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-fd5ac533-4046-4c98-8219-1771e9e6f817 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:17.478136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-fd5ac533-4046-4c98-8219-1771e9e6f817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:17.513771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:17.543643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:17.588317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 134/271] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:17.589643 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ac533-4046-4c98-8219-1771e9e6f817 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:17.749028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c056e5-eedd-4c26-9a48-1f58fb386de5 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:17.761150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ac533-4046-4c98-8219-1771e9e6f817 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:17.797394 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e454509-981d-4803-a991-64cbe3171323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:17.798915 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454509-981d-4803-a991-64cbe3171323 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.799521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454509-981d-4803-a991-64cbe3171323 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.800013 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454509-981d-4803-a991-64cbe3171323 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.803893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e454509-981d-4803-a991-64cbe3171323 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:17.805608 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e454509-981d-4803-a991-64cbe3171323 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:17.807432 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7831067-62f6-44e5-8901-cd9cdeacbc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:17.808505 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831067-62f6-44e5-8901-cd9cdeacbc9a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.808748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831067-62f6-44e5-8901-cd9cdeacbc9a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.809113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831067-62f6-44e5-8901-cd9cdeacbc9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.809324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 138/272] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:17 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:17.810667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e454509-981d-4803-a991-64cbe3171323 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:17.830597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7831067-62f6-44e5-8901-cd9cdeacbc9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-a7831067-62f6-44e5-8901-cd9cdeacbc9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:17.836346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:17.837384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:17.838336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:17.841249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 135/273] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:17.842430 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7831067-62f6-44e5-8901-cd9cdeacbc9a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:17.877676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:17.944762 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:17.953645 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e454509-981d-4803-a991-64cbe3171323 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:17.968049 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7831067-62f6-44e5-8901-cd9cdeacbc9a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:17.995744 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ff645-530b-4bba-bcfa-68574744818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:17.996427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ff645-530b-4bba-bcfa-68574744818b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:17.996710 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ff645-530b-4bba-bcfa-68574744818b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:17.996975 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ff645-530b-4bba-bcfa-68574744818b None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:17.998291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ff645-530b-4bba-bcfa-68574744818b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:17.999838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ff645-530b-4bba-bcfa-68574744818b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:18.001422 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 139/274] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:18.002608 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f3aa-955d-417d-b10f-cccdc1d01fba None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.002889 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f3aa-955d-417d-b10f-cccdc1d01fba None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.003166 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f3aa-955d-417d-b10f-cccdc1d01fba None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.004641 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 140/275] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:18.005183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2f3aa-955d-417d-b10f-cccdc1d01fba None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:18.009045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77393118-ab5c-45ba-87b0-387f8d713860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:18.009668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393118-ab5c-45ba-87b0-387f8d713860 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.009944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393118-ab5c-45ba-87b0-387f8d713860 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.010388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393118-ab5c-45ba-87b0-387f8d713860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.011191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393118-ab5c-45ba-87b0-387f8d713860 None admin] RBAC: Adding query filter params (role.id=d883529ef5284c35b27b759c57353383, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93, user.id=740ebdac18f444bb8f90ad4538f791ee) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:18.011499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393118-ab5c-45ba-87b0-387f8d713860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:18.012992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393118-ab5c-45ba-87b0-387f8d713860 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:18.019489 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 136/276] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:17 2022] GET /identity/v3/role_assignments?user.id=740ebdac18f444bb8f90ad4538f791ee&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=d883529ef5284c35b27b759c57353383 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:18.021928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7c323-1483-440f-ae8e-9364d83d1b62 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.022267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7c323-1483-440f-ae8e-9364d83d1b62 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.022833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7c323-1483-440f-ae8e-9364d83d1b62 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.024054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 137/277] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:18.024705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7c323-1483-440f-ae8e-9364d83d1b62 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:18.131388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2f3aa-955d-417d-b10f-cccdc1d01fba None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:18.140578 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:18.160283 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7c323-1483-440f-ae8e-9364d83d1b62 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:18.177670 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207691f7-5a91-499f-b736-71dcd3db0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:18.178505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207691f7-5a91-499f-b736-71dcd3db0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.178794 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207691f7-5a91-499f-b736-71dcd3db0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.179075 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207691f7-5a91-499f-b736-71dcd3db0ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.179820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207691f7-5a91-499f-b736-71dcd3db0ba1 None admin] RBAC: Adding query filter params (name=member) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:18.180206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207691f7-5a91-499f-b736-71dcd3db0ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:18.181680 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207691f7-5a91-499f-b736-71dcd3db0ba1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:18.197341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 141/278] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:18.198931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a14b9-783c-4b14-beb5-a492a5e50bcb None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.199090 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a14b9-783c-4b14-beb5-a492a5e50bcb None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.199343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a14b9-783c-4b14-beb5-a492a5e50bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.214176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131b954-a3b0-4cdb-9fbe-7153c1d64b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:18.214825 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131b954-a3b0-4cdb-9fbe-7153c1d64b94 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.215106 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131b954-a3b0-4cdb-9fbe-7153c1d64b94 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.215381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131b954-a3b0-4cdb-9fbe-7153c1d64b94 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.216100 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131b954-a3b0-4cdb-9fbe-7153c1d64b94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:18.217569 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131b954-a3b0-4cdb-9fbe-7153c1d64b94 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:18.220250 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-682a14b9-783c-4b14-beb5-a492a5e50bcb None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:18.221468 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-682a14b9-783c-4b14-beb5-a492a5e50bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:18.246367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:18.254374 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 138/279] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:18 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:18.260424 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e44d-fc14-407c-ba5c-8815a732b2bf None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.260778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e44d-fc14-407c-ba5c-8815a732b2bf None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.261005 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e44d-fc14-407c-ba5c-8815a732b2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.284801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-3d28e44d-fc14-407c-ba5c-8815a732b2bf None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:18.286585 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-3d28e44d-fc14-407c-ba5c-8815a732b2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:18.324212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 142/280] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:18.324931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a14b9-783c-4b14-beb5-a492a5e50bcb None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:18.399516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 139/281] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:18.409462 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28e44d-fc14-407c-ba5c-8815a732b2bf None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:18.469389 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a14b9-783c-4b14-beb5-a492a5e50bcb None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:18.502416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:18.523115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a270ba8-871e-4fad-939c-a420a5b60dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:18.523115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a270ba8-871e-4fad-939c-a420a5b60dbb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.523550 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a270ba8-871e-4fad-939c-a420a5b60dbb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.523923 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a270ba8-871e-4fad-939c-a420a5b60dbb None admin] PATH_INFO: `/v3/users/7c255b5885684adcbb1ea47428d52bf2` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.540635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28e44d-fc14-407c-ba5c-8815a732b2bf None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:18.542885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a270ba8-871e-4fad-939c-a420a5b60dbb None admin] RBAC: Authorizing `identity:get_user(user_id=7c255b5885684adcbb1ea47428d52bf2)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:18.544964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a270ba8-871e-4fad-939c-a420a5b60dbb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:18.547434 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 143/282] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:18 2022] GET /identity/v3/users/7c255b5885684adcbb1ea47428d52bf2 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:18.548491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a270ba8-871e-4fad-939c-a420a5b60dbb None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:18.568014 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:18.588994 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc69f-efc3-4c52-b707-22b45916ec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:18.589680 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc69f-efc3-4c52-b707-22b45916ec81 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.589959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc69f-efc3-4c52-b707-22b45916ec81 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.590309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc69f-efc3-4c52-b707-22b45916ec81 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.605242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbc69f-efc3-4c52-b707-22b45916ec81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-aebbc69f-efc3-4c52-b707-22b45916ec81 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:18.610765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:18.611995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:18.613098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:18.615175 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 140/283] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:18.616261 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc69f-efc3-4c52-b707-22b45916ec81 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:18.686472 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a270ba8-871e-4fad-939c-a420a5b60dbb None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:18.726815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf4dfc-3d37-439c-a510-f90615d83440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:18.727789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf4dfc-3d37-439c-a510-f90615d83440 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.727789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf4dfc-3d37-439c-a510-f90615d83440 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.728021 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf4dfc-3d37-439c-a510-f90615d83440 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.740194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf4dfc-3d37-439c-a510-f90615d83440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-83bf4dfc-3d37-439c-a510-f90615d83440 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:18.745437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:18.746743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 144/284] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:18.747986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf4dfc-3d37-439c-a510-f90615d83440 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:18.768226 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc69f-efc3-4c52-b707-22b45916ec81 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:18.808711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:18.809619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.810096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.810654 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.811764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] RBAC: Adding query filter params (role.id=cc0fab8f90a44acc90de9af43e2bd7df, user.id=7c255b5885684adcbb1ea47428d52bf2, scope.system=all) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:18.812235 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:18.813986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:18.816226 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:18.823191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 141/285] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:26:18 2022] GET /identity/v3/role_assignments?user.id=7c255b5885684adcbb1ea47428d52bf2&scope.system=all&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:18.824354 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:18.856117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:18.925303 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf4dfc-3d37-439c-a510-f90615d83440 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:18.952018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b62bc0-3807-40c5-82eb-c4813b5860dd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:18.993158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:18.994160 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:18.994609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:18.995089 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:18.996032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:18.996502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:18.998322 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:19.001969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:19.002793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:19.003173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:19.003553 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:19.004486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:19.004898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:19.006618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:19.010309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 142/286] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:19.016621 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:19.030018 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 145/287] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:19.036143 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:19.106612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:19.144623 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57914c8-d6a3-47da-88f2-4f8c7283e10e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:19.182033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0741bc6-1028-43c1-ad92-556d253217e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:19.183925 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0741bc6-1028-43c1-ad92-556d253217e1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:19.184211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0741bc6-1028-43c1-ad92-556d253217e1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:19.184491 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0741bc6-1028-43c1-ad92-556d253217e1 None admin] PATH_INFO: `/v3/users/18e8396b4a734ad6bea128b4117494d1` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:19.187439 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0741bc6-1028-43c1-ad92-556d253217e1 None admin] RBAC: Authorizing `identity:get_user(user_id=18e8396b4a734ad6bea128b4117494d1)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:19.188936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0741bc6-1028-43c1-ad92-556d253217e1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:19.190860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 143/288] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:19 2022] GET /identity/v3/users/18e8396b4a734ad6bea128b4117494d1 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:19.199017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0741bc6-1028-43c1-ad92-556d253217e1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:19.249621 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2b866-5133-409b-884c-6f122b27f2b0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:19.320978 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dffc7-176a-4e08-92e5-2aad455956ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:19.321645 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dffc7-176a-4e08-92e5-2aad455956ea None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:19.321926 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dffc7-176a-4e08-92e5-2aad455956ea None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:19.322227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dffc7-176a-4e08-92e5-2aad455956ea None admin] PATH_INFO: `/v3/users/2c271af4e6324d4c9608780f154f7cd6` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:19.326549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dffc7-176a-4e08-92e5-2aad455956ea None admin] RBAC: Authorizing `identity:get_user(user_id=2c271af4e6324d4c9608780f154f7cd6)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:19.328031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dffc7-176a-4e08-92e5-2aad455956ea None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:19.329702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 146/289] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:19 2022] GET /identity/v3/users/2c271af4e6324d4c9608780f154f7cd6 => generated 297 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:19.336898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dffc7-176a-4e08-92e5-2aad455956ea None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:19.341544 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:19.412526 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0741bc6-1028-43c1-ad92-556d253217e1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:19.467801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:19.478181 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a478b-028b-4fae-be56-9b4de2d5e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:19.479088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a478b-028b-4fae-be56-9b4de2d5e66d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:19.479526 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a478b-028b-4fae-be56-9b4de2d5e66d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:19.479978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a478b-028b-4fae-be56-9b4de2d5e66d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:19.527403 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dffc7-176a-4e08-92e5-2aad455956ea None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:19.557007 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a478b-028b-4fae-be56-9b4de2d5e66d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:19.558599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a478b-028b-4fae-be56-9b4de2d5e66d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:19.634911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3ded9-2c74-4616-bf6c-c733fd8ae860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:19.635599 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3ded9-2c74-4616-bf6c-c733fd8ae860 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:19.635879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3ded9-2c74-4616-bf6c-c733fd8ae860 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:19.636152 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3ded9-2c74-4616-bf6c-c733fd8ae860 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:19.637427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3ded9-2c74-4616-bf6c-c733fd8ae860 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:19.639039 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3ded9-2c74-4616-bf6c-c733fd8ae860 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:19.641242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 147/290] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:19 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:19.659521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3ded9-2c74-4616-bf6c-c733fd8ae860 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:19.678618 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 144/291] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:19.684648 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a478b-028b-4fae-be56-9b4de2d5e66d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:19.716531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:19.799833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a478b-028b-4fae-be56-9b4de2d5e66d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:19.823465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:19.845075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf2f6-731b-40d8-a8b5-0d1febe0e612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:19.845743 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf2f6-731b-40d8-a8b5-0d1febe0e612 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:19.846025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf2f6-731b-40d8-a8b5-0d1febe0e612 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:19.846330 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf2f6-731b-40d8-a8b5-0d1febe0e612 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:19.847668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cf2f6-731b-40d8-a8b5-0d1febe0e612 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:19.849150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cf2f6-731b-40d8-a8b5-0d1febe0e612 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:19.856261 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 145/292] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:19 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:19.857785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf2f6-731b-40d8-a8b5-0d1febe0e612 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:19.891269 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3ded9-2c74-4616-bf6c-c733fd8ae860 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:19.963656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794b28c-6fb7-4820-a30f-67cd7873aace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:19.964287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b28c-6fb7-4820-a30f-67cd7873aace None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:19.964571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b28c-6fb7-4820-a30f-67cd7873aace None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:19.964864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b28c-6fb7-4820-a30f-67cd7873aace None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/users/2c271af4e6324d4c9608780f154f7cd6/roles/e4367af82d094e189d2679fe6b2c6f9f` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:19.966357 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:19.971594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794b28c-6fb7-4820-a30f-67cd7873aace None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93, user_id=2c271af4e6324d4c9608780f154f7cd6, role_id=e4367af82d094e189d2679fe6b2c6f9f)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:19.973226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794b28c-6fb7-4820-a30f-67cd7873aace None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:19.973435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf2f6-731b-40d8-a8b5-0d1febe0e612 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:19.996820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 148/293] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:19 2022] PUT /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/users/2c271af4e6324d4c9608780f154f7cd6/roles/e4367af82d094e189d2679fe6b2c6f9f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:26:20.015999 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c5e36-a5e5-4581-b441-9d341c63961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:20.016666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5e36-a5e5-4581-b441-9d341c63961c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:20.016948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5e36-a5e5-4581-b441-9d341c63961c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:20.017224 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5e36-a5e5-4581-b441-9d341c63961c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:20.018054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c5e36-a5e5-4581-b441-9d341c63961c None admin] RBAC: Adding query filter params (role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93, user.id=18e8396b4a734ad6bea128b4117494d1) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:20.018435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c5e36-a5e5-4581-b441-9d341c63961c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:20.020012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c5e36-a5e5-4581-b441-9d341c63961c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:20.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 146/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:19 2022] GET /identity/v3/role_assignments?user.id=18e8396b4a734ad6bea128b4117494d1&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:20.857161 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5fb6-0b5b-4d02-9466-6659249c0fde None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:20.857966 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5fb6-0b5b-4d02-9466-6659249c0fde None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:20.857966 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5fb6-0b5b-4d02-9466-6659249c0fde None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:20.859285 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 149/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:20.862748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd579-c7ef-462c-8b12-8afdbb145ac2 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:20.863533 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd579-c7ef-462c-8b12-8afdbb145ac2 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:20.863533 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd579-c7ef-462c-8b12-8afdbb145ac2 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:20.864764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 147/296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:20.870073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d511-9fc9-4b6b-a004-52242f87e687 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:20.870541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633b92-d3a5-4363-af20-95351304e7b7 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:20.870726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633b92-d3a5-4363-af20-95351304e7b7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:20.871035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633b92-d3a5-4363-af20-95351304e7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:20.871807 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d511-9fc9-4b6b-a004-52242f87e687 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:20.872569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d511-9fc9-4b6b-a004-52242f87e687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:20.890351 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-67633b92-d3a5-4363-af20-95351304e7b7 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:20.891571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-67633b92-d3a5-4363-af20-95351304e7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:20.897013 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-80b3d511-9fc9-4b6b-a004-52242f87e687 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:20.898512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-80b3d511-9fc9-4b6b-a004-52242f87e687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:20.983132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 150/297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:20.988257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 148/298] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:21.006947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67633b92-d3a5-4363-af20-95351304e7b7 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:21.011142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3d511-9fc9-4b6b-a004-52242f87e687 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:21.083774 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:21.118680 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:21.129690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67633b92-d3a5-4363-af20-95351304e7b7 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:21.130444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3d511-9fc9-4b6b-a004-52242f87e687 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:21.168324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a190fb-ca47-4ca8-a334-a1a0462b7888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:21.168949 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba5f40-54fd-4a7c-b540-826659e9b0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:21.169494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a190fb-ca47-4ca8-a334-a1a0462b7888 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.169754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba5f40-54fd-4a7c-b540-826659e9b0dd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.170126 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba5f40-54fd-4a7c-b540-826659e9b0dd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.170277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a190fb-ca47-4ca8-a334-a1a0462b7888 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.170684 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a190fb-ca47-4ca8-a334-a1a0462b7888 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.170958 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba5f40-54fd-4a7c-b540-826659e9b0dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.172532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a190fb-ca47-4ca8-a334-a1a0462b7888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:21.174410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a190fb-ca47-4ca8-a334-a1a0462b7888 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:21.176685 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 151/299] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:21.183482 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a190fb-ca47-4ca8-a334-a1a0462b7888 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:21.189677 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba5f40-54fd-4a7c-b540-826659e9b0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-c0ba5f40-54fd-4a7c-b540-826659e9b0dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:21.196233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:21.197442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:21.198388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 149/300] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:21.204783 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba5f40-54fd-4a7c-b540-826659e9b0dd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:21.316943 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba5f40-54fd-4a7c-b540-826659e9b0dd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:21.317213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a190fb-ca47-4ca8-a334-a1a0462b7888 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:21.353989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:21.360780 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40da6a-1f14-4a94-a092-2eee04dbbb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:21.361433 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40da6a-1f14-4a94-a092-2eee04dbbb51 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.361713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40da6a-1f14-4a94-a092-2eee04dbbb51 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.361996 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40da6a-1f14-4a94-a092-2eee04dbbb51 None admin] PATH_INFO: `/v3/groups` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.362716 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40da6a-1f14-4a94-a092-2eee04dbbb51 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:21.364076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af567ebc-f76e-42e1-9e33-d31e556adbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:21.364280 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40da6a-1f14-4a94-a092-2eee04dbbb51 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:21.365078 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567ebc-f76e-42e1-9e33-d31e556adbed None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.365527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567ebc-f76e-42e1-9e33-d31e556adbed None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.365964 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567ebc-f76e-42e1-9e33-d31e556adbed None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.367037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af567ebc-f76e-42e1-9e33-d31e556adbed None admin] RBAC: Adding query filter params (name=reader) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:21.367544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af567ebc-f76e-42e1-9e33-d31e556adbed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:21.369378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af567ebc-f76e-42e1-9e33-d31e556adbed None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:21.380659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 150/301] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:21.381977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a07cf9-0011-46ef-86c8-7cd45f927262 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.382371 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a07cf9-0011-46ef-86c8-7cd45f927262 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.382697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a07cf9-0011-46ef-86c8-7cd45f927262 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.384273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 152/303] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:26:21 2022] POST /identity/v3/groups => generated 229 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:21.384395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 151/303] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:21.396196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40da6a-1f14-4a94-a092-2eee04dbbb51 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:21.396624 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02981-f9a1-44e4-9b9f-cbf8250de5d5 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.397081 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02981-f9a1-44e4-9b9f-cbf8250de5d5 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.397358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02981-f9a1-44e4-9b9f-cbf8250de5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.425980 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-22b02981-f9a1-44e4-9b9f-cbf8250de5d5 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:21.428247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-22b02981-f9a1-44e4-9b9f-cbf8250de5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:21.441055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:21.542298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 152/304] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:21.578882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b02981-f9a1-44e4-9b9f-cbf8250de5d5 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:21.595564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40da6a-1f14-4a94-a092-2eee04dbbb51 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:21.646453 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1294cd-c81e-432c-8dc7-2c2b93057f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:21.647442 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1294cd-c81e-432c-8dc7-2c2b93057f2a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.647915 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1294cd-c81e-432c-8dc7-2c2b93057f2a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.648200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1294cd-c81e-432c-8dc7-2c2b93057f2a None admin] PATH_INFO: `/v3/users/338b57e89a934128afd5d93b53551d6a` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.663564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1294cd-c81e-432c-8dc7-2c2b93057f2a None admin] RBAC: Authorizing `identity:get_user(user_id=338b57e89a934128afd5d93b53551d6a)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:21.665070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1294cd-c81e-432c-8dc7-2c2b93057f2a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:21.667095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 153/305] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:21 2022] GET /identity/v3/users/338b57e89a934128afd5d93b53551d6a => generated 293 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:21.673469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1294cd-c81e-432c-8dc7-2c2b93057f2a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b02981-f9a1-44e4-9b9f-cbf8250de5d5 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:21.730039 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:21.771950 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b37cb-c5f7-43ae-9275-68cfe07f7bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:21.773289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b37cb-c5f7-43ae-9275-68cfe07f7bb2 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.774360 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b37cb-c5f7-43ae-9275-68cfe07f7bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.774879 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b37cb-c5f7-43ae-9275-68cfe07f7bb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.790095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b37cb-c5f7-43ae-9275-68cfe07f7bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-567b37cb-c5f7-43ae-9275-68cfe07f7bb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:21.796372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:21.797444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:21.798580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 153/306] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:21.806776 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b37cb-c5f7-43ae-9275-68cfe07f7bb2 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:21.806990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1294cd-c81e-432c-8dc7-2c2b93057f2a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:21.814024 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:21.847573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:21.848253 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.848529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.848800 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.849612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] RBAC: Adding query filter params (role.id=e4367af82d094e189d2679fe6b2c6f9f, user.id=338b57e89a934128afd5d93b53551d6a, scope.system=all) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:21.849921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:21.851539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:21.859218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 154/307] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:26:21 2022] GET /identity/v3/role_assignments?user.id=338b57e89a934128afd5d93b53551d6a&scope.system=all&role.id=e4367af82d094e189d2679fe6b2c6f9f => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:21.941604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b37cb-c5f7-43ae-9275-68cfe07f7bb2 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:21.984789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:21.985458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:21.985741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:21.986019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:21.986801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:21.987113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:21.988627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:21.999449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 154/308] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:22.005428 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:22.119784 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01815f3-3599-4c22-94ad-cfd0be4e38df None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:22.134299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:22.158530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486041-2a56-49cc-ae2a-1a2c650c2b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:22.159200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486041-2a56-49cc-ae2a-1a2c650c2b73 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:22.159485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486041-2a56-49cc-ae2a-1a2c650c2b73 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:22.159767 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486041-2a56-49cc-ae2a-1a2c650c2b73 None admin] PATH_INFO: `/v3/users/7c255b5885684adcbb1ea47428d52bf2` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:22.161132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486041-2a56-49cc-ae2a-1a2c650c2b73 None admin] RBAC: Authorizing `identity:get_user(user_id=7c255b5885684adcbb1ea47428d52bf2)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:22.162648 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486041-2a56-49cc-ae2a-1a2c650c2b73 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:22.164503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 155/309] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:22 2022] GET /identity/v3/users/7c255b5885684adcbb1ea47428d52bf2 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:22.170129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:22.241682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:22.269157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8443f-c943-4fd5-8ade-4736945cec27 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:22.269612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8443f-c943-4fd5-8ade-4736945cec27 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:22.270051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8443f-c943-4fd5-8ade-4736945cec27 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:22.276759 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 156/310] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:22.278770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe864-dccb-4dcf-8fbc-37758b9c4601 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:22.279058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe864-dccb-4dcf-8fbc-37758b9c4601 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:22.279334 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe864-dccb-4dcf-8fbc-37758b9c4601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:22.287592 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823f3a7-7e6f-410e-af52-0e665cfee586 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:22.302899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-b0cbe864-dccb-4dcf-8fbc-37758b9c4601 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:22.304402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-b0cbe864-dccb-4dcf-8fbc-37758b9c4601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:22.327935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fe359-de98-438c-b702-b9e9163983d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:22.328654 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fe359-de98-438c-b702-b9e9163983d3 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:22.328944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fe359-de98-438c-b702-b9e9163983d3 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:22.329227 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fe359-de98-438c-b702-b9e9163983d3 None admin] PATH_INFO: `/v3/system/users/7c255b5885684adcbb1ea47428d52bf2/roles/cc0fab8f90a44acc90de9af43e2bd7df` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:22.331027 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fe359-de98-438c-b702-b9e9163983d3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7c255b5885684adcbb1ea47428d52bf2, role_id=cc0fab8f90a44acc90de9af43e2bd7df)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:22.332465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fe359-de98-438c-b702-b9e9163983d3 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:22.345591 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 155/311] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:26:22 2022] PUT /identity/v3/system/users/7c255b5885684adcbb1ea47428d52bf2/roles/cc0fab8f90a44acc90de9af43e2bd7df => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:22.395114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 157/312] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:22.418586 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fe359-de98-438c-b702-b9e9163983d3 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:22.551334 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fe359-de98-438c-b702-b9e9163983d3 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:22.590343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:22.600894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5db5-ec11-4708-8625-e2d8dd83ce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:22.601619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5db5-ec11-4708-8625-e2d8dd83ce78 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:22.601907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5db5-ec11-4708-8625-e2d8dd83ce78 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:22.602218 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5db5-ec11-4708-8625-e2d8dd83ce78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:22.614619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a5db5-ec11-4708-8625-e2d8dd83ce78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-160a5db5-ec11-4708-8625-e2d8dd83ce78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:22.620337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:22.621366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:22.622770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 156/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:22.629284 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbe864-dccb-4dcf-8fbc-37758b9c4601 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:22.747668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbe864-dccb-4dcf-8fbc-37758b9c4601 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:22.760043 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:22.791634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:22.792269 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:22.792730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:22.793080 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:22.793819 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:22.794161 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:22.795658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:22.808146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 158/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:22.814234 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5db5-ec11-4708-8625-e2d8dd83ce78 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:22.930594 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5db5-ec11-4708-8625-e2d8dd83ce78 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:22.967544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25475a9-b4bc-4882-ac3b-f410498d527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:22.968244 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25475a9-b4bc-4882-ac3b-f410498d527d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:22.968481 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25475a9-b4bc-4882-ac3b-f410498d527d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:22.968764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25475a9-b4bc-4882-ac3b-f410498d527d None admin] PATH_INFO: `/v3/users/2c271af4e6324d4c9608780f154f7cd6` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:22.969974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25475a9-b4bc-4882-ac3b-f410498d527d None admin] RBAC: Authorizing `identity:get_user(user_id=2c271af4e6324d4c9608780f154f7cd6)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:22.971497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25475a9-b4bc-4882-ac3b-f410498d527d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:22.973224 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 157/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:22 2022] GET /identity/v3/users/2c271af4e6324d4c9608780f154f7cd6 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:22.979006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:23.088267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c0be2-f80f-4357-bf7b-1fb2afd050e0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:23.132639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922830a-c442-497e-8607-9f62c49c0b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:23.134215 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922830a-c442-497e-8607-9f62c49c0b07 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:23.134996 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922830a-c442-497e-8607-9f62c49c0b07 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:23.135742 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922830a-c442-497e-8607-9f62c49c0b07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:23.213135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:23.261027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922830a-c442-497e-8607-9f62c49c0b07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:23.264328 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922830a-c442-497e-8607-9f62c49c0b07 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:23.404475 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 159/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:23.409828 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25475a9-b4bc-4882-ac3b-f410498d527d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:23.477315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:23.525753 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25475a9-b4bc-4882-ac3b-f410498d527d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:23.563648 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5178776-7485-4ea7-9d45-308e379e5c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:23.564407 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5178776-7485-4ea7-9d45-308e379e5c6e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:23.564688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5178776-7485-4ea7-9d45-308e379e5c6e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:23.564962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5178776-7485-4ea7-9d45-308e379e5c6e None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:23.566595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5178776-7485-4ea7-9d45-308e379e5c6e None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:23.568133 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5178776-7485-4ea7-9d45-308e379e5c6e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:23.570853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 158/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:23 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:23.577534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922830a-c442-497e-8607-9f62c49c0b07 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:23.658825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:23.693892 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922830a-c442-497e-8607-9f62c49c0b07 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:23.730736 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b323e-9551-42cb-9114-ab53bc656db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:23.731388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b323e-9551-42cb-9114-ab53bc656db4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:23.731670 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b323e-9551-42cb-9114-ab53bc656db4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:23.731942 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b323e-9551-42cb-9114-ab53bc656db4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:23.732756 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b323e-9551-42cb-9114-ab53bc656db4 None admin] RBAC: Adding query filter params (role.id=e4367af82d094e189d2679fe6b2c6f9f, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93, user.id=2c271af4e6324d4c9608780f154f7cd6) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:23.733068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b323e-9551-42cb-9114-ab53bc656db4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:23.734576 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b323e-9551-42cb-9114-ab53bc656db4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:23.741642 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 160/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:23 2022] GET /identity/v3/role_assignments?user.id=2c271af4e6324d4c9608780f154f7cd6&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=e4367af82d094e189d2679fe6b2c6f9f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:23.785896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05413e5a-4baa-46b7-9817-45ed32e9a262 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:23.786211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05413e5a-4baa-46b7-9817-45ed32e9a262 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:23.786489 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05413e5a-4baa-46b7-9817-45ed32e9a262 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:23.788236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 159/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:23.794060 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce244-cfd4-4c04-93ec-ecc71dcca04c None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:23.794374 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce244-cfd4-4c04-93ec-ecc71dcca04c None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:23.794719 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce244-cfd4-4c04-93ec-ecc71dcca04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:23.815184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-bcdce244-cfd4-4c04-93ec-ecc71dcca04c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:23.816565 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-bcdce244-cfd4-4c04-93ec-ecc71dcca04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:23.899861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 161/320] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:23.923572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05413e5a-4baa-46b7-9817-45ed32e9a262 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:24.037590 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05413e5a-4baa-46b7-9817-45ed32e9a262 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:24.060682 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322b760-9578-4401-8af1-41f4d410aa98 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.061261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322b760-9578-4401-8af1-41f4d410aa98 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.061261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322b760-9578-4401-8af1-41f4d410aa98 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.062809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 162/321] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:24.064585 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:24.068155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93f2a-74f1-42ef-969b-61693f363c6f None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.068816 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93f2a-74f1-42ef-969b-61693f363c6f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.069088 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93f2a-74f1-42ef-969b-61693f363c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.078028 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ecf49-27a5-4b38-b93a-8c419fe0500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:24.078724 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ecf49-27a5-4b38-b93a-8c419fe0500b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.079006 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ecf49-27a5-4b38-b93a-8c419fe0500b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.079276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ecf49-27a5-4b38-b93a-8c419fe0500b None admin] PATH_INFO: `/v3/domains/default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.080457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ecf49-27a5-4b38-b93a-8c419fe0500b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:24.082070 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ecf49-27a5-4b38-b93a-8c419fe0500b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:24.084121 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 160/322] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:24.087753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-acb93f2a-74f1-42ef-969b-61693f363c6f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:24.088968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-acb93f2a-74f1-42ef-969b-61693f363c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:24.089408 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ecf49-27a5-4b38-b93a-8c419fe0500b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:24.154703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:24.164496 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 163/323] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:24.186409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93f2a-74f1-42ef-969b-61693f363c6f None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:24.198292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ecf49-27a5-4b38-b93a-8c419fe0500b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:24.236054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf93dd-7bd1-4423-9a7e-c5c9019e3505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:24.236808 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf93dd-7bd1-4423-9a7e-c5c9019e3505 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.237088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf93dd-7bd1-4423-9a7e-c5c9019e3505 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.237373 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf93dd-7bd1-4423-9a7e-c5c9019e3505 None admin] PATH_INFO: `/v3/groups` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.238042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf93dd-7bd1-4423-9a7e-c5c9019e3505 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:24.239705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf93dd-7bd1-4423-9a7e-c5c9019e3505 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:24.264798 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 161/324] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:26:24 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:24.292682 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93f2a-74f1-42ef-969b-61693f363c6f None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:24.333454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f289769-bade-4136-8f4e-2615d660768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:24.334139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f289769-bade-4136-8f4e-2615d660768d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.335556 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f289769-bade-4136-8f4e-2615d660768d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.335556 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f289769-bade-4136-8f4e-2615d660768d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.349204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f289769-bade-4136-8f4e-2615d660768d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-6f289769-bade-4136-8f4e-2615d660768d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:24.354529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:24.355663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:24.362435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 164/325] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:24.365239 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf93dd-7bd1-4423-9a7e-c5c9019e3505 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:24.430484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:24.488588 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf93dd-7bd1-4423-9a7e-c5c9019e3505 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:24.528519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c6856-f9be-4e58-b124-990bdfa4fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:24.529176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c6856-f9be-4e58-b124-990bdfa4fa44 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.529585 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c6856-f9be-4e58-b124-990bdfa4fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.529887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c6856-f9be-4e58-b124-990bdfa4fa44 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.531853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c6856-f9be-4e58-b124-990bdfa4fa44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:24.531853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c6856-f9be-4e58-b124-990bdfa4fa44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:24.532412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c6856-f9be-4e58-b124-990bdfa4fa44 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:24.541874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 162/326] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:24.547879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f289769-bade-4136-8f4e-2615d660768d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:24.550929 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe02aff-8dc8-4865-9a57-6f896df3f67d None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.551218 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe02aff-8dc8-4865-9a57-6f896df3f67d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.551740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe02aff-8dc8-4865-9a57-6f896df3f67d None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.553312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 163/327] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:24.560134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745146e3-bb40-450b-bf21-203e5404d05a None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.560475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745146e3-bb40-450b-bf21-203e5404d05a None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.561339 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745146e3-bb40-450b-bf21-203e5404d05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.584438 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-745146e3-bb40-450b-bf21-203e5404d05a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:24.585686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-745146e3-bb40-450b-bf21-203e5404d05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:24.621771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:24.692552 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f289769-bade-4136-8f4e-2615d660768d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:24.699568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 164/328] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:24.727544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745146e3-bb40-450b-bf21-203e5404d05a None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:24.740264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34feda66-0a69-454c-bb46-d435e66ad210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:24.741838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feda66-0a69-454c-bb46-d435e66ad210 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.742386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feda66-0a69-454c-bb46-d435e66ad210 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.742990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feda66-0a69-454c-bb46-d435e66ad210 None admin] PATH_INFO: `/v3/users/338b57e89a934128afd5d93b53551d6a` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.744990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34feda66-0a69-454c-bb46-d435e66ad210 None admin] RBAC: Authorizing `identity:get_user(user_id=338b57e89a934128afd5d93b53551d6a)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:24.747665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34feda66-0a69-454c-bb46-d435e66ad210 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:24.750160 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 165/329] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:24 2022] GET /identity/v3/users/338b57e89a934128afd5d93b53551d6a => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:24.755888 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34feda66-0a69-454c-bb46-d435e66ad210 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:24.793753 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:24.890458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745146e3-bb40-450b-bf21-203e5404d05a None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:24.916245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34feda66-0a69-454c-bb46-d435e66ad210 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:24.955865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:24.973086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0301564-ac4e-4254-867a-662504b4e2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:24.973502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045aba0-e2a5-45fe-af61-3f7ebcbb15d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:24.974574 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045aba0-e2a5-45fe-af61-3f7ebcbb15d6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.974874 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0301564-ac4e-4254-867a-662504b4e2e9 None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:24.975062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045aba0-e2a5-45fe-af61-3f7ebcbb15d6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.977508 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045aba0-e2a5-45fe-af61-3f7ebcbb15d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.978063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0301564-ac4e-4254-867a-662504b4e2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:24.978590 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0301564-ac4e-4254-867a-662504b4e2e9 None admin] PATH_INFO: `/v3/system/users/338b57e89a934128afd5d93b53551d6a/roles/e4367af82d094e189d2679fe6b2c6f9f` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:24.980499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0301564-ac4e-4254-867a-662504b4e2e9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=338b57e89a934128afd5d93b53551d6a, role_id=e4367af82d094e189d2679fe6b2c6f9f)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:24.982505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0301564-ac4e-4254-867a-662504b4e2e9 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:24.995564 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045aba0-e2a5-45fe-af61-3f7ebcbb15d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b045aba0-e2a5-45fe-af61-3f7ebcbb15d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:25.004607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:25.005851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:25.006849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 165/330] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:25.008097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 166/331] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:26:24 2022] PUT /identity/v3/system/users/338b57e89a934128afd5d93b53551d6a/roles/e4367af82d094e189d2679fe6b2c6f9f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:25.022039 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045aba0-e2a5-45fe-af61-3f7ebcbb15d6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:25.186514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045aba0-e2a5-45fe-af61-3f7ebcbb15d6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:25.189657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:25.232680 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:25.233344 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:25.233682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:25.233959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:25.234741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:25.235046 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:25.236514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:25.247083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 166/332] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:25.255321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0301564-ac4e-4254-867a-662504b4e2e9 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:25.378667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0301564-ac4e-4254-867a-662504b4e2e9 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:25.435443 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8ccdd-5ca5-4685-99cc-1b2433fb006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:25.436533 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ccdd-5ca5-4685-99cc-1b2433fb006b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:25.437021 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ccdd-5ca5-4685-99cc-1b2433fb006b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:25.437500 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ccdd-5ca5-4685-99cc-1b2433fb006b None admin] PATH_INFO: `/v3/users/7c255b5885684adcbb1ea47428d52bf2` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:25.438960 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8ccdd-5ca5-4685-99cc-1b2433fb006b None admin] RBAC: Authorizing `identity:get_user(user_id=7c255b5885684adcbb1ea47428d52bf2)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:25.440660 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8ccdd-5ca5-4685-99cc-1b2433fb006b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:25.443019 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 167/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:25 2022] GET /identity/v3/users/7c255b5885684adcbb1ea47428d52bf2 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:25.452009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:25.583840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8c7f0-c220-453c-a107-d56dfc6269d1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:25.633855 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc84092-2371-45ea-87e5-119a611aef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:25.634549 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc84092-2371-45ea-87e5-119a611aef5f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:25.634833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc84092-2371-45ea-87e5-119a611aef5f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:25.635108 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc84092-2371-45ea-87e5-119a611aef5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:25.635918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc84092-2371-45ea-87e5-119a611aef5f None admin] RBAC: Adding query filter params (role.id=cc0fab8f90a44acc90de9af43e2bd7df, user.id=7c255b5885684adcbb1ea47428d52bf2, scope.system=all) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:25.636233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc84092-2371-45ea-87e5-119a611aef5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:25.637715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc84092-2371-45ea-87e5-119a611aef5f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:25.654414 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 167/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:26:25 2022] GET /identity/v3/role_assignments?user.id=7c255b5885684adcbb1ea47428d52bf2&scope.system=all&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:26.842808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329852-2ae4-436c-aafa-5984af468528 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:26.843364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329852-2ae4-436c-aafa-5984af468528 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:26.843364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329852-2ae4-436c-aafa-5984af468528 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:26.844967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 168/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:26.850771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0b66-81c0-401c-83cd-fcbc67164abd None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:26.851252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0b66-81c0-401c-83cd-fcbc67164abd None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:26.851360 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0b66-81c0-401c-83cd-fcbc67164abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:26.881693 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-fc6b0b66-81c0-401c-83cd-fcbc67164abd None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:26.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-fc6b0b66-81c0-401c-83cd-fcbc67164abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:26.908789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:26.971900 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 168/336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:27.005713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b329852-2ae4-436c-aafa-5984af468528 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:27.084886 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:27.152018 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b329852-2ae4-436c-aafa-5984af468528 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:27.227503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7363645-98b5-45ea-931d-4506266fd419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:27.228553 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7363645-98b5-45ea-931d-4506266fd419 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:27.229004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7363645-98b5-45ea-931d-4506266fd419 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:27.229446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7363645-98b5-45ea-931d-4506266fd419 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:27.256057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7363645-98b5-45ea-931d-4506266fd419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-b7363645-98b5-45ea-931d-4506266fd419 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:27.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:27.269523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:27.271115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 169/337] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:27.277295 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b0b66-81c0-401c-83cd-fcbc67164abd None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:27.419733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b0b66-81c0-401c-83cd-fcbc67164abd None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:27.471296 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1e08c-3d82-4bb6-bda3-7f3910bed08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:27.471995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1e08c-3d82-4bb6-bda3-7f3910bed08b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:27.472278 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1e08c-3d82-4bb6-bda3-7f3910bed08b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:27.472639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1e08c-3d82-4bb6-bda3-7f3910bed08b None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:27.473380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1e08c-3d82-4bb6-bda3-7f3910bed08b None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:27.473689 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1e08c-3d82-4bb6-bda3-7f3910bed08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:27.475135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1e08c-3d82-4bb6-bda3-7f3910bed08b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:27.487673 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 169/338] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:27.494306 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7363645-98b5-45ea-931d-4506266fd419 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:27.715028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7363645-98b5-45ea-931d-4506266fd419 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:27.748125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:27.791725 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a08010-4365-47ef-b11b-3635503f67ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:27.792792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a08010-4365-47ef-b11b-3635503f67ca None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:27.793284 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a08010-4365-47ef-b11b-3635503f67ca None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:27.793760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a08010-4365-47ef-b11b-3635503f67ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:27.962512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a08010-4365-47ef-b11b-3635503f67ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:27.965588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a08010-4365-47ef-b11b-3635503f67ca None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:28.077190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3f973-7956-4f56-ae8e-144e97a805aa None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.077748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3f973-7956-4f56-ae8e-144e97a805aa None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.077748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3f973-7956-4f56-ae8e-144e97a805aa None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.079443 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 170/339] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:28.085234 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a125e2-f5d5-41fa-ad4e-01631dd1538a None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.085539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a125e2-f5d5-41fa-ad4e-01631dd1538a None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.085788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a125e2-f5d5-41fa-ad4e-01631dd1538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.143395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-04a125e2-f5d5-41fa-ad4e-01631dd1538a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:28.144720 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-04a125e2-f5d5-41fa-ad4e-01631dd1538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:28.172400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:28.175584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 170/340] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:28.191325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a08010-4365-47ef-b11b-3635503f67ca None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:28.232998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:28.250819 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 171/341] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:28.252207 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6129d15-d3d2-494f-94e1-fcbd8fc1e66e None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.252498 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6129d15-d3d2-494f-94e1-fcbd8fc1e66e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.252778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6129d15-d3d2-494f-94e1-fcbd8fc1e66e None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.254436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 172/342] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:28.260041 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a85bf-e3c6-4a04-98da-e46650b8654a None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.260450 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a85bf-e3c6-4a04-98da-e46650b8654a None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.260955 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a85bf-e3c6-4a04-98da-e46650b8654a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.284186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-ea7a85bf-e3c6-4a04-98da-e46650b8654a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:28.285659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-ea7a85bf-e3c6-4a04-98da-e46650b8654a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:28.367653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 173/343] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:28.368734 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a85bf-e3c6-4a04-98da-e46650b8654a None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:28.443893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a08010-4365-47ef-b11b-3635503f67ca None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:28.475040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:28.504157 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a85bf-e3c6-4a04-98da-e46650b8654a None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:28.510760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701a8f4-96aa-464e-994a-2c0143591b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:28.512036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701a8f4-96aa-464e-994a-2c0143591b32 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.512538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701a8f4-96aa-464e-994a-2c0143591b32 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.513034 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701a8f4-96aa-464e-994a-2c0143591b32 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.515321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701a8f4-96aa-464e-994a-2c0143591b32 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:28.518669 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701a8f4-96aa-464e-994a-2c0143591b32 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:28.523969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 171/344] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:28 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:28.525861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701a8f4-96aa-464e-994a-2c0143591b32 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:28.578873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85830045-354d-4c38-aeed-089fa4e9db74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:28.579616 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85830045-354d-4c38-aeed-089fa4e9db74 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.579904 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85830045-354d-4c38-aeed-089fa4e9db74 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.580183 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85830045-354d-4c38-aeed-089fa4e9db74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.604876 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85830045-354d-4c38-aeed-089fa4e9db74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-85830045-354d-4c38-aeed-089fa4e9db74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:28.610165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:28.611381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:28.612521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:28.616703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 174/345] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:28.618465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85830045-354d-4c38-aeed-089fa4e9db74 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:28.645814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:28.665071 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701a8f4-96aa-464e-994a-2c0143591b32 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:28.717202 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eeba3-6046-4f65-a3db-95543169b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:28.718025 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eeba3-6046-4f65-a3db-95543169b0a4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.718424 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eeba3-6046-4f65-a3db-95543169b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.718792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eeba3-6046-4f65-a3db-95543169b0a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.737346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eeba3-6046-4f65-a3db-95543169b0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-7b5eeba3-6046-4f65-a3db-95543169b0a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:28.745489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:28.746595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:28.747501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:28.755530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 172/346] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:28.756357 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eeba3-6046-4f65-a3db-95543169b0a4 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:28.840581 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85830045-354d-4c38-aeed-089fa4e9db74 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:28.881666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8b314-ee7e-4bd8-9175-e6135e34fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:28.882388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b314-ee7e-4bd8-9175-e6135e34fa9a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.882668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b314-ee7e-4bd8-9175-e6135e34fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.882945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b314-ee7e-4bd8-9175-e6135e34fa9a None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.887475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:28.890721 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8b314-ee7e-4bd8-9175-e6135e34fa9a None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:28.892358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8b314-ee7e-4bd8-9175-e6135e34fa9a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:28.894463 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 175/347] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:28 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:28.897799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8b314-ee7e-4bd8-9175-e6135e34fa9a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:28.902944 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eeba3-6046-4f65-a3db-95543169b0a4 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:28.941667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05343c23-1eb8-4e6d-ac57-c8d87e6cc71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:28.942340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05343c23-1eb8-4e6d-ac57-c8d87e6cc71a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.942622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05343c23-1eb8-4e6d-ac57-c8d87e6cc71a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.942895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05343c23-1eb8-4e6d-ac57-c8d87e6cc71a None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.943996 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05343c23-1eb8-4e6d-ac57-c8d87e6cc71a None admin] RBAC: Adding query filter params (name=member) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:28.944301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05343c23-1eb8-4e6d-ac57-c8d87e6cc71a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:28.945721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05343c23-1eb8-4e6d-ac57-c8d87e6cc71a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:28.955047 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 173/348] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:28.956378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76561f-f72e-438c-9171-f5a46f0b5652 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:28.956663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76561f-f72e-438c-9171-f5a46f0b5652 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:28.956993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76561f-f72e-438c-9171-f5a46f0b5652 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:28.958709 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 174/349] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:28.959460 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76561f-f72e-438c-9171-f5a46f0b5652 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:28.969103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:29.036739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8b314-ee7e-4bd8-9175-e6135e34fa9a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:29.077656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c099b0-ca2b-4aa9-8128-2500e9edc0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:29.078397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c099b0-ca2b-4aa9-8128-2500e9edc0dc None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.078771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c099b0-ca2b-4aa9-8128-2500e9edc0dc None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.078894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c099b0-ca2b-4aa9-8128-2500e9edc0dc None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.079606 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c099b0-ca2b-4aa9-8128-2500e9edc0dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:29.079919 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c099b0-ca2b-4aa9-8128-2500e9edc0dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:29.081323 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c099b0-ca2b-4aa9-8128-2500e9edc0dc None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:29.091278 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 176/350] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:29.092645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b98f-c007-4965-bdee-fd478cef3e6b None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.092936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b98f-c007-4965-bdee-fd478cef3e6b None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.093247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b98f-c007-4965-bdee-fd478cef3e6b None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.094806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 177/351] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:29.095790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9b98f-c007-4965-bdee-fd478cef3e6b None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:29.105276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76561f-f72e-438c-9171-f5a46f0b5652 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:29.145101 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641b4f3-47b4-4220-9474-bed97ec530dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:29.146160 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641b4f3-47b4-4220-9474-bed97ec530dd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.146453 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641b4f3-47b4-4220-9474-bed97ec530dd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.146734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641b4f3-47b4-4220-9474-bed97ec530dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.147883 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641b4f3-47b4-4220-9474-bed97ec530dd None admin] RBAC: Adding query filter params (group.id=2d2fccf3bb484b9dafb73ec9e138909b, role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=278f44bb741f4075b58db40b154bb64d) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:29.148491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641b4f3-47b4-4220-9474-bed97ec530dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:29.150184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641b4f3-47b4-4220-9474-bed97ec530dd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:29.157242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 175/352] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:28 2022] GET /identity/v3/role_assignments?group.id=2d2fccf3bb484b9dafb73ec9e138909b&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:29.158768 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b93fa-69d7-49fb-ad5b-e000fb2be35b None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.159129 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b93fa-69d7-49fb-ad5b-e000fb2be35b None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.159413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b93fa-69d7-49fb-ad5b-e000fb2be35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.179305 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:29.183903 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-040b93fa-69d7-49fb-ad5b-e000fb2be35b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:29.185187 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-040b93fa-69d7-49fb-ad5b-e000fb2be35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:29.241546 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9b98f-c007-4965-bdee-fd478cef3e6b None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:29.279384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:29.294585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 176/353] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:29.295377 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b93fa-69d7-49fb-ad5b-e000fb2be35b None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:29.299215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a1ab3-cee4-40ba-86e2-53fc71795f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:29.300329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a1ab3-cee4-40ba-86e2-53fc71795f0a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.300904 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a1ab3-cee4-40ba-86e2-53fc71795f0a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.301458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a1ab3-cee4-40ba-86e2-53fc71795f0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.399929 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a1ab3-cee4-40ba-86e2-53fc71795f0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:29.401469 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a1ab3-cee4-40ba-86e2-53fc71795f0a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:29.441552 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b93fa-69d7-49fb-ad5b-e000fb2be35b None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:29.489095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:29.494191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c37161-e99e-48d8-9dd2-8a159c100cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:29.494688 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37161-e99e-48d8-9dd2-8a159c100cc8 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.494974 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37161-e99e-48d8-9dd2-8a159c100cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.495248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37161-e99e-48d8-9dd2-8a159c100cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.553967 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 178/354] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:29.556872 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161789-7889-4f65-bd75-fd6c77edc978 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.557670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161789-7889-4f65-bd75-fd6c77edc978 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.558289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161789-7889-4f65-bd75-fd6c77edc978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.584196 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-77161789-7889-4f65-bd75-fd6c77edc978 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:29.585428 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-77161789-7889-4f65-bd75-fd6c77edc978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:29.601659 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:29.606156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c37161-e99e-48d8-9dd2-8a159c100cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:29.606156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c37161-e99e-48d8-9dd2-8a159c100cc8 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:29.692517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 179/355] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:29.693793 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161789-7889-4f65-bd75-fd6c77edc978 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:29.735558 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 177/356] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:29.737388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188139c-b678-4e7e-b8df-50f87f86bc9d None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.737893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188139c-b678-4e7e-b8df-50f87f86bc9d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.738458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188139c-b678-4e7e-b8df-50f87f86bc9d None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.740384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 178/357] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:29.741343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188139c-b678-4e7e-b8df-50f87f86bc9d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:29.812089 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:29.823648 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161789-7889-4f65-bd75-fd6c77edc978 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:29.863535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:29.865715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188139c-b678-4e7e-b8df-50f87f86bc9d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:29.869308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c49f73-b4b3-4781-b691-bc828e4cb6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:29.870011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c49f73-b4b3-4781-b691-bc828e4cb6ea None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.870324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c49f73-b4b3-4781-b691-bc828e4cb6ea None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.870599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c49f73-b4b3-4781-b691-bc828e4cb6ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.884753 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c49f73-b4b3-4781-b691-bc828e4cb6ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-d0c49f73-b4b3-4781-b691-bc828e4cb6ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:29.888670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:29.889714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:29.890722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 180/358] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:29.891976 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c49f73-b4b3-4781-b691-bc828e4cb6ea None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:29.916347 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee842e0-dbb4-4e20-89d3-a76f734ca953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:29.917009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee842e0-dbb4-4e20-89d3-a76f734ca953 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:29.917393 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee842e0-dbb4-4e20-89d3-a76f734ca953 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:29.917587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee842e0-dbb4-4e20-89d3-a76f734ca953 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:29.919022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee842e0-dbb4-4e20-89d3-a76f734ca953 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:29.920473 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee842e0-dbb4-4e20-89d3-a76f734ca953 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:29.927721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 179/359] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:29 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:29.927721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee842e0-dbb4-4e20-89d3-a76f734ca953 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.037744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c49f73-b4b3-4781-b691-bc828e4cb6ea None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.064375 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee842e0-dbb4-4e20-89d3-a76f734ca953 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.090124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3227-ad0a-4ee9-a186-f1e7ec6b8f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.090124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3227-ad0a-4ee9-a186-f1e7ec6b8f4b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.090124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3227-ad0a-4ee9-a186-f1e7ec6b8f4b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.090124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3227-ad0a-4ee9-a186-f1e7ec6b8f4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.104773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b3227-ad0a-4ee9-a186-f1e7ec6b8f4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-422b3227-ad0a-4ee9-a186-f1e7ec6b8f4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:30.111074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:30.112209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:30.113232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:30.115055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb769fcb-60c8-4151-b2c9-7fa4c291b817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.115734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb769fcb-60c8-4151-b2c9-7fa4c291b817 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.116018 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb769fcb-60c8-4151-b2c9-7fa4c291b817 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.116284 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb769fcb-60c8-4151-b2c9-7fa4c291b817 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.116939 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 181/360] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:30.118236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb769fcb-60c8-4151-b2c9-7fa4c291b817 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:30.118379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42be9a-43dd-4f6b-a995-8f047ef8624e None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.118612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42be9a-43dd-4f6b-a995-8f047ef8624e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.118898 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42be9a-43dd-4f6b-a995-8f047ef8624e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.119753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb769fcb-60c8-4151-b2c9-7fa4c291b817 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:30.121583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 180/361] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:29 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:30.127187 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb769fcb-60c8-4151-b2c9-7fa4c291b817 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.146083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-8a42be9a-43dd-4f6b-a995-8f047ef8624e None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:30.147375 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-8a42be9a-43dd-4f6b-a995-8f047ef8624e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:30.156671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:30.183522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:30.247842 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 182/362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:30.248795 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42be9a-43dd-4f6b-a995-8f047ef8624e None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.283121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb769fcb-60c8-4151-b2c9-7fa4c291b817 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.331905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.332594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.332945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.333290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.334091 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] RBAC: Adding query filter params (name=reader) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:30.334515 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:30.336029 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:30.345962 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 181/363] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:30.346794 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.401707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42be9a-43dd-4f6b-a995-8f047ef8624e None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.440044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199ce00-1ee3-4e87-8741-9978f2d172f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.440715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199ce00-1ee3-4e87-8741-9978f2d172f7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.440996 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199ce00-1ee3-4e87-8741-9978f2d172f7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.441358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199ce00-1ee3-4e87-8741-9978f2d172f7 None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.442597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199ce00-1ee3-4e87-8741-9978f2d172f7 None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:30.444444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199ce00-1ee3-4e87-8741-9978f2d172f7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:30.447149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 183/364] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:30.447957 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199ce00-1ee3-4e87-8741-9978f2d172f7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.462991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:30.476275 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:30.480717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae60d22-a53f-4ac9-a86a-f141955ccaac None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.519534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.520179 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.520458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.520733 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.521485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:30.521792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:30.526397 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:30.538690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 182/365] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:30.540195 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.585133 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199ce00-1ee3-4e87-8741-9978f2d172f7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.624575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd97b6d-39b3-4872-9b3b-0a6fb4cad74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.625274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd97b6d-39b3-4872-9b3b-0a6fb4cad74f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.625570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd97b6d-39b3-4872-9b3b-0a6fb4cad74f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.625840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd97b6d-39b3-4872-9b3b-0a6fb4cad74f None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.629057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd97b6d-39b3-4872-9b3b-0a6fb4cad74f None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:30.629392 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd97b6d-39b3-4872-9b3b-0a6fb4cad74f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:30.631216 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 184/366] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:30.631989 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd97b6d-39b3-4872-9b3b-0a6fb4cad74f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.682439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208d770-92f3-405b-9c35-7d992e4418c1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.732020 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d2c4c-2381-44d4-b9a6-8015a7082bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.732683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d2c4c-2381-44d4-b9a6-8015a7082bbe None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.732961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d2c4c-2381-44d4-b9a6-8015a7082bbe None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.733231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d2c4c-2381-44d4-b9a6-8015a7082bbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.747478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:30.752576 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d2c4c-2381-44d4-b9a6-8015a7082bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-dc2d2c4c-2381-44d4-b9a6-8015a7082bbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:30.760697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:30.761791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:30.762857 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:30.763572 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 183/367] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:30.764559 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d2c4c-2381-44d4-b9a6-8015a7082bbe None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.778390 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd97b6d-39b3-4872-9b3b-0a6fb4cad74f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.796509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:30.832915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20585359-626a-4842-b990-530164563527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.832915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20585359-626a-4842-b990-530164563527 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.833253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20585359-626a-4842-b990-530164563527 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.833253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20585359-626a-4842-b990-530164563527 None admin] PATH_INFO: `/v3/users/338b57e89a934128afd5d93b53551d6a` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.835114 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20585359-626a-4842-b990-530164563527 None admin] RBAC: Authorizing `identity:get_user(user_id=338b57e89a934128afd5d93b53551d6a)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:30.836585 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20585359-626a-4842-b990-530164563527 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:30.838264 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 185/368] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/users/338b57e89a934128afd5d93b53551d6a => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:30.839049 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20585359-626a-4842-b990-530164563527 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:30.913151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d2c4c-2381-44d4-b9a6-8015a7082bbe None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.960585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:30.961280 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:30.961565 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:30.961845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:30.962828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] RBAC: Adding query filter params (group.id=2d2fccf3bb484b9dafb73ec9e138909b, role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:30.963151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:30.965347 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:30.972327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20585359-626a-4842-b990-530164563527 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:30.976076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 184/369] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/role_assignments?group.id=2d2fccf3bb484b9dafb73ec9e138909b&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:30.977010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:31.024177 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3c10a-9ddd-40f6-a090-feb17b4299dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:31.025836 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3c10a-9ddd-40f6-a090-feb17b4299dd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:31.026376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3c10a-9ddd-40f6-a090-feb17b4299dd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:31.026883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3c10a-9ddd-40f6-a090-feb17b4299dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:31.051601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:31.122623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:31.125034 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a09ba6-4336-4c0d-b9c2-311d0c5df998 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:31.142792 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3c10a-9ddd-40f6-a090-feb17b4299dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:31.146434 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3c10a-9ddd-40f6-a090-feb17b4299dd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:31.175536 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:31.176191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:31.176477 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:31.176763 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:31.178247 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] RBAC: Adding query filter params (group.id=2d2fccf3bb484b9dafb73ec9e138909b, role.id=d883529ef5284c35b27b759c57353383, scope.project.id=278f44bb741f4075b58db40b154bb64d) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:31.178570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:31.180365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:31.194147 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 185/370] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/role_assignments?group.id=2d2fccf3bb484b9dafb73ec9e138909b&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=d883529ef5284c35b27b759c57353383 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:31.194147 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:31.297009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 186/371] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:31.297908 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3c10a-9ddd-40f6-a090-feb17b4299dd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:31.352103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:31.394612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555acee-82bb-48ea-98d1-5112059920c1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:31.425818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3c10a-9ddd-40f6-a090-feb17b4299dd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:31.430673 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:31.431319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:31.431597 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:31.431870 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:31.432594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:31.432896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:31.434344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:31.448963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 186/372] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:31.449903 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:31.465386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:31.475958 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334a407-cb4a-4c2a-9136-ee77a6a79065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:31.476910 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a407-cb4a-4c2a-9136-ee77a6a79065 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:31.477670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a407-cb4a-4c2a-9136-ee77a6a79065 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:31.477992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a407-cb4a-4c2a-9136-ee77a6a79065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:31.479277 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a407-cb4a-4c2a-9136-ee77a6a79065 None admin] RBAC: Adding query filter params (role.id=e4367af82d094e189d2679fe6b2c6f9f, user.id=338b57e89a934128afd5d93b53551d6a, scope.system=all) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:31.479868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a407-cb4a-4c2a-9136-ee77a6a79065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:31.482118 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a407-cb4a-4c2a-9136-ee77a6a79065 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:31.498486 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 187/373] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:26:31 2022] GET /identity/v3/role_assignments?user.id=338b57e89a934128afd5d93b53551d6a&scope.system=all&role.id=e4367af82d094e189d2679fe6b2c6f9f => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:31.499764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9cb9-b35c-46f0-952a-c04f9e27d341 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:31.500048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9cb9-b35c-46f0-952a-c04f9e27d341 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:31.500321 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9cb9-b35c-46f0-952a-c04f9e27d341 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:31.501728 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 188/374] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:31.502298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f9cb9-b35c-46f0-952a-c04f9e27d341 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:31.639319 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f9cb9-b35c-46f0-952a-c04f9e27d341 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:31.643388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:31.663545 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c34f02-6837-4768-8bb4-6c24dc2ac130 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:31.701002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb3b4-2b50-4958-a290-b37c9a232106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:31.701732 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb3b4-2b50-4958-a290-b37c9a232106 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:31.702017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb3b4-2b50-4958-a290-b37c9a232106 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:31.702318 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb3b4-2b50-4958-a290-b37c9a232106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:31.711546 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a078a2a-d536-40d2-b474-a0c957d128be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:31.712223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078a2a-d536-40d2-b474-a0c957d128be None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:31.712509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078a2a-d536-40d2-b474-a0c957d128be None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:31.712790 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078a2a-d536-40d2-b474-a0c957d128be None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:31.714001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a078a2a-d536-40d2-b474-a0c957d128be None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:31.715559 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a078a2a-d536-40d2-b474-a0c957d128be None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:31.717603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 187/375] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:31 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:31.718876 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7ccf-ea2e-4922-b11d-6389a7a78096 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:31.719168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7ccf-ea2e-4922-b11d-6389a7a78096 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:31.719447 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7ccf-ea2e-4922-b11d-6389a7a78096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:31.742165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-c39f7ccf-ea2e-4922-b11d-6389a7a78096 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:31.743389 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-c39f7ccf-ea2e-4922-b11d-6389a7a78096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:31.787619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb3b4-2b50-4958-a290-b37c9a232106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:31.789193 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb3b4-2b50-4958-a290-b37c9a232106 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:31.828253 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 188/376] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:31.829178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7ccf-ea2e-4922-b11d-6389a7a78096 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:31.842522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:31.912136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 189/377] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:31.913031 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb3b4-2b50-4958-a290-b37c9a232106 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:31.923106 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:31.973761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7ccf-ea2e-4922-b11d-6389a7a78096 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.015946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096df24-97a9-498f-9a3b-3094a1802e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.016622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096df24-97a9-498f-9a3b-3094a1802e30 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.016907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096df24-97a9-498f-9a3b-3094a1802e30 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.017188 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096df24-97a9-498f-9a3b-3094a1802e30 None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.018492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096df24-97a9-498f-9a3b-3094a1802e30 None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.020200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096df24-97a9-498f-9a3b-3094a1802e30 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.024323 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 189/378] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:31 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:32.025249 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096df24-97a9-498f-9a3b-3094a1802e30 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:32.050262 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb3b4-2b50-4958-a290-b37c9a232106 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.092801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91bb6a-a936-4708-a698-b96007d50037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.093476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91bb6a-a936-4708-a698-b96007d50037 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.093758 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91bb6a-a936-4708-a698-b96007d50037 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.094036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91bb6a-a936-4708-a698-b96007d50037 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.109064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91bb6a-a936-4708-a698-b96007d50037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-6c91bb6a-a936-4708-a698-b96007d50037 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:32.117077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:32.119531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 190/379] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:32.121667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91bb6a-a936-4708-a698-b96007d50037 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:32.127630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:32.163702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096df24-97a9-498f-9a3b-3094a1802e30 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.204206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392e410-5352-44e4-adae-8a69367d4e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.204824 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e410-5352-44e4-adae-8a69367d4e65 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.205185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e410-5352-44e4-adae-8a69367d4e65 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.205470 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e410-5352-44e4-adae-8a69367d4e65 None admin] PATH_INFO: `/v3/projects/52cdb50da43d48649726866000557d76` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.206657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:32.208028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392e410-5352-44e4-adae-8a69367d4e65 None admin] RBAC: Authorizing `identity:get_project(project_id=52cdb50da43d48649726866000557d76)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.211980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392e410-5352-44e4-adae-8a69367d4e65 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.216682 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 190/380] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:32 2022] GET /identity/v3/projects/52cdb50da43d48649726866000557d76 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:32.217645 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392e410-5352-44e4-adae-8a69367d4e65 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:32.257584 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91bb6a-a936-4708-a698-b96007d50037 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.303285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.304387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.304870 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.305341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.307008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] RBAC: Adding query filter params (group.id=2d2fccf3bb484b9dafb73ec9e138909b, role.id=d883529ef5284c35b27b759c57353383, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:32.307595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.309646 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.322264 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 191/381] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:32 2022] GET /identity/v3/role_assignments?group.id=2d2fccf3bb484b9dafb73ec9e138909b&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=d883529ef5284c35b27b759c57353383 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:32.322264 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:32.346420 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392e410-5352-44e4-adae-8a69367d4e65 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.392276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.392943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.393273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.393559 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.394553 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] RBAC: Adding query filter params (name=member) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:32.394870 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.396366 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.406885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 191/382] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:32.414136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:32.429359 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:32.512353 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebf35-e8ce-4bff-b03a-f4a41b66b29c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.546405 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c64387-a957-4166-8f73-d7b40ebef179 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.589806 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5fa16-755f-4781-88b9-585c1a33edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.590520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5fa16-755f-4781-88b9-585c1a33edfb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.590808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5fa16-755f-4781-88b9-585c1a33edfb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.591088 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5fa16-755f-4781-88b9-585c1a33edfb None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.592554 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5fa16-755f-4781-88b9-585c1a33edfb None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.594544 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5fa16-755f-4781-88b9-585c1a33edfb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.597427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 192/383] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:32 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:32.602748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:32.604074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5fa16-755f-4781-88b9-585c1a33edfb None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:32.619771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743ad49-49d5-4ed7-9d48-313c6e6713fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.620833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743ad49-49d5-4ed7-9d48-313c6e6713fb None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.621270 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743ad49-49d5-4ed7-9d48-313c6e6713fb None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.621742 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743ad49-49d5-4ed7-9d48-313c6e6713fb None admin] PATH_INFO: `/v3/groups/d8deb47a8872416b8d027b7ddcc0ffbd` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.631289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743ad49-49d5-4ed7-9d48-313c6e6713fb None admin] RBAC: Authorizing `identity:get_group(group_id=d8deb47a8872416b8d027b7ddcc0ffbd)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.633052 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743ad49-49d5-4ed7-9d48-313c6e6713fb None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.635405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 192/384] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:32 2022] GET /identity/v3/groups/d8deb47a8872416b8d027b7ddcc0ffbd => generated 229 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:32.643387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743ad49-49d5-4ed7-9d48-313c6e6713fb None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:32.698568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:32.723607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5fa16-755f-4781-88b9-585c1a33edfb None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.781994 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743ad49-49d5-4ed7-9d48-313c6e6713fb None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.788303 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152709a4-bfa5-4713-bc07-b65dd58630cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.789070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152709a4-bfa5-4713-bc07-b65dd58630cd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.789419 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152709a4-bfa5-4713-bc07-b65dd58630cd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.789773 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152709a4-bfa5-4713-bc07-b65dd58630cd None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.792182 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152709a4-bfa5-4713-bc07-b65dd58630cd None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.793890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152709a4-bfa5-4713-bc07-b65dd58630cd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.796190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 193/385] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:32 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:32.801616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152709a4-bfa5-4713-bc07-b65dd58630cd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:32.821823 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3de1c4-e815-416c-93a0-5e76d1153f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.822877 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3de1c4-e815-416c-93a0-5e76d1153f5a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.823346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3de1c4-e815-416c-93a0-5e76d1153f5a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.823800 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3de1c4-e815-416c-93a0-5e76d1153f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.824792 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3de1c4-e815-416c-93a0-5e76d1153f5a None admin] RBAC: Adding query filter params (group.id=d8deb47a8872416b8d027b7ddcc0ffbd, role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=52cdb50da43d48649726866000557d76) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:32.825396 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3de1c4-e815-416c-93a0-5e76d1153f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.827740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3de1c4-e815-416c-93a0-5e76d1153f5a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.836621 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 193/386] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:32 2022] GET /identity/v3/role_assignments?group.id=d8deb47a8872416b8d027b7ddcc0ffbd&scope.project.id=52cdb50da43d48649726866000557d76&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:32.923484 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152709a4-bfa5-4713-bc07-b65dd58630cd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:32.961729 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b7951-b95b-41fc-8b80-a45c7eeda51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:32.962488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b7951-b95b-41fc-8b80-a45c7eeda51a None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:32.962777 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b7951-b95b-41fc-8b80-a45c7eeda51a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:32.963057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b7951-b95b-41fc-8b80-a45c7eeda51a None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d/groups/2d2fccf3bb484b9dafb73ec9e138909b/roles/cc0fab8f90a44acc90de9af43e2bd7df` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:32.965220 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b7951-b95b-41fc-8b80-a45c7eeda51a None admin] RBAC: Authorizing `identity:create_grant(project_id=278f44bb741f4075b58db40b154bb64d, group_id=2d2fccf3bb484b9dafb73ec9e138909b, role_id=cc0fab8f90a44acc90de9af43e2bd7df)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:32.966775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b7951-b95b-41fc-8b80-a45c7eeda51a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:32.987007 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 194/387] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:26:32 2022] PUT /identity/v3/projects/278f44bb741f4075b58db40b154bb64d/groups/2d2fccf3bb484b9dafb73ec9e138909b/roles/cc0fab8f90a44acc90de9af43e2bd7df => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:33.535525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aeb97-487e-472f-9513-408b775d34b4 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:33.536567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aeb97-487e-472f-9513-408b775d34b4 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:33.537057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aeb97-487e-472f-9513-408b775d34b4 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:33.539117 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 194/388] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:33.545227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68140e-8727-4b2e-a45c-c846fa9a4256 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:33.546282 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68140e-8727-4b2e-a45c-c846fa9a4256 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:33.546765 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68140e-8727-4b2e-a45c-c846fa9a4256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:33.553913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:33.567773 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-bf68140e-8727-4b2e-a45c-c846fa9a4256 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:33.569444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-bf68140e-8727-4b2e-a45c-c846fa9a4256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:33.581747 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5d2c-078e-4f09-be26-340b4d63cc6a None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:33.582728 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5d2c-078e-4f09-be26-340b4d63cc6a None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:33.582728 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5d2c-078e-4f09-be26-340b4d63cc6a None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:33.584050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 195/389] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:33.589879 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ffc4-a555-4a16-a072-0f3229e3de5d None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:33.590236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ffc4-a555-4a16-a072-0f3229e3de5d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:33.590456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ffc4-a555-4a16-a072-0f3229e3de5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:33.617488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-8b27ffc4-a555-4a16-a072-0f3229e3de5d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:33.619966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-8b27ffc4-a555-4a16-a072-0f3229e3de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:33.698478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 195/390] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:33.730855 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:33.739788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 196/391] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:33.744651 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68140e-8727-4b2e-a45c-c846fa9a4256 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:33.783029 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ffc4-a555-4a16-a072-0f3229e3de5d None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:33.906981 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ffc4-a555-4a16-a072-0f3229e3de5d None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:33.927109 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68140e-8727-4b2e-a45c-c846fa9a4256 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:33.944417 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8d200-a0da-491a-b31f-370af7bdf557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:33.945441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8d200-a0da-491a-b31f-370af7bdf557 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:33.945948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8d200-a0da-491a-b31f-370af7bdf557 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:33.946596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8d200-a0da-491a-b31f-370af7bdf557 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:33.957659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8d200-a0da-491a-b31f-370af7bdf557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-bbf8d200-a0da-491a-b31f-370af7bdf557 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:33.963297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:33.964298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:33.965219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:33.972639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 197/392] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:33.974186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8d200-a0da-491a-b31f-370af7bdf557 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:33.987186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:33.999713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9acac2-5326-4951-9360-f02a130f3744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:34.000396 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9acac2-5326-4951-9360-f02a130f3744 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.000677 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9acac2-5326-4951-9360-f02a130f3744 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.000951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9acac2-5326-4951-9360-f02a130f3744 None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.022427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9acac2-5326-4951-9360-f02a130f3744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-9a9acac2-5326-4951-9360-f02a130f3744 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:34.027513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:34.028610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:34.034643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 196/393] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:34.038491 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:34.039651 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9acac2-5326-4951-9360-f02a130f3744 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:34.114265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8d200-a0da-491a-b31f-370af7bdf557 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:34.151191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:34.152291 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.152795 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.153290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.154294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:34.154819 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:34.156539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:34.170230 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 198/394] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:34.172717 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:34.282377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc12ec-b4b2-48f2-a73a-9ac893a88a8c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:34.294586 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:34.324134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84cf30-cebe-46bb-8ce1-754c1405e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:34.325448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cf30-cebe-46bb-8ce1-754c1405e4a9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.326054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cf30-cebe-46bb-8ce1-754c1405e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.327082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cf30-cebe-46bb-8ce1-754c1405e4a9 None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.328813 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84cf30-cebe-46bb-8ce1-754c1405e4a9 None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:34.330601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84cf30-cebe-46bb-8ce1-754c1405e4a9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:34.336840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 199/395] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:34 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:34.339038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84cf30-cebe-46bb-8ce1-754c1405e4a9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:34.360410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9acac2-5326-4951-9360-f02a130f3744 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:34.424943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:34.426065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.426579 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.427087 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.428025 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] RBAC: Adding query filter params (name=member) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:34.428555 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:34.430196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:34.440060 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 197/396] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:34.446407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:34.502470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84cf30-cebe-46bb-8ce1-754c1405e4a9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:34.538508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:34.545871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4133e-85ea-4e85-b975-2b48bb63ff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:34.547401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4133e-85ea-4e85-b975-2b48bb63ff38 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.547910 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4133e-85ea-4e85-b975-2b48bb63ff38 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.548405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4133e-85ea-4e85-b975-2b48bb63ff38 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.550034 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4133e-85ea-4e85-b975-2b48bb63ff38 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:34.552009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4133e-85ea-4e85-b975-2b48bb63ff38 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:34.558317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 200/397] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:34 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:34.561359 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4133e-85ea-4e85-b975-2b48bb63ff38 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:34.635289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:34.641604 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844040-6700-4a2f-95f6-8c0bc7908adb None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:34.713534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba4a3b-4d2d-4941-834d-0269ade65d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:34.714263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4a3b-4d2d-4941-834d-0269ade65d7b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.714744 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4a3b-4d2d-4941-834d-0269ade65d7b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.715023 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4a3b-4d2d-4941-834d-0269ade65d7b None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.716371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba4a3b-4d2d-4941-834d-0269ade65d7b None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:34.718049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba4a3b-4d2d-4941-834d-0269ade65d7b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:34.720377 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 198/398] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:34 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:34.729517 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba4a3b-4d2d-4941-834d-0269ade65d7b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:34.741494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4133e-85ea-4e85-b975-2b48bb63ff38 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:34.777345 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a23a9-8c69-4153-b841-414baf410fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:34.778475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a23a9-8c69-4153-b841-414baf410fb7 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.778974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a23a9-8c69-4153-b841-414baf410fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.779469 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a23a9-8c69-4153-b841-414baf410fb7 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d/groups/2d2fccf3bb484b9dafb73ec9e138909b/roles/d883529ef5284c35b27b759c57353383` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.781987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a23a9-8c69-4153-b841-414baf410fb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=278f44bb741f4075b58db40b154bb64d, group_id=2d2fccf3bb484b9dafb73ec9e138909b, role_id=d883529ef5284c35b27b759c57353383)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:34.783776 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a23a9-8c69-4153-b841-414baf410fb7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:34.802369 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 201/399] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:26:34 2022] PUT /identity/v3/projects/278f44bb741f4075b58db40b154bb64d/groups/2d2fccf3bb484b9dafb73ec9e138909b/roles/d883529ef5284c35b27b759c57353383 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:26:34.804468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2e20-2c53-4d53-af83-9aa9987c978f None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.805057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2e20-2c53-4d53-af83-9aa9987c978f None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.805588 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2e20-2c53-4d53-af83-9aa9987c978f None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.811619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 202/400] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:34.813453 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead9e82-7eb8-4942-931f-5fab13003cdb None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.813961 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead9e82-7eb8-4942-931f-5fab13003cdb None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.814484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead9e82-7eb8-4942-931f-5fab13003cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.842673 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-3ead9e82-7eb8-4942-931f-5fab13003cdb None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:34.844285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-3ead9e82-7eb8-4942-931f-5fab13003cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:34.859444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba4a3b-4d2d-4941-834d-0269ade65d7b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:34.905204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:34.933068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 203/401] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:34.937710 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc35f9e-99d8-4fc4-892b-8195593d816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:34.938722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc35f9e-99d8-4fc4-892b-8195593d816e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:34.939207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc35f9e-99d8-4fc4-892b-8195593d816e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:34.939721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc35f9e-99d8-4fc4-892b-8195593d816e None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:34.941211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc35f9e-99d8-4fc4-892b-8195593d816e None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:34.942973 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc35f9e-99d8-4fc4-892b-8195593d816e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:34.950168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 199/402] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:34 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:34.951912 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead9e82-7eb8-4942-931f-5fab13003cdb None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:34.968796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc35f9e-99d8-4fc4-892b-8195593d816e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:35.002609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:35.084974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead9e82-7eb8-4942-931f-5fab13003cdb None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:35.128269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56550ee-10db-499f-99dc-75c18fddf811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:35.129358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56550ee-10db-499f-99dc-75c18fddf811 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.129644 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56550ee-10db-499f-99dc-75c18fddf811 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.129929 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56550ee-10db-499f-99dc-75c18fddf811 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/groups/2d2fccf3bb484b9dafb73ec9e138909b/roles/cc0fab8f90a44acc90de9af43e2bd7df` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.138995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56550ee-10db-499f-99dc-75c18fddf811 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93, group_id=2d2fccf3bb484b9dafb73ec9e138909b, role_id=cc0fab8f90a44acc90de9af43e2bd7df)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:35.140500 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56550ee-10db-499f-99dc-75c18fddf811 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:35.152650 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc35f9e-99d8-4fc4-892b-8195593d816e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:35.167037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 204/403] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:26:34 2022] PUT /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/groups/2d2fccf3bb484b9dafb73ec9e138909b/roles/cc0fab8f90a44acc90de9af43e2bd7df => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:26:35.198986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7731b-4ee3-41a1-ba91-03b5a6536e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:35.199729 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7731b-4ee3-41a1-ba91-03b5a6536e15 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.200019 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7731b-4ee3-41a1-ba91-03b5a6536e15 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.200301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7731b-4ee3-41a1-ba91-03b5a6536e15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.213701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7731b-4ee3-41a1-ba91-03b5a6536e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-57c7731b-4ee3-41a1-ba91-03b5a6536e15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:35.221998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:35.223216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:35.224384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 200/404] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:35.230412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56550ee-10db-499f-99dc-75c18fddf811 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:35.358757 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:35.366327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56550ee-10db-499f-99dc-75c18fddf811 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:35.415037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb3838-4a41-41d7-b5ac-60a8b42372c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:35.415725 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3838-4a41-41d7-b5ac-60a8b42372c4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.416011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3838-4a41-41d7-b5ac-60a8b42372c4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.416286 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3838-4a41-41d7-b5ac-60a8b42372c4 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.417135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb3838-4a41-41d7-b5ac-60a8b42372c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:35.417447 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb3838-4a41-41d7-b5ac-60a8b42372c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:35.418997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb3838-4a41-41d7-b5ac-60a8b42372c4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:35.429349 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 205/405] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:35.435184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7731b-4ee3-41a1-ba91-03b5a6536e15 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:35.493193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:35.514642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c119d-1fb8-48e6-8205-0af5eae76e5f None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.515016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c119d-1fb8-48e6-8205-0af5eae76e5f None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.515186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c119d-1fb8-48e6-8205-0af5eae76e5f None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.517868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 206/406] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:35.522324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ce0d-dcb8-4301-b4eb-3b9900ac8841 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.522736 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ce0d-dcb8-4301-b4eb-3b9900ac8841 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.523020 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ce0d-dcb8-4301-b4eb-3b9900ac8841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.544332 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-5404ce0d-dcb8-4301-b4eb-3b9900ac8841 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:35.545547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-5404ce0d-dcb8-4301-b4eb-3b9900ac8841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:35.555793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7731b-4ee3-41a1-ba91-03b5a6536e15 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:35.598609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dae53-0bae-4c88-af88-fb7f3b2d9129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:35.599759 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dae53-0bae-4c88-af88-fb7f3b2d9129 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.600098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dae53-0bae-4c88-af88-fb7f3b2d9129 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.600437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dae53-0bae-4c88-af88-fb7f3b2d9129 None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.601670 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dae53-0bae-4c88-af88-fb7f3b2d9129 None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:35.605622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dae53-0bae-4c88-af88-fb7f3b2d9129 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:35.610208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 201/407] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:35 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:35.612695 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dae53-0bae-4c88-af88-fb7f3b2d9129 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:35.639804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 207/408] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:35.641354 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706458b-c726-4696-ae41-287972b15e0c None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.641645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706458b-c726-4696-ae41-287972b15e0c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.641920 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706458b-c726-4696-ae41-287972b15e0c None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.643761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 208/409] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:35.649448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c0813-419e-4981-a119-518265b886ed None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.650204 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c0813-419e-4981-a119-518265b886ed None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.650928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c0813-419e-4981-a119-518265b886ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.671188 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-bd6c0813-419e-4981-a119-518265b886ed None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:35.672698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-bd6c0813-419e-4981-a119-518265b886ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:35.769276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dae53-0bae-4c88-af88-fb7f3b2d9129 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:35.783223 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 209/410] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:35.785204 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c0813-419e-4981-a119-518265b886ed None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:35.813213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:35.824733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:35.844132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daf17b-c36a-4e9e-b8fe-1baeb7f5a597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:35.845308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daf17b-c36a-4e9e-b8fe-1baeb7f5a597 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:35.846497 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daf17b-c36a-4e9e-b8fe-1baeb7f5a597 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:35.846776 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daf17b-c36a-4e9e-b8fe-1baeb7f5a597 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:35.850865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95daf17b-c36a-4e9e-b8fe-1baeb7f5a597 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:35.853237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95daf17b-c36a-4e9e-b8fe-1baeb7f5a597 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:35.856057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 202/411] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:35 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:35.857071 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daf17b-c36a-4e9e-b8fe-1baeb7f5a597 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:35.945237 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c0813-419e-4981-a119-518265b886ed None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:35.997826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daf17b-c36a-4e9e-b8fe-1baeb7f5a597 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:36.003491 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1bfdd-4e71-4401-901c-1df90cc99e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:36.004696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1bfdd-4e71-4401-901c-1df90cc99e9b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:36.005218 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1bfdd-4e71-4401-901c-1df90cc99e9b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:36.005862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1bfdd-4e71-4401-901c-1df90cc99e9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:36.020397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1bfdd-4e71-4401-901c-1df90cc99e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-e4c1bfdd-4e71-4401-901c-1df90cc99e9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:36.025990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:36.027275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:36.028892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 210/412] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:26:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:36.030655 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1bfdd-4e71-4401-901c-1df90cc99e9b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:36.034195 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6c3ca-1472-4999-aa75-e2bc8f52d41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:36.035878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6c3ca-1472-4999-aa75-e2bc8f52d41e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:36.035878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6c3ca-1472-4999-aa75-e2bc8f52d41e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:36.035878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6c3ca-1472-4999-aa75-e2bc8f52d41e None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:36.047351 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6c3ca-1472-4999-aa75-e2bc8f52d41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-ddc6c3ca-1472-4999-aa75-e2bc8f52d41e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:36.053289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:36.054611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:36.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:36.062148 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 203/413] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:36.062148 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6c3ca-1472-4999-aa75-e2bc8f52d41e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:36.108398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:36.118647 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:36.144875 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1bfdd-4e71-4401-901c-1df90cc99e9b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:36.167062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6c3ca-1472-4999-aa75-e2bc8f52d41e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:36.183079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0758-4253-45de-89c1-ab30103adaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:36.184027 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0758-4253-45de-89c1-ab30103adaf8 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:36.184569 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0758-4253-45de-89c1-ab30103adaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:36.185000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0758-4253-45de-89c1-ab30103adaf8 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/groups/2d2fccf3bb484b9dafb73ec9e138909b/roles/d883529ef5284c35b27b759c57353383` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:36.187631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a0758-4253-45de-89c1-ab30103adaf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93, group_id=2d2fccf3bb484b9dafb73ec9e138909b, role_id=d883529ef5284c35b27b759c57353383)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:36.189392 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a0758-4253-45de-89c1-ab30103adaf8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:36.205468 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce899889-75ea-43da-807a-9da466673646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:36.206221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899889-75ea-43da-807a-9da466673646 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:36.206606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899889-75ea-43da-807a-9da466673646 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:36.207083 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899889-75ea-43da-807a-9da466673646 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:36.207818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 211/414] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:26:36 2022] PUT /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93/groups/2d2fccf3bb484b9dafb73ec9e138909b/roles/d883529ef5284c35b27b759c57353383 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:36.209034 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0758-4253-45de-89c1-ab30103adaf8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:36.214504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce899889-75ea-43da-807a-9da466673646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:36.215032 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce899889-75ea-43da-807a-9da466673646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:36.216671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce899889-75ea-43da-807a-9da466673646 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:36.234078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 204/415] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:36.240267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce899889-75ea-43da-807a-9da466673646 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:36.337582 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0758-4253-45de-89c1-ab30103adaf8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:36.382328 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:36.383318 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:36.383883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:36.384430 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:36.385331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] RBAC: Adding query filter params (name=member) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:36.385786 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:36.387686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:36.394519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:36.398846 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 212/416] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:36.407618 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:36.514974 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce899889-75ea-43da-807a-9da466673646 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:36.532835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:36.550625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faf744-7a8d-4e5d-bb60-0d697565d7dc None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:36.578611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0165a-464c-4554-ac9f-8259b6a13bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:36.579308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0165a-464c-4554-ac9f-8259b6a13bd5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:36.579596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0165a-464c-4554-ac9f-8259b6a13bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:36.579875 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0165a-464c-4554-ac9f-8259b6a13bd5 None admin] PATH_INFO: `/v3/groups/d8deb47a8872416b8d027b7ddcc0ffbd` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:36.586140 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0165a-464c-4554-ac9f-8259b6a13bd5 None admin] RBAC: Authorizing `identity:get_group(group_id=d8deb47a8872416b8d027b7ddcc0ffbd)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:36.586140 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0165a-464c-4554-ac9f-8259b6a13bd5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:36.588069 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 205/417] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:36 2022] GET /identity/v3/groups/d8deb47a8872416b8d027b7ddcc0ffbd => generated 229 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:36.594156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0165a-464c-4554-ac9f-8259b6a13bd5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:36.613897 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d187-34a1-4b1e-bc5e-10110734fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:36.616974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d187-34a1-4b1e-bc5e-10110734fb96 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:36.617470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d187-34a1-4b1e-bc5e-10110734fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:36.617995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d187-34a1-4b1e-bc5e-10110734fb96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:36.738265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:36.750345 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4d187-34a1-4b1e-bc5e-10110734fb96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:36.751889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4d187-34a1-4b1e-bc5e-10110734fb96 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:36.818146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0165a-464c-4554-ac9f-8259b6a13bd5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:36.873683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d351eb-7045-4a52-8cc1-44d93ad4f861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:36.875686 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d351eb-7045-4a52-8cc1-44d93ad4f861 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:36.876832 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d351eb-7045-4a52-8cc1-44d93ad4f861 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:36.879077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d351eb-7045-4a52-8cc1-44d93ad4f861 None admin] PATH_INFO: `/v3/projects/52cdb50da43d48649726866000557d76` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:36.882571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d351eb-7045-4a52-8cc1-44d93ad4f861 None admin] RBAC: Authorizing `identity:get_project(project_id=52cdb50da43d48649726866000557d76)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:36.885511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d351eb-7045-4a52-8cc1-44d93ad4f861 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:36.888623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 206/418] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:36 2022] GET /identity/v3/projects/52cdb50da43d48649726866000557d76 => generated 344 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:36.894852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d351eb-7045-4a52-8cc1-44d93ad4f861 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:36.965140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 213/419] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:36.971077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d187-34a1-4b1e-bc5e-10110734fb96 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:36.976029 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:37.090431 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:37.095552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d187-34a1-4b1e-bc5e-10110734fb96 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:37.120354 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d351eb-7045-4a52-8cc1-44d93ad4f861 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:37.139076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45787e4-0425-41e1-9460-2e9b5639c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:37.139767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45787e4-0425-41e1-9460-2e9b5639c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.140054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45787e4-0425-41e1-9460-2e9b5639c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.140334 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45787e4-0425-41e1-9460-2e9b5639c8a4 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.143033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45787e4-0425-41e1-9460-2e9b5639c8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:37.144589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45787e4-0425-41e1-9460-2e9b5639c8a4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:37.146734 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 214/420] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:36 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:37.152578 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45787e4-0425-41e1-9460-2e9b5639c8a4 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:37.201761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ad740-91f7-4842-aa2d-bc66101d05ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:37.202512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad740-91f7-4842-aa2d-bc66101d05ca None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.202798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad740-91f7-4842-aa2d-bc66101d05ca None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.203085 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad740-91f7-4842-aa2d-bc66101d05ca None admin] PATH_INFO: `/v3/projects/52cdb50da43d48649726866000557d76/groups/d8deb47a8872416b8d027b7ddcc0ffbd/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.208518 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ad740-91f7-4842-aa2d-bc66101d05ca None admin] RBAC: Authorizing `identity:create_grant(project_id=52cdb50da43d48649726866000557d76, group_id=d8deb47a8872416b8d027b7ddcc0ffbd, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:37.210001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ad740-91f7-4842-aa2d-bc66101d05ca None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:37.238268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 207/421] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:26:36 2022] PUT /identity/v3/projects/52cdb50da43d48649726866000557d76/groups/d8deb47a8872416b8d027b7ddcc0ffbd/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:37.278768 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45787e4-0425-41e1-9460-2e9b5639c8a4 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:37.323311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86bd13-248a-41e7-bcb8-ee7641e245e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:37.324458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86bd13-248a-41e7-bcb8-ee7641e245e6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.325005 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86bd13-248a-41e7-bcb8-ee7641e245e6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.325512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86bd13-248a-41e7-bcb8-ee7641e245e6 None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.328846 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86bd13-248a-41e7-bcb8-ee7641e245e6 None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:37.330926 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86bd13-248a-41e7-bcb8-ee7641e245e6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:37.334645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 215/422] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:37 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:37.340421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ad740-91f7-4842-aa2d-bc66101d05ca None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:37.413211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a681778-02db-4d58-9044-6c780d469871 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.413871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a681778-02db-4d58-9044-6c780d469871 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.414442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a681778-02db-4d58-9044-6c780d469871 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.416091 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 216/423] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:37.443237 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3fc63-054e-4219-8ab0-92ee336088ce None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.443995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3fc63-054e-4219-8ab0-92ee336088ce None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.444175 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3fc63-054e-4219-8ab0-92ee336088ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.471419 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-8df3fc63-054e-4219-8ab0-92ee336088ce None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:37.472631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-8df3fc63-054e-4219-8ab0-92ee336088ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:37.503889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:37.528484 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ad740-91f7-4842-aa2d-bc66101d05ca None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:37.534479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:37.561011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 217/424] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:37.571720 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6627cd5-0946-42d3-aac9-32c600959c8d None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.572074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6627cd5-0946-42d3-aac9-32c600959c8d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.572359 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6627cd5-0946-42d3-aac9-32c600959c8d None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.574145 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 218/425] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:37.583915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865190b-1169-40fe-a095-ea31ba986970 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.583915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865190b-1169-40fe-a095-ea31ba986970 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.584209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865190b-1169-40fe-a095-ea31ba986970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.595900 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:37.596847 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.601603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.601603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.601603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] RBAC: Adding query filter params (group.id=2d2fccf3bb484b9dafb73ec9e138909b, role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=278f44bb741f4075b58db40b154bb64d) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:37.601603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:37.601603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:37.607153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 208/426] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:37 2022] GET /identity/v3/role_assignments?group.id=2d2fccf3bb484b9dafb73ec9e138909b&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 611 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:37.607988 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:37.614444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-9865190b-1169-40fe-a095-ea31ba986970 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:37.615855 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-9865190b-1169-40fe-a095-ea31ba986970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:37.735392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919da42-02fe-4ab3-a9e3-1bac244da030 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:37.777193 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 219/427] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:37.791540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58349a-d60f-43ca-9c30-1e568db41545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:37.792228 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58349a-d60f-43ca-9c30-1e568db41545 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.792508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58349a-d60f-43ca-9c30-1e568db41545 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.792782 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58349a-d60f-43ca-9c30-1e568db41545 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:37.805739 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58349a-d60f-43ca-9c30-1e568db41545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-5b58349a-d60f-43ca-9c30-1e568db41545 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:37.811569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:37.813058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:37.814208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 209/428] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:37.819824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865190b-1169-40fe-a095-ea31ba986970 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:37.821955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58349a-d60f-43ca-9c30-1e568db41545 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:37.853689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:37.940425 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58349a-d60f-43ca-9c30-1e568db41545 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:37.984505 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865190b-1169-40fe-a095-ea31ba986970 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:37.987727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a810d-ca85-4a6f-8631-1f222a5a019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:37.988215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:37.988573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a810d-ca85-4a6f-8631-1f222a5a019e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:37.988957 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a810d-ca85-4a6f-8631-1f222a5a019e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:37.989318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a810d-ca85-4a6f-8631-1f222a5a019e None admin] PATH_INFO: `/v3/roles/member` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.003851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a810d-ca85-4a6f-8631-1f222a5a019e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-453a810d-ca85-4a6f-8631-1f222a5a019e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:38.009035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:38.013808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:26:38.014933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:38.016119 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 210/429] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:38.017427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a810d-ca85-4a6f-8631-1f222a5a019e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:38.043824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:38.045205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.045720 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.046459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.048095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:38.048614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:38.051227 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:38.066697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 220/430] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:38.074719 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:38.127157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:38.141767 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a810d-ca85-4a6f-8631-1f222a5a019e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:38.179574 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:38.180566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.181055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.181511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.182462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] RBAC: Adding query filter params (name=member) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:38.182954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:38.184556 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:38.194432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 211/431] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:38.198901 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f94de8-8994-4f91-8e66-aeb607b83fb1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:38.200072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:38.244086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c241698-413f-4695-806a-f08ee27e21a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:38.244652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c241698-413f-4695-806a-f08ee27e21a2 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.244941 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c241698-413f-4695-806a-f08ee27e21a2 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.245219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c241698-413f-4695-806a-f08ee27e21a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.282616 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:38.331710 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f9838-b036-47e4-8cbe-dae1021de9cb None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:38.345702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c241698-413f-4695-806a-f08ee27e21a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:38.348229 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c241698-413f-4695-806a-f08ee27e21a2 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:38.373480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5fd56-66c1-48d0-810f-917b234e4eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:38.374555 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5fd56-66c1-48d0-810f-917b234e4eed None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.375068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5fd56-66c1-48d0-810f-917b234e4eed None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.375571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5fd56-66c1-48d0-810f-917b234e4eed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.397639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:38.454836 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5fd56-66c1-48d0-810f-917b234e4eed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:38.456817 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5fd56-66c1-48d0-810f-917b234e4eed None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:38.492368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 221/432] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:38.498739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c241698-413f-4695-806a-f08ee27e21a2 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:38.571910 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:38.596788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 212/433] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:38.603311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5fd56-66c1-48d0-810f-917b234e4eed None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:38.627263 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c241698-413f-4695-806a-f08ee27e21a2 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:38.658522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:38.676372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d23ff-359e-4821-817d-5e14da060141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:38.677156 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d23ff-359e-4821-817d-5e14da060141 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.677544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d23ff-359e-4821-817d-5e14da060141 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.677824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d23ff-359e-4821-817d-5e14da060141 None admin] PATH_INFO: `/v3/projects/278f44bb741f4075b58db40b154bb64d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.679094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d23ff-359e-4821-817d-5e14da060141 None admin] RBAC: Authorizing `identity:get_project(project_id=278f44bb741f4075b58db40b154bb64d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:38.680598 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d23ff-359e-4821-817d-5e14da060141 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:38.688737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 222/434] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/projects/278f44bb741f4075b58db40b154bb64d => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:38.689870 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d23ff-359e-4821-817d-5e14da060141 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:38.723631 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5fd56-66c1-48d0-810f-917b234e4eed None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:38.763968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f943814-f540-44c4-9170-ffef0c5440d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:38.765747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f943814-f540-44c4-9170-ffef0c5440d5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.766221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f943814-f540-44c4-9170-ffef0c5440d5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.766666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f943814-f540-44c4-9170-ffef0c5440d5 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.768487 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f943814-f540-44c4-9170-ffef0c5440d5 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:38.770810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f943814-f540-44c4-9170-ffef0c5440d5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:38.774026 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 213/435] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:38.775940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e5cf-0c76-4fa8-bbc9-7bccdb128dd3 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.776747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e5cf-0c76-4fa8-bbc9-7bccdb128dd3 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.777565 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e5cf-0c76-4fa8-bbc9-7bccdb128dd3 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.779444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 214/436] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:38.780773 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806e5cf-0c76-4fa8-bbc9-7bccdb128dd3 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:38.810670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d23ff-359e-4821-817d-5e14da060141 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:38.836178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:38.852933 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598feea9-4da3-41a5-b78c-4ea0ca41ca74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:38.853637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598feea9-4da3-41a5-b78c-4ea0ca41ca74 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.853924 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598feea9-4da3-41a5-b78c-4ea0ca41ca74 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.854241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598feea9-4da3-41a5-b78c-4ea0ca41ca74 None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.855494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598feea9-4da3-41a5-b78c-4ea0ca41ca74 None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:38.857022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598feea9-4da3-41a5-b78c-4ea0ca41ca74 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:38.863940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 223/437] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:38.865215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cded1-2dff-4a53-bd0a-fda2063e452d None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.865506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cded1-2dff-4a53-bd0a-fda2063e452d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.865791 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cded1-2dff-4a53-bd0a-fda2063e452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.884881 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-b81cded1-2dff-4a53-bd0a-fda2063e452d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:38.886167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-b81cded1-2dff-4a53-bd0a-fda2063e452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:38.904192 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806e5cf-0c76-4fa8-bbc9-7bccdb128dd3 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:38.922770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:38.946571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1facc-574f-44fa-8ed8-82497d1a8f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:38.947234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1facc-574f-44fa-8ed8-82497d1a8f9f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:38.947568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1facc-574f-44fa-8ed8-82497d1a8f9f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:38.947777 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1facc-574f-44fa-8ed8-82497d1a8f9f None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:38.949141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1facc-574f-44fa-8ed8-82497d1a8f9f None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:38.950631 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1facc-574f-44fa-8ed8-82497d1a8f9f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:38.957013 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 215/438] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:38.957013 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1facc-574f-44fa-8ed8-82497d1a8f9f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:38.974129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 224/439] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:38.974129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cded1-2dff-4a53-bd0a-fda2063e452d None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:39.070662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1facc-574f-44fa-8ed8-82497d1a8f9f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:39.083303 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cded1-2dff-4a53-bd0a-fda2063e452d None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:39.113686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:39.118622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:39.119263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:39.119549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:39.119814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:39.120634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] RBAC: Adding query filter params (group.id=2d2fccf3bb484b9dafb73ec9e138909b, role.id=d883529ef5284c35b27b759c57353383, scope.project.id=278f44bb741f4075b58db40b154bb64d) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:39.120946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:39.122456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:39.129015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 216/440] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/role_assignments?group.id=2d2fccf3bb484b9dafb73ec9e138909b&scope.project.id=278f44bb741f4075b58db40b154bb64d&role.id=d883529ef5284c35b27b759c57353383 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:39.129859 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:39.130878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:39.131569 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:39.131853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:39.132126 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:39.132937 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] RBAC: Adding query filter params (group.id=2d2fccf3bb484b9dafb73ec9e138909b, role.id=cc0fab8f90a44acc90de9af43e2bd7df, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:39.133249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:39.134761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:39.148646 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 225/441] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:38 2022] GET /identity/v3/role_assignments?group.id=2d2fccf3bb484b9dafb73ec9e138909b&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=cc0fab8f90a44acc90de9af43e2bd7df => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:39.204436 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:39.272811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369deaa0-d6a4-4938-9cc9-f310bea6d89e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:39.319269 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78824d1e-0625-4668-b7f3-751406db61bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:39.320378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824d1e-0625-4668-b7f3-751406db61bd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:39.320938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824d1e-0625-4668-b7f3-751406db61bd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:39.321444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824d1e-0625-4668-b7f3-751406db61bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:39.335927 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78824d1e-0625-4668-b7f3-751406db61bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-78824d1e-0625-4668-b7f3-751406db61bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:39.343066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:39.344222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:39.345191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:39.347634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 217/442] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:39.352167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:39.473775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c039a05-e88e-4525-bca5-97d97d1d5b6e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:39.509239 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeb57a-e200-4082-ab3c-9eec60ee3e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:39.509863 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeb57a-e200-4082-ab3c-9eec60ee3e16 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:39.510178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeb57a-e200-4082-ab3c-9eec60ee3e16 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:39.510460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeb57a-e200-4082-ab3c-9eec60ee3e16 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:39.511200 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeb57a-e200-4082-ab3c-9eec60ee3e16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:39.511516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeb57a-e200-4082-ab3c-9eec60ee3e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:39.512923 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeb57a-e200-4082-ab3c-9eec60ee3e16 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:39.524062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 226/443] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:39.530237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78824d1e-0625-4668-b7f3-751406db61bd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:39.620610 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0398c-2e50-49f6-8182-6b865828fe03 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:39.620830 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0398c-2e50-49f6-8182-6b865828fe03 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:39.621075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0398c-2e50-49f6-8182-6b865828fe03 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:39.622655 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 227/444] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:39.628003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642817f-1f4f-448e-80ad-d931cabccc22 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:39.628298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642817f-1f4f-448e-80ad-d931cabccc22 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:39.628667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642817f-1f4f-448e-80ad-d931cabccc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:39.648156 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-4642817f-1f4f-448e-80ad-d931cabccc22 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:39.649396 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-4642817f-1f4f-448e-80ad-d931cabccc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:39.650339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78824d1e-0625-4668-b7f3-751406db61bd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:39.662667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:39.689979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c9018-1501-44e8-80a4-ffeaf9314f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:39.690671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c9018-1501-44e8-80a4-ffeaf9314f1c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:39.691002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c9018-1501-44e8-80a4-ffeaf9314f1c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:39.691307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c9018-1501-44e8-80a4-ffeaf9314f1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:39.702034 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:39.732568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 228/445] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:39.754936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642817f-1f4f-448e-80ad-d931cabccc22 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:39.774688 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c9018-1501-44e8-80a4-ffeaf9314f1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:39.776191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c9018-1501-44e8-80a4-ffeaf9314f1c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:39.856553 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642817f-1f4f-448e-80ad-d931cabccc22 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:39.876338 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 218/446] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:26:39.882512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c9018-1501-44e8-80a4-ffeaf9314f1c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:39.895143 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:39.896487 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaa8e6-c745-43f1-bf45-40bc9fca72ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:39.897158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa8e6-c745-43f1-bf45-40bc9fca72ad None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:39.897449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa8e6-c745-43f1-bf45-40bc9fca72ad None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:39.897715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa8e6-c745-43f1-bf45-40bc9fca72ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:39.909948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaa8e6-c745-43f1-bf45-40bc9fca72ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-91eaa8e6-c745-43f1-bf45-40bc9fca72ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:39.915180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:39.916326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:39.917289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 229/447] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:26:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:39.922752 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaa8e6-c745-43f1-bf45-40bc9fca72ad None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:39.966470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:39.995219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c9018-1501-44e8-80a4-ffeaf9314f1c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:40.032895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c452c4c-2bb6-4636-a09f-c67037c99042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:40.033569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c452c4c-2bb6-4636-a09f-c67037c99042 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:40.033846 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c452c4c-2bb6-4636-a09f-c67037c99042 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:40.034077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaa8e6-c745-43f1-bf45-40bc9fca72ad None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:40.034211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c452c4c-2bb6-4636-a09f-c67037c99042 None admin] PATH_INFO: `/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:40.035412 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c452c4c-2bb6-4636-a09f-c67037c99042 None admin] RBAC: Authorizing `identity:get_project(project_id=ae4b1bf645fd425ba9d22c9d4b08bd93)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:40.036893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c452c4c-2bb6-4636-a09f-c67037c99042 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:40.038669 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 219/448] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:39 2022] GET /identity/v3/projects/ae4b1bf645fd425ba9d22c9d4b08bd93 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:40.044579 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c452c4c-2bb6-4636-a09f-c67037c99042 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:40.071833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e240f-2def-4015-a600-879f171744e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:40.072478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e240f-2def-4015-a600-879f171744e0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:40.072760 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e240f-2def-4015-a600-879f171744e0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:40.073038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e240f-2def-4015-a600-879f171744e0 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:40.073770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e240f-2def-4015-a600-879f171744e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:40.074080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e240f-2def-4015-a600-879f171744e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:40.075527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e240f-2def-4015-a600-879f171744e0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:40.084595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 230/449] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:40.090528 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e240f-2def-4015-a600-879f171744e0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:40.129036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:40.153257 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c452c4c-2bb6-4636-a09f-c67037c99042 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:40.190828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345adfce-4a7e-4f5e-90f5-63314d845a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:40.191572 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345adfce-4a7e-4f5e-90f5-63314d845a02 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:40.191855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345adfce-4a7e-4f5e-90f5-63314d845a02 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:40.192135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345adfce-4a7e-4f5e-90f5-63314d845a02 None admin] PATH_INFO: `/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:40.193326 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345adfce-4a7e-4f5e-90f5-63314d845a02 None admin] RBAC: Authorizing `identity:get_group(group_id=2d2fccf3bb484b9dafb73ec9e138909b)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:40.195184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345adfce-4a7e-4f5e-90f5-63314d845a02 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:40.196817 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 220/450] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:40 2022] GET /identity/v3/groups/2d2fccf3bb484b9dafb73ec9e138909b => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:40.202761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345adfce-4a7e-4f5e-90f5-63314d845a02 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:40.204082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:40.207519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e240f-2def-4015-a600-879f171744e0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:40.242143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812f8c2-e40f-40db-b72d-689b2296b2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:40.242802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812f8c2-e40f-40db-b72d-689b2296b2bc None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:40.243076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812f8c2-e40f-40db-b72d-689b2296b2bc None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:40.243360 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812f8c2-e40f-40db-b72d-689b2296b2bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:40.304247 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345adfce-4a7e-4f5e-90f5-63314d845a02 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:40.314420 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812f8c2-e40f-40db-b72d-689b2296b2bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:40.315887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812f8c2-e40f-40db-b72d-689b2296b2bc None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:40.339971 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:40.340587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:40.340892 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:40.341154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:40.341950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] RBAC: Adding query filter params (group.id=2d2fccf3bb484b9dafb73ec9e138909b, role.id=d883529ef5284c35b27b759c57353383, scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:40.342296 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:40.343768 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:40.350127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 221/451] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:40 2022] GET /identity/v3/role_assignments?group.id=2d2fccf3bb484b9dafb73ec9e138909b&scope.project.id=ae4b1bf645fd425ba9d22c9d4b08bd93&role.id=d883529ef5284c35b27b759c57353383 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:40.421417 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:40.425673 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 231/452] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:26:40.431924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:40.447779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:40.547493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf29b63-a5bf-4f84-b982-d22bfc64f876 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:40.583189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eba874-1e55-4863-b95c-30ebf10159d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:40.583862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eba874-1e55-4863-b95c-30ebf10159d3 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:40.584146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eba874-1e55-4863-b95c-30ebf10159d3 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:40.586465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eba874-1e55-4863-b95c-30ebf10159d3 None admin] PATH_INFO: `/v3/projects/52cdb50da43d48649726866000557d76` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:40.586465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eba874-1e55-4863-b95c-30ebf10159d3 None admin] RBAC: Authorizing `identity:get_project(project_id=52cdb50da43d48649726866000557d76)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:40.587117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eba874-1e55-4863-b95c-30ebf10159d3 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:40.588841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 222/453] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:26:40 2022] GET /identity/v3/projects/52cdb50da43d48649726866000557d76 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:40.594719 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812f8c2-e40f-40db-b72d-689b2296b2bc None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:40.702585 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812f8c2-e40f-40db-b72d-689b2296b2bc None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:40.739087 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cee02f-eff3-4a5f-a3f5-4c7b157cff52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:40.739752 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee02f-eff3-4a5f-a3f5-4c7b157cff52 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:40.740034 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee02f-eff3-4a5f-a3f5-4c7b157cff52 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:40.740309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee02f-eff3-4a5f-a3f5-4c7b157cff52 None admin] PATH_INFO: `/v3/groups/d8deb47a8872416b8d027b7ddcc0ffbd` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:40.741511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cee02f-eff3-4a5f-a3f5-4c7b157cff52 None admin] RBAC: Authorizing `identity:get_group(group_id=d8deb47a8872416b8d027b7ddcc0ffbd)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:40.743006 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cee02f-eff3-4a5f-a3f5-4c7b157cff52 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:40.744685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 232/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:26:40 2022] GET /identity/v3/groups/d8deb47a8872416b8d027b7ddcc0ffbd => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:40.750823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eba874-1e55-4863-b95c-30ebf10159d3 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:40.837587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:40.861436 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eba874-1e55-4863-b95c-30ebf10159d3 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:40.897837 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcc23d-08bc-4a69-bb34-b216999f7557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:40.898506 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcc23d-08bc-4a69-bb34-b216999f7557 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:40.898789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcc23d-08bc-4a69-bb34-b216999f7557 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:40.899065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcc23d-08bc-4a69-bb34-b216999f7557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:40.899869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcc23d-08bc-4a69-bb34-b216999f7557 None admin] RBAC: Adding query filter params (group.id=d8deb47a8872416b8d027b7ddcc0ffbd, role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=52cdb50da43d48649726866000557d76) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:40.900187 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcc23d-08bc-4a69-bb34-b216999f7557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:40.901658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcc23d-08bc-4a69-bb34-b216999f7557 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:40.907999 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 223/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:26:40 2022] GET /identity/v3/role_assignments?group.id=d8deb47a8872416b8d027b7ddcc0ffbd&scope.project.id=52cdb50da43d48649726866000557d76&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:43.367499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a4bbf-dabd-4d91-8135-6a4ee2f42019 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:43.368590 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a4bbf-dabd-4d91-8135-6a4ee2f42019 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:43.369080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a4bbf-dabd-4d91-8135-6a4ee2f42019 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:43.371161 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 233/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:43.377411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd552fa-5d8c-49f7-87a6-6c447b6f3dd2 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:43.378616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd552fa-5d8c-49f7-87a6-6c447b6f3dd2 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:43.379103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd552fa-5d8c-49f7-87a6-6c447b6f3dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:43.406531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-9cd552fa-5d8c-49f7-87a6-6c447b6f3dd2 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:43.408165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-9cd552fa-5d8c-49f7-87a6-6c447b6f3dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:43.508718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 224/457] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:43.553154 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a4bbf-dabd-4d91-8135-6a4ee2f42019 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:43.574016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843a95d-50f5-4105-b438-ecba1770ea3c None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:43.574413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843a95d-50f5-4105-b438-ecba1770ea3c None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:43.574584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843a95d-50f5-4105-b438-ecba1770ea3c None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:43.577850 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 225/458] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:43.585687 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ceaa7-2889-4d15-8d42-3a5bca8322b8 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:43.586405 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ceaa7-2889-4d15-8d42-3a5bca8322b8 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:43.586927 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ceaa7-2889-4d15-8d42-3a5bca8322b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:43.607959 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-d38ceaa7-2889-4d15-8d42-3a5bca8322b8 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:43.609564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-d38ceaa7-2889-4d15-8d42-3a5bca8322b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:43.689064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:43.736329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a4bbf-dabd-4d91-8135-6a4ee2f42019 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:43.759888 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 226/459] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:43.786388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a5110-27dc-4e28-8025-858335ef605a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:43.787580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a5110-27dc-4e28-8025-858335ef605a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:43.788083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a5110-27dc-4e28-8025-858335ef605a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:43.788560 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a5110-27dc-4e28-8025-858335ef605a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:43.805157 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a5110-27dc-4e28-8025-858335ef605a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:43.807034 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a5110-27dc-4e28-8025-858335ef605a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:43.807679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a790f-08b5-4409-8291-618c5da17803 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:43.808027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a790f-08b5-4409-8291-618c5da17803 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:43.808305 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a790f-08b5-4409-8291-618c5da17803 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:43.810094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 227/460] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:43.810832 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a790f-08b5-4409-8291-618c5da17803 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:43.817028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-637a5110-27dc-4e28-8025-858335ef605a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:43.827182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:43.828211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:43.829190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:43.829190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:43.829190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:43.829190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:43.831861 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 234/461] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:43.833494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453daed3-b20c-4166-8909-606bf329e365 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:43.838146 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453daed3-b20c-4166-8909-606bf329e365 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:43.838863 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453daed3-b20c-4166-8909-606bf329e365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:43.860026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-453daed3-b20c-4166-8909-606bf329e365 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:43.861625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-453daed3-b20c-4166-8909-606bf329e365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:43.941396 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a790f-08b5-4409-8291-618c5da17803 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:43.957337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 235/462] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:43.958520 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453daed3-b20c-4166-8909-606bf329e365 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:43.981410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:43.982600 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:43.983116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:43.983668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:43.995843 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:43.997790 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:44.016005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:44.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:44.018027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:44.018027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:44.018027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:44.018027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:44.020334 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 228/463] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:44.020953 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:44.022332 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:44.090461 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:44.091603 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453daed3-b20c-4166-8909-606bf329e365 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:44.128754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:44.129398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.129673 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.129945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.130673 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:44.130978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:44.132369 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.139167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ba7de-6578-4167-be6f-2d21f46d6c49 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:44.145056 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 236/464] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:44.146124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:44.178584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc2bee-0917-4c2e-8720-fd2d6fe4d4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:44.179548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2bee-0917-4c2e-8720-fd2d6fe4d4e8 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.179824 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2bee-0917-4c2e-8720-fd2d6fe4d4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.180106 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2bee-0917-4c2e-8720-fd2d6fe4d4e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.191022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc2bee-0917-4c2e-8720-fd2d6fe4d4e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:44.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc2bee-0917-4c2e-8720-fd2d6fe4d4e8 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-52bc2bee-0917-4c2e-8720-fd2d6fe4d4e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:44.208100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:44.209168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:44.210197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:44.210197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:44.210197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:44.210197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:44.210197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 229/465] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:44.210976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71bcd2-2515-455b-85f4-fba9c9a0b3c5 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.211144 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71bcd2-2515-455b-85f4-fba9c9a0b3c5 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.211429 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71bcd2-2515-455b-85f4-fba9c9a0b3c5 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.212768 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 230/466] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:44.217263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71bcd2-2515-455b-85f4-fba9c9a0b3c5 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:44.265066 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf7040-b7fc-4379-8b4d-e91ed42ffb65 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:44.276398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:44.303823 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:44.304462 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.304747 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.305028 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.305735 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:44.306046 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:44.307522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.327470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 237/467] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:44.327470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:44.332245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71bcd2-2515-455b-85f4-fba9c9a0b3c5 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:44.341076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:44.370769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d43749-39fe-4804-b7b6-1a4ff3cb8d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:44.371412 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d43749-39fe-4804-b7b6-1a4ff3cb8d90 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.371694 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d43749-39fe-4804-b7b6-1a4ff3cb8d90 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.371970 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d43749-39fe-4804-b7b6-1a4ff3cb8d90 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.372666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d43749-39fe-4804-b7b6-1a4ff3cb8d90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:44.374182 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d43749-39fe-4804-b7b6-1a4ff3cb8d90 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.397145 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 231/468] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:44 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:44.399216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ac4f2-8aec-44f1-91f6-14e3cdf859e3 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.399731 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ac4f2-8aec-44f1-91f6-14e3cdf859e3 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.400210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ac4f2-8aec-44f1-91f6-14e3cdf859e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.421117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-ff2ac4f2-8aec-44f1-91f6-14e3cdf859e3 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:44.422900 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-ff2ac4f2-8aec-44f1-91f6-14e3cdf859e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:44.462840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d1d98-4091-42ea-aec0-a78e1d3a704c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:44.503141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 232/469] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:44.504831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ac4f2-8aec-44f1-91f6-14e3cdf859e3 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:44.534416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:44.535556 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.536146 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.536629 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.537572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:44.538130 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:44.539868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.554987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 238/470] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:44.560730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:44.600143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:44.636099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:44.647337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ac4f2-8aec-44f1-91f6-14e3cdf859e3 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:44.686179 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6103-1785-4a05-8c31-02166d8c1915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:44.686834 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6103-1785-4a05-8c31-02166d8c1915 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.687118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6103-1785-4a05-8c31-02166d8c1915 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.687395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6103-1785-4a05-8c31-02166d8c1915 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.688097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f6103-1785-4a05-8c31-02166d8c1915 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:44.689566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f6103-1785-4a05-8c31-02166d8c1915 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.692820 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18408b4b-b298-40f2-bfd6-147dce0da10a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:44.716955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 233/471] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:44 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:44.717565 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6103-1785-4a05-8c31-02166d8c1915 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:44.771519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93cb08-3d26-4beb-a6f3-ef38459cbcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:44.772216 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93cb08-3d26-4beb-a6f3-ef38459cbcee None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.772486 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93cb08-3d26-4beb-a6f3-ef38459cbcee None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.772766 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93cb08-3d26-4beb-a6f3-ef38459cbcee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.803360 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93cb08-3d26-4beb-a6f3-ef38459cbcee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:44.804867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93cb08-3d26-4beb-a6f3-ef38459cbcee None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-aa93cb08-3d26-4beb-a6f3-ef38459cbcee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:44.839699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:44.840784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:44.841762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:44.841762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:44.841762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:44.841762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:44.841762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 239/472] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:44.843609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e98cf3-cdeb-4f3f-ab16-72b752e24466 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.843898 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e98cf3-cdeb-4f3f-ab16-72b752e24466 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.844177 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e98cf3-cdeb-4f3f-ab16-72b752e24466 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.845465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 240/473] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:44.848732 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6103-1785-4a05-8c31-02166d8c1915 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:44.850146 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e98cf3-cdeb-4f3f-ab16-72b752e24466 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:44.897247 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991f8aa-4844-44d4-abbe-61b5f365b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:44.897910 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991f8aa-4844-44d4-abbe-61b5f365b47f None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.898236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991f8aa-4844-44d4-abbe-61b5f365b47f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.898516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991f8aa-4844-44d4-abbe-61b5f365b47f None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.899231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991f8aa-4844-44d4-abbe-61b5f365b47f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:44.900748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991f8aa-4844-44d4-abbe-61b5f365b47f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:44.930023 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 234/474] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:44 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:44.935216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9ec0c-88e3-46f4-a54e-821d54aeb82f None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:44.935570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9ec0c-88e3-46f4-a54e-821d54aeb82f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:44.938159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9ec0c-88e3-46f4-a54e-821d54aeb82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:44.952714 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:44.973591 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-63f9ec0c-88e3-46f4-a54e-821d54aeb82f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:44.974860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-63f9ec0c-88e3-46f4-a54e-821d54aeb82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:44.987002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:45.014268 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e98cf3-cdeb-4f3f-ab16-72b752e24466 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:45.064996 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:45.066351 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:45.066788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:45.067216 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:45.068093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:45.068946 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:45.069708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 235/475] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:45.070782 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:45.090580 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 241/476] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:45.092623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9ec0c-88e3-46f4-a54e-821d54aeb82f None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:45.095222 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:45.214153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9ec0c-88e3-46f4-a54e-821d54aeb82f None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:45.219210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f8249-bd68-4047-bc93-c900aeba383c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:45.260779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:45.261538 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:45.261892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:45.262301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:45.263117 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:45.264090 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c043f1-0ff0-434e-8937-aee0aa402c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:45.264788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:45.265093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c043f1-0ff0-434e-8937-aee0aa402c82 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:45.265540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c043f1-0ff0-434e-8937-aee0aa402c82 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:45.265964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c043f1-0ff0-434e-8937-aee0aa402c82 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:45.266904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c043f1-0ff0-434e-8937-aee0aa402c82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:45.268509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c043f1-0ff0-434e-8937-aee0aa402c82 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:45.279374 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:45.283552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.sql.core [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:26:45.283552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:26:45.283552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [parameters: {'id': 'ed3e7872cf0f4254852610d71f9ce524', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:26:45.283552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=207769) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:26:45.300537 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 236/477] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:45 2022] POST /identity/v3/users => generated 249 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application result.read() Jul 28 08:26:45.301163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:26:45.302203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application yield resource Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise self.value Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application session.commit() Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:26:45.303320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:26:45.304308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:26:45.305315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application result.read() Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:26:45.307241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [parameters: {'id': 'ed3e7872cf0f4254852610d71f9ce524', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:45.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:26:45.311748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:45.314596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 242/478] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:26:45 2022] POST /identity/v3/roles => generated 144 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:45.314743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c043f1-0ff0-434e-8937-aee0aa402c82 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:45.445445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c043f1-0ff0-434e-8937-aee0aa402c82 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:45.489822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141787e-e697-464f-b8f5-24ae3847423b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:45.490546 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141787e-e697-464f-b8f5-24ae3847423b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:45.490825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141787e-e697-464f-b8f5-24ae3847423b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:45.491099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141787e-e697-464f-b8f5-24ae3847423b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:45.504671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141787e-e697-464f-b8f5-24ae3847423b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-d141787e-e697-464f-b8f5-24ae3847423b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:45.510114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:45.512183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:45.513294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 237/479] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:26:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:45.522513 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:45.532387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:45.638550 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50be3c-e26a-4bfa-bc3e-eb6c9719dfe2 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:45.676364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35349de5-7fd3-48cd-b5ba-5d62736922d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:45.677075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35349de5-7fd3-48cd-b5ba-5d62736922d0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:45.677358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35349de5-7fd3-48cd-b5ba-5d62736922d0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:45.677628 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35349de5-7fd3-48cd-b5ba-5d62736922d0 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:45.678421 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35349de5-7fd3-48cd-b5ba-5d62736922d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:45.678733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35349de5-7fd3-48cd-b5ba-5d62736922d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:45.680197 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35349de5-7fd3-48cd-b5ba-5d62736922d0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:45.691709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 243/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:46.951149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6d66-099b-4aef-9bd4-3559fbb4ae25 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:46.951732 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6d66-099b-4aef-9bd4-3559fbb4ae25 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:46.951732 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6d66-099b-4aef-9bd4-3559fbb4ae25 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:46.953375 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 238/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:46.959176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8c8c-2e32-4d29-8c98-1b2b2ac2ed4b None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:46.959662 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8c8c-2e32-4d29-8c98-1b2b2ac2ed4b None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:46.959770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8c8c-2e32-4d29-8c98-1b2b2ac2ed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:46.978927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-921f8c8c-2e32-4d29-8c98-1b2b2ac2ed4b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:46.980173 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-921f8c8c-2e32-4d29-8c98-1b2b2ac2ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:47.070722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:47.077102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 244/482] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:47.099900 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d6d66-099b-4aef-9bd4-3559fbb4ae25 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:47.154948 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:47.210865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d6d66-099b-4aef-9bd4-3559fbb4ae25 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:47.246554 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0c6af-3d9c-4d19-bed4-73bc893fdd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:47.247213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0c6af-3d9c-4d19-bed4-73bc893fdd35 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.247492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0c6af-3d9c-4d19-bed4-73bc893fdd35 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.247768 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0c6af-3d9c-4d19-bed4-73bc893fdd35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.258666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0c6af-3d9c-4d19-bed4-73bc893fdd35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-1bb0c6af-3d9c-4d19-bed4-73bc893fdd35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:47.264658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:47.265588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 239/483] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:47.271153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f8c8c-2e32-4d29-8c98-1b2b2ac2ed4b None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:47.324251 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5ec9-46f2-4508-96f5-80240eb0f492 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.324618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5ec9-46f2-4508-96f5-80240eb0f492 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.324869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5ec9-46f2-4508-96f5-80240eb0f492 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.326544 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 240/484] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:47.335320 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c54f2-ac02-4713-aaef-190c71bf30f0 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.335611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c54f2-ac02-4713-aaef-190c71bf30f0 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.335888 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c54f2-ac02-4713-aaef-190c71bf30f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.368994 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-6a9c54f2-ac02-4713-aaef-190c71bf30f0 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:47.371254 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-6a9c54f2-ac02-4713-aaef-190c71bf30f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:47.446083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f8c8c-2e32-4d29-8c98-1b2b2ac2ed4b None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:47.490030 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:47.490629 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.490921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.491200 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.491940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:47.492252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:47.493658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:47.503294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 245/485] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:47.513482 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:47.521401 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 241/486] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:47.546780 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c54f2-ac02-4713-aaef-190c71bf30f0 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:47.629038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:47.650309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:47.696307 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca70e1a-9882-41c7-89dd-c19154f3e842 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:47.716631 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c54f2-ac02-4713-aaef-190c71bf30f0 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:47.736498 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34a604-75fe-4cbf-b075-50e43d24b6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:47.737350 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a604-75fe-4cbf-b075-50e43d24b6f8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.737641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a604-75fe-4cbf-b075-50e43d24b6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.738271 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a604-75fe-4cbf-b075-50e43d24b6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.753008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34a604-75fe-4cbf-b075-50e43d24b6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:47.754653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34a604-75fe-4cbf-b075-50e43d24b6f8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:47.765445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da24d67-3a96-4ab4-aed1-6042dae719cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:47.766474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da24d67-3a96-4ab4-aed1-6042dae719cf None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.767105 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da24d67-3a96-4ab4-aed1-6042dae719cf None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.767594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da24d67-3a96-4ab4-aed1-6042dae719cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-5a34a604-75fe-4cbf-b075-50e43d24b6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:47.770804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:47.771787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:47.772759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:47.772759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:47.772759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:47.772759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:47.772759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 246/487] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:47.773656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00397b03-a971-4a79-acca-fe957388b3f6 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.773952 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00397b03-a971-4a79-acca-fe957388b3f6 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.774265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00397b03-a971-4a79-acca-fe957388b3f6 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.775816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 247/488] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:47.781871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4857-49a3-4ed7-ab95-d3791972e772 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.782284 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4857-49a3-4ed7-ab95-d3791972e772 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.782480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4857-49a3-4ed7-ab95-d3791972e772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.787097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da24d67-3a96-4ab4-aed1-6042dae719cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-6da24d67-3a96-4ab4-aed1-6042dae719cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:47.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:47.795142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:47.796082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:47.797828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 242/489] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:47.798468 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da24d67-3a96-4ab4-aed1-6042dae719cf None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:47.804353 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-098c4857-49a3-4ed7-ab95-d3791972e772 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:47.806471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-098c4857-49a3-4ed7-ab95-d3791972e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:47.882583 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 248/490] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:47.883499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c4857-49a3-4ed7-ab95-d3791972e772 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:47.930430 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:47.931745 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da24d67-3a96-4ab4-aed1-6042dae719cf None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:47.934434 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:47.971292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:47.971994 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:47.972277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:47.972568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:47.973274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:47.973581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:47.975023 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:47.992953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 243/491] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:47.993935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.002633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c4857-49a3-4ed7-ab95-d3791972e772 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.040405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c87050-df71-446b-be63-9002790e63b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.041138 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87050-df71-446b-be63-9002790e63b7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.041255 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87050-df71-446b-be63-9002790e63b7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.041512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87050-df71-446b-be63-9002790e63b7 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.042294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c87050-df71-446b-be63-9002790e63b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:48.042609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c87050-df71-446b-be63-9002790e63b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:48.044070 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c87050-df71-446b-be63-9002790e63b7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:48.052917 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 249/492] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:48.055537 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92008d-3bf0-41c0-a659-e02b7c8039f7 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.056666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92008d-3bf0-41c0-a659-e02b7c8039f7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.057153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92008d-3bf0-41c0-a659-e02b7c8039f7 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.059136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 250/493] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:48.060430 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92008d-3bf0-41c0-a659-e02b7c8039f7 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.105392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede70283-3115-4eff-a057-7f8dc28f206d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.139963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680771ce-b75e-4344-b690-986c3b0ba69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.140619 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680771ce-b75e-4344-b690-986c3b0ba69b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.140895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680771ce-b75e-4344-b690-986c3b0ba69b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.141170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680771ce-b75e-4344-b690-986c3b0ba69b None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.152040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680771ce-b75e-4344-b690-986c3b0ba69b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-680771ce-b75e-4344-b690-986c3b0ba69b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:48.157036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:48.158093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 244/494] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:48.163935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680771ce-b75e-4344-b690-986c3b0ba69b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.182373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:48.185321 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92008d-3bf0-41c0-a659-e02b7c8039f7 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.188320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:48.225604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1eea4-01e3-4b3e-9c26-7a94fc30469f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.226471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1eea4-01e3-4b3e-9c26-7a94fc30469f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.227035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1eea4-01e3-4b3e-9c26-7a94fc30469f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.227331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1eea4-01e3-4b3e-9c26-7a94fc30469f None admin] PATH_INFO: `/v3/users/nova` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.236476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1eea4-01e3-4b3e-9c26-7a94fc30469f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:48.238762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1eea4-01e3-4b3e-9c26-7a94fc30469f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-17a1eea4-01e3-4b3e-9c26-7a94fc30469f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:48.252701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:48.253864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:48.254956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 251/495] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:48.255631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2702e-3c9e-4bdd-9d91-ffa4d8ce2221 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.255631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2702e-3c9e-4bdd-9d91-ffa4d8ce2221 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.256465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2702e-3c9e-4bdd-9d91-ffa4d8ce2221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.275214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680771ce-b75e-4344-b690-986c3b0ba69b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.275897 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-6dc2702e-3c9e-4bdd-9d91-ffa4d8ce2221 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:48.277121 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-6dc2702e-3c9e-4bdd-9d91-ffa4d8ce2221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:48.312377 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.312944 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.313228 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.313504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.326416 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:48.327963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:48.343992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:48.345174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:48.346173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:48.346173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:48.346173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:48.346173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:48.346173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 245/496] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:48.346173 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.357595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 252/497] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:48.358514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2702e-3c9e-4bdd-9d91-ffa4d8ce2221 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.434346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:48.470366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:48.474913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe09be-6122-468c-b178-3852c147eb39 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.478657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2702e-3c9e-4bdd-9d91-ffa4d8ce2221 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.510404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.511027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.511304 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.511581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.512301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:48.512603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:48.514004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:48.518299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.518299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.518299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.518299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.518695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:48.518695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:48.522080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:48.525786 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 246/498] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:48.531118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.564956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 253/499] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:48.565742 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.636115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253f7b3-d686-4513-88aa-eb60569ff3db None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.669509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.670159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.670441 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.670713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.671403 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:48.671706 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:48.672094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e562d35-f762-4ee5-ae6d-ef93d2d972cf None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.673110 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:48.674632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:48.686019 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 247/500] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:48.686877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.714777 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ce58f-d310-49d2-a634-4f4a0cedf45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.715434 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce58f-d310-49d2-a634-4f4a0cedf45d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.715738 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce58f-d310-49d2-a634-4f4a0cedf45d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.716019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce58f-d310-49d2-a634-4f4a0cedf45d None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.726625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ce58f-d310-49d2-a634-4f4a0cedf45d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-1c4ce58f-d310-49d2-a634-4f4a0cedf45d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:48.731656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:48.732833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:48.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 254/501] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:48.734599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda3ab2-0a4e-4094-9d82-fc132e3c773a None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.734599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda3ab2-0a4e-4094-9d82-fc132e3c773a None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.734730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda3ab2-0a4e-4094-9d82-fc132e3c773a None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.736725 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 255/502] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:48.737739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda3ab2-0a4e-4094-9d82-fc132e3c773a None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.754892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:48.790056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762eeeed-f5d6-4730-b008-ad8fba3055b9 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.825963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.826544 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.826823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.827094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.837264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:48.838850 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:48.845310 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda3ab2-0a4e-4094-9d82-fc132e3c773a None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:48.855286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:48.856289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:48.857316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:48.857316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:48.857316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:48.857316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:48.857316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 248/503] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:48.858042 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:48.884558 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdc36d-cfdf-4183-9078-080d834992fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:48.885227 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdc36d-cfdf-4183-9078-080d834992fd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:48.885520 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdc36d-cfdf-4183-9078-080d834992fd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:48.885801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdc36d-cfdf-4183-9078-080d834992fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:48.910327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:48.962761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdc36d-cfdf-4183-9078-080d834992fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:48.964364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdc36d-cfdf-4183-9078-080d834992fd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:48.969089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577334d9-891f-4d6d-ac7e-bdd4c67b0aff None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:48.978670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:49.005071 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.005740 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.005983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.006288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.013519 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.015015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:49.028185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:49.029235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 249/504] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:49.030313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.074915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 256/505] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:49.076307 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d05f-a73d-4330-9ff0-0caa11541afa None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.076540 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d05f-a73d-4330-9ff0-0caa11541afa None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.076819 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d05f-a73d-4330-9ff0-0caa11541afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.095339 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-1c60d05f-a73d-4330-9ff0-0caa11541afa None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:49.096589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-1c60d05f-a73d-4330-9ff0-0caa11541afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:49.141075 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac35d8-11c3-4e2f-9e6d-10ca0b6445b0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.148341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:49.166706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 257/506] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:49.167579 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d05f-a73d-4330-9ff0-0caa11541afa None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.187942 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.188580 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.188865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.189150 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.190300 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:49.190613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.192092 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.200981 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 250/507] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:49.201888 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.213173 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:49.274529 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d05f-a73d-4330-9ff0-0caa11541afa None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.307624 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2653522-3c02-497b-aa8b-259a82f2dddc None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.309465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.310128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.310432 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.310709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.311421 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:49.311728 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.313151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.325512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 258/508] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:49.326510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.344790 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.345435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.345721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.346010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.346867 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:49.347297 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.348801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.385722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 251/509] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:49.386619 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.413803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:49.435059 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7330ca-9edd-4138-9a61-4471c0ccb496 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.436652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:49.471289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.471948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.472282 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.472576 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.484835 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.486639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.488709 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03c3b1-3876-4f22-b5b2-018c85ae14e8 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:49.502079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:49.503164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:49.504233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:49.504233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:49.504233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:49.504233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:49.504233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 259/510] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:49.504679 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.527040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.528102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.528557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.529002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.543240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.544750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:49.559700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:49.560823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:49.562056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:49.562056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:49.562056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:49.562056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:49.562056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 252/511] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:49.562056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.620738 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba4a6-bb04-47a8-b4e1-12e58dfeab0b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.652585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:49.662319 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.663004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.663282 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.663554 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.676098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.676423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30658f1e-0129-40f8-8ad9-367aa7641155 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.677592 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:49.693044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:49.694605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:49.696045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:49.696045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:49.696045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:49.696045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:49.696045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 260/512] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:49.696045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.704606 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:49.716154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.716807 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.717115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.717384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.725141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.726640 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:49.738365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:49.739529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 253/513] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:49.741107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.803455 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750338f-c89c-47ef-9649-c01d44a02397 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.842055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739672d4-e88e-4df8-99f5-ff84d111b107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.842726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739672d4-e88e-4df8-99f5-ff84d111b107 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.843010 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739672d4-e88e-4df8-99f5-ff84d111b107 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.843290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739672d4-e88e-4df8-99f5-ff84d111b107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.844309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cfd1e-defa-4854-9b38-dd5fcdce2522 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:49.881959 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:49.882650 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:49.882928 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:49.883199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:49.883922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:49.884225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.885654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.893341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:49.904024 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 254/514] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:49.904024 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:49.926883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739672d4-e88e-4df8-99f5-ff84d111b107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:49.928377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739672d4-e88e-4df8-99f5-ff84d111b107 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:49.937485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.008021 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827260d8-aabc-4e54-ba54-6440f42d69d4 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.039458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 261/515] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:50.040348 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739672d4-e88e-4df8-99f5-ff84d111b107 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.047998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.048708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.048973 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.049246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.049990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:50.050332 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.051780 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.064501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 255/516] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:50.065313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.129311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.150743 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739672d4-e88e-4df8-99f5-ff84d111b107 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.158591 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.173593 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfed77-6f4f-4656-b3b5-2e0c19269db6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.185982 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.186696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.186986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.187264 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.187966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:50.188289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.189663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.201969 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 262/517] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:50.202832 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.208934 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.209573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.209856 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.210156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.210965 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:50.211271 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.212753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.230600 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 256/518] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:50.232566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.307624 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eabedb-924b-4e44-b9e8-d647fbf5ba65 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.333074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46970fc-9b19-4698-838a-06c098d4838c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.343198 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.343878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.344164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.344443 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.350340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.351467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.352953 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:50.365778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:50.366885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:50.367860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 263/519] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:50.368440 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.373371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.374044 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.374626 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.385305 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.386874 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.397208 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:50.402036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:50.403016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:50.404000 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:50.404000 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:50.404000 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:50.404000 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:50.404000 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 257/520] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:50.406973 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.476425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671af1c-6300-4231-a2f2-0e207a125321 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.511403 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e15d60-ee86-4b4a-a82e-53cba74ce232 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.511633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc47e45-19b7-4646-bb18-6de84bb0b4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.512656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc47e45-19b7-4646-bb18-6de84bb0b4e3 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.512942 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc47e45-19b7-4646-bb18-6de84bb0b4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.513218 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc47e45-19b7-4646-bb18-6de84bb0b4e3 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.513900 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc47e45-19b7-4646-bb18-6de84bb0b4e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.515424 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc47e45-19b7-4646-bb18-6de84bb0b4e3 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.540872 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 264/521] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:26:50 2022] POST /identity/v3/users => generated 247 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:26:50.541728 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc47e45-19b7-4646-bb18-6de84bb0b4e3 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.555874 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.556785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.557143 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.557496 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.567565 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.569560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:50.585826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:50.587277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:26:50.588503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:50.592022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 258/522] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:50.592899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.626572 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.650310 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.673819 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc47e45-19b7-4646-bb18-6de84bb0b4e3 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.712250 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097013e2-6ca5-43bc-9383-ecc759baafb3 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.714837 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc7e7-3820-4df0-9da7-4eff2b9165d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.715496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc7e7-3820-4df0-9da7-4eff2b9165d0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.715779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc7e7-3820-4df0-9da7-4eff2b9165d0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.716057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc7e7-3820-4df0-9da7-4eff2b9165d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.756890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.757530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.757808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.758080 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.758887 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:50.759199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.760781 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.779925 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 259/523] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:50.780978 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.801890 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cc7e7-3820-4df0-9da7-4eff2b9165d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.803431 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cc7e7-3820-4df0-9da7-4eff2b9165d0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.893095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.894508 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:50.912989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7d0d-7fc6-4f6b-b0b7-c2ede06c8877 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:50.918078 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 265/524] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:50.919327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc7e7-3820-4df0-9da7-4eff2b9165d0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:50.955104 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:50.956044 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:50.956534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:50.957023 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:50.958294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:50.958825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:50.961516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:50.987292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 260/525] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:50.987292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.036533 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc7e7-3820-4df0-9da7-4eff2b9165d0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.073162 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.073791 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.074075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.074381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.075203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:51.075571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.077131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.095521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 266/526] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:51.096182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.102499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7d65-3e60-4583-8e5b-c6db2cb99293 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.141892 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.142959 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.143408 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.143702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.144572 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=c5ba9e7780f5437caac949931ca55b80) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:51.144886 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.146598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.152557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 261/527] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:50 2022] GET /identity/v3/role_assignments?user.id=c5ba9e7780f5437caac949931ca55b80&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:51.153560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.165948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:51.179466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:51.262689 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c959d4-8dab-4087-98ae-4321d83b4027 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.282388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987f12a-f2c3-45d1-84be-068f684d0611 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.321190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.321868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.322176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.322458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.334470 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.335159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.335442 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.335717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.337423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.338933 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.346642 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.348505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:51.359312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:51.360341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:51.361301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:51.361301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:51.361301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:51.361301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:51.361301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 267/528] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:51.362140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:51.365210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:51.366159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:51.367126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:51.367126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:51.367126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:51.367126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:51.367126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:51.367126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:51.367126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:51.367126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:51.369613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 262/529] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:51.370778 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.458818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:51.468348 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:51.479532 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b4099-04a1-4aeb-ab6e-4b8e7ea40c47 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.496838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f2d17-7543-413d-80d5-e60f92f84312 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.522125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a8253-afc1-45b5-91f1-6e797180402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.525591 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8253-afc1-45b5-91f1-6e797180402a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.526006 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8253-afc1-45b5-91f1-6e797180402a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.526398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8253-afc1-45b5-91f1-6e797180402a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.544635 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.545016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.545386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.546527 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:51.546934 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.548757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.565151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 263/530] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:51.566467 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.624988 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a8253-afc1-45b5-91f1-6e797180402a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.627496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a8253-afc1-45b5-91f1-6e797180402a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.698632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8983b7-6b09-4213-92a3-51ec20dd702d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.741253 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:51.748897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.749900 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.750331 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.750721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.751747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:51.752152 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.754211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.760718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:51.770149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 264/531] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:51.771629 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.776834 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 268/532] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:51.778195 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a8253-afc1-45b5-91f1-6e797180402a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.903154 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a8253-afc1-45b5-91f1-6e797180402a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.908113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7caee-3e2f-42fa-a7cf-67750096facd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:51.945447 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.946088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.946387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.946658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.947488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=40c475d21adc4ba4bd48bed9ebb0b445) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:51.947799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.949252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.955945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 269/533] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:51 2022] GET /identity/v3/role_assignments?user.id=40c475d21adc4ba4bd48bed9ebb0b445&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:51.956902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:51.964502 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:51.965594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:51.966929 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:51.967465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:51.978765 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:51.981079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:51.999521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:52.000567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:52.001567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:52.001567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:52.001567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:52.001567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:52.001567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:52.001567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:52.001567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:52.001567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:52.004950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 265/534] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:52.015214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:52.025727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:52.076108 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee01b3-4163-4efb-9775-bab9b34bd965 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:52.109076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:52.121643 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:52.122363 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:52.122644 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:52.122916 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:52.139722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:52.141330 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:52.157706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:52.158802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:52.164280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:52.164280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:52.164280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:52.164280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:52.164280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 270/535] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:52.168589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:52.225149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82943369-115b-48b2-b101-b3c80e876eff None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:52.287461 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:52.289342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:52.290181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:52.291542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:52.292585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:52.293321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:52.296447 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:52.325889 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 266/536] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:52.332461 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:52.397250 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9830a-9658-4cff-9b31-b233553dcab0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:52.420929 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:52.489193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3ee96-1975-404a-8c2f-51f41e21766f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:52.505924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:52.528255 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:52.529070 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:52.529458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:52.529839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:52.530693 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:52.531125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:52.532968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:52.559200 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 271/537] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:52.564803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:52.567455 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eb709-9ec1-4b70-ac00-b58ad17acf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:52.568110 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eb709-9ec1-4b70-ac00-b58ad17acf99 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:52.568418 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eb709-9ec1-4b70-ac00-b58ad17acf99 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:52.569061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eb709-9ec1-4b70-ac00-b58ad17acf99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:52.569876 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43eb709-9ec1-4b70-ac00-b58ad17acf99 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=f2076e2caf434975a590a7766cb575f6) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:52.570215 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43eb709-9ec1-4b70-ac00-b58ad17acf99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:52.571698 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43eb709-9ec1-4b70-ac00-b58ad17acf99 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:52.589417 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 267/538] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:52 2022] GET /identity/v3/role_assignments?user.id=f2076e2caf434975a590a7766cb575f6&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:52.651736 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a5f8-15c8-4d0a-848d-ff4121791e56 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:52.652306 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a5f8-15c8-4d0a-848d-ff4121791e56 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:52.652785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a5f8-15c8-4d0a-848d-ff4121791e56 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:52.654801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 268/539] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:52.664433 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057406ef-2baa-4c64-a24e-32d9edb58d33 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:52.669480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057406ef-2baa-4c64-a24e-32d9edb58d33 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:52.669943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057406ef-2baa-4c64-a24e-32d9edb58d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:52.704533 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-057406ef-2baa-4c64-a24e-32d9edb58d33 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:52.705808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-057406ef-2baa-4c64-a24e-32d9edb58d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:52.732393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e76a00-3adf-46c4-8de5-4f4b4fea767f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:52.812196 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:52.812794 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:52.813076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:52.813351 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:52.829032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:52.830585 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:52.854567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 269/540] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:52.860038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:52.861042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:52.862026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 272/541] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:52.867671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057406ef-2baa-4c64-a24e-32d9edb58d33 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:52.911935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:52.964003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:53.054289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057406ef-2baa-4c64-a24e-32d9edb58d33 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:53.082571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:53.100902 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:53.101557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:53.101843 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:53.102137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:53.102917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:53.103230 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:53.104701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:53.120861 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3509f1-793a-4635-b245-53336438b4b8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:53.144562 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 270/542] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:53.154542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:53.170276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3052230-8e7e-49b2-b9cc-d5a1e1cbcab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:53.171797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052230-8e7e-49b2-b9cc-d5a1e1cbcab2 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:53.172296 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052230-8e7e-49b2-b9cc-d5a1e1cbcab2 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:53.172789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052230-8e7e-49b2-b9cc-d5a1e1cbcab2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:53.195181 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3052230-8e7e-49b2-b9cc-d5a1e1cbcab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-f3052230-8e7e-49b2-b9cc-d5a1e1cbcab2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:53.201299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:53.202393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:53.203301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:53.212683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 273/543] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:53.218783 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3052230-8e7e-49b2-b9cc-d5a1e1cbcab2 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:53.278903 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5215fa8-c63c-4fa9-ae37-e703c52957cc None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:53.318302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2ae84-2261-4068-b26e-0f4eb08ee606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:53.319620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2ae84-2261-4068-b26e-0f4eb08ee606 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:53.319911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2ae84-2261-4068-b26e-0f4eb08ee606 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:53.320188 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2ae84-2261-4068-b26e-0f4eb08ee606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:53.321150 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2ae84-2261-4068-b26e-0f4eb08ee606 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=78394014f5904021823a96d12902b2ec) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:53.321476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2ae84-2261-4068-b26e-0f4eb08ee606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:53.323126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2ae84-2261-4068-b26e-0f4eb08ee606 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:53.329594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 271/544] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:53 2022] GET /identity/v3/role_assignments?user.id=78394014f5904021823a96d12902b2ec&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:53.390515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:53.406581 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3052230-8e7e-49b2-b9cc-d5a1e1cbcab2 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:53.419763 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a2862-508e-4d0e-99bc-285d6042cf97 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:53.420235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a2862-508e-4d0e-99bc-285d6042cf97 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:53.420616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a2862-508e-4d0e-99bc-285d6042cf97 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:53.422494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 272/545] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:53.431639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c114e0c-6286-4134-b404-cd601cbe0a7d None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:53.432072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c114e0c-6286-4134-b404-cd601cbe0a7d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:53.432455 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c114e0c-6286-4134-b404-cd601cbe0a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:53.465364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:53.466094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:53.466405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:53.466726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:53.467773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:53.468084 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:53.469534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:53.475643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-7c114e0c-6286-4134-b404-cd601cbe0a7d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:53.478053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-7c114e0c-6286-4134-b404-cd601cbe0a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:53.492765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 274/546] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:53.498754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:53.538679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:53.630875 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 273/547] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:53.663683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c114e0c-6286-4134-b404-cd601cbe0a7d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:53.679461 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c4290-3725-4cfb-8016-ed0a762fcefd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:53.753951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:53.755770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:53.756057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:53.756335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:53.782686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:53.784186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:53.814251 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c114e0c-6286-4134-b404-cd601cbe0a7d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:53.816172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:53.817119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:53.818065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:53.818065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:53.818065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:53.818065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:53.820711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 275/548] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:53.826295 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:53.838134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:53.878139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597aea6c-08ba-4b43-80bd-4883def6643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:53.878139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597aea6c-08ba-4b43-80bd-4883def6643a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:53.878139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597aea6c-08ba-4b43-80bd-4883def6643a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:53.878139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597aea6c-08ba-4b43-80bd-4883def6643a None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:53.886679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597aea6c-08ba-4b43-80bd-4883def6643a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-597aea6c-08ba-4b43-80bd-4883def6643a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:53.898139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:53.899240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 274/549] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:53.900167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597aea6c-08ba-4b43-80bd-4883def6643a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:53.962268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:53.987642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983b5e-c611-454f-9824-919eeee4103a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:54.029849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:54.030549 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.030835 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.031113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.031834 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:54.032143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:54.033590 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:54.048936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 276/550] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:54.054826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:54.161733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:54.179414 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d446b-653b-46f2-853c-e7f5f43b812e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:54.221198 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:54.222443 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.222732 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.223044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] PATH_INFO: `/v3/users/swift` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.231278 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:54.232955 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:54.246707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:54.247751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:26:54.248775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:54.249350 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 277/551] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:26:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:54.255103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:54.276170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597aea6c-08ba-4b43-80bd-4883def6643a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:54.329943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:54.330621 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.330911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.331192 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.332006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:54.332324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:54.333871 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:54.343163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 275/552] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:54.349139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:54.458586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:54.463627 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1728434-3a5e-456e-bbad-26b7bf73a50c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:54.475843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daabbd0-87ad-4d54-b6b1-714b891f8880 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:54.516735 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:54.524912 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e63d9-faa8-4950-8e3a-e1248d9a38ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:54.525695 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e63d9-faa8-4950-8e3a-e1248d9a38ad None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.525983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e63d9-faa8-4950-8e3a-e1248d9a38ad None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.526292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e63d9-faa8-4950-8e3a-e1248d9a38ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.532383 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132fa30-3d55-4f48-ad58-a69a83e207fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:54.533334 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132fa30-3d55-4f48-ad58-a69a83e207fe None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.533832 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132fa30-3d55-4f48-ad58-a69a83e207fe None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.534404 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132fa30-3d55-4f48-ad58-a69a83e207fe None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.535507 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132fa30-3d55-4f48-ad58-a69a83e207fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:54.536129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132fa30-3d55-4f48-ad58-a69a83e207fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:54.537816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132fa30-3d55-4f48-ad58-a69a83e207fe None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:54.538163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e63d9-faa8-4950-8e3a-e1248d9a38ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:54.539652 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e63d9-faa8-4950-8e3a-e1248d9a38ad None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-7c2e63d9-faa8-4950-8e3a-e1248d9a38ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:54.555976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:54.557011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:54.558009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:54.558009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:54.558009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:54.558009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:54.558009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 276/553] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:54.559179 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910f55-e2b2-406c-a6df-c874a29fb01f None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.559179 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910f55-e2b2-406c-a6df-c874a29fb01f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.559459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910f55-e2b2-406c-a6df-c874a29fb01f None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.561217 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 277/554] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:54.562410 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 278/555] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:26:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:54.566660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1686-ad81-4851-9eaa-846badbbf56f None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.567129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1686-ad81-4851-9eaa-846badbbf56f None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.567558 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1686-ad81-4851-9eaa-846badbbf56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.567677 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93910f55-e2b2-406c-a6df-c874a29fb01f None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:54.594329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-a4fc1686-ad81-4851-9eaa-846badbbf56f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:54.595921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-a4fc1686-ad81-4851-9eaa-846badbbf56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:54.673405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 279/556] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:54.674807 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc1686-ad81-4851-9eaa-846badbbf56f None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:54.685587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93910f55-e2b2-406c-a6df-c874a29fb01f None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:54.727235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:54.727986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.728430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.728830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.729742 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:54.730233 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:54.731901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:54.733763 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:54.746620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 278/557] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:54.747833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:54.792708 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc1686-ad81-4851-9eaa-846badbbf56f None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:54.806271 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:54.835039 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992cb02-dbd4-4e1c-b9a4-2e3e4082ef52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:54.836031 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992cb02-dbd4-4e1c-b9a4-2e3e4082ef52 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.836465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992cb02-dbd4-4e1c-b9a4-2e3e4082ef52 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.836891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992cb02-dbd4-4e1c-b9a4-2e3e4082ef52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.865461 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c859f5-ae10-49e7-afd7-c9dd1f13125c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:54.904407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1494bd-7722-48cf-883f-81a99fbc9f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:54.905358 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1494bd-7722-48cf-883f-81a99fbc9f5d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:54.905650 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1494bd-7722-48cf-883f-81a99fbc9f5d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:54.905932 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1494bd-7722-48cf-883f-81a99fbc9f5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:54.918282 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1494bd-7722-48cf-883f-81a99fbc9f5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:54.918641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992cb02-dbd4-4e1c-b9a4-2e3e4082ef52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:54.920126 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992cb02-dbd4-4e1c-b9a4-2e3e4082ef52 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-bd1494bd-7722-48cf-883f-81a99fbc9f5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:54.923901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:54.925165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 279/558] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:54.926146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1494bd-7722-48cf-883f-81a99fbc9f5d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:54.985667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:55.038839 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1494bd-7722-48cf-883f-81a99fbc9f5d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.040820 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 280/559] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:26:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:26:55.042017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866215f-db7c-4be3-894b-639688041685 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.042367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866215f-db7c-4be3-894b-639688041685 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.042647 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866215f-db7c-4be3-894b-639688041685 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.044054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 281/560] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:55.046933 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866215f-db7c-4be3-894b-639688041685 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.063405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:55.082210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.082870 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.083155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.083464 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] PATH_INFO: `/v3/users/nova` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.091686 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:55.093159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:55.104951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:55.106065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 280/561] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:26:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:55.107779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.174606 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866215f-db7c-4be3-894b-639688041685 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.218374 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27855704-fcfa-4d10-8552-06cd214efbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.219037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855704-fcfa-4d10-8552-06cd214efbe5 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.219322 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855704-fcfa-4d10-8552-06cd214efbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.219570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855704-fcfa-4d10-8552-06cd214efbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.220308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27855704-fcfa-4d10-8552-06cd214efbe5 None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:55.220626 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27855704-fcfa-4d10-8552-06cd214efbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:55.222076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27855704-fcfa-4d10-8552-06cd214efbe5 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:55.226008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59242a2d-d0c7-4912-9624-3879a2e1b8de None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.232651 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 282/562] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:55.234026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b33f8b-f329-485a-9e73-5308f40fdd51 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.234352 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b33f8b-f329-485a-9e73-5308f40fdd51 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.234651 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b33f8b-f329-485a-9e73-5308f40fdd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.246290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:55.254488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-28b33f8b-f329-485a-9e73-5308f40fdd51 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:55.255717 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-28b33f8b-f329-485a-9e73-5308f40fdd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:55.268638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.269297 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.269584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.269861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.281356 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:55.282941 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:55.297935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:55.299127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:55.300138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:55.300138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:55.300138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:55.300138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:55.304067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 281/563] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:55.304937 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.330164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 283/564] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:55.331044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b33f8b-f329-485a-9e73-5308f40fdd51 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.333365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:55.429446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4277dd8-d53b-48b2-bfb2-a556621d445e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.441789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b33f8b-f329-485a-9e73-5308f40fdd51 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.469272 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.469916 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.470231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.470507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.471325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:55.471643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:55.473210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:55.479235 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.479902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.480191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.480470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.491532 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:55.497626 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 282/565] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:55.497840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:55.497963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:55.516589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:55.517663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:55.518740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:55.518740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:55.518740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:55.518740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:55.522769 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 284/566] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:55.523778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.531914 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:55.585056 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:55.619391 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c0044-966e-4a16-a804-b217ef9c9ad9 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.632640 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9121e6c-c178-4f4e-8198-f893f57f110b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.655835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.656482 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.656767 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.657047 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.657760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:55.658072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:55.659586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:55.667050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420baab-ded5-4ec5-b61a-184dff7319bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.667723 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420baab-ded5-4ec5-b61a-184dff7319bb None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.668001 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420baab-ded5-4ec5-b61a-184dff7319bb None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.668281 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420baab-ded5-4ec5-b61a-184dff7319bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.671490 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 283/567] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:55.672325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.679649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420baab-ded5-4ec5-b61a-184dff7319bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-7420baab-ded5-4ec5-b61a-184dff7319bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:55.685174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:55.686545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 285/568] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:55.687682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420baab-ded5-4ec5-b61a-184dff7319bb None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.769749 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:55.786395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651007ce-5565-4292-a1f2-310da1c38bf7 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.789975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420baab-ded5-4ec5-b61a-184dff7319bb None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.804987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:55.821789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.822480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.822772 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.823043 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.827102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f9b09-b7dd-41d8-8ffc-a9a89592f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:55.827738 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9b09-b7dd-41d8-8ffc-a9a89592f591 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.828018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9b09-b7dd-41d8-8ffc-a9a89592f591 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.828292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9b09-b7dd-41d8-8ffc-a9a89592f591 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.828991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f9b09-b7dd-41d8-8ffc-a9a89592f591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:55.829299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f9b09-b7dd-41d8-8ffc-a9a89592f591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:55.830726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f9b09-b7dd-41d8-8ffc-a9a89592f591 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:55.834138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:55.835613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:55.843223 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 286/569] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:55.844468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5a02d-1fdd-4c9b-86f3-6803e45edc99 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:55.844755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5a02d-1fdd-4c9b-86f3-6803e45edc99 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:55.845030 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5a02d-1fdd-4c9b-86f3-6803e45edc99 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:55.846254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 287/570] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:55.846938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5a02d-1fdd-4c9b-86f3-6803e45edc99 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:55.854963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:55.856067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:55.857068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:55.857068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:55.857068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:55.857068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:55.857068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 284/571] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:55.857375 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:55.963421 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5a02d-1fdd-4c9b-86f3-6803e45edc99 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:55.972096 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf42af-04a8-4f6e-a504-603935e2625a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.002000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc076f37-fdd3-447a-8a08-9eb7e60b7892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.002701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076f37-fdd3-447a-8a08-9eb7e60b7892 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.002982 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076f37-fdd3-447a-8a08-9eb7e60b7892 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.003323 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076f37-fdd3-447a-8a08-9eb7e60b7892 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.008522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.009166 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.009446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.009720 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.010491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:56.010798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.011534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076f37-fdd3-447a-8a08-9eb7e60b7892 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.012218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.012996 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076f37-fdd3-447a-8a08-9eb7e60b7892 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.014855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.022202 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 285/572] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:56.022974 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-cc076f37-fdd3-447a-8a08-9eb7e60b7892 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:56.025771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:56.027570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 288/573] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08e20-0de0-4010-b554-c13adfce3e4c None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.028843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08e20-0de0-4010-b554-c13adfce3e4c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.029637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08e20-0de0-4010-b554-c13adfce3e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.047713 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-a2a08e20-0de0-4010-b554-c13adfce3e4c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:56.048931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-a2a08e20-0de0-4010-b554-c13adfce3e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:56.087221 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.130526 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 289/574] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:56.131727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a08e20-0de0-4010-b554-c13adfce3e4c None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.134215 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387378d-44d6-400e-88a4-59c08e8bc9c1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.171819 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.172700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.172975 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.173263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] PATH_INFO: `/v3/users/glance` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.181027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.182540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:56.194386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:56.195592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 286/575] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:56.196658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.235794 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a08e20-0de0-4010-b554-c13adfce3e4c None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.257383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.270874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.271504 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.271790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.272061 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.272813 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:56.273064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.274501 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.286276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 290/576] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:56.287107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.306459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b9d6e-f7d8-495c-acc8-4eadf8722a49 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.313617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.344439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.345103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.345367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.345668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.356726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.358243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:56.373081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:56.374200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:56.375186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:56.375186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:56.375186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:56.375186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:56.375186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 287/577] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:56.375186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.395352 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608964e2-1b9f-4512-92ef-fb6c81245a28 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.429855 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.430551 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.430846 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.431108 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.431862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:56.432188 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.433617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.451894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 291/578] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:56.452747 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.484352 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814c4eb-a6cd-4d15-814e-a05cf29e12cf None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.495989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.521544 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180e1f3-4ef6-451e-beb5-ec1319e7b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.522244 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180e1f3-4ef6-451e-beb5-ec1319e7b8aa None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.522553 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180e1f3-4ef6-451e-beb5-ec1319e7b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.522888 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180e1f3-4ef6-451e-beb5-ec1319e7b8aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.533403 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180e1f3-4ef6-451e-beb5-ec1319e7b8aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-b180e1f3-4ef6-451e-beb5-ec1319e7b8aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:56.539002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:56.540053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 288/579] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:56.546410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180e1f3-4ef6-451e-beb5-ec1319e7b8aa None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.547268 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.559634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189c0f-f2fb-43f0-9327-a3bc5c09b744 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.597262 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.597897 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.598204 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.598481 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.599398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:56.599696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.601953 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.624778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 292/580] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:56.626415 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.650072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180e1f3-4ef6-451e-beb5-ec1319e7b8aa None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.685938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.686619 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.686899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.687170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.694458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.695921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:56.707520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:56.708505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:56.709508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:56.709508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:56.709508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:56.709508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:56.709508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:56.709508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:56.709508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:56.709508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:56.710414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 289/581] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:56.711439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.734330 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0288d7-7ea3-4e07-a753-c837da63f07e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.746794 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.770928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.771566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.771842 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.772111 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.772810 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:56.773170 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.774542 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.786263 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 293/582] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:56.787119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.789833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.824176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799be6f6-fa7f-4910-a6df-42844955718f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.858433 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.859064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.859346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.859622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.860414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=ec86b4f3ad1d4f75a685c4ff209051ce) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:56.860729 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.862236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.867955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 290/583] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/role_assignments?user.id=ec86b4f3ad1d4f75a685c4ff209051ce&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:56.868791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.907050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9f0db-5862-437e-a7eb-fd1dcdd87b77 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:56.945625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:56.946451 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:56.946737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:56.947014 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:56.947765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:56.948079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:56.949528 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:56.959025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 294/584] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:56.959938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:56.988325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:56.997912 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825271dd-4530-4c76-a1b0-20d10428b224 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.047281 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:57.048479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.049144 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.050432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.050432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.062928 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.064377 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:57.079596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:57.080713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:57.082074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:57.082074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:57.082074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:57.082074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:57.082074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 291/585] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:57.082074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.098660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c84e32-ca40-4f87-b09c-d4061ea5b15f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.134559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.135314 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.135598 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.135879 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.136638 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:57.136946 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.138553 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.156601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 295/586] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:57.156601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.188458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976e14f-846a-4d68-a6b6-1258e2ca08b7 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.224101 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.224765 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.225047 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.225322 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.233263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.234772 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:57.246503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:57.247722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 292/587] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:57.248866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.261040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:57.267561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619bb22-a4fa-4f8b-afb4-debdfa5245e4 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.295033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:57.306829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.307409 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.307767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.307976 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.319938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.321412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:57.339544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:57.340674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:57.341676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:57.341676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:57.341676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:57.341676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:57.341676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 296/588] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:57.341676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.371419 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3c86-344d-4021-8d88-d9f66ca14cbd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.410329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.410967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.411244 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.411512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.412211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:57.412513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.413899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.430744 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 293/589] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:57.430744 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.467495 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57790d-4eca-4b38-8239-489efb5d79fa None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.496344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:57.505285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09159cdf-9986-4e88-83ba-2ed75f1e5d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.505982 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09159cdf-9986-4e88-83ba-2ed75f1e5d4f None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.506293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09159cdf-9986-4e88-83ba-2ed75f1e5d4f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.506570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09159cdf-9986-4e88-83ba-2ed75f1e5d4f None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/c5ba9e7780f5437caac949931ca55b80/roles/070199642db14984a418795ea5af54cb` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.526789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09159cdf-9986-4e88-83ba-2ed75f1e5d4f None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=c5ba9e7780f5437caac949931ca55b80, role_id=070199642db14984a418795ea5af54cb)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.528419 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09159cdf-9986-4e88-83ba-2ed75f1e5d4f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.536201 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6282d8-6127-40e2-ba93-0ae9338aea4d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.550134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 297/590] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:57 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/c5ba9e7780f5437caac949931ca55b80/roles/070199642db14984a418795ea5af54cb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:57.550134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09159cdf-9986-4e88-83ba-2ed75f1e5d4f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.586098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.587489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.587823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.588151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.589295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:57.589654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.591963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.594536 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:57.621512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 294/591] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:57.622833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.691910 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09159cdf-9986-4e88-83ba-2ed75f1e5d4f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.732121 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.732762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.733043 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.733317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.734022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:57.734361 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.735762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.749212 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 298/592] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:57.749895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f277c-ee9f-4c90-8be9-6d2972d4d097 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.750097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.785700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:57.805420 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.806253 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.806538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.807074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.816705 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.819572 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:57.835809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:57.836940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:57.837946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 295/593] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:57.839071 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.856222 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:57.874028 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f7806-5974-4bd1-ba71-717e3ba1f83b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:57.912575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:57.913395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:57.913772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:57.914188 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:57.926572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:57.928205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:57.947215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:57.949000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:57.950977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:57.950977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:57.950977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:57.950977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:57.950977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 299/594] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:57.951577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:57.975643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c843fdb-f3f7-4ec8-b3ce-2ae96d829c51 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.012533 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.013662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.014057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.014462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.024469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.026120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:26:58.039727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:58.041048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:26:58.042342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:58.043429 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 296/595] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:58.044476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:58.093645 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:58.106847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f89d0f-1473-4ede-8f4d-93f478bd54d8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.149040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.149817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.150199 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.150667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.151489 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:58.152075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.153544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:58.162821 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:58.163861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c9e41-94b4-4ef0-99f8-becca13dff7e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.169696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 300/596] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:58.170730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:58.211783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.212619 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.212998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.213363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.214192 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:58.214597 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.216402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:58.230155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 297/597] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:58.230652 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:58.290370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab5508-4d6a-4775-814a-4b9e4996358b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.333251 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.333904 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.334233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.334511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.335428 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:58.335849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.337420 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:58.350383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc89856-df62-49e2-bb35-a15257922010 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.353189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:58.361406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 301/598] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:26:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:58.363003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:58.392138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beba672-9fc5-44af-a35a-50cb97f4fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.392883 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beba672-9fc5-44af-a35a-50cb97f4fcea None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.393099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beba672-9fc5-44af-a35a-50cb97f4fcea None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.393356 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beba672-9fc5-44af-a35a-50cb97f4fcea None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/40c475d21adc4ba4bd48bed9ebb0b445/roles/070199642db14984a418795ea5af54cb` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.408971 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beba672-9fc5-44af-a35a-50cb97f4fcea None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=40c475d21adc4ba4bd48bed9ebb0b445, role_id=070199642db14984a418795ea5af54cb)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.410572 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beba672-9fc5-44af-a35a-50cb97f4fcea None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:58.430183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 298/599] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:58 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/40c475d21adc4ba4bd48bed9ebb0b445/roles/070199642db14984a418795ea5af54cb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:58.431541 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beba672-9fc5-44af-a35a-50cb97f4fcea None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:58.440671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:58.532247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265f8be-395b-4b0b-a269-fe5ef71e6354 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.623270 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.624375 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.624896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.625389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.642732 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beba672-9fc5-44af-a35a-50cb97f4fcea None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.643866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.645601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:58.664685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:58.665763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:58.666797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:58.666797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:58.666797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:58.666797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:58.666797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:58.666797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:58.666797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:58.666797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:58.669370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 302/600] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:58.675555 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:58.685064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.685785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.686069 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.686424 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.701999 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.703540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:26:58.724170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:58.725245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:58.726327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:26:58.726327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:26:58.726327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:26:58.726327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:58.726327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 299/601] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:58.733439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:58.753105 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:58.846051 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f48b4-09d4-4b88-9ba8-50d746c0f073 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.865364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:58.888776 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.889792 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.890326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.890876 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.891866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:58.892410 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.894442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:58.908760 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 303/602] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:58.914599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:58.917502 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863d7af-9a53-4379-8477-35424e7b76af None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:58.988431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bea15-1201-44be-85a5-b54627895a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:58.989089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bea15-1201-44be-85a5-b54627895a54 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:58.989370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bea15-1201-44be-85a5-b54627895a54 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:58.989643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bea15-1201-44be-85a5-b54627895a54 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:58.990405 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bea15-1201-44be-85a5-b54627895a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:58.990711 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bea15-1201-44be-85a5-b54627895a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:58.992179 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bea15-1201-44be-85a5-b54627895a54 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:59.015830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 300/603] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:59.017226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c3e0b-7bcc-42a2-8ed5-8b749361dba8 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.017512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c3e0b-7bcc-42a2-8ed5-8b749361dba8 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.017788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c3e0b-7bcc-42a2-8ed5-8b749361dba8 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.024452 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 301/604] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:26:59.025338 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c3e0b-7bcc-42a2-8ed5-8b749361dba8 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:59.090022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101a183-3016-425d-9a24-df9b2592c110 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:59.113206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:59.132793 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4293e-c816-4e1a-8423-0745938c9ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:59.133501 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4293e-c816-4e1a-8423-0745938c9ab7 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.133779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4293e-c816-4e1a-8423-0745938c9ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.134195 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4293e-c816-4e1a-8423-0745938c9ab7 None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/f2076e2caf434975a590a7766cb575f6/roles/070199642db14984a418795ea5af54cb` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.152502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4293e-c816-4e1a-8423-0745938c9ab7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=f2076e2caf434975a590a7766cb575f6, role_id=070199642db14984a418795ea5af54cb)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:59.153984 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4293e-c816-4e1a-8423-0745938c9ab7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:59.160346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c3e0b-7bcc-42a2-8ed5-8b749361dba8 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:59.173706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 304/605] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:58 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/f2076e2caf434975a590a7766cb575f6/roles/070199642db14984a418795ea5af54cb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:26:59.175620 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dc93c-c069-48ec-9bf2-c9a671165f8b None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.175941 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dc93c-c069-48ec-9bf2-c9a671165f8b None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.176213 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dc93c-c069-48ec-9bf2-c9a671165f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.198911 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-731dc93c-c069-48ec-9bf2-c9a671165f8b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:26:59.200598 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-731dc93c-c069-48ec-9bf2-c9a671165f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:26:59.200966 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:59.201667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.201943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.202257 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.210881 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:59.212442 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:26:59.226212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:59.228841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:59.231512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:26:59.231512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:26:59.231512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:26:59.231512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:26:59.231512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:26:59.231512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:26:59.231512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:26:59.231512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:26:59.232000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:59.232083 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 302/606] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:26:59.234845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:59.294230 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 305/607] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:26:59.318296 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dc93c-c069-48ec-9bf2-c9a671165f8b None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:59.402933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c581f6-d91c-417b-8486-598f522f3899 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:59.435669 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:59.446952 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:59.447755 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.448039 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.448311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.449229 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:59.450029 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:59.451858 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:59.477467 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 303/608] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:26:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:26:59.483392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:59.510824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dc93c-c069-48ec-9bf2-c9a671165f8b None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:59.577470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35334346-4391-46c4-a49f-e70e18108110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:59.578612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35334346-4391-46c4-a49f-e70e18108110 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.579921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35334346-4391-46c4-a49f-e70e18108110 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.580612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35334346-4391-46c4-a49f-e70e18108110 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.601389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35334346-4391-46c4-a49f-e70e18108110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-35334346-4391-46c4-a49f-e70e18108110 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:26:59.607358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:26:59.608514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:26:59.609449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:26:59.612404 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 306/609] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:26:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:26:59.623378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35334346-4391-46c4-a49f-e70e18108110 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:59.646910 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248122d-356a-4263-9605-cd5ddbdfdbd3 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:59.659449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:59.686292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1162-730c-4fad-bdbc-8d086931e77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:59.687448 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1162-730c-4fad-bdbc-8d086931e77b None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.688051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1162-730c-4fad-bdbc-8d086931e77b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.688530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1162-730c-4fad-bdbc-8d086931e77b None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/78394014f5904021823a96d12902b2ec/roles/070199642db14984a418795ea5af54cb` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.706825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee1162-730c-4fad-bdbc-8d086931e77b None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=78394014f5904021823a96d12902b2ec, role_id=070199642db14984a418795ea5af54cb)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:59.708739 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee1162-730c-4fad-bdbc-8d086931e77b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:59.726907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 304/610] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:26:59 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/78394014f5904021823a96d12902b2ec/roles/070199642db14984a418795ea5af54cb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:26:59.808111 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35334346-4391-46c4-a49f-e70e18108110 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:26:59.872636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a711f-c729-41a4-b39e-af9f6872a685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:26:59.873734 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a711f-c729-41a4-b39e-af9f6872a685 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.874267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a711f-c729-41a4-b39e-af9f6872a685 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.874748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a711f-c729-41a4-b39e-af9f6872a685 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.875806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a711f-c729-41a4-b39e-af9f6872a685 None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:26:59.876396 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a711f-c729-41a4-b39e-af9f6872a685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:26:59.878359 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a711f-c729-41a4-b39e-af9f6872a685 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:26:59.895416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 307/611] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:26:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:26:59.905175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1162-730c-4fad-bdbc-8d086931e77b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:26:59.949884 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:26:59.967521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137054a-fe09-42cd-8f75-439b08841525 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.968178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137054a-fe09-42cd-8f75-439b08841525 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.968668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137054a-fe09-42cd-8f75-439b08841525 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:26:59.970308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 308/612] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:26:59.976125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3d0f-57ed-4ff8-a0f4-cfbb57e4b70c None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:26:59.976793 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3d0f-57ed-4ff8-a0f4-cfbb57e4b70c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:26:59.977270 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3d0f-57ed-4ff8-a0f4-cfbb57e4b70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:00.003264 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-2f5a3d0f-57ed-4ff8-a0f4-cfbb57e4b70c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:00.004476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-2f5a3d0f-57ed-4ff8-a0f4-cfbb57e4b70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:00.044302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1162-730c-4fad-bdbc-8d086931e77b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:00.094247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 309/613] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:00.106909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:00.107980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:00.108497 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:00.109004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:00.114511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a3d0f-57ed-4ff8-a0f4-cfbb57e4b70c None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:00.123181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:00.125028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:00.134719 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:00.155894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:00.157010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:00.157979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:00.157979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:00.157979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:00.157979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:00.157979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 305/614] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:00.171825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:00.230387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a3d0f-57ed-4ff8-a0f4-cfbb57e4b70c None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:00.276834 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bf56f-b205-43c2-a383-4d13b5fc124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:00.277575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bf56f-b205-43c2-a383-4d13b5fc124a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:00.277862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bf56f-b205-43c2-a383-4d13b5fc124a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:00.278161 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bf56f-b205-43c2-a383-4d13b5fc124a None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:00.299720 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bf56f-b205-43c2-a383-4d13b5fc124a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:00.307300 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-c13bf56f-b205-43c2-a383-4d13b5fc124a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:00.312517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:00.313593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:00.314547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:00.319442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 310/615] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:00.321309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bf56f-b205-43c2-a383-4d13b5fc124a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:00.326577 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ffba8-3088-4a01-8cfe-e645c1f395a8 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:00.385415 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:00.386066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:00.386381 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:00.386660 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:00.387371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:00.387686 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:00.389104 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:00.414906 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 306/616] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:00.423178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:00.444214 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:00.483969 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bf56f-b205-43c2-a383-4d13b5fc124a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:00.521686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:00.522790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:00.523951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:00.524484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:00.525440 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:00.525960 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:00.527589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:00.537465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 311/617] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:27:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:00.544261 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:00.667255 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08da17b-f290-43ba-95dc-933f086511ac None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:00.703505 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:00.713604 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25bf0-4f38-4a75-8f66-f125c9d3fb9b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:00.724020 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:00.725047 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:00.725532 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:00.726166 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:00.744682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:00.748020 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:00.773775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:00.774868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:00.775859 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:00.775859 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:00.775859 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:00.775859 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:00.778405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 312/618] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:00.784179 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:00.788223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:00.788920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:00.789215 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:00.789484 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] PATH_INFO: `/v3/users/swift` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:00.802070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:00.803595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:00.835409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:00.836601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:00.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 307/619] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:00.850213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:00.870514 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:00.936323 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08998a-8572-4a2a-bcc9-cbc3fd155968 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:00.962868 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dae7b-2cb9-480f-8384-ffd394aa989b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:00.982247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1afaf-94f0-416b-bc3e-ac34320bdf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:00.982938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1afaf-94f0-416b-bc3e-ac34320bdf9a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:00.983222 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1afaf-94f0-416b-bc3e-ac34320bdf9a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:00.983502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1afaf-94f0-416b-bc3e-ac34320bdf9a None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:00.984217 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1afaf-94f0-416b-bc3e-ac34320bdf9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:00.984523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1afaf-94f0-416b-bc3e-ac34320bdf9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:00.986573 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1afaf-94f0-416b-bc3e-ac34320bdf9a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:01.003651 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 313/620] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:01.004921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7baca4-2448-4685-9be7-63f5483fa6d3 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.005211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7baca4-2448-4685-9be7-63f5483fa6d3 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.005488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7baca4-2448-4685-9be7-63f5483fa6d3 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.007057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 314/621] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:01.009644 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7baca4-2448-4685-9be7-63f5483fa6d3 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:01.014816 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8314cd-eaf9-4d42-a785-8eaac5653dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.015458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8314cd-eaf9-4d42-a785-8eaac5653dde None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.015737 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8314cd-eaf9-4d42-a785-8eaac5653dde None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.016007 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8314cd-eaf9-4d42-a785-8eaac5653dde None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.016821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8314cd-eaf9-4d42-a785-8eaac5653dde None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:01.017123 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8314cd-eaf9-4d42-a785-8eaac5653dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:01.018619 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8314cd-eaf9-4d42-a785-8eaac5653dde None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:01.044029 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 308/622] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:27:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:01.045628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d24b8c-abbd-4880-b1d9-dbaffde421b3 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.045919 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d24b8c-abbd-4880-b1d9-dbaffde421b3 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.046362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d24b8c-abbd-4880-b1d9-dbaffde421b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.064485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:01.067243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-f0d24b8c-abbd-4880-b1d9-dbaffde421b3 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:01.069319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-f0d24b8c-abbd-4880-b1d9-dbaffde421b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:01.147469 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7baca4-2448-4685-9be7-63f5483fa6d3 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:01.152403 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:01.170775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 309/623] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:01.172099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d24b8c-abbd-4880-b1d9-dbaffde421b3 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:01.194534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.195659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.195957 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.196238 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.206039 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:01.208502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:01.225090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:01.227233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:01.229265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:01.234151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 315/624] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:27:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:01.234151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:01.321184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d24b8c-abbd-4880-b1d9-dbaffde421b3 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:01.363905 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20414aec-dd51-40e0-a05b-d17f473b35c1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:01.365741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:01.369704 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.370891 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.371414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.371911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.388873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:01.391319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:01.407137 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a8aa-01e4-493a-bf34-6716e7503661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.408185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a8aa-01e4-493a-bf34-6716e7503661 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.408700 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a8aa-01e4-493a-bf34-6716e7503661 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.409201 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a8aa-01e4-493a-bf34-6716e7503661 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:01.412064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:01.413040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:01.414050 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:01.414050 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:01.414050 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:01.414050 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:01.418236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 310/625] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:01.419161 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:01.429374 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0a8aa-01e4-493a-bf34-6716e7503661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-8ff0a8aa-01e4-493a-bf34-6716e7503661 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:01.437523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:01.438637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:01.439577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:01.443468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 316/626] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:01.444922 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a8aa-01e4-493a-bf34-6716e7503661 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:01.491164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:01.556181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e339fc0-8b64-4d33-99dc-2899ceb3e2ef None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:01.589559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a8aa-01e4-493a-bf34-6716e7503661 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:01.595997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.596859 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.597243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.597608 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.598567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:01.598973 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:01.600530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:01.624868 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 311/627] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:27:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:01.624868 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:01.645156 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.645804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.646092 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.646403 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.647293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:01.647415 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:01.648789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:01.663144 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 317/628] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:01.664044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:01.696516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:01.753974 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69266-241f-4c43-8360-96902b72c833 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:01.762499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:01.782749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a16a6-2142-4b9b-a3e4-f10062dfdbcd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:01.799027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.799682 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.800127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.800315 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.801055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:01.801369 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:01.802804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:01.814390 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 312/629] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:27:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:01.814390 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:01.828972 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6924fa-9868-4150-bde5-76577cab77a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.829967 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6924fa-9868-4150-bde5-76577cab77a3 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.830680 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6924fa-9868-4150-bde5-76577cab77a3 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.830868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6924fa-9868-4150-bde5-76577cab77a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:01.921781 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6924fa-9868-4150-bde5-76577cab77a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:01.923429 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6924fa-9868-4150-bde5-76577cab77a3 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:01.950265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d08922-9c02-4e07-b2d5-a6d4ac3950d9 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:01.952424 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:01.994232 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313dab1-defb-4666-a75c-8f73e70bccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:01.994904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313dab1-defb-4666-a75c-8f73e70bccb4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:01.995183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313dab1-defb-4666-a75c-8f73e70bccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:01.995458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313dab1-defb-4666-a75c-8f73e70bccb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.005202 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313dab1-defb-4666-a75c-8f73e70bccb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:02.006719 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313dab1-defb-4666-a75c-8f73e70bccb4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-2313dab1-defb-4666-a75c-8f73e70bccb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:02.022291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:02.029423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 313/630] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37342607-932a-4540-82f0-f7ade55485d9 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.030478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37342607-932a-4540-82f0-f7ade55485d9 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.031139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37342607-932a-4540-82f0-f7ade55485d9 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.031139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 314/631] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:02.031139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37342607-932a-4540-82f0-f7ade55485d9 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.056479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 318/632] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:27:02.057716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6924fa-9868-4150-bde5-76577cab77a3 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.074350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:02.161644 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37342607-932a-4540-82f0-f7ade55485d9 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.177102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6924fa-9868-4150-bde5-76577cab77a3 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.196066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9df9ba-0f32-49f5-878a-613ccce1fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.196726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df9ba-0f32-49f5-878a-613ccce1fcff None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.197012 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df9ba-0f32-49f5-878a-613ccce1fcff None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.197290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df9ba-0f32-49f5-878a-613ccce1fcff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.200933 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:02.208430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9df9ba-0f32-49f5-878a-613ccce1fcff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:02.209918 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9df9ba-0f32-49f5-878a-613ccce1fcff None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:02.214247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.214889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.215172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.215447 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.216191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:02.216502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:02.217974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-1e9df9ba-0f32-49f5-878a-613ccce1fcff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:02.225095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:02.226241 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:02.227237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:02.227237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:02.227237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:02.227237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:02.227237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 315/633] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:02.227839 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0375d-fbd2-41ff-b0d0-40f4185215b8 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.228122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0375d-fbd2-41ff-b0d0-40f4185215b8 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.228399 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0375d-fbd2-41ff-b0d0-40f4185215b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.234878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 319/634] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:02.235818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.251021 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-d5e0375d-fbd2-41ff-b0d0-40f4185215b8 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:02.252292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-d5e0375d-fbd2-41ff-b0d0-40f4185215b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:02.320677 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 316/635] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:02.321955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56c885-4141-499f-882b-fb5251eb2e0d None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.322279 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56c885-4141-499f-882b-fb5251eb2e0d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.322566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56c885-4141-499f-882b-fb5251eb2e0d None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.328126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 317/636] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:02.328904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56c885-4141-499f-882b-fb5251eb2e0d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.339046 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1409f81-4b43-4cd8-a08c-1f7211465dfe None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.345791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:02.376726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.377376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.377658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.377930 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.389262 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:02.390768 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:02.405441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:02.406672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:02.407862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:02.407862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:02.407862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:02.407862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:02.407862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 320/637] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:02.407862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.443218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56c885-4141-499f-882b-fb5251eb2e0d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.446518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:02.484094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e6507-d9cb-4856-a918-983e1cde972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.484820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e6507-d9cb-4856-a918-983e1cde972e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.485184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e6507-d9cb-4856-a918-983e1cde972e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.485531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e6507-d9cb-4856-a918-983e1cde972e None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.486334 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e6507-d9cb-4856-a918-983e1cde972e None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:02.486710 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e6507-d9cb-4856-a918-983e1cde972e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:02.488906 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e6507-d9cb-4856-a918-983e1cde972e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:02.510162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 318/638] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:02.510162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e97a0-a77c-4063-b9c4-2dd6d81644b6 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.510162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e97a0-a77c-4063-b9c4-2dd6d81644b6 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.510162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e97a0-a77c-4063-b9c4-2dd6d81644b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.527563 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-e83e97a0-a77c-4063-b9c4-2dd6d81644b6 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:02.528812 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-e83e97a0-a77c-4063-b9c4-2dd6d81644b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:02.536416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430c24c-ab57-41b1-9f42-8a421545b3dc None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.575370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6c3b3-b638-479d-a15f-6bb506374cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.576201 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c3b3-b638-479d-a15f-6bb506374cda None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.576561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c3b3-b638-479d-a15f-6bb506374cda None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.576915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c3b3-b638-479d-a15f-6bb506374cda None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/ec86b4f3ad1d4f75a685c4ff209051ce/roles/070199642db14984a418795ea5af54cb` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.596290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6c3b3-b638-479d-a15f-6bb506374cda None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=ec86b4f3ad1d4f75a685c4ff209051ce, role_id=070199642db14984a418795ea5af54cb)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:02.597829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6c3b3-b638-479d-a15f-6bb506374cda None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:02.604813 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:02.610692 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 319/639] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:02.611391 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e97a0-a77c-4063-b9c4-2dd6d81644b6 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.618351 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 321/640] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:27:02 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/ec86b4f3ad1d4f75a685c4ff209051ce/roles/070199642db14984a418795ea5af54cb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:27:02.619674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6c3b3-b638-479d-a15f-6bb506374cda None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.748779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:02.750317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e97a0-a77c-4063-b9c4-2dd6d81644b6 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.776171 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6c3b3-b638-479d-a15f-6bb506374cda None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.804883 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7ee8-7423-46e4-a0a2-201d4741419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.805557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7ee8-7423-46e4-a0a2-201d4741419d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.805859 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7ee8-7423-46e4-a0a2-201d4741419d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.806141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7ee8-7423-46e4-a0a2-201d4741419d None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.821100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7ee8-7423-46e4-a0a2-201d4741419d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:02.824173 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.824823 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.825109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.825389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.826097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-579a7ee8-7423-46e4-a0a2-201d4741419d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:02.826246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:02.827219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:02.828157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:02.829542 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:02.829638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 320/641] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:02.829737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:02.829838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7ee8-7423-46e4-a0a2-201d4741419d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.841112 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 322/642] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:02.842049 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:02.898511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:02.943339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7ee8-7423-46e4-a0a2-201d4741419d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.950530 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4079d7-9270-4d72-9c68-a795e395c706 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:02.979081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.979748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.980030 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.980301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] PATH_INFO: `/v3/users/glance` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.985579 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e79234-2120-4f91-80dd-7dfdc1195da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:02.986274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79234-2120-4f91-80dd-7dfdc1195da5 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:02.986554 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79234-2120-4f91-80dd-7dfdc1195da5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:02.986825 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79234-2120-4f91-80dd-7dfdc1195da5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:02.988189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:02.989679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.002215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e79234-2120-4f91-80dd-7dfdc1195da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:03.002411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:03.003494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 321/643] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:03.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-37e79234-2120-4f91-80dd-7dfdc1195da5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:03.010816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:03.011755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 323/644] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:03.012664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e79234-2120-4f91-80dd-7dfdc1195da5 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.017012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:03.130182 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff18174-21a3-491d-8bb0-f43e57ff69de None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.130707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e79234-2120-4f91-80dd-7dfdc1195da5 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.153631 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:03.168060 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.169379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.169878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.170141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.170549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.170836 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.171046 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.171172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.171858 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:03.172147 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.173555 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.179829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.181628 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.182627 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 322/645] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:03.183474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:03.199285 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:03.200566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:03.201652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:03.201652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:03.201652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:03.201652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:03.201652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:03.201652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:03.201652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:03.201652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:03.207293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 324/646] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:03.207293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.278981 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:03.297665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391c948-30ce-4e41-9601-3a33fb7c56f1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.320416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96b60f-3983-4b84-bc8a-cf984734ec8a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.336344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.337544 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.338068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.338583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.339584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:03.340157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.341987 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.354497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.355134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.355413 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.355685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.356411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:03.356710 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.358091 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.363542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 323/647] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:03.364646 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.371038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 325/648] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:03.371896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.412961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:03.477162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25445560-786e-4690-b37c-7f7f1edfb50d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.486618 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d64bd2-a0d8-404c-b805-7024ffe27081 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.514544 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.515200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.515501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.515776 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.521089 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:03.529649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.531200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.531768 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.532408 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.532688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.532959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.533717 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:03.534025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.535550 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.552822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 326/649] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:03.553061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:03.553969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:03.554980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:03.554980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:03.554980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:03.554980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:03.555236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.555321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 324/650] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:03.555321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.661747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:03.665751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc61cd-d7a4-4179-af98-a3ae45fdfff9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.671553 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6beb-e0df-4e00-ade3-bdbca97b6a5a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.703164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ce905-88f7-4f06-8c3f-be04393f2c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.703789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ce905-88f7-4f06-8c3f-be04393f2c82 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.704070 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ce905-88f7-4f06-8c3f-be04393f2c82 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.704342 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ce905-88f7-4f06-8c3f-be04393f2c82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.713841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.714983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.716121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.716686 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.729532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.731447 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:03.747337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:03.748581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:03.749792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:03.749792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:03.749792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:03.749792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:03.756618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 325/651] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:03.756618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.775685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:03.792554 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ce905-88f7-4f06-8c3f-be04393f2c82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.794152 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ce905-88f7-4f06-8c3f-be04393f2c82 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.860664 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e914656-1f06-4d78-9bda-23ae0920f12e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:03.897112 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:03.897740 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:03.898017 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:03.898323 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:03.899118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=c5ba9e7780f5437caac949931ca55b80) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:03.899414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:03.900862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:03.907313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 327/652] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:27:03.908267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ce905-88f7-4f06-8c3f-be04393f2c82 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.911603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 326/653] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/role_assignments?user.id=c5ba9e7780f5437caac949931ca55b80&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:03.912470 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:03.933080 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:04.026297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:04.055518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ce905-88f7-4f06-8c3f-be04393f2c82 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.064288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49f30d-1721-4bf9-b82f-449dfbc6f846 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.101851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.103019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.103019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.103216 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.103798 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:04.104107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.106359 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.123468 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.123840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 328/654] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:04.124217 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.126143 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.126323 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.126421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.126421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:04.126421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.127654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.141871 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 327/655] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:04.142878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.220502 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:04.252133 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414800f5-0331-4caf-8f70-09a14f61c3bf None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.264868 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf6b03-b5ee-4763-9203-0f5d7aaf167b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.292459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.293131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.293418 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.293698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.305678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.307113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.308259 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.309131 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.309606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.310083 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.314258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:04.319182 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.321144 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:04.324705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:04.325873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:04.326940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:04.326940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:04.326940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:04.326940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:04.331278 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 329/656] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:04.336502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:04.340683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:04.341713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:27:04.342917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:04.345327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 328/657] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:04.351002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.446758 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1919e-344f-47ee-9cd6-302d8a71b05a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.468111 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e0cfd-8158-46f3-bf31-fc86fe3df871 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.484901 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.485570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.485857 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.486161 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.496147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.497317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:04.498936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.509851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.510776 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.511216 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.511646 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.512536 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:04.513009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:04.514397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:04.515683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:27:04.516986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:04.518024 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.521515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 330/658] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:04.521515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.539098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 329/659] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:04.540920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.604671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:04.663093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbc2fc-9ce4-4a26-a0f9-c8fd15abe70e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.670531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6712ada-2cbc-4882-a45d-f11f831993bb None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.704035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.704682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.704964 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.705241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.706044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:04.706525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.707030 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a32e7-5513-4259-b172-f802afbdfb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.707672 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a32e7-5513-4259-b172-f802afbdfb63 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.707943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a32e7-5513-4259-b172-f802afbdfb63 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.708059 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.708278 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a32e7-5513-4259-b172-f802afbdfb63 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.709016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a32e7-5513-4259-b172-f802afbdfb63 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:04.709324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a32e7-5513-4259-b172-f802afbdfb63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.710806 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a32e7-5513-4259-b172-f802afbdfb63 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.738128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 331/660] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:04.738128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.738469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 330/661] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:04.738469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3133-9dff-442f-81c8-0ffefa0e3f82 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.738469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3133-9dff-442f-81c8-0ffefa0e3f82 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.738469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3133-9dff-442f-81c8-0ffefa0e3f82 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.738469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 331/662] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:04.741089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579170db-b7cc-4229-8a39-e4572e3c3b24 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.741089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579170db-b7cc-4229-8a39-e4572e3c3b24 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.741460 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579170db-b7cc-4229-8a39-e4572e3c3b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.761035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-579170db-b7cc-4229-8a39-e4572e3c3b24 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:04.762344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-579170db-b7cc-4229-8a39-e4572e3c3b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:04.808736 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:04.836142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 332/663] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:27:04.837361 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579170db-b7cc-4229-8a39-e4572e3c3b24 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.852245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02f94e-6165-4d98-8ae1-dbabd0b9021b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:04.871291 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:04.891219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:04.891863 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:04.892149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:04.892425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:04.899770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:04.901236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:04.913206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:04.914473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 332/664] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:04.915703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:04.977813 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579170db-b7cc-4229-8a39-e4572e3c3b24 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:05.015734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074129b2-30d5-417e-9322-72ef5a81c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:05.016808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074129b2-30d5-417e-9322-72ef5a81c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:05.017286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074129b2-30d5-417e-9322-72ef5a81c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:05.017734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074129b2-30d5-417e-9322-72ef5a81c4a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:05.022078 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32075f6-fd1f-442f-8fde-0b02b3c5dc9c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:05.059402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f44bb-415c-4fc8-961f-bc96df528e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:05.060051 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f44bb-415c-4fc8-961f-bc96df528e11 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:05.060335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f44bb-415c-4fc8-961f-bc96df528e11 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:05.060609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f44bb-415c-4fc8-961f-bc96df528e11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:05.072963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:05.100690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074129b2-30d5-417e-9322-72ef5a81c4a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:05.102445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074129b2-30d5-417e-9322-72ef5a81c4a2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:05.115140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:05.144271 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f44bb-415c-4fc8-961f-bc96df528e11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:05.145800 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f44bb-415c-4fc8-961f-bc96df528e11 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:05.220707 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 333/665] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:27:05.220707 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074129b2-30d5-417e-9322-72ef5a81c4a2 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:05.272675 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 333/666] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:27:05.272675 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f44bb-415c-4fc8-961f-bc96df528e11 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:05.313193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:05.341120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074129b2-30d5-417e-9322-72ef5a81c4a2 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:05.357509 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:05.379299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcc79a-ee48-4cf6-abec-5bb2e01d48d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:05.379970 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcc79a-ee48-4cf6-abec-5bb2e01d48d5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:05.380253 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcc79a-ee48-4cf6-abec-5bb2e01d48d5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:05.380634 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcc79a-ee48-4cf6-abec-5bb2e01d48d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:05.383786 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f44bb-415c-4fc8-961f-bc96df528e11 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:05.393002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcc79a-ee48-4cf6-abec-5bb2e01d48d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-bcdcc79a-ee48-4cf6-abec-5bb2e01d48d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:05.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:05.399276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:05.400240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:05.402601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 334/667] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:05.404167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcc79a-ee48-4cf6-abec-5bb2e01d48d5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:05.429328 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:05.429988 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:05.430308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:05.430589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:05.431380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:05.431695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:05.433212 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:05.447602 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 334/668] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:05.448500 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:05.550335 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcc79a-ee48-4cf6-abec-5bb2e01d48d5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:05.578163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9cd81-caf2-48bb-ba52-cdbced38ccc9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:05.594182 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:05.594726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:05.595006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:05.595283 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:05.598896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:05.608585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:05.610022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:05.621715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:05.622408 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:05.622720 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:05.622966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:05.626427 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:05.629194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:05.630240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:05.631822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:05.631822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:05.631822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:05.631822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:05.631822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 335/669] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:05.631822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:05.641592 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:05.643615 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:05.660185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:05.661947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:05.663841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:05.663841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:05.663841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:05.663841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:05.666130 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 335/670] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:05.667136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:05.770798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e960e40-c092-4e09-a1c7-6f02726d66dc None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:05.793963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb336d-d426-4cdf-99fb-60585241407d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:05.812065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:05.812712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:05.812992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:05.813266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:05.814000 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:05.814347 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:05.815828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:05.829068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 336/671] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:27:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:05.830531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:05.840804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:05.841465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:05.841739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:05.842014 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:05.842840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=40c475d21adc4ba4bd48bed9ebb0b445) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:05.843153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:05.844616 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:05.852478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 336/672] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:05 2022] GET /identity/v3/role_assignments?user.id=40c475d21adc4ba4bd48bed9ebb0b445&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:05.852819 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:05.924066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:05.937783 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:06.005242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62811e43-832c-4a86-a82f-f6a066ca7aef None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.009094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84783293-2cf5-43e1-84db-ee43fc3b459a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.054259 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.055383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.055383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.055584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.056142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:06.056450 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.057874 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.067707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.068365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.069010 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.069966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.070716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:06.071023 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.072882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.079170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 337/673] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:06.080463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.090283 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 337/674] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:06.091538 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.219982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff91c3-48b5-4468-8f7c-2f05328d03ab None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.231985 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:06.238584 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3964ce-11aa-4794-8062-478c3047df76 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.254979 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:06.262319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.262992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.263771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.264051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.276903 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.278416 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.279309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa9ab3-c65e-42b8-859d-9c1698b9a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.279983 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa9ab3-c65e-42b8-859d-9c1698b9a6da None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.280312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa9ab3-c65e-42b8-859d-9c1698b9a6da None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.280546 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa9ab3-c65e-42b8-859d-9c1698b9a6da None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.288821 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa9ab3-c65e-42b8-859d-9c1698b9a6da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.290478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa9ab3-c65e-42b8-859d-9c1698b9a6da None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:06.294808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:06.295808 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:06.296851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:06.296851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:06.296851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:06.296851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:06.300935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 338/675] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:06.301583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-83aa9ab3-c65e-42b8-859d-9c1698b9a6da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:06.304737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:06.305696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 338/676] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df74d-b57f-4f53-a947-720a5a6147a7 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.308009 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df74d-b57f-4f53-a947-720a5a6147a7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.308665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df74d-b57f-4f53-a947-720a5a6147a7 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.313159 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 339/677] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:06.313159 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150df74d-b57f-4f53-a947-720a5a6147a7 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.413421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbff505-5881-4035-bbd6-cbca4e14e13b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.432338 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150df74d-b57f-4f53-a947-720a5a6147a7 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.451286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.451939 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.452217 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.452486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.459301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.460768 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:06.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:06.474098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 339/678] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:06.475141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.479730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea70d16-4e0e-404a-8935-186875b1192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.481310 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea70d16-4e0e-404a-8935-186875b1192f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.481579 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea70d16-4e0e-404a-8935-186875b1192f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.481850 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea70d16-4e0e-404a-8935-186875b1192f None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.482597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea70d16-4e0e-404a-8935-186875b1192f None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:06.482898 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea70d16-4e0e-404a-8935-186875b1192f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.484291 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea70d16-4e0e-404a-8935-186875b1192f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.499138 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 340/679] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:06.500541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e7514-adc7-4c58-8188-b9e7885adc4f None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.500839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e7514-adc7-4c58-8188-b9e7885adc4f None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.501121 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e7514-adc7-4c58-8188-b9e7885adc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.507381 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:06.521404 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-584e7514-adc7-4c58-8188-b9e7885adc4f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:06.522588 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-584e7514-adc7-4c58-8188-b9e7885adc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:06.552799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:06.595607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511437-a6c1-4ef5-95dc-bb4d7f116a33 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.606160 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 341/680] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:06.606160 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e7514-adc7-4c58-8188-b9e7885adc4f None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.636686 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.637331 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.637623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.637901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.638702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:06.639019 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.640477 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.653690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 340/681] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:06.654483 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.728628 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e7514-adc7-4c58-8188-b9e7885adc4f None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.739965 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:06.759726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefb124-150f-44dd-9346-c02beaa671b5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.764736 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.765381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.765666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.765945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.766756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:06.767070 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.768535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.785528 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 342/682] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:06.785528 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.804036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.804722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.804961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.805250 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.813993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:06.815175 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:06.816359 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:06.911376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bba84-9597-4f27-b2f4-f77c88efc850 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:06.955497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae5b58-761c-4e6d-b986-fe5cc7a06a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:06.956245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae5b58-761c-4e6d-b986-fe5cc7a06a84 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:06.956540 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae5b58-761c-4e6d-b986-fe5cc7a06a84 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:06.956826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae5b58-761c-4e6d-b986-fe5cc7a06a84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:06.960630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:06.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 341/683] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:06.963371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:06.973499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae5b58-761c-4e6d-b986-fe5cc7a06a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-a1ae5b58-761c-4e6d-b986-fe5cc7a06a84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:06.979739 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:06.980775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:06.981688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:06.985844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 343/684] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:06.986559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae5b58-761c-4e6d-b986-fe5cc7a06a84 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:07.075738 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:07.078591 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2dc7a-c056-4443-bbee-913d9e3c8d73 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:07.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae5b58-761c-4e6d-b986-fe5cc7a06a84 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:07.118451 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:07.119097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:07.119409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:07.119648 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:07.120480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=78394014f5904021823a96d12902b2ec) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:07.120793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:07.122477 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:07.138213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 342/685] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/role_assignments?user.id=78394014f5904021823a96d12902b2ec&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:07.138213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:07.143492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:07.144337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:07.144692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:07.145036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:07.145960 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=f2076e2caf434975a590a7766cb575f6) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:07.146401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:07.148063 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:07.149257 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:07.155608 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 344/686] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:06 2022] GET /identity/v3/role_assignments?user.id=f2076e2caf434975a590a7766cb575f6&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:07.157002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:07.304382 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ff48f-b8dc-427b-bb82-22cfd6cabec8 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:07.379995 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:07.380807 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:07.382529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:07.382529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:07.382529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:07.382825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:07.383593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd10e5-c513-4be0-889e-387a3f8f626b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:07.384718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:07.407627 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 343/687] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:27:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:07.414816 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:07.438759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:07.438759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:07.438759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:07.438759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:07.439181 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:07.439181 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:07.439181 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:07.448259 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 345/688] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:07.454273 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:07.470366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:07.493894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:07.541666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fd547-7639-451d-89c0-d90c2f10dfb5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:07.579737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c48a03-58e6-4a14-b3d5-dec92a6fa9a6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:07.585871 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866e05a-2c4a-4a1a-b463-6332910c19f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:07.586650 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866e05a-2c4a-4a1a-b463-6332910c19f6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:07.587008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866e05a-2c4a-4a1a-b463-6332910c19f6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:07.587359 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866e05a-2c4a-4a1a-b463-6332910c19f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:07.624473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:07.626060 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:07.626612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:07.627148 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:07.640599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:07.642467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:07.659927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:07.660998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:07.661972 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:07.661972 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:07.661972 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:07.661972 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:07.664274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 346/689] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:07.671142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:07.673743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866e05a-2c4a-4a1a-b463-6332910c19f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:07.675504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866e05a-2c4a-4a1a-b463-6332910c19f6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:07.785928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:07.788654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:07.858580 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 344/690] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:27:07.864536 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866e05a-2c4a-4a1a-b463-6332910c19f6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:07.869604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8289e86-3ab4-42a8-9c57-543455f3326f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:07.932179 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:07.933338 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:07.933338 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:07.933338 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:07.934072 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:07.934415 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:07.935822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:07.949900 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 347/691] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:07.955508 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.055674 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866e05a-2c4a-4a1a-b463-6332910c19f6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:08.062938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cd6a1-a219-407d-897c-3b740f444a5f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:08.084917 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:08.099775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d091-a7ae-443b-8d4e-4d7a2e4454b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:08.100457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d091-a7ae-443b-8d4e-4d7a2e4454b9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.100747 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d091-a7ae-443b-8d4e-4d7a2e4454b9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.101031 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d091-a7ae-443b-8d4e-4d7a2e4454b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.109682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76d091-a7ae-443b-8d4e-4d7a2e4454b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:08.111121 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76d091-a7ae-443b-8d4e-4d7a2e4454b9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:08.123276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb647f2-b893-4fbe-8dd0-bbfe7070d38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:08.124008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb647f2-b893-4fbe-8dd0-bbfe7070d38b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.124356 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb647f2-b893-4fbe-8dd0-bbfe7070d38b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.124692 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb647f2-b893-4fbe-8dd0-bbfe7070d38b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-be76d091-a7ae-443b-8d4e-4d7a2e4454b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:08.125944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:08.127158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:08.128499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 348/692] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:27:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:08.131162 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59489f8d-0d2f-4d88-aa43-49f9c88bc5ab None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.131451 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59489f8d-0d2f-4d88-aa43-49f9c88bc5ab None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.131737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59489f8d-0d2f-4d88-aa43-49f9c88bc5ab None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.134187 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 349/693] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:08.134955 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59489f8d-0d2f-4d88-aa43-49f9c88bc5ab None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.147261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb647f2-b893-4fbe-8dd0-bbfe7070d38b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:08.149138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb647f2-b893-4fbe-8dd0-bbfe7070d38b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-3bb647f2-b893-4fbe-8dd0-bbfe7070d38b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:08.169062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:08.170125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:08.171135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:08.171135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:08.171135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:08.171135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:08.171986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 345/694] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:08.175503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037780a-3784-47ec-9a8e-d12d9636bc6a None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.175793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037780a-3784-47ec-9a8e-d12d9636bc6a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.176076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037780a-3784-47ec-9a8e-d12d9636bc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.195278 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-4037780a-3784-47ec-9a8e-d12d9636bc6a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:08.196745 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-4037780a-3784-47ec-9a8e-d12d9636bc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:08.210545 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:08.266638 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59489f8d-0d2f-4d88-aa43-49f9c88bc5ab None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:08.290554 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 346/695] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:08.290554 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037780a-3784-47ec-9a8e-d12d9636bc6a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.333370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:08.334022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.334440 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.334728 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.335534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:08.335842 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:08.337315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:08.356204 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 350/696] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:27:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:08.357058 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.402980 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:08.420646 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037780a-3784-47ec-9a8e-d12d9636bc6a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:08.459237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:08.460507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.461046 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.461549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.462627 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:08.463244 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:08.464943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:08.466032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0f6f9-7a0b-4856-8166-9e5409822193 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:08.482858 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 347/697] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:08.484218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.494502 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:08.501990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:08.502695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.503012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.503295 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.514843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:08.521784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:08.538974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:08.540016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:08.541425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:08.541425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:08.541425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:08.541425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:08.541425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 351/698] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:08.541425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.602413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2f7dc-b04a-4378-84f6-a95fe5f9a0fd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:08.640475 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d1dc-62ee-4b63-9cc0-e5b3c4802579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:08.641643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d1dc-62ee-4b63-9cc0-e5b3c4802579 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.642269 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d1dc-62ee-4b63-9cc0-e5b3c4802579 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.642772 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d1dc-62ee-4b63-9cc0-e5b3c4802579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.651883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:08.662481 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356117e4-c114-4e41-8427-504c654ef86c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:08.702892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:08.703764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.704150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.704527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.712984 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:08.714886 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:08.727745 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2d1dc-62ee-4b63-9cc0-e5b3c4802579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:08.728516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:08.729649 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:08.730882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:08.730882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:08.730882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:08.730882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:08.730882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:08.730882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:08.730882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:08.730882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:08.737391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 352/699] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:08.737391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.737622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2d1dc-62ee-4b63-9cc0-e5b3c4802579 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:08.748774 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:08.864264 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3106cfb-8f00-4a49-87f7-19ba060a9867 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:08.870558 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 348/700] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:27:08.871757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d1dc-62ee-4b63-9cc0-e5b3c4802579 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.922443 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:08.923014 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:08.923298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:08.923581 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:08.924295 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:08.924604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:08.926043 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:08.947393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 353/701] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:08.947393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:08.961024 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:09.015284 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d1dc-62ee-4b63-9cc0-e5b3c4802579 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:09.027309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:09.065862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:09.067073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.067514 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.067986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.069568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:09.070491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:09.072683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:09.079591 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c585a6-caf4-4236-b2d5-b9933f3a61c0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:09.088778 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 349/702] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:09.089845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:09.126775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:09.127790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.128432 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.128874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.140907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:09.142856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:09.214615 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e77ee-b2f1-4222-afeb-e7becdaa77fc None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:09.257243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e867-01c4-421e-a793-9a722eee0e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:09.258064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e867-01c4-421e-a793-9a722eee0e72 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.258469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e867-01c4-421e-a793-9a722eee0e72 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.258840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e867-01c4-421e-a793-9a722eee0e72 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.259675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12e867-01c4-421e-a793-9a722eee0e72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:09.261416 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12e867-01c4-421e-a793-9a722eee0e72 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:09.294169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 350/703] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:27:09 2022] POST /identity/v3/users => generated 254 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:09.294169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e867-01c4-421e-a793-9a722eee0e72 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:09.310796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:09.312080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:09.313065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:09.313065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:09.313065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:09.313065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:09.315330 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 354/704] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:09.322047 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:09.323863 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:09.416471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e867-01c4-421e-a793-9a722eee0e72 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:09.452075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:09.459617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbfc16-61c2-4fd5-8e1b-4dce5fd1e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:09.459617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbfc16-61c2-4fd5-8e1b-4dce5fd1e2fa None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.459617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbfc16-61c2-4fd5-8e1b-4dce5fd1e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.459617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbfc16-61c2-4fd5-8e1b-4dce5fd1e2fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.460495 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbfc16-61c2-4fd5-8e1b-4dce5fd1e2fa None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=ec86b4f3ad1d4f75a685c4ff209051ce) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:09.461106 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbfc16-61c2-4fd5-8e1b-4dce5fd1e2fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:09.462748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbfc16-61c2-4fd5-8e1b-4dce5fd1e2fa None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:09.474489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 351/705] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:09 2022] GET /identity/v3/role_assignments?user.id=ec86b4f3ad1d4f75a685c4ff209051ce&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:09.534263 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf84a-7cb1-4a27-8371-59994da11bf9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:09.577485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e5ed-62a4-464b-8c58-1dfa94cb52c8 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.577485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e5ed-62a4-464b-8c58-1dfa94cb52c8 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.577485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e5ed-62a4-464b-8c58-1dfa94cb52c8 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.577485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 352/706] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:09.585652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:09.586039 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.586344 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.586625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.587360 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:09.587466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d7fc2-a77b-4897-bef0-d54e5e335a92 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.587670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:09.587783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d7fc2-a77b-4897-bef0-d54e5e335a92 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.587994 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d7fc2-a77b-4897-bef0-d54e5e335a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.589079 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:09.604229 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 355/707] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:09.611528 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:09.631437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-690d7fc2-a77b-4897-bef0-d54e5e335a92 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:09.633920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-690d7fc2-a77b-4897-bef0-d54e5e335a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:09.740324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 353/708] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:09.742085 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8c028-3f06-49cd-a3e7-cfc2444ed876 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.742844 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8c028-3f06-49cd-a3e7-cfc2444ed876 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.743189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8c028-3f06-49cd-a3e7-cfc2444ed876 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.745658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 354/709] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:09.751168 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203b88-447f-4229-a452-2c13cd18295c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:09.752447 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99c63c-2fd5-4a7e-9761-0ffade74eeff None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.752813 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99c63c-2fd5-4a7e-9761-0ffade74eeff None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.753253 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99c63c-2fd5-4a7e-9761-0ffade74eeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.778897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-6a99c63c-2fd5-4a7e-9761-0ffade74eeff None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:09.780601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-6a99c63c-2fd5-4a7e-9761-0ffade74eeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:09.790535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:09.794660 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:09.799945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:09.801236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:09.801236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:09.801236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:09.814261 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:09.815753 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:09.831309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:09.833381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 356/710] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:09.835294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:09.885567 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 355/711] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:09.886557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99c63c-2fd5-4a7e-9761-0ffade74eeff None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:09.999136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9266a4-1243-4932-9a3d-ec8e52aaf023 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:10.012424 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99c63c-2fd5-4a7e-9761-0ffade74eeff None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:10.046952 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:10.047897 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:10.048323 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:10.048663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] PATH_INFO: `/v3/services/network` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:10.048937 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:10.049433 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:10.049714 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:10.050149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:10.050502 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:10.051300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:10.051420 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:10.051766 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:10.053326 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:10.065856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:10.067476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:10.069467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:10.070498 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 356/712] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:10.070498 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:10.070676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 357/713] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:09 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:10.078127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:10.078843 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:10.135020 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:10.196055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78a5dc-cc90-4c82-b7ad-31cef22e1594 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:10.209983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2e82c-bf64-4d6e-b6cd-2f5e1dfb5859 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:10.230638 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:10.231362 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:10.231715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:10.232054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:10.232935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=c5ba9e7780f5437caac949931ca55b80) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:10.233313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:10.234856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:10.243539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 358/714] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:10 2022] GET /identity/v3/role_assignments?user.id=c5ba9e7780f5437caac949931ca55b80&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:10.244686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:10.263874 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:10.269132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:10.269548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:10.269658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:10.270371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:10.271915 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:10.294665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:10.295793 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:10.296721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:10.299611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 357/715] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:27:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:10.306281 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:10.406804 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb3351-406a-423c-87b4-07da3237f9ca None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:10.420666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:10.449758 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:10.454019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:10.454687 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:10.455124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:10.455401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:10.456054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] RBAC: Adding query filter params (name=network) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:10.456364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:10.457743 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:10.465695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 359/716] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:10.471861 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:10.491850 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c9076-90a1-4373-b705-7bd80d427125 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:10.560288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:10.561429 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:10.562926 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:10.563413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:10.567404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:10.567975 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:10.570445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:10.585606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 358/717] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:27:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:10.592073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:10.681242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c176-66b0-4c29-a742-9fbf57389a80 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:10.724696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274830ba-84df-4097-b458-986b66b24be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:10.725849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274830ba-84df-4097-b458-986b66b24be3 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:10.726653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274830ba-84df-4097-b458-986b66b24be3 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:10.727150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274830ba-84df-4097-b458-986b66b24be3 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:10.728098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274830ba-84df-4097-b458-986b66b24be3 None admin] RBAC: Adding query filter params (type=network) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:10.728752 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274830ba-84df-4097-b458-986b66b24be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:10.730573 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274830ba-84df-4097-b458-986b66b24be3 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:10.737852 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 360/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:10.780616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac539e55-3521-4180-9eb5-eb54d719c776 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:10.825888 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbe9f8-c1ff-4a5d-98a7-2b03681213b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:10.826589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbe9f8-c1ff-4a5d-98a7-2b03681213b1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:10.826877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbe9f8-c1ff-4a5d-98a7-2b03681213b1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:10.827156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbe9f8-c1ff-4a5d-98a7-2b03681213b1 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:10.827877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbe9f8-c1ff-4a5d-98a7-2b03681213b1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:10.828190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbe9f8-c1ff-4a5d-98a7-2b03681213b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:10.829654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbe9f8-c1ff-4a5d-98a7-2b03681213b1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:10.831556 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:10.837690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 359/719] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:27:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:11.732759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26479230-aa08-405f-b2d1-bd4f87f7b3a9 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:11.733801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26479230-aa08-405f-b2d1-bd4f87f7b3a9 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:11.734350 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26479230-aa08-405f-b2d1-bd4f87f7b3a9 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:11.736185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 361/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:11.742028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd24852-62e3-44f6-84e4-132307d6258d None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:11.742364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd24852-62e3-44f6-84e4-132307d6258d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:11.742684 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd24852-62e3-44f6-84e4-132307d6258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:11.767194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-0dd24852-62e3-44f6-84e4-132307d6258d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:11.768990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-0dd24852-62e3-44f6-84e4-132307d6258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:11.863181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 360/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:11.903140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26479230-aa08-405f-b2d1-bd4f87f7b3a9 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:11.951841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:12.032143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26479230-aa08-405f-b2d1-bd4f87f7b3a9 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:12.069999 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0e664-7e4c-450c-8882-c506e23b0f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:12.071215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0e664-7e4c-450c-8882-c506e23b0f77 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.071726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0e664-7e4c-450c-8882-c506e23b0f77 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.072216 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0e664-7e4c-450c-8882-c506e23b0f77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.084452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0e664-7e4c-450c-8882-c506e23b0f77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-79e0e664-7e4c-450c-8882-c506e23b0f77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:12.090449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:12.091483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:12.092398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:12.095231 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 362/722] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:12.101522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd24852-62e3-44f6-84e4-132307d6258d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:12.158132 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7e4de-15d5-4d9f-aaa4-d24ba8c9cd2c None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.158132 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7e4de-15d5-4d9f-aaa4-d24ba8c9cd2c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.158132 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7e4de-15d5-4d9f-aaa4-d24ba8c9cd2c None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.160626 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 363/723] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:12.165575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182dc5-079c-43b1-b3d5-7fa9cb1d3d61 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.165575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182dc5-079c-43b1-b3d5-7fa9cb1d3d61 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.165575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182dc5-079c-43b1-b3d5-7fa9cb1d3d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.182802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-6b182dc5-079c-43b1-b3d5-7fa9cb1d3d61 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:12.184163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-6b182dc5-079c-43b1-b3d5-7fa9cb1d3d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:12.267301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd24852-62e3-44f6-84e4-132307d6258d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:12.295420 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:12.311904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:12.312170 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 364/724] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:27:12.312577 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.312860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.313132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.313871 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:12.318141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:12.318141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:12.334505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 361/725] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:27:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:12.340297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b182dc5-079c-43b1-b3d5-7fa9cb1d3d61 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:12.354618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:12.401026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:12.515676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e6702-a977-4b1f-9114-75073cf54169 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:12.534379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b182dc5-079c-43b1-b3d5-7fa9cb1d3d61 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:12.554675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2974cb9-18e5-4dde-8c42-751592ca4f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:12.555398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2974cb9-18e5-4dde-8c42-751592ca4f9e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.555844 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2974cb9-18e5-4dde-8c42-751592ca4f9e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.555981 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2974cb9-18e5-4dde-8c42-751592ca4f9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.568944 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2974cb9-18e5-4dde-8c42-751592ca4f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-c2974cb9-18e5-4dde-8c42-751592ca4f9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:12.577456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:12.578589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 362/726] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:12.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5d6ef-ed66-4701-86d4-4b7ef549f3e2 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5d6ef-ed66-4701-86d4-4b7ef549f3e2 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5d6ef-ed66-4701-86d4-4b7ef549f3e2 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.586235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 363/727] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:12.586235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5d6ef-ed66-4701-86d4-4b7ef549f3e2 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:12.591055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620df8eb-0b2e-4af6-953a-a44a76431c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:12.591971 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620df8eb-0b2e-4af6-953a-a44a76431c7f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.593503 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620df8eb-0b2e-4af6-953a-a44a76431c7f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.594057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620df8eb-0b2e-4af6-953a-a44a76431c7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.606576 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620df8eb-0b2e-4af6-953a-a44a76431c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:12.608242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620df8eb-0b2e-4af6-953a-a44a76431c7f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-620df8eb-0b2e-4af6-953a-a44a76431c7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:12.625065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:12.626312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:12.627276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:12.627276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:12.627276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:12.627276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:12.632104 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 365/728] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:12.633961 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f46110-a099-4e25-98dd-67a2115710cf None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.634358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f46110-a099-4e25-98dd-67a2115710cf None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.634726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f46110-a099-4e25-98dd-67a2115710cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.653706 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:12.655379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-b7f46110-a099-4e25-98dd-67a2115710cf None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:12.656707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-b7f46110-a099-4e25-98dd-67a2115710cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:12.714815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5d6ef-ed66-4701-86d4-4b7ef549f3e2 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:12.750478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 366/729] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:12.751721 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f46110-a099-4e25-98dd-67a2115710cf None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:12.761362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:12.762364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.762828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.763304 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.764230 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:12.764720 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:12.766491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:12.777139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 364/730] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:12.781903 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:12.783228 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:12.877656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f46110-a099-4e25-98dd-67a2115710cf None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:12.903957 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb02e0-4a9d-4ba2-8d2d-34767a2a0475 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:12.916300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:12.916964 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.917244 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.917519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.918559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:12.918694 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:12.920022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:12.923627 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:12.937069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 367/731] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:12.938175 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:12.950973 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:12.952100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:12.952592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:12.953089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:12.966287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:12.968201 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:12.985239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:12.986339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:12.987462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:12.987462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:12.987462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:12.987462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:12.994287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 365/732] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:12.995770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:13.021844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:13.058846 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a55a7e-3d01-4f96-892e-ca62ef130513 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.106686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86d713-8816-4394-8aeb-33d5e2d2043c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.107378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86d713-8816-4394-8aeb-33d5e2d2043c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.107657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86d713-8816-4394-8aeb-33d5e2d2043c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.107936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86d713-8816-4394-8aeb-33d5e2d2043c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.120520 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86d713-8816-4394-8aeb-33d5e2d2043c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-fc86d713-8816-4394-8aeb-33d5e2d2043c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:13.126276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:13.127512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:13.128744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:13.132790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 368/733] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:13.133800 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86d713-8816-4394-8aeb-33d5e2d2043c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:13.134179 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e9850-37bd-43c7-a47f-d0faf6d5bc49 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.180834 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.182386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.182893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.183367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.194550 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:13.196410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:13.211162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:13.212378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:13.213491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:13.216657 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 366/734] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:27:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:13.217855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:13.249601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:13.260239 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86d713-8816-4394-8aeb-33d5e2d2043c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.294519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:13.301125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.302639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.302980 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.303262 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.304032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:13.304355 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:13.305935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:13.325757 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 369/735] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:13.325757 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:13.353348 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30670c-9ea3-457b-ba25-a67a0f263771 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.407605 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.408610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.409096 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.409588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.410880 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:13.411366 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:13.413551 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:13.429832 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 367/736] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:13.431454 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:13.464502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b760e-2e62-4dc4-b900-b5490a732592 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.509734 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.510423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.510704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.510979 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.511893 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:13.512207 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:13.513770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:13.550169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 370/737] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:27:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:13.550169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:13.551907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:13.570021 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37652a-6131-4e4a-8eb4-817371103147 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.612436 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c430-6a52-480f-9446-942d565cf733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.613208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c430-6a52-480f-9446-942d565cf733 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.613524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c430-6a52-480f-9446-942d565cf733 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.613886 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c430-6a52-480f-9446-942d565cf733 None admin] PATH_INFO: `/v3/users/swift` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.619942 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:13.624100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472c430-6a52-480f-9446-942d565cf733 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:13.626238 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472c430-6a52-480f-9446-942d565cf733 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-8472c430-6a52-480f-9446-942d565cf733 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:13.646535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:13.647895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:13.649121 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 368/738] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:13.650610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469693f8-e852-40f2-a35c-f123954e215d None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.651046 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469693f8-e852-40f2-a35c-f123954e215d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.651520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469693f8-e852-40f2-a35c-f123954e215d None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.653995 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 369/739] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:13.655989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e48b5-599e-4d91-9e60-ca7fbcc1b407 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.656440 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e48b5-599e-4d91-9e60-ca7fbcc1b407 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.656879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e48b5-599e-4d91-9e60-ca7fbcc1b407 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.659046 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 370/740] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:13.660193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e48b5-599e-4d91-9e60-ca7fbcc1b407 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:13.713916 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a624b-56bc-4e07-8260-5ba2c9005204 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.763171 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.764016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.764163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.764462 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.776599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:13.778360 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:13.796785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:13.798748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:13.800802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:13.800802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:13.800802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:13.800802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:13.800802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 371/741] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:13.800802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:13.819577 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e48b5-599e-4d91-9e60-ca7fbcc1b407 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.865576 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5db22-1dea-4c06-bcab-1dc5b7545a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.866640 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5db22-1dea-4c06-bcab-1dc5b7545a24 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.867131 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5db22-1dea-4c06-bcab-1dc5b7545a24 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.867611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5db22-1dea-4c06-bcab-1dc5b7545a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.888286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:13.927993 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:13.932090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88446fe3-41f1-4d7b-9bb2-3343d4ae882b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:13.951575 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5db22-1dea-4c06-bcab-1dc5b7545a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:13.953264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5db22-1dea-4c06-bcab-1dc5b7545a24 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:13.978329 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcb3c3-f28d-4629-9004-afd372b1ac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:13.979320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb3c3-f28d-4629-9004-afd372b1ac4f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:13.979764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb3c3-f28d-4629-9004-afd372b1ac4f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:13.980203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb3c3-f28d-4629-9004-afd372b1ac4f None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:13.981572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcb3c3-f28d-4629-9004-afd372b1ac4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:13.982070 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcb3c3-f28d-4629-9004-afd372b1ac4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:13.984252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcb3c3-f28d-4629-9004-afd372b1ac4f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:14.005039 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 372/742] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:27:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:14.006456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2825-30cc-4e1e-96e6-435e544545ef None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:14.006758 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2825-30cc-4e1e-96e6-435e544545ef None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:14.007001 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2825-30cc-4e1e-96e6-435e544545ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:14.027516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-4c4e2825-30cc-4e1e-96e6-435e544545ef None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:14.028956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-4c4e2825-30cc-4e1e-96e6-435e544545ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:14.112570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 373/743] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:14.113798 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34801a4-63b8-48e1-bf71-d27f9220134e None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:14.114083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34801a4-63b8-48e1-bf71-d27f9220134e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:14.114420 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34801a4-63b8-48e1-bf71-d27f9220134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:14.125367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 371/744] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:27:14.125367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5db22-1dea-4c06-bcab-1dc5b7545a24 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:14.142575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-c34801a4-63b8-48e1-bf71-d27f9220134e None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:14.143922 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-c34801a4-63b8-48e1-bf71-d27f9220134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:14.212175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:14.227685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 374/745] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:27:14.229069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34801a4-63b8-48e1-bf71-d27f9220134e None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:14.256051 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:14.293736 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5db22-1dea-4c06-bcab-1dc5b7545a24 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:14.343958 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:14.344986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:14.345517 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:14.345878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:14.347189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:14.347667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:14.350155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:14.377954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 372/746] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:14.377954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:14.392593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34801a4-63b8-48e1-bf71-d27f9220134e None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:14.436714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a660cbc-6d15-4047-94f5-27cfd08a3a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:14.438761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660cbc-6d15-4047-94f5-27cfd08a3a9c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:14.438761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660cbc-6d15-4047-94f5-27cfd08a3a9c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:14.438761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660cbc-6d15-4047-94f5-27cfd08a3a9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:14.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1faa57-833a-4b81-bf01-f2d0ee8eeea1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:14.535623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:14.535941 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a660cbc-6d15-4047-94f5-27cfd08a3a9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:14.537960 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a660cbc-6d15-4047-94f5-27cfd08a3a9c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:14.562523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:14.581375 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:14.582473 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:14.582964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:14.583388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:14.602941 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:14.605543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:14.632643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:14.633791 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:14.634954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:14.634954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:14.634954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:14.634954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:14.635242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 373/747] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:14.636628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:14.670635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 375/748] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:27:14.672072 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a660cbc-6d15-4047-94f5-27cfd08a3a9c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:14.790530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a093f0-c649-4c84-b83f-82497384df41 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:14.802174 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a660cbc-6d15-4047-94f5-27cfd08a3a9c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:14.827740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:14.837308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8216e7b-318c-4eeb-8cdb-871d8c80671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:14.838249 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216e7b-318c-4eeb-8cdb-871d8c80671c None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:14.838827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216e7b-318c-4eeb-8cdb-871d8c80671c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:14.839288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216e7b-318c-4eeb-8cdb-871d8c80671c None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:14.840711 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8216e7b-318c-4eeb-8cdb-871d8c80671c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:14.842561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8216e7b-318c-4eeb-8cdb-871d8c80671c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:14.862377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde87e3-735a-4f74-a9ae-01bf56dc9f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:14.863404 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde87e3-735a-4f74-a9ae-01bf56dc9f48 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:14.863822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde87e3-735a-4f74-a9ae-01bf56dc9f48 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:14.864266 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde87e3-735a-4f74-a9ae-01bf56dc9f48 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:14.865173 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde87e3-735a-4f74-a9ae-01bf56dc9f48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:14.867958 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde87e3-735a-4f74-a9ae-01bf56dc9f48 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:14.868945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 374/749] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:27:14 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:14.869818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8216e7b-318c-4eeb-8cdb-871d8c80671c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:14.898212 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 376/750] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:27:14 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:14.900576 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde87e3-735a-4f74-a9ae-01bf56dc9f48 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:14.934823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:15.083120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8216e7b-318c-4eeb-8cdb-871d8c80671c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:15.135640 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:15.136679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:15.137388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:15.138168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] PATH_INFO: `/v3/users/nova` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:15.148110 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:15.150035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:15.161579 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde87e3-735a-4f74-a9ae-01bf56dc9f48 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:15.168318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:15.170583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:15.172612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:15.176373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 375/751] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:27:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:15.177896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:15.252299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:15.256548 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:15.256548 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:15.256548 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:15.256548 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:15.257439 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:15.259112 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:15.284475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 377/752] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:15.285434 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:15.331571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:15.371636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bc2f1-cdcd-46a1-be95-8e651019bf93 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:15.372803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:15.440825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:15.441818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:15.442186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:15.442571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:15.456142 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:15.458062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:15.475708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:15.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:15.479835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:15.479835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:15.479835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:15.479835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:15.480789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d506dd8-0fc8-4036-9f04-87f75ea63584 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:15.485924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 376/753] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:15.485924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:15.564327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:15.566015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:15.566857 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:15.567633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:15.569776 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:15.570726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:15.573692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:15.605686 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409fb26-d70d-47ed-a553-f01682e584b6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:15.612767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 378/754] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:27:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:15.614839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:15.634558 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:15.646240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:15.647114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:15.647505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:15.647947 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:15.656243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:15.658082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:15.672492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:15.674717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:15.676771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:15.676771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:15.676771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:15.676771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:15.676771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:15.676771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:15.676771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:15.676771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:15.680923 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 377/755] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:15.681964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:15.758594 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:15.766034 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a11fc-e95e-4992-9b11-008b1eb54b88 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:15.804422 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:15.805937 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:15.806468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:15.806942 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:15.807642 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d39ab-a624-44e5-9476-9948fb7ee7e7 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:15.808112 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:15.808634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:15.810289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:15.824056 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 379/756] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:15.825276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:15.856540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:15.857769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:15.858402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:15.858881 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:15.875477 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:15.877908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:15.897206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:15.898307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:15.899302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:15.899302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:15.899302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:15.899302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:15.902354 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 378/757] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:15.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:15.962762 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:15.984644 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebf6ce-f961-4f7c-9e46-a10b7ad5fe77 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:16.038209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:16.039127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:16.039559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:16.039998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:16.040928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:16.041394 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:16.041798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdd24d-b552-4295-81e1-f08ab038f098 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:16.043454 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:16.058317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 380/758] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:16.059485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:16.092149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:16.097363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:16.098644 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:16.099074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:16.099525 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:16.109973 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:16.112057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:16.128181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:16.129264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:16.130274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:16.130274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:16.130274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:16.130274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:16.130274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:16.130274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:16.130274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:16.130274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:16.134261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 379/759] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:16.135488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:16.221438 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78f17a-5bdb-47c4-a595-0dab3f2b331e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:16.276164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:16.282915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:16.283936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:16.284889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:16.285390 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:16.286902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:16.287405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:16.289358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:16.291924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488e88d-3bf7-4fae-92d3-6769b3f9eff7 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:16.313193 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 381/760] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:16.315015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:16.364839 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:16.366331 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:16.367066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:16.367828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:16.369587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=2569457ff8f6431daab71dd6083183a5) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:16.370435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:16.373395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:16.385004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 380/761] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:16 2022] GET /identity/v3/role_assignments?user.id=2569457ff8f6431daab71dd6083183a5&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:16.386823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:16.548427 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:16.598529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488c32d-af82-48a9-9c70-dbd8e4c86860 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:16.628949 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd169b7-27dc-4be1-a2cb-6b785b146df7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:16.668422 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:16.669446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:16.669880 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:16.670342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:16.683099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:16.685566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:16.706869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:16.707922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:16.708905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:16.708905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:16.708905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:16.708905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:16.708905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:16.708905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:16.708905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:16.708905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:16.709386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 381/762] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:16.716666 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:16.724212 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:16.725307 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:16.725815 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:16.726435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:16.727944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:16.728504 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:16.731450 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:16.756903 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 382/763] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:16.766384 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:16.783778 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:16.995605 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb25f2-e0dc-401b-ae3e-5f3b53edde1c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:17.047515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1a917-5256-4ee6-83dd-f6482daa313b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:17.058795 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b519e-9e93-4807-8281-b0844386793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:17.059994 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b519e-9e93-4807-8281-b0844386793a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:17.060601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b519e-9e93-4807-8281-b0844386793a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:17.061085 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b519e-9e93-4807-8281-b0844386793a None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:17.062378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b519e-9e93-4807-8281-b0844386793a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:17.062845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b519e-9e93-4807-8281-b0844386793a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:17.065096 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b519e-9e93-4807-8281-b0844386793a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:17.087437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 382/764] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:17.093730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b519e-9e93-4807-8281-b0844386793a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:17.182438 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f7769-fd3a-4d4f-896f-a524043986ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:17.183516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7769-fd3a-4d4f-896f-a524043986ca None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:17.183994 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7769-fd3a-4d4f-896f-a524043986ca None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:17.184472 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7769-fd3a-4d4f-896f-a524043986ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:17.185965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f7769-fd3a-4d4f-896f-a524043986ca None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=ec86b4f3ad1d4f75a685c4ff209051ce) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:17.186544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f7769-fd3a-4d4f-896f-a524043986ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:17.189405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f7769-fd3a-4d4f-896f-a524043986ca None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:17.208655 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:17.222025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 383/765] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:16 2022] GET /identity/v3/role_assignments?user.id=ec86b4f3ad1d4f75a685c4ff209051ce&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 259 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:17.280237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:17.321923 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b519e-9e93-4807-8281-b0844386793a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:17.385351 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90073184-9f5e-4f39-9444-dd4354d6acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:17.386376 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90073184-9f5e-4f39-9444-dd4354d6acca None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:17.386685 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90073184-9f5e-4f39-9444-dd4354d6acca None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:17.387141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90073184-9f5e-4f39-9444-dd4354d6acca None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/c5ba9e7780f5437caac949931ca55b80/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:17.390513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90073184-9f5e-4f39-9444-dd4354d6acca None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=c5ba9e7780f5437caac949931ca55b80, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:17.393191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90073184-9f5e-4f39-9444-dd4354d6acca None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:17.422155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 383/766] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:27:17 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/c5ba9e7780f5437caac949931ca55b80/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:27:17.664604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac236f3-4b5c-4138-aef6-62a0f60a0bda None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:17.665075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac236f3-4b5c-4138-aef6-62a0f60a0bda None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:17.665564 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac236f3-4b5c-4138-aef6-62a0f60a0bda None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:17.668762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 384/767] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:17.674398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca893081-d1fd-424e-a993-c869a25eeb4f None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:17.674743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca893081-d1fd-424e-a993-c869a25eeb4f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:17.675035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca893081-d1fd-424e-a993-c869a25eeb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:17.704164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-ca893081-d1fd-424e-a993-c869a25eeb4f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:17.705449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-ca893081-d1fd-424e-a993-c869a25eeb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:17.810471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 384/768] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:17.829425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac236f3-4b5c-4138-aef6-62a0f60a0bda None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:17.858387 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a9fe-134c-4ed3-b381-87212dbd2437 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:17.858752 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a9fe-134c-4ed3-b381-87212dbd2437 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:17.858926 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a9fe-134c-4ed3-b381-87212dbd2437 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:17.860688 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 385/769] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:17.882471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83367e-e998-409f-90da-0e5de9665eac None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:17.882807 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83367e-e998-409f-90da-0e5de9665eac None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:17.883136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83367e-e998-409f-90da-0e5de9665eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:17.902040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-0f83367e-e998-409f-90da-0e5de9665eac None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:17.903389 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-0f83367e-e998-409f-90da-0e5de9665eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:17.976790 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 386/770] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:17.978532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:18.028456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83367e-e998-409f-90da-0e5de9665eac None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:18.097880 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac236f3-4b5c-4138-aef6-62a0f60a0bda None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:18.164020 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83367e-e998-409f-90da-0e5de9665eac None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:18.167198 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:18.167884 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:18.168172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:18.168452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] PATH_INFO: `/v3/services/network` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:18.169103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:18.170557 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:18.195292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:18.196471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:18.197416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:18.200670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 385/771] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:17 2022] GET /identity/v3/services/network => generated 88 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:18.200914 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:18.206129 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:18.206129 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:18.206129 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:18.207963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:18.208601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:18.209520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:18.235489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:18.236638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:18.237647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 387/772] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:27:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:18.249816 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:18.348788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:18.419710 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8639da-e18e-4619-8345-870e2ae6d2e6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:18.450568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:18.483624 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:18.484707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:18.485198 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:18.485726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:18.486885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] RBAC: Adding query filter params (name=network) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:18.487422 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:18.489917 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:18.499506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 386/773] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:18.506641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:18.572311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d056779-733b-414b-99c8-09b49d23bb3d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:18.686873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:18.687956 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:18.688486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:18.688999 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:18.689907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:18.690523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:18.692149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:18.710288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 388/774] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:27:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:18.716286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:18.716436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e4ac-5c38-4467-b200-37ac95b11af8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:18.788939 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:18.810697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:18.811374 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:18.811660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:18.811939 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:18.812670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] RBAC: Adding query filter params (type=network) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:18.812987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:18.814472 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:18.828242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 387/775] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:18.833518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:18.951720 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ed963-e943-4b9b-8048-569e3d3b5456 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:18.988465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca20eb5-8a42-4443-adb0-e2ead7786f31 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:18.998514 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:19.013419 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:19.017362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.017362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.017362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.017362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:19.017362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:19.017362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:19.038463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 389/776] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:27:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:19.039833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:19.059784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:19.060594 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.060878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.061205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.061974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] RBAC: Adding query filter params (interface=public, service_id=4b4091ed229c4bf7930849214fa169b1, region_id=RegionOne) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:19.062312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:19.063698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:19.088980 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 388/777] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:27:18 2022] GET /identity/v3/endpoints?service_id=4b4091ed229c4bf7930849214fa169b1&interface=public®ion_id=RegionOne => generated 194 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:19.111865 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:19.229135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a40ba3-e12b-4ca9-8992-283c97dc8151 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:19.230680 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:19.268066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:19.268764 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.269102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.269378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.270184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] RBAC: Adding query filter params (interface=public, service_id=df3b8be0042543a9a905be0f9a1679ec, region_id=RegionOne) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:19.270510 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:19.271931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:19.280707 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 390/778] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:27:19 2022] GET /identity/v3/endpoints?service_id=df3b8be0042543a9a905be0f9a1679ec&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:19.286834 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:19.349930 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53286c3a-0732-43ac-9124-c75d43921788 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:19.361227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:19.402580 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d1449-0dd1-4a3b-95fc-97d5e521fa5c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:19.423537 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7dec-c9f3-41fb-96e5-a1b02acdc85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:19.424192 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7dec-c9f3-41fb-96e5-a1b02acdc85b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.424478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7dec-c9f3-41fb-96e5-a1b02acdc85b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.424808 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7dec-c9f3-41fb-96e5-a1b02acdc85b None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.425598 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e7dec-c9f3-41fb-96e5-a1b02acdc85b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:19.427061 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e7dec-c9f3-41fb-96e5-a1b02acdc85b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:19.433128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 389/779] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:27:19 2022] GET /identity/v3/services => generated 772 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:19.479539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ef4eb-70cb-4540-9a6d-b1a75a67bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:19.480252 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ef4eb-70cb-4540-9a6d-b1a75a67bc4a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.480458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ef4eb-70cb-4540-9a6d-b1a75a67bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.480748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ef4eb-70cb-4540-9a6d-b1a75a67bc4a None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.481446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ef4eb-70cb-4540-9a6d-b1a75a67bc4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:19.482892 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ef4eb-70cb-4540-9a6d-b1a75a67bc4a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:19.497317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 391/780] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:27:19 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:19.639136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda53be-6798-486b-95a4-114ba9b473cc None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.639528 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda53be-6798-486b-95a4-114ba9b473cc None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.639706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda53be-6798-486b-95a4-114ba9b473cc None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.641549 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 390/781] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:19.647972 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ed0c3-7959-45e8-8d0b-b3f4ce5a185b None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.648359 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ed0c3-7959-45e8-8d0b-b3f4ce5a185b None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.648536 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ed0c3-7959-45e8-8d0b-b3f4ce5a185b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.672693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-928ed0c3-7959-45e8-8d0b-b3f4ce5a185b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:19.673988 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-928ed0c3-7959-45e8-8d0b-b3f4ce5a185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:19.770057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16c2bd-aab7-412b-a9b3-f6360eca3981 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.770428 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16c2bd-aab7-412b-a9b3-f6360eca3981 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.770665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16c2bd-aab7-412b-a9b3-f6360eca3981 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.772180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 391/782] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:19.779562 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d6f1d-3dbf-42e5-9c2f-751303afdc7b None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:19.780276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d6f1d-3dbf-42e5-9c2f-751303afdc7b None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:19.780817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d6f1d-3dbf-42e5-9c2f-751303afdc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:19.794309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 392/783] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:19.806073 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-073d6f1d-3dbf-42e5-9c2f-751303afdc7b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:19.807841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-073d6f1d-3dbf-42e5-9c2f-751303afdc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:19.823222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ed0c3-7959-45e8-8d0b-b3f4ce5a185b None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:19.888691 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:19.915434 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 392/784] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:19.938692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d6f1d-3dbf-42e5-9c2f-751303afdc7b None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:19.969933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ed0c3-7959-45e8-8d0b-b3f4ce5a185b None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:20.022192 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:20.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ba7bc-0e4d-4c9c-81b6-2a4c83e5684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:20.034330 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ba7bc-0e4d-4c9c-81b6-2a4c83e5684b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.034617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ba7bc-0e4d-4c9c-81b6-2a4c83e5684b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.034895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ba7bc-0e4d-4c9c-81b6-2a4c83e5684b None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.057123 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ba7bc-0e4d-4c9c-81b6-2a4c83e5684b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-fb4ba7bc-0e4d-4c9c-81b6-2a4c83e5684b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:20.062325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:20.064718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:20.065825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 393/785] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:20.087761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ba7bc-0e4d-4c9c-81b6-2a4c83e5684b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:20.101647 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d6f1d-3dbf-42e5-9c2f-751303afdc7b None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:20.157539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332e9ad-cff1-4bb7-9e4d-848d0b522cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:20.158408 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332e9ad-cff1-4bb7-9e4d-848d0b522cb4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.158588 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332e9ad-cff1-4bb7-9e4d-848d0b522cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.158768 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332e9ad-cff1-4bb7-9e4d-848d0b522cb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.186037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332e9ad-cff1-4bb7-9e4d-848d0b522cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-1332e9ad-cff1-4bb7-9e4d-848d0b522cb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:20.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:20.199294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:20.200531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 393/786] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:20.208731 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332e9ad-cff1-4bb7-9e4d-848d0b522cb4 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:20.270213 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ba7bc-0e4d-4c9c-81b6-2a4c83e5684b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:20.280702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:20.344464 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:20.345127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.345445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.345730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.346620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:20.346941 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:20.348396 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:20.359223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 394/787] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:27:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:20.367257 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:20.409298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332e9ad-cff1-4bb7-9e4d-848d0b522cb4 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:20.432323 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:20.478724 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:20.479701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.480204 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.480683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.486131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:20.486131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:20.486131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:20.503156 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 394/788] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:20.514717 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:20.549535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7214ef-e328-4664-abac-165b35b6ff8a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:20.656656 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d194541-02c1-48c9-83a4-3bd91f8a83f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:20.657373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d194541-02c1-48c9-83a4-3bd91f8a83f1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.657662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d194541-02c1-48c9-83a4-3bd91f8a83f1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.657992 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d194541-02c1-48c9-83a4-3bd91f8a83f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.670941 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d194541-02c1-48c9-83a4-3bd91f8a83f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:20.672903 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d194541-02c1-48c9-83a4-3bd91f8a83f1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-3d194541-02c1-48c9-83a4-3bd91f8a83f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:20.690564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:20.691628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:20.692617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:20.692617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:20.692617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:20.692617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:20.692617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 395/789] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:20.694018 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1638906-7243-4f48-971b-562915909550 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.699628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1638906-7243-4f48-971b-562915909550 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.700085 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1638906-7243-4f48-971b-562915909550 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.701993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 396/790] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:20.702865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1638906-7243-4f48-971b-562915909550 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:20.745802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a70f9-f8c7-4aaa-b6ca-5aed8d5692dd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:20.751136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:20.816104 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4b7fd-c68c-4952-87c8-1f89b5f30cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:20.817235 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b7fd-c68c-4952-87c8-1f89b5f30cab None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.817745 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b7fd-c68c-4952-87c8-1f89b5f30cab None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.818411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b7fd-c68c-4952-87c8-1f89b5f30cab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.823621 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1638906-7243-4f48-971b-562915909550 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:20.841273 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4b7fd-c68c-4952-87c8-1f89b5f30cab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:20.844311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4b7fd-c68c-4952-87c8-1f89b5f30cab None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:20.864284 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:20.864981 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.865281 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.865582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.866412 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:20.866753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:20.868306 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-12c4b7fd-c68c-4952-87c8-1f89b5f30cab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:20.877991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:20.879402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:20.880523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:20.880523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:20.880523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:20.880523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:20.881658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 395/791] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:20.883710 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561098c2-295d-4872-80c9-90698fb56024 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:20.890134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561098c2-295d-4872-80c9-90698fb56024 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:20.890134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561098c2-295d-4872-80c9-90698fb56024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:20.890561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 397/792] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:20.890561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:20.918772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-561098c2-295d-4872-80c9-90698fb56024 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:20.920685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-561098c2-295d-4872-80c9-90698fb56024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:20.946951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:21.016438 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 396/793] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:21.018539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561098c2-295d-4872-80c9-90698fb56024 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:21.075350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:21.100011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91de847-68b5-476c-9e75-78b9954fc610 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:21.141414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:21.142495 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:21.143026 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:21.143508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:21.144476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:21.144951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:21.146733 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:21.162323 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 398/794] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:21.163578 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:21.220304 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561098c2-295d-4872-80c9-90698fb56024 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:21.263388 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:21.264463 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:21.264463 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:21.264771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:21.273816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:21.275229 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:21.289008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:21.290506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:21.291887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:21.295163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 397/795] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:27:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:21.296236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:21.315056 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:21.334164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f93de0-c4d3-4d4e-8f5c-3f67d616ee10 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:21.392779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ab1a3-d9be-451a-80b8-9160bae44091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:21.393895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ab1a3-d9be-451a-80b8-9160bae44091 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:21.394513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ab1a3-d9be-451a-80b8-9160bae44091 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:21.394942 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ab1a3-d9be-451a-80b8-9160bae44091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:21.408089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ab1a3-d9be-451a-80b8-9160bae44091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-0d7ab1a3-d9be-451a-80b8-9160bae44091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:21.414607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:21.415897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:21.416833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:21.419785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 399/796] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:21.420849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ab1a3-d9be-451a-80b8-9160bae44091 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:21.429977 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:21.433342 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da72a21-5fb4-4e5a-a9a1-d19c58470ff8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:21.509332 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:21.511517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:21.512541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:21.513463 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:21.532088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:21.535614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:21.546693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ab1a3-d9be-451a-80b8-9160bae44091 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:21.561249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:21.563288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:21.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:21.571336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 398/797] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:21.573596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:21.598591 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:21.599933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:21.599933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:21.600461 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:21.601091 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:21.603295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:21.603295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:21.634153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 400/798] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:27:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:21.634153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:21.711446 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:21.748062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673baaa-7de5-47e5-99a1-3bd1e0042cd0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:21.757698 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:21.803616 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:21.805327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:21.806156 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:21.807036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:21.809054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:21.810006 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:21.813350 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:21.826180 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c96ba-b6ef-4ca5-8c4e-0bdfbaabafb6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:21.838381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 399/799] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:21.840690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:21.871078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:21.871883 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:21.872181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:21.872444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:21.873530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:21.873841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:21.875446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:21.902078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 401/800] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:27:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:21.903164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:22.056809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779dde-04aa-416b-ba12-7b1f371daf79 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:22.060398 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c334f39-62da-4bdf-8c9e-a2af75b7cc34 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:22.092807 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:22.123989 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:22.124685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.124975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.125234 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.129116 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:22.130695 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:22.132435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.133255 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.140783 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:22.142342 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:22.156396 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:22.160370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:22.160633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:22.162393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:22.165891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:22.165891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:22.165891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:22.165891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:22.167547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 400/801] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:22.167547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:22.200613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:22.202524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:22.210980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:22.210980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:22.210980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:22.210980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:22.210980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 402/802] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:22.210980 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:22.303531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092e398-12ad-4e17-b49f-6994637f1703 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:22.344823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d3ce0-fdf7-4ef1-a56e-4c902408c9e8 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:22.349693 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9950f-ec6b-4214-9324-c57ebbaf23f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:22.350518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9950f-ec6b-4214-9324-c57ebbaf23f9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.350954 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9950f-ec6b-4214-9324-c57ebbaf23f9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.351166 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9950f-ec6b-4214-9324-c57ebbaf23f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.371380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9950f-ec6b-4214-9324-c57ebbaf23f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:22.374036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9950f-ec6b-4214-9324-c57ebbaf23f9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:22.385075 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445c2a2-c4e0-4baf-959b-5ba2cc367832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:22.385732 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c2a2-c4e0-4baf-959b-5ba2cc367832 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.386009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c2a2-c4e0-4baf-959b-5ba2cc367832 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.386318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c2a2-c4e0-4baf-959b-5ba2cc367832 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.387038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445c2a2-c4e0-4baf-959b-5ba2cc367832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:22.387346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445c2a2-c4e0-4baf-959b-5ba2cc367832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:22.388775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445c2a2-c4e0-4baf-959b-5ba2cc367832 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-0ba9950f-ec6b-4214-9324-c57ebbaf23f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:22.401471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:22.402645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:22.403811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:22.403811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:22.403811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:22.403811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:22.404259 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 403/803] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:22.404612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021132-f283-4c85-87d3-40ae59ea0489 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.404899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021132-f283-4c85-87d3-40ae59ea0489 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.405246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021132-f283-4c85-87d3-40ae59ea0489 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.407542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 404/804] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:22.407542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c021132-f283-4c85-87d3-40ae59ea0489 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:22.413186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 401/805] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:22.415084 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbec45a-e47e-4a69-9ad6-55b56dad0b83 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.415593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbec45a-e47e-4a69-9ad6-55b56dad0b83 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.416047 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbec45a-e47e-4a69-9ad6-55b56dad0b83 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.418491 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 402/806] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:22.419685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbec45a-e47e-4a69-9ad6-55b56dad0b83 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:22.442004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:22.472892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:22.528790 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c021132-f283-4c85-87d3-40ae59ea0489 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:22.551573 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbec45a-e47e-4a69-9ad6-55b56dad0b83 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:22.568192 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a171e-f0fe-4387-be9a-5d49b73dbf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:22.569177 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a171e-f0fe-4387-be9a-5d49b73dbf47 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.569488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a171e-f0fe-4387-be9a-5d49b73dbf47 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.569772 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a171e-f0fe-4387-be9a-5d49b73dbf47 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.570578 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a171e-f0fe-4387-be9a-5d49b73dbf47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:22.570848 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a171e-f0fe-4387-be9a-5d49b73dbf47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:22.572338 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a171e-f0fe-4387-be9a-5d49b73dbf47 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:22.586607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 405/807] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:22.587985 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba827-c1f3-4f61-8edb-24ab1dcb5b0f None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.588283 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba827-c1f3-4f61-8edb-24ab1dcb5b0f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.588559 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba827-c1f3-4f61-8edb-24ab1dcb5b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.601671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:22.602754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.603165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.603637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.608010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-e84ba827-c1f3-4f61-8edb-24ab1dcb5b0f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:22.609253 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-e84ba827-c1f3-4f61-8edb-24ab1dcb5b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:22.613666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:22.615750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:22.631109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:22.633062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 403/808] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:22.635376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:22.697836 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 406/809] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:22.699397 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b166a3-bc40-4b14-95f9-8bbe2cbd8eec None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.699728 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b166a3-bc40-4b14-95f9-8bbe2cbd8eec None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.700004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b166a3-bc40-4b14-95f9-8bbe2cbd8eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.731105 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-08b166a3-bc40-4b14-95f9-8bbe2cbd8eec None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:22.732612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-08b166a3-bc40-4b14-95f9-8bbe2cbd8eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:22.756930 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:22.775057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:22.778125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49808694-2a3b-4359-ab57-0e6d540438af None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:22.818252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:22.819371 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:22.819661 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:22.819948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:22.820711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:22.821029 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:22.822494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:22.837366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 404/810] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:22.838293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:22.851969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 407/811] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:22.853396 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b166a3-bc40-4b14-95f9-8bbe2cbd8eec None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:22.972955 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ba4ce-2ae1-46a3-b1e4-4314611b5eee None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:22.998223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b166a3-bc40-4b14-95f9-8bbe2cbd8eec None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.018569 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.019299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.019613 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.019901 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] PATH_INFO: `/v3/users/nova` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.031322 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.040523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.041238 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.041619 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.041894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.042913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:23.043300 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.044781 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:23.045376 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:23.046756 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 405/812] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:27:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:23.048473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.062799 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 408/813] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:23.063787 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.081447 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:23.085768 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:23.172729 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a233-bc9d-4a5a-85a1-9e72839f317d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.203976 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72230ded-e8c9-4532-a7da-2cddd9459ca4 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.219142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.220229 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.220688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.221176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.222432 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.225073 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:23.242641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:23.243891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:27:23.244907 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:23.245856 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.245856 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.245856 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.245856 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.246226 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 406/814] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:23.246226 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.254664 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.257491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:23.272196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:23.273370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:23.274509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 409/815] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:23.275101 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.357077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:23.380135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:23.402520 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f14b0-fe32-4b24-b18f-d19f958e055f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.404705 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91ce3c-d16b-406a-be4c-a8ce8a4f6fc7 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.447942 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.448553 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.448863 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.449141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] PATH_INFO: `/v3/services/network` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.449879 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.451435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.455734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.456660 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.457063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.457513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.458816 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:23.459353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.461663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:23.462147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:23.463210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:23.464968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 407/816] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:23.467490 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.491197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 410/817] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:23.491197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.617887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4e5d8-b0e6-4092-b08b-1387ca4ef84f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.632418 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d27b46-47c3-4220-902d-69244518550f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.655390 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf00f5a-eaf9-4d84-bb9d-f8dd55c6e8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.656090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf00f5a-eaf9-4d84-bb9d-f8dd55c6e8e9 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.656373 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf00f5a-eaf9-4d84-bb9d-f8dd55c6e8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.656653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf00f5a-eaf9-4d84-bb9d-f8dd55c6e8e9 None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/2569457ff8f6431daab71dd6083183a5/roles/070199642db14984a418795ea5af54cb` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.667465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:23.673330 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf00f5a-eaf9-4d84-bb9d-f8dd55c6e8e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=2569457ff8f6431daab71dd6083183a5, role_id=070199642db14984a418795ea5af54cb)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.675220 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf00f5a-eaf9-4d84-bb9d-f8dd55c6e8e9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.677818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.678741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.679055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.679355 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.680587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:23.680995 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.684929 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:23.691029 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 411/818] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:23.691998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.706349 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 408/819] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:27:23 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/2569457ff8f6431daab71dd6083183a5/roles/070199642db14984a418795ea5af54cb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:27:23.707567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf00f5a-eaf9-4d84-bb9d-f8dd55c6e8e9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.818581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc3d5c-2884-4ae9-87fb-89c8039e0b82 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.862523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf00f5a-eaf9-4d84-bb9d-f8dd55c6e8e9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:23.864088 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.868486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.868486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.868486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.868486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:23.868486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.868486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.890432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 412/820] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:23.890432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.929604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:23.931232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:23.931232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:23.931546 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:23.932380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] RBAC: Adding query filter params (name=network) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:23.933852 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:23.934903 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:23.942145 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 409/821] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:23.943433 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:23.957619 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:24.022386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25041a-bf75-4bf7-b1ea-87469e4d03f5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:24.022627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:24.066812 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12035c25-50a1-4f8d-9d74-f01adcd46669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:24.067465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12035c25-50a1-4f8d-9d74-f01adcd46669 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:24.067744 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12035c25-50a1-4f8d-9d74-f01adcd46669 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:24.068015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12035c25-50a1-4f8d-9d74-f01adcd46669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:24.074176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530d406-bb04-4f9b-a949-2b2a470bd59f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:24.113846 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:24.114812 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:24.115237 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:24.115614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:24.116462 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:24.116874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:24.118614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:24.129541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 410/822] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:24.129541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:24.167693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12035c25-50a1-4f8d-9d74-f01adcd46669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:24.169275 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12035c25-50a1-4f8d-9d74-f01adcd46669 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:24.235136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:24.254460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4b341-55a3-4b85-8c09-4eee27cde1b7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:24.287187 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:24.294837 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 413/823] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:27:24.296118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12035c25-50a1-4f8d-9d74-f01adcd46669 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:24.301393 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de5f16-7cb5-4c64-9709-58d759bd3d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:24.302313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de5f16-7cb5-4c64-9709-58d759bd3d0a None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:24.302726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de5f16-7cb5-4c64-9709-58d759bd3d0a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:24.303161 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de5f16-7cb5-4c64-9709-58d759bd3d0a None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/ec86b4f3ad1d4f75a685c4ff209051ce/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:24.305554 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de5f16-7cb5-4c64-9709-58d759bd3d0a None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=ec86b4f3ad1d4f75a685c4ff209051ce, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:24.307587 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de5f16-7cb5-4c64-9709-58d759bd3d0a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:24.334167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 411/824] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:27:24 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/ec86b4f3ad1d4f75a685c4ff209051ce/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:27:24.334167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de5f16-7cb5-4c64-9709-58d759bd3d0a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:24.448280 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12035c25-50a1-4f8d-9d74-f01adcd46669 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:24.494336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de5f16-7cb5-4c64-9709-58d759bd3d0a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:24.510230 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:24.510479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:24.510932 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:24.511402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:24.512676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] RBAC: Adding query filter params (type=network) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:24.514913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:24.520640 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:24.530129 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 414/825] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:24.531775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:24.563515 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:24.573560 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fc4ed-af39-4b86-9a19-796ccb9079bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:24.574259 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fc4ed-af39-4b86-9a19-796ccb9079bf None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:24.574541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fc4ed-af39-4b86-9a19-796ccb9079bf None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:24.574818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fc4ed-af39-4b86-9a19-796ccb9079bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:24.577643 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fc4ed-af39-4b86-9a19-796ccb9079bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:24.577643 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fc4ed-af39-4b86-9a19-796ccb9079bf None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:24.625441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 412/826] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:27:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:24.626555 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fc4ed-af39-4b86-9a19-796ccb9079bf None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:24.738745 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:24.749458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4cf5-48ff-41ec-8653-6d7331e18762 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:24.828266 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fc4ed-af39-4b86-9a19-796ccb9079bf None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:24.875155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:24.875530 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6026f4d-c8b4-42f8-922a-39181f3dd36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:24.876258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6026f4d-c8b4-42f8-922a-39181f3dd36f None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:24.876544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6026f4d-c8b4-42f8-922a-39181f3dd36f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:24.876815 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6026f4d-c8b4-42f8-922a-39181f3dd36f None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:24.877767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6026f4d-c8b4-42f8-922a-39181f3dd36f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:24.878402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:24.879294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6026f4d-c8b4-42f8-922a-39181f3dd36f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:24.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:24.880237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:24.905901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:24.906767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 413/827] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:27:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:24.914509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:24.944921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:24.946873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:24.948818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:24.948818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:24.948818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:24.948818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:24.948818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 415/828] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:24.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6026f4d-c8b4-42f8-922a-39181f3dd36f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:25.079137 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:25.096524 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6026f4d-c8b4-42f8-922a-39181f3dd36f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:25.137146 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:25.137808 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:25.138091 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:25.138408 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:25.139140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:25.139447 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:25.140956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:25.156208 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 414/829] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:25.162180 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:25.249524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:25.294898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e175a-a4b8-47d1-b915-ea7ddb8fe866 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:25.334021 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:25.334965 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:25.335395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:25.335844 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:25.345365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:25.347291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:25.360137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:25.361371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:25.362663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:25.362663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:25.362663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:25.362663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:25.362663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:25.362663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:25.362663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:25.362663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:25.364961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 416/830] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:25.372169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:25.507838 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75af38-40ac-408b-9195-0203d78776b7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:25.549793 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59deb7c-7c5d-4cc9-b16a-6403f3ee8486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:25.550501 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59deb7c-7c5d-4cc9-b16a-6403f3ee8486 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:25.550784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59deb7c-7c5d-4cc9-b16a-6403f3ee8486 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:25.551061 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59deb7c-7c5d-4cc9-b16a-6403f3ee8486 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:25.551853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59deb7c-7c5d-4cc9-b16a-6403f3ee8486 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:25.552165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59deb7c-7c5d-4cc9-b16a-6403f3ee8486 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:25.553664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59deb7c-7c5d-4cc9-b16a-6403f3ee8486 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:25.567662 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 415/831] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:25.574283 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:25.721168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fed03-a372-4c8b-8dac-0befcbb78c8d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:25.753347 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:25.769286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176717c-c681-4c98-bbdf-5b8e6b81e667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:25.770727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176717c-c681-4c98-bbdf-5b8e6b81e667 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:25.771439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176717c-c681-4c98-bbdf-5b8e6b81e667 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:25.772046 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176717c-c681-4c98-bbdf-5b8e6b81e667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:25.773897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176717c-c681-4c98-bbdf-5b8e6b81e667 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=c5ba9e7780f5437caac949931ca55b80) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:25.774879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176717c-c681-4c98-bbdf-5b8e6b81e667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:25.777622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176717c-c681-4c98-bbdf-5b8e6b81e667 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:25.786969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 417/832] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:25 2022] GET /identity/v3/role_assignments?user.id=c5ba9e7780f5437caac949931ca55b80&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:26.516974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045eda-1dee-4296-b9cb-afe78c616cdf None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:26.517780 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045eda-1dee-4296-b9cb-afe78c616cdf None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:26.517780 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045eda-1dee-4296-b9cb-afe78c616cdf None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:26.525871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 416/833] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:26.532695 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7c9e-681d-40b7-b45a-a303f26c65b6 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:26.533016 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7c9e-681d-40b7-b45a-a303f26c65b6 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:26.533310 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7c9e-681d-40b7-b45a-a303f26c65b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:26.552950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-901a7c9e-681d-40b7-b45a-a303f26c65b6 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:26.554233 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-901a7c9e-681d-40b7-b45a-a303f26c65b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:26.628077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 418/834] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:26.653812 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35045eda-1dee-4296-b9cb-afe78c616cdf None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:26.669463 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:26.719135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3534173-b613-459b-99b7-1f6a1a635f40 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:26.719531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3534173-b613-459b-99b7-1f6a1a635f40 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:26.719861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3534173-b613-459b-99b7-1f6a1a635f40 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:26.725823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 419/835] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:26.728506 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d558ee-82ae-40d4-8714-6b0d7ca66146 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:26.728971 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d558ee-82ae-40d4-8714-6b0d7ca66146 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:26.729316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d558ee-82ae-40d4-8714-6b0d7ca66146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:26.754473 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-92d558ee-82ae-40d4-8714-6b0d7ca66146 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:26.756021 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-92d558ee-82ae-40d4-8714-6b0d7ca66146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:26.834529 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35045eda-1dee-4296-b9cb-afe78c616cdf None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:26.866381 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 420/836] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:26.888902 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d558ee-82ae-40d4-8714-6b0d7ca66146 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:26.916462 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f778a-a4aa-402d-8a65-9d790a7663a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:26.918470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f778a-a4aa-402d-8a65-9d790a7663a0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:26.919488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f778a-a4aa-402d-8a65-9d790a7663a0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:26.920211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f778a-a4aa-402d-8a65-9d790a7663a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:26.942746 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f778a-a4aa-402d-8a65-9d790a7663a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-a37f778a-a4aa-402d-8a65-9d790a7663a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:26.950698 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:26.951805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:27:26.952726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:26.955417 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 417/837] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:27:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:26.962477 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f778a-a4aa-402d-8a65-9d790a7663a0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:26.967037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:27.023242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d558ee-82ae-40d4-8714-6b0d7ca66146 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:27.063326 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e794a9c-008f-45d5-8c85-0cab8a2b0c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:27.064363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e794a9c-008f-45d5-8c85-0cab8a2b0c2e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.064812 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e794a9c-008f-45d5-8c85-0cab8a2b0c2e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.065265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e794a9c-008f-45d5-8c85-0cab8a2b0c2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.078739 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e794a9c-008f-45d5-8c85-0cab8a2b0c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:27.080770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-5e794a9c-008f-45d5-8c85-0cab8a2b0c2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:27.087227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:27.088582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:27.089683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:27.094746 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 421/838] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:27.098341 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e794a9c-008f-45d5-8c85-0cab8a2b0c2e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:27.138269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f778a-a4aa-402d-8a65-9d790a7663a0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:27.193193 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:27.194551 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.195021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.195506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.196545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:27.197101 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:27.198780 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:27.209735 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 418/839] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:27:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:27.215372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:27.236152 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e794a9c-008f-45d5-8c85-0cab8a2b0c2e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:27.262916 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:27.297628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:27.298335 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.298641 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.298928 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.300108 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:27.300445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:27.302357 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:27.318604 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 422/840] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:27.319951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:27.382256 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ff5cf-6fed-405f-84d3-d20c2203a80a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:27.427405 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:27.434959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c48282-17c3-4476-830b-3f04f0c1acb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:27.436076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48282-17c3-4476-830b-3f04f0c1acb3 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.436515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48282-17c3-4476-830b-3f04f0c1acb3 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.437107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48282-17c3-4476-830b-3f04f0c1acb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.463917 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c48282-17c3-4476-830b-3f04f0c1acb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:27.466076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c48282-17c3-4476-830b-3f04f0c1acb3 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:27.476711 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccd290-ac7a-4b6c-a255-c6292bfdd3a3 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-63c48282-17c3-4476-830b-3f04f0c1acb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:27.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:27.486096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:27.487441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:27.487441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:27.487441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:27.487441 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:27.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 419/841] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:27.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902ddc9-7782-400c-baf0-304130bdff6d None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902ddc9-7782-400c-baf0-304130bdff6d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902ddc9-7782-400c-baf0-304130bdff6d None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.495378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 420/842] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:27.496884 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902ddc9-7782-400c-baf0-304130bdff6d None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:27.534058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b9e22-265f-4ce2-9fbb-b739d577515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:27.534862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9e22-265f-4ce2-9fbb-b739d577515a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.535152 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9e22-265f-4ce2-9fbb-b739d577515a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.535425 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9e22-265f-4ce2-9fbb-b739d577515a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.549759 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b9e22-265f-4ce2-9fbb-b739d577515a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:27.551298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b9e22-265f-4ce2-9fbb-b739d577515a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-f14b9e22-265f-4ce2-9fbb-b739d577515a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:27.569584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:27.570904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:27.572078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:27.572078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:27.572078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:27.572078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:27.572078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 423/843] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:27.572430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f711d-371e-4c62-b936-9b41742187a3 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.572430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f711d-371e-4c62-b936-9b41742187a3 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.573096 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f711d-371e-4c62-b936-9b41742187a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.599103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-3f8f711d-371e-4c62-b936-9b41742187a3 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:27.600790 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-3f8f711d-371e-4c62-b936-9b41742187a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:27.626877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:27.667833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902ddc9-7782-400c-baf0-304130bdff6d None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:27.713079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 424/844] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:27.713079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f711d-371e-4c62-b936-9b41742187a3 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:27.724994 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:27.726293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.726887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.727473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.728881 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:27.729543 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:27.732486 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:27.753700 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 421/845] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:27.755727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:27.836900 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:27.866054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f711d-371e-4c62-b936-9b41742187a3 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:27.918044 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:27.919001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.919445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.919867 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.920755 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:27.921219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:27.921646 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcd08c-72cc-4fd5-8170-c73c2d6f3aae None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:27.923570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:27.952328 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 425/846] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:27.952328 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:27.983395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:27.983754 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:27.984745 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:27.985231 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:27.985626 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:27.986442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:27.988124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:27.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:28.000492 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:28.001741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:28.007324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 422/847] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:28.008082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:28.096802 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace8163-414e-46e4-a385-bf9c95977276 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:28.125466 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d6a2a-8fb6-4e92-8fda-0eab936e7426 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:28.137385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:28.138061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:28.139667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.140053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.140386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.151799 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:28.153324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:28.167543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:28.169337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 426/848] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:27:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:28.170898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:28.181169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfe819-7a51-4daf-9844-2a620844d1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:28.181872 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfe819-7a51-4daf-9844-2a620844d1cc None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.182188 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfe819-7a51-4daf-9844-2a620844d1cc None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.182466 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfe819-7a51-4daf-9844-2a620844d1cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.193176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfe819-7a51-4daf-9844-2a620844d1cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:28.195071 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfe819-7a51-4daf-9844-2a620844d1cc None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-10cfe819-7a51-4daf-9844-2a620844d1cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:28.209004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:28.215656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 423/849] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5bdd-27a3-4bf0-a4ac-5e2b70d4c7f8 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.217120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5bdd-27a3-4bf0-a4ac-5e2b70d4c7f8 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.218236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5bdd-27a3-4bf0-a4ac-5e2b70d4c7f8 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.218236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 424/850] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:28.218236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e5bdd-27a3-4bf0-a4ac-5e2b70d4c7f8 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:28.268308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:28.299961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817dbae7-2efe-415f-a263-7b72d34b8b53 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:28.338601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e5bdd-27a3-4bf0-a4ac-5e2b70d4c7f8 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:28.339455 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:28.340176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.340551 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.351103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.351103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:28.351103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:28.351103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:28.355806 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 427/851] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:28.356721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:28.380162 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a207cb1-a813-4678-9375-a72b1b410713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:28.380816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a207cb1-a813-4678-9375-a72b1b410713 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.381096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a207cb1-a813-4678-9375-a72b1b410713 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.381372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a207cb1-a813-4678-9375-a72b1b410713 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.382223 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a207cb1-a813-4678-9375-a72b1b410713 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:28.382591 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a207cb1-a813-4678-9375-a72b1b410713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:28.384008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a207cb1-a813-4678-9375-a72b1b410713 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:28.402543 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 425/852] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:28.403870 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07390c-9883-494e-8912-fdfffd10ca14 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.404231 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07390c-9883-494e-8912-fdfffd10ca14 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.404666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07390c-9883-494e-8912-fdfffd10ca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.424011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-bb07390c-9883-494e-8912-fdfffd10ca14 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:28.425171 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-bb07390c-9883-494e-8912-fdfffd10ca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:28.453145 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:28.488642 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b7fcd-124d-40ce-8a16-636faff45312 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:28.530134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 426/853] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:28.530134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07390c-9883-494e-8912-fdfffd10ca14 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:28.530609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:28.530809 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.531022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.531294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.532165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:28.532474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:28.534194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:28.545550 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:28.555551 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 428/854] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:28.556568 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:28.639597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07390c-9883-494e-8912-fdfffd10ca14 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:28.677921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:28.678658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.678777 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.679074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.679823 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:28.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:28.681653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:28.686882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 427/855] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:28.687707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:28.700093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a8cab-25f9-4825-8de8-d05fb32ffa56 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:28.721019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:28.749393 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443cbe3-6a04-4a52-ad81-fc7593d802ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:28.750070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443cbe3-6a04-4a52-ad81-fc7593d802ad None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.750489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443cbe3-6a04-4a52-ad81-fc7593d802ad None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.750840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443cbe3-6a04-4a52-ad81-fc7593d802ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.848045 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:28.849032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f10f-8f8d-44ce-bced-fb0e57e2c7b1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:28.854812 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443cbe3-6a04-4a52-ad81-fc7593d802ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:28.856340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443cbe3-6a04-4a52-ad81-fc7593d802ad None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:28.911510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:28.912249 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:28.912482 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:28.913309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:28.913969 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:28.915837 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:28.931518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:28.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:28.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 428/856] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:28.936060 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:28.997350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 429/857] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:27:28.998427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443cbe3-6a04-4a52-ad81-fc7593d802ad None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:29.046064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:29.075326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8a73-b95a-426c-81e1-ece93fdd38d0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:29.126709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8ec44-5e8c-4635-8fa0-aa66a03e62c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:29.129245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8ec44-5e8c-4635-8fa0-aa66a03e62c5 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:29.129245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8ec44-5e8c-4635-8fa0-aa66a03e62c5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:29.129245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8ec44-5e8c-4635-8fa0-aa66a03e62c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:29.130828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:29.136679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443cbe3-6a04-4a52-ad81-fc7593d802ad None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:29.183705 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:29.185281 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:29.186414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:29.186414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:29.187422 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:29.187776 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:29.190055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:29.198476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 430/858] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:29.199687 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:29.242692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8ec44-5e8c-4635-8fa0-aa66a03e62c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:29.244737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8ec44-5e8c-4635-8fa0-aa66a03e62c5 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:29.357956 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d026052-2d5d-4212-bd9c-a8ca74693e32 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:29.381894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:29.404245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 429/859] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:27:29.404672 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:29.405132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:29.405390 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:29.405709 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:29.409454 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8ec44-5e8c-4635-8fa0-aa66a03e62c5 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:29.419079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:29.421098 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:29.439178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:29.440294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:29.441383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:29.441383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:29.441383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:29.441383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:29.441383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 431/860] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:29.441383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:29.462657 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:29.542267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8ec44-5e8c-4635-8fa0-aa66a03e62c5 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:29.587002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:29.587814 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:29.588083 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:29.588354 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:29.589043 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:29.589380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:29.590764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:29.597110 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 430/861] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:29.598225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:29.616431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01fec3-d72c-48e3-8dd2-05a6c2dc2079 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:29.663923 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:29.666717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:29.667604 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:29.668023 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:29.668405 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:29.683979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:29.685811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:29.705008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:29.706346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:29.707722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:29.707722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:29.707722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:29.707722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:29.713428 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 432/862] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:29.715077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:29.752101 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58436866-4774-4840-bf12-59afc854a5a6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:29.801063 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:29.802459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:29.802459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:29.802459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:29.803289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:29.803646 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:29.805234 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:29.826098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 431/863] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:29.827500 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:29.833900 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:29.892146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec036-b3e0-4291-983a-125b780571e5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:29.935827 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:29.936588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:29.937041 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:29.937386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:29.938431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:29.938825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:29.940611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:29.956126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 433/864] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:29.962320 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:29.969515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8434b73-3f1f-40fa-9b10-c28e567d19ab None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:29.982264 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:30.023253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:30.024301 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:30.024670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:30.025014 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:30.033689 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:30.035535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:30.049202 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:30.050458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:30.051559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:30.051559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:30.051559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:30.051559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:30.051559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:30.051559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:30.051559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:30.051559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:30.056271 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 432/865] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:30.057427 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:30.096586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc031098-8735-4380-b11c-aceff19d375f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:30.142174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:30.152687 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:30.153779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:30.153779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:30.154204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:30.163886 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:30.165522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:30.179938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:30.181386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:30.182679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:30.182679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:30.182679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:30.182679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:30.182679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:30.182679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:30.182679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:30.182679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:30.188856 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 434/866] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:30.189993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:30.202893 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb250317-7912-4681-9a59-47c1a181ed1b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:30.253997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:30.255414 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:30.255950 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:30.256409 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:30.257406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:30.257960 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:30.259782 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:30.289117 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 433/867] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:30.290378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:30.325195 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:30.332498 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cbb1a-c30a-47e7-9219-e5c836bf82b4 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:30.374571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:30.375673 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:30.375673 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:30.376049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:30.376647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:30.376967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:30.378555 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:30.398172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 435/868] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:30.399522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:30.424575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e81f8-4863-4dc7-a5f2-7f0750bc861f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:30.460051 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:30.467662 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94c231-9cd3-4168-908c-894bce093e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:30.468875 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94c231-9cd3-4168-908c-894bce093e27 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:30.469370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94c231-9cd3-4168-908c-894bce093e27 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:30.469893 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94c231-9cd3-4168-908c-894bce093e27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:30.470939 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94c231-9cd3-4168-908c-894bce093e27 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=2569457ff8f6431daab71dd6083183a5) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:30.471593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94c231-9cd3-4168-908c-894bce093e27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:30.473355 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94c231-9cd3-4168-908c-894bce093e27 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:30.486864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 434/869] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:30 2022] GET /identity/v3/role_assignments?user.id=2569457ff8f6431daab71dd6083183a5&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:30.533524 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976aef24-4a81-4de5-a012-ea1a07b3e7ab None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:30.583693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f36b3a-4459-4982-be6a-1de0acc6687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:30.584398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f36b3a-4459-4982-be6a-1de0acc6687e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:30.584704 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f36b3a-4459-4982-be6a-1de0acc6687e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:30.585010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f36b3a-4459-4982-be6a-1de0acc6687e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:30.585896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f36b3a-4459-4982-be6a-1de0acc6687e None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=ec86b4f3ad1d4f75a685c4ff209051ce) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:30.586282 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f36b3a-4459-4982-be6a-1de0acc6687e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:30.587947 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f36b3a-4459-4982-be6a-1de0acc6687e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:30.595699 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 436/870] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:30 2022] GET /identity/v3/role_assignments?user.id=ec86b4f3ad1d4f75a685c4ff209051ce&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:31.020903 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc85e4-1576-4944-80ba-cc6215fc244c None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:31.021481 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc85e4-1576-4944-80ba-cc6215fc244c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:31.021481 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc85e4-1576-4944-80ba-cc6215fc244c None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:31.023497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 435/871] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:31.029252 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b2a5-658c-47ae-b082-dbc3efea4366 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:31.029535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b2a5-658c-47ae-b082-dbc3efea4366 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:31.029811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b2a5-658c-47ae-b082-dbc3efea4366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:31.049212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-a857b2a5-658c-47ae-b082-dbc3efea4366 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:31.050546 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-a857b2a5-658c-47ae-b082-dbc3efea4366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:31.134312 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 437/872] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:31.156221 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc85e4-1576-4944-80ba-cc6215fc244c None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:31.265826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:31.277379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc85e4-1576-4944-80ba-cc6215fc244c None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:31.318306 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02989159-cae6-43a3-9092-2a4f0f363ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:31.319744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02989159-cae6-43a3-9092-2a4f0f363ed6 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:31.320228 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02989159-cae6-43a3-9092-2a4f0f363ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:31.320712 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02989159-cae6-43a3-9092-2a4f0f363ed6 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:31.321645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02989159-cae6-43a3-9092-2a4f0f363ed6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:31.323313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02989159-cae6-43a3-9092-2a4f0f363ed6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:31.342058 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 436/873] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:27:31 2022] POST /identity/v3/services => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:31.953671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b88f5-9840-4e93-9f7f-beab02da670e None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:31.954128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b88f5-9840-4e93-9f7f-beab02da670e None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:31.954251 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b88f5-9840-4e93-9f7f-beab02da670e None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:31.956068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 438/874] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:31.963253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f483d-f7cf-48a9-8b5f-59aedf7b3afd None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:31.963568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f483d-f7cf-48a9-8b5f-59aedf7b3afd None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:31.963927 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f483d-f7cf-48a9-8b5f-59aedf7b3afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:31.983687 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-917f483d-f7cf-48a9-8b5f-59aedf7b3afd None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:31.984919 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-917f483d-f7cf-48a9-8b5f-59aedf7b3afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:32.074510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 437/875] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:32.096948 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b88f5-9840-4e93-9f7f-beab02da670e None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:32.158966 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:32.210442 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b88f5-9840-4e93-9f7f-beab02da670e None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:32.247575 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bdfc6-e82b-451a-a95a-115e55153b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:32.248224 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bdfc6-e82b-451a-a95a-115e55153b12 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:32.248504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bdfc6-e82b-451a-a95a-115e55153b12 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:32.248779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bdfc6-e82b-451a-a95a-115e55153b12 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:32.249503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bdfc6-e82b-451a-a95a-115e55153b12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:32.250937 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bdfc6-e82b-451a-a95a-115e55153b12 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:32.272510 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 439/876] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:27:32 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:32.832283 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b61e2-85c0-4b11-96c1-81007eab6458 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:32.833045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b61e2-85c0-4b11-96c1-81007eab6458 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:32.833553 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b61e2-85c0-4b11-96c1-81007eab6458 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:32.835655 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 438/877] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:32.841728 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7310f7d-7439-4cbd-9f83-398ab21ad96d None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:32.842012 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7310f7d-7439-4cbd-9f83-398ab21ad96d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:32.842325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7310f7d-7439-4cbd-9f83-398ab21ad96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:32.861067 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25511c-289b-49df-9c19-32562ce41d91 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:32.861760 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25511c-289b-49df-9c19-32562ce41d91 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:32.862331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25511c-289b-49df-9c19-32562ce41d91 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:32.863513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-f7310f7d-7439-4cbd-9f83-398ab21ad96d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:32.864622 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 439/878] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:32.865885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-f7310f7d-7439-4cbd-9f83-398ab21ad96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:32.870878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eddcb7-34d3-4442-b420-b5ebf99434b4 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:32.871520 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eddcb7-34d3-4442-b420-b5ebf99434b4 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:32.872069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eddcb7-34d3-4442-b420-b5ebf99434b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:32.898659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-07eddcb7-34d3-4442-b420-b5ebf99434b4 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:32.900660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-07eddcb7-34d3-4442-b420-b5ebf99434b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:32.978155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 440/879] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:33.007295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7310f7d-7439-4cbd-9f83-398ab21ad96d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.018609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 440/880] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:33.039569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:33.060103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eddcb7-34d3-4442-b420-b5ebf99434b4 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.078491 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:33.153319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7310f7d-7439-4cbd-9f83-398ab21ad96d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:33.204252 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:33.205106 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.205288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.205620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.206633 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:33.208925 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:33.223606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:33.224810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:33.225741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:33.230730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 441/881] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:33.233951 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.247575 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eddcb7-34d3-4442-b420-b5ebf99434b4 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:33.312387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4b394-fa02-4c8c-814e-cbf173f4c318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:33.313208 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b394-fa02-4c8c-814e-cbf173f4c318 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.313663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b394-fa02-4c8c-814e-cbf173f4c318 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.314116 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b394-fa02-4c8c-814e-cbf173f4c318 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.332349 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4b394-fa02-4c8c-814e-cbf173f4c318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-74b4b394-fa02-4c8c-814e-cbf173f4c318 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:33.339636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:33.340867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:33.341931 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:33.346702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 441/882] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:33.350563 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4b394-fa02-4c8c-814e-cbf173f4c318 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.374933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:33.390413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e8159-0b04-418d-9271-2122a641a179 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:33.437753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:33.439243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.439784 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.440365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.441639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:33.442842 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:33.445303 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:33.453858 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 442/883] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:33.458688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:33.461560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.535967 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4b394-fa02-4c8c-814e-cbf173f4c318 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:33.572387 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51311e8e-4a23-477a-9a85-e7a95dae1d2b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:33.588102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:33.589163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.589473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.589800 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.590723 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:33.591075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:33.593075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:33.611614 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e56a3-2b63-4c46-b3fc-ae1e00019eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:33.612170 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 442/884] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:33.612979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e56a3-2b63-4c46-b3fc-ae1e00019eba None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.613477 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e56a3-2b63-4c46-b3fc-ae1e00019eba None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.613594 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.614087 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e56a3-2b63-4c46-b3fc-ae1e00019eba None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.615033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e56a3-2b63-4c46-b3fc-ae1e00019eba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:33.615559 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e56a3-2b63-4c46-b3fc-ae1e00019eba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:33.617551 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e56a3-2b63-4c46-b3fc-ae1e00019eba None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:33.623349 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 443/885] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:33.624891 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154f17-e8f4-4c31-aa78-5af3748474a2 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.625382 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154f17-e8f4-4c31-aa78-5af3748474a2 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.625893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154f17-e8f4-4c31-aa78-5af3748474a2 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.627853 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 444/886] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:33.642267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe76e00-111a-4f78-ab8f-ef16dc473aa1 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.642960 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe76e00-111a-4f78-ab8f-ef16dc473aa1 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.643536 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe76e00-111a-4f78-ab8f-ef16dc473aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.671107 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-4fe76e00-111a-4f78-ab8f-ef16dc473aa1 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:33.673823 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-4fe76e00-111a-4f78-ab8f-ef16dc473aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:33.696221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:33.770561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 445/887] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:33.773619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea2a69-116c-4b79-abd6-4d9b22ab65b9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:33.789605 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe76e00-111a-4f78-ab8f-ef16dc473aa1 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.828071 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:33.835002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:33.835682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.836294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.848737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:33.850359 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:33.866468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:33.867525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:33.868534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:33.868534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:33.868534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:33.868534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:33.868534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 443/888] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:33.873507 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.912243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe76e00-111a-4f78-ab8f-ef16dc473aa1 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:33.953731 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:33.954924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:33.955658 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:33.956155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:33.957074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:33.958936 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:33.969833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:33.971267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:33.972311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:33.981112 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 446/889] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:33.982513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:33.998074 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937abfc-c075-4a43-9b83-37a9893dc8ed None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:34.002738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:34.035645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:34.036324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.036612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.036887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.037624 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:34.037939 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:34.039364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:34.057932 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 444/890] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:34.058901 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:34.105076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879dca92-822f-4559-8be6-1b66dc22df6c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:34.106810 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:34.151170 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:34.152360 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.152959 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.153438 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.154385 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:34.154948 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:34.156935 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:34.169217 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5187f8-1e13-474a-9985-a8b2f85346a6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:34.169625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 447/891] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:34.171682 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:34.208141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:34.208805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.209050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.209325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.217765 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:34.219123 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:34.232596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:34.233845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:34.234918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:34.238837 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 445/892] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:27:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:34.239938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:34.273907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:34.304007 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5018726-91f8-4c65-a3c2-596eec37e3cd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:34.347549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:34.348833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.349363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.349955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.350912 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:34.351659 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:34.353361 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:34.365599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13093b0d-010b-4c5f-afab-07764523eee3 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:34.366275 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 448/893] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:34.367464 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:34.369375 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:34.415982 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:34.417242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.417780 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.418303 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.419571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:34.420091 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:34.421868 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:34.450304 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 446/894] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:27:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:34.452161 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:34.511149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40beba08-16a5-4d13-b027-4acdb3a7260b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:34.562318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333a7a9-8f7b-4728-8cbe-f5fc918748fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:34.563100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333a7a9-8f7b-4728-8cbe-f5fc918748fd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.563392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333a7a9-8f7b-4728-8cbe-f5fc918748fd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.563837 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333a7a9-8f7b-4728-8cbe-f5fc918748fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.564693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333a7a9-8f7b-4728-8cbe-f5fc918748fd None admin] RBAC: Adding query filter params (interface=public, service_id=4278ab7358f14e6390e3da510a061997, region_id=RegionOne) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:34.565082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333a7a9-8f7b-4728-8cbe-f5fc918748fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:34.566752 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333a7a9-8f7b-4728-8cbe-f5fc918748fd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:34.582290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 449/895] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:27:34 2022] GET /identity/v3/endpoints?service_id=4278ab7358f14e6390e3da510a061997&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:34.585055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446560-ab96-42ce-a5d0-65f06fbf0655 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.585394 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446560-ab96-42ce-a5d0-65f06fbf0655 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.585732 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446560-ab96-42ce-a5d0-65f06fbf0655 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.587639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 450/896] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:34.593122 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dbffd-7c22-4bec-9a0a-764d92866045 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:34.593288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b446560-ab96-42ce-a5d0-65f06fbf0655 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:34.598244 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:34.630333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad40b3-e2b1-4b02-af09-9b299d1d95e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:34.631366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad40b3-e2b1-4b02-af09-9b299d1d95e9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.631824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad40b3-e2b1-4b02-af09-9b299d1d95e9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.632402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad40b3-e2b1-4b02-af09-9b299d1d95e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.678527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:34.724716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad40b3-e2b1-4b02-af09-9b299d1d95e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:34.726253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad40b3-e2b1-4b02-af09-9b299d1d95e9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:34.742423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b446560-ab96-42ce-a5d0-65f06fbf0655 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:34.787379 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfe2a7-054e-4ec2-a8e8-ebf6109e6108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:34.788311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe2a7-054e-4ec2-a8e8-ebf6109e6108 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.788759 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe2a7-054e-4ec2-a8e8-ebf6109e6108 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.789239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe2a7-054e-4ec2-a8e8-ebf6109e6108 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.789913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfe2a7-054e-4ec2-a8e8-ebf6109e6108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:34.791404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfe2a7-054e-4ec2-a8e8-ebf6109e6108 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:34.800516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 451/897] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:27:34 2022] GET /identity/v3/services => generated 1268 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:27:34.802527 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468659c-354a-40b0-bbd0-1644ed5b1cab None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:34.802958 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468659c-354a-40b0-bbd0-1644ed5b1cab None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:34.803536 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468659c-354a-40b0-bbd0-1644ed5b1cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:34.838421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-d468659c-354a-40b0-bbd0-1644ed5b1cab None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:34.841661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-d468659c-354a-40b0-bbd0-1644ed5b1cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:34.870316 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 447/898] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:34 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:27:34.877503 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad40b3-e2b1-4b02-af09-9b299d1d95e9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:34.949613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:34.957502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:34.959486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 452/899] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:34.987120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468659c-354a-40b0-bbd0-1644ed5b1cab None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.018203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad40b3-e2b1-4b02-af09-9b299d1d95e9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:35.075238 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:35.076288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:35.076727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:35.077377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:35.101945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:35.104753 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:35.121231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468659c-354a-40b0-bbd0-1644ed5b1cab None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:35.132336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:35.134018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:35.135141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:35.135141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:35.135141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:35.135141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:35.135862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 448/900] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:35.140224 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.174993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:35.175936 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:35.176387 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:35.176797 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:35.177643 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:35.179275 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:35.190909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:35.192344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:35.194073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 453/901] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:27:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:35.199279 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.281939 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:35.324471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad2301-0198-42a6-a75c-d6477f749760 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:35.327664 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726325a-2950-4438-a9f5-14302ece80e6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:35.344211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:35.370940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:35.371350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:35.371713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:35.372015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:35.372750 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:35.373097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:35.374552 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:35.378845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:35.379841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:35.380341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:35.380851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:35.381943 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:35.382593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:35.384248 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:35.384956 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 454/902] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:35.385921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.404364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 449/903] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:35.409648 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.530943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966891-1a4b-4f76-8656-ff7ee2df199c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:35.556341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f5dcd-a7a3-43cc-aec5-ccd04d1c2379 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:35.571136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:35.592929 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:35.593492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:35.593825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:35.594099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:35.594855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:35.595164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:35.597298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:35.604836 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 455/904] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:35.614292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.619125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:35.619871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:35.620151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:35.620473 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:35.630406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:35.632169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:35.648727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:35.649822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:35.650847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:35.650847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:35.650847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:35.650847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:35.650847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:35.650847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:35.650847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:35.650847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:35.652315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 450/905] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:35.659771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.703883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:35.763821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5ff4d-f1ad-4d1d-a9ad-1158eae69705 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:35.788541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1b32-d29a-4db9-b124-a549b7945c0a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:35.802126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:35.802965 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:35.803391 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:35.803673 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] PATH_INFO: `/v3/endpoints` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:35.804445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] RBAC: Adding query filter params (interface=public, service_id=e023ad9dcc5b4620b555a7427c10e0b2, region_id=RegionOne) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:35.804758 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:35.806421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:35.814161 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 456/906] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:27:35 2022] GET /identity/v3/endpoints?service_id=e023ad9dcc5b4620b555a7427c10e0b2&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:35.817950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.834312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:35.835395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:35.835847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:35.836545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:35.838064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:35.838590 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:35.841114 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:35.862357 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 451/907] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:35.870086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:35.903257 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:35.957611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1ba09-562e-4e11-aae0-877d05120aff None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:36.000903 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786c7ac-f569-4a7a-9301-783b509859e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:36.001557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786c7ac-f569-4a7a-9301-783b509859e2 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:36.001840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786c7ac-f569-4a7a-9301-783b509859e2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:36.002135 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786c7ac-f569-4a7a-9301-783b509859e2 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:36.002886 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786c7ac-f569-4a7a-9301-783b509859e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:36.004294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786c7ac-f569-4a7a-9301-783b509859e2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:36.010620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 457/908] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:27:35 2022] GET /identity/v3/services => generated 1268 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:27:36.011918 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3828-4e54-4007-90af-30e19391a27a None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:36.012211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3828-4e54-4007-90af-30e19391a27a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:36.012488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3828-4e54-4007-90af-30e19391a27a None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:36.013983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 458/909] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:36.020043 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c43b76-54eb-4f98-b4dc-66a917188f59 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:36.020371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c43b76-54eb-4f98-b4dc-66a917188f59 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:36.020623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c43b76-54eb-4f98-b4dc-66a917188f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:36.028397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:36.041852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-73c43b76-54eb-4f98-b4dc-66a917188f59 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:36.043425 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-73c43b76-54eb-4f98-b4dc-66a917188f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:36.051419 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5252181-b5e6-43b8-b242-d88e7b23e764 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:36.117621 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:36.118342 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:36.118857 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:36.119218 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:36.120327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] RBAC: Adding query filter params (role.id=678a59243ee345488ef8c869ef84f779, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=2569457ff8f6431daab71dd6083183a5) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:36.120726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:36.122438 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:36.138149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 452/910] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:35 2022] GET /identity/v3/role_assignments?user.id=2569457ff8f6431daab71dd6083183a5&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=678a59243ee345488ef8c869ef84f779 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:36.174406 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 459/911] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:36.196122 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:36.360262 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff6ddd-560f-4900-9251-ab8ef32389eb None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:36.402814 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd612f17-e0d2-48e4-a943-46b6c84353b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:36.403607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd612f17-e0d2-48e4-a943-46b6c84353b6 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:36.403949 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd612f17-e0d2-48e4-a943-46b6c84353b6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:36.404300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd612f17-e0d2-48e4-a943-46b6c84353b6 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:36.405039 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd612f17-e0d2-48e4-a943-46b6c84353b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:36.406583 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd612f17-e0d2-48e4-a943-46b6c84353b6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:36.434372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 453/912] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:27:36 2022] POST /identity/v3/services => generated 241 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:37.115940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da05291-9a40-45a3-8381-f88aeff45a98 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:37.116548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da05291-9a40-45a3-8381-f88aeff45a98 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:37.116548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da05291-9a40-45a3-8381-f88aeff45a98 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:37.118180 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 460/913] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:37.124106 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b305c53-6e07-4a67-b2df-b160e488c569 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:37.124417 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b305c53-6e07-4a67-b2df-b160e488c569 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:37.124668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b305c53-6e07-4a67-b2df-b160e488c569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:37.144599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-6b305c53-6e07-4a67-b2df-b160e488c569 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:37.145829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-6b305c53-6e07-4a67-b2df-b160e488c569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:37.162289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:37.244653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 454/914] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:37.274467 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da05291-9a40-45a3-8381-f88aeff45a98 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:37.348065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:37.421182 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da05291-9a40-45a3-8381-f88aeff45a98 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:37.464496 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:37.465276 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:37.465561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:37.465836 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:37.466628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:37.468119 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:37.478901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:37.480163 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:37.481165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 461/915] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:37.487310 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b305c53-6e07-4a67-b2df-b160e488c569 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:37.601670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b305c53-6e07-4a67-b2df-b160e488c569 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:37.639624 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:37.640273 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:37.640552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:37.640824 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:37.641522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:37.641822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:37.643395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:37.649241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 455/916] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:37.656889 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:37.803018 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092dcee-17f7-4dbb-9031-8ed732a781c6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:37.817550 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:37.846779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d26249-24f1-4135-b86e-34a1276d863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:37.847969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26249-24f1-4135-b86e-34a1276d863b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:37.848485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26249-24f1-4135-b86e-34a1276d863b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:37.849049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26249-24f1-4135-b86e-34a1276d863b None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:37.849982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d26249-24f1-4135-b86e-34a1276d863b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:37.850639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d26249-24f1-4135-b86e-34a1276d863b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:37.852586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d26249-24f1-4135-b86e-34a1276d863b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:37.860026 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 462/917] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:27:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:37.867472 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:37.884179 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:38.047743 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d4f6d-9d67-4569-8db6-b9010c39f2fa None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:38.093364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b573bd4-02ca-4082-863a-42cab4f00ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:38.093923 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b573bd4-02ca-4082-863a-42cab4f00ba2 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.094238 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b573bd4-02ca-4082-863a-42cab4f00ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.094512 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b573bd4-02ca-4082-863a-42cab4f00ba2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.095246 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b573bd4-02ca-4082-863a-42cab4f00ba2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:38.096796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b573bd4-02ca-4082-863a-42cab4f00ba2 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:38.123279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 456/918] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:27:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:38.360726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddfad2-d102-4ae3-b4cb-b31dab88beb9 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.361877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddfad2-d102-4ae3-b4cb-b31dab88beb9 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.362483 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddfad2-d102-4ae3-b4cb-b31dab88beb9 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.364647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 463/919] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:38.371279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adc6b6-544c-4654-88a6-1086bd0f05bb None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.371860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adc6b6-544c-4654-88a6-1086bd0f05bb None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.372001 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adc6b6-544c-4654-88a6-1086bd0f05bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.396716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-c3adc6b6-544c-4654-88a6-1086bd0f05bb None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:38.398028 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-c3adc6b6-544c-4654-88a6-1086bd0f05bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:38.477570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:38.499246 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 457/920] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:38.532220 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddfad2-d102-4ae3-b4cb-b31dab88beb9 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:38.583124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa163f1-ce9e-4ee4-83a6-38316beb71a5 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.583124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa163f1-ce9e-4ee4-83a6-38316beb71a5 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.583124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa163f1-ce9e-4ee4-83a6-38316beb71a5 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.585632 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 458/921] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:38.594890 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd575a8-fb1c-46cd-b3e0-78b15e9ee40b None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.595354 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd575a8-fb1c-46cd-b3e0-78b15e9ee40b None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.595809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd575a8-fb1c-46cd-b3e0-78b15e9ee40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.621139 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-ffd575a8-fb1c-46cd-b3e0-78b15e9ee40b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:38.622300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-ffd575a8-fb1c-46cd-b3e0-78b15e9ee40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:38.670939 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddfad2-d102-4ae3-b4cb-b31dab88beb9 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:38.706183 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 459/922] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:38.713508 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7fa78-ce22-4aeb-b99f-3da170a9ba89 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.713801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7fa78-ce22-4aeb-b99f-3da170a9ba89 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.714080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7fa78-ce22-4aeb-b99f-3da170a9ba89 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.715620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:38.715763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 460/923] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:38.719143 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.719703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.720258 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.721412 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:38.722183 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9104-352a-4e46-903f-fd4d299fcfae None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.722535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9104-352a-4e46-903f-fd4d299fcfae None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.722817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9104-352a-4e46-903f-fd4d299fcfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.722927 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:38.733676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:38.738266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:38.741485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 464/924] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:27:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:38.747462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:38.758988 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-1ced9104-352a-4e46-903f-fd4d299fcfae None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:38.760291 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-1ced9104-352a-4e46-903f-fd4d299fcfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:38.798887 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:38.872675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f7351-d011-4b6a-bd63-c1435678b0cd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:38.898293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 461/925] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:38.899420 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced9104-352a-4e46-903f-fd4d299fcfae None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:38.928181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fe903-f325-4196-956a-ca03d8f3ab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:38.929261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fe903-f325-4196-956a-ca03d8f3ab4e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:38.929795 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fe903-f325-4196-956a-ca03d8f3ab4e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:38.930337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fe903-f325-4196-956a-ca03d8f3ab4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:38.951119 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fe903-f325-4196-956a-ca03d8f3ab4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-7e3fe903-f325-4196-956a-ca03d8f3ab4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:38.963066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:38.964913 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 465/926] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:27:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:38.966860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fe903-f325-4196-956a-ca03d8f3ab4e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:38.999094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:39.067908 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced9104-352a-4e46-903f-fd4d299fcfae None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:39.127145 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:39.128347 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:39.128860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:39.129267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:39.130658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:39.131223 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:39.132937 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:39.146153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 462/927] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:39.146153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:39.152028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:39.161136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fe903-f325-4196-956a-ca03d8f3ab4e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:39.205115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:39.205887 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:39.206293 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:39.206595 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:39.207252 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:39.208727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:39.219665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:39.220997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:39.222094 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 466/928] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:39.223048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:39.280066 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1465d-2d77-4ccd-aa03-a37df9f8c3c7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:39.296692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:39.328811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:39.329764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:39.330278 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:39.330687 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:39.331546 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:39.331869 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:39.333604 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:39.349481 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 463/929] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:27:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:39.349723 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49613a-03c2-408f-b8fa-ca822d35fa2d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:39.349850 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:39.398009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:39.399299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:39.399812 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:39.400211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:39.401268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:39.401548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:39.403761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:39.414200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 467/930] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:39.414200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:39.452924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:39.510470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd223ba1-1903-47b6-8393-6535e60e0913 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:39.554353 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:39.555178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:39.555486 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:39.556215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:39.556692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:39.556993 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:39.558771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:39.559942 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fabe1-16dd-449c-b0ed-a96b017a329c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:39.570494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 464/931] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:39.571755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:39.608252 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:39.609005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:39.609398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:39.609665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:39.621538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:39.623070 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:39.624774 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:39.640265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:39.641792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:39.643076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:39.643076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:39.643076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:39.643076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:39.646280 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 468/932] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:39.647620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:39.704506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511be856-78a1-487b-b757-df08249b0dfb None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:39.742770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f343b7-fb37-4a4c-887d-eb70abc4fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:39.743718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f343b7-fb37-4a4c-887d-eb70abc4fecd None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:39.744003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f343b7-fb37-4a4c-887d-eb70abc4fecd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:39.744437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f343b7-fb37-4a4c-887d-eb70abc4fecd None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:39.745340 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f343b7-fb37-4a4c-887d-eb70abc4fecd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:39.746584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:39.746930 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f343b7-fb37-4a4c-887d-eb70abc4fecd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:39.773621 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 465/933] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:27:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:39.773965 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b7e9-db40-4291-8017-25cc13ac05a2 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:39.775945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f343b7-fb37-4a4c-887d-eb70abc4fecd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:39.858376 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:39.860278 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:39.860968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:39.861983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:39.863594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:39.864148 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:39.865969 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:39.873918 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 469/934] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:39.880066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:39.933988 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:39.947586 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f343b7-fb37-4a4c-887d-eb70abc4fecd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:40.000132 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:40.000687 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.001525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.001823 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.002709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:40.003023 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:40.005932 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:40.020577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 466/935] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:40.026459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:40.040419 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1dfd4-c8f2-406d-b848-f6aef9fa7f76 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:40.092462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:40.093269 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.093542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.093847 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.094843 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] RBAC: Adding query filter params (interface=public, service_id=e4d7eb37a40c4090806d0e2c5cceefe9, region_id=RegionOne) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:40.095174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:40.096726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:40.107838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 470/936] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:27:39 2022] GET /identity/v3/endpoints?service_id=e4d7eb37a40c4090806d0e2c5cceefe9&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:40.118152 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:40.132019 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:40.165253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bcd04-95ba-4207-9f73-5178d526e887 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:40.206989 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:40.207670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.207956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.208236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.217389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:40.219072 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:40.237561 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:40.239844 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:40.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 467/937] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:27:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:40.255657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:40.261702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadfe09-ec3b-4323-b949-a09e5ae18b4f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:40.276565 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:40.303574 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b793820-3c35-4786-b800-5ebec8c1420f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:40.304226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793820-3c35-4786-b800-5ebec8c1420f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.304511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793820-3c35-4786-b800-5ebec8c1420f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.304783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793820-3c35-4786-b800-5ebec8c1420f None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.305461 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b793820-3c35-4786-b800-5ebec8c1420f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:40.306899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b793820-3c35-4786-b800-5ebec8c1420f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:40.313916 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 471/938] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:27:40 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:27:40.402035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ac14-9266-48e0-a10c-41fb70df6f9d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:40.413109 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a1be-9fe1-47b4-bdcd-452c70364707 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.413738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a1be-9fe1-47b4-bdcd-452c70364707 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.413738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a1be-9fe1-47b4-bdcd-452c70364707 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.415509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 472/939] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:40.421413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5e30-ca60-4d59-9f4c-cbe7983b24fd None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.421683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5e30-ca60-4d59-9f4c-cbe7983b24fd None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.421978 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5e30-ca60-4d59-9f4c-cbe7983b24fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.444084 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-dc4f5e30-ca60-4d59-9f4c-cbe7983b24fd None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:40.445686 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-dc4f5e30-ca60-4d59-9f4c-cbe7983b24fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:40.462027 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:40.463414 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.464394 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.464920 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.467135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:40.468367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:40.470760 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:40.501048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 468/940] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:27:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:40.508510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:40.551312 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:40.574157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 473/941] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:40.591338 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f5e30-ca60-4d59-9f4c-cbe7983b24fd None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:40.653371 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:40.711058 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee470f2d-2214-4536-bbe0-c6e54816497b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:40.748798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f5e30-ca60-4d59-9f4c-cbe7983b24fd None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:40.755417 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:40.756095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.756382 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.756658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.769780 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:40.771413 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:40.789048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:40.790331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:40.791521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:40.791521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:40.791521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:40.791521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:40.791521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 469/942] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:40.792665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:40.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:40.793584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:40.794022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:40.794875 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:40.797685 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:40.799728 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:40.799904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: WARNING keystone.server.flask.common [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 08:27:40.821463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.sql.core [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:27:40.821463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:27:40.821463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [parameters: {'id': '26ee6f121dec46de880679a12339e1da', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:27:40.821463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=207770) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 26ee6f121dec46de880679a12339e1da.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 26ee6f121dec46de880679a12339e1da. Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:27:40.841990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result.read() Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 08:27:40.843221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application yield resource Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise self.value Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application session.commit() Jul 28 08:27:40.844538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:27:40.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:27:40.847102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result.read() Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:27:40.848057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [parameters: {'id': '26ee6f121dec46de880679a12339e1da', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:40.849126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:40.850531 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 26ee6f121dec46de880679a12339e1da. Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:40.852898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 474/943] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:27:40 2022] POST /identity/v3/projects => generated 262 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:40.854792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:40.925662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:40.948891 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e607-217a-41ca-8f4d-bd6709415412 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:40.985564 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:40.988148 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60905a7e-a3ea-4b44-80e8-e80a36eb5b4e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:41.006611 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:41.007290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:41.007665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:41.008008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:41.008797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:41.009211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:41.011325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:41.032205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 470/944] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:41.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:41.033795 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:41.034410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:41.034891 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:41.037957 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:41.044207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:41.046457 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:41.060049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:41.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:41.062299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:41.062299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:41.062299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:41.062299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:41.062299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:41.062299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:41.062299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:41.062299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:41.068410 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 475/945] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:41.071832 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:41.162478 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ade1d8-2ba0-4926-b411-1b0a9d24b33a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:41.201442 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901c4d-22cd-41ec-bc81-d939200c840a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:41.202813 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:41.203535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:41.203822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:41.204104 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:41.215151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:41.217740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:41.223765 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:41.236519 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:41.237704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:41.238936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:41.238936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:41.238936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:41.238936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:41.238936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:41.238936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:41.238936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:41.238936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:41.242206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 471/946] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:41.244818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:41.258204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:41.258921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:41.259227 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:41.259504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:41.260465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:41.260801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:41.262537 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:41.302444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 476/947] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:41.322032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:41.409578 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f7894-7f30-441a-9629-bce61323799c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:41.456147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed22842-9a16-4fd8-b60b-3dd277dc52d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:41.456885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed22842-9a16-4fd8-b60b-3dd277dc52d7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:41.457327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed22842-9a16-4fd8-b60b-3dd277dc52d7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:41.457612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed22842-9a16-4fd8-b60b-3dd277dc52d7 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:41.458635 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed22842-9a16-4fd8-b60b-3dd277dc52d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:41.458952 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed22842-9a16-4fd8-b60b-3dd277dc52d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:41.460468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed22842-9a16-4fd8-b60b-3dd277dc52d7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:41.476173 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 472/948] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:41.483928 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:41.626565 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981eb8d-6f85-4318-9cda-72c1c9e88fa1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:41.672232 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7622e4-740f-4597-806b-2154f0a874c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:41.673340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7622e4-740f-4597-806b-2154f0a874c9 None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:41.673781 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7622e4-740f-4597-806b-2154f0a874c9 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:41.674359 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7622e4-740f-4597-806b-2154f0a874c9 None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/2569457ff8f6431daab71dd6083183a5/roles/678a59243ee345488ef8c869ef84f779` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:41.677144 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7622e4-740f-4597-806b-2154f0a874c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=2569457ff8f6431daab71dd6083183a5, role_id=678a59243ee345488ef8c869ef84f779)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:41.678887 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7622e4-740f-4597-806b-2154f0a874c9 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:41.695858 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 477/949] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:27:41 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/2569457ff8f6431daab71dd6083183a5/roles/678a59243ee345488ef8c869ef84f779 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:27:42.278557 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0929c-d505-4d50-805c-19500b744572 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:42.279134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0929c-d505-4d50-805c-19500b744572 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:42.279134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0929c-d505-4d50-805c-19500b744572 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:42.280954 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 473/950] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:42.287841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502a6dd-e62e-44e6-a2a9-931515c64fe4 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:42.288150 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502a6dd-e62e-44e6-a2a9-931515c64fe4 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:42.288402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502a6dd-e62e-44e6-a2a9-931515c64fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:42.310194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-4502a6dd-e62e-44e6-a2a9-931515c64fe4 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:42.312009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-4502a6dd-e62e-44e6-a2a9-931515c64fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:42.344665 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:42.398234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 478/951] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:42.418362 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0929c-d505-4d50-805c-19500b744572 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:42.549858 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0929c-d505-4d50-805c-19500b744572 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:42.561224 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:42.594780 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a2b57-aa14-4c56-85ab-807bc9762a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:42.594780 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a2b57-aa14-4c56-85ab-807bc9762a00 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:42.595863 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a2b57-aa14-4c56-85ab-807bc9762a00 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:42.595953 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a2b57-aa14-4c56-85ab-807bc9762a00 None admin] PATH_INFO: `/v3/services/compute` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:42.596642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a2b57-aa14-4c56-85ab-807bc9762a00 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:42.598275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a2b57-aa14-4c56-85ab-807bc9762a00 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-032a2b57-aa14-4c56-85ab-807bc9762a00 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:42.610401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:42.611609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:42.612567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 474/952] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:42.617911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502a6dd-e62e-44e6-a2a9-931515c64fe4 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:42.736176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502a6dd-e62e-44e6-a2a9-931515c64fe4 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:42.755109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7bd9-b1af-4a4f-9237-09c2cb618e4a None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:42.755493 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7bd9-b1af-4a4f-9237-09c2cb618e4a None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:42.755806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7bd9-b1af-4a4f-9237-09c2cb618e4a None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:42.757594 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 475/953] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:42.764551 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ff91b-13c9-4f16-9f73-ce88de0790f1 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:42.764977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ff91b-13c9-4f16-9f73-ce88de0790f1 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:42.765342 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ff91b-13c9-4f16-9f73-ce88de0790f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:42.779162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:42.779957 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:42.780340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:42.780697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:42.781538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:42.781851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:42.783326 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:42.791321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 479/954] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:42.792328 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-954ff91b-13c9-4f16-9f73-ce88de0790f1 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:42.794052 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-954ff91b-13c9-4f16-9f73-ce88de0790f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:42.798260 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:42.842476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:42.917650 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 476/955] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:42.943687 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ff91b-13c9-4f16-9f73-ce88de0790f1 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:42.959887 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05e259-c701-4dc0-8234-3efee04e8ad0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:43.004598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:43.007092 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:43.009373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:43.009963 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:43.012574 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:43.013129 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:43.014811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:43.021872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 480/956] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:43.071406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:43.077552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ff91b-13c9-4f16-9f73-ce88de0790f1 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:43.125307 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:43.126008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:43.126333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:43.126613 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:43.127451 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:43.128986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:43.141215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:43.142448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:43.143671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 477/957] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:43.152554 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:43.266892 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:43.277188 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f00bcf-cbf3-4abb-a5bc-05f9d28827a6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:43.323185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:43.323870 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:43.324151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:43.324423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:43.325159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:43.325468 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:43.326910 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:43.335117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 481/958] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:43.343513 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:43.485034 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64655514-2335-4a0a-b1b3-b7b3bee51c1d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:43.531556 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a880128-a49c-43d9-bc90-7d390c01f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:43.532496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a880128-a49c-43d9-bc90-7d390c01f4ae None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:43.532817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a880128-a49c-43d9-bc90-7d390c01f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:43.533108 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a880128-a49c-43d9-bc90-7d390c01f4ae None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:43.534025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a880128-a49c-43d9-bc90-7d390c01f4ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:43.534381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a880128-a49c-43d9-bc90-7d390c01f4ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:43.535928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a880128-a49c-43d9-bc90-7d390c01f4ae None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:43.543538 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 478/959] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:43.552196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:43.691296 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d600-da89-4a57-a21f-212693f0cfe6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:43.733893 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c20c9-6e7f-43b3-b688-2ba529931921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:43.734835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c20c9-6e7f-43b3-b688-2ba529931921 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:43.735405 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c20c9-6e7f-43b3-b688-2ba529931921 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:43.735847 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c20c9-6e7f-43b3-b688-2ba529931921 None admin] PATH_INFO: `/v3/endpoints` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:43.736832 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c20c9-6e7f-43b3-b688-2ba529931921 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:43.738490 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c20c9-6e7f-43b3-b688-2ba529931921 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:43.770733 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 482/960] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:27:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:44.051370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fed4d4-6150-4468-b1b3-73c23de28776 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:44.051939 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fed4d4-6150-4468-b1b3-73c23de28776 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:44.052539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fed4d4-6150-4468-b1b3-73c23de28776 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:44.055699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 479/961] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:44.064703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3c45f-81e4-44de-8559-89c39346b20f None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:44.065436 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3c45f-81e4-44de-8559-89c39346b20f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:44.066044 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3c45f-81e4-44de-8559-89c39346b20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:44.098005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-65c3c45f-81e4-44de-8559-89c39346b20f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:44.100073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-65c3c45f-81e4-44de-8559-89c39346b20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:44.115354 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:44.129589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54be17-2e6b-4b97-8f08-c7f5f5a47fd4 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:44.129991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54be17-2e6b-4b97-8f08-c7f5f5a47fd4 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:44.130446 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54be17-2e6b-4b97-8f08-c7f5f5a47fd4 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:44.132368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 480/962] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:44.141951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1267ba-35e8-41c4-9bf5-9df6d2a5c980 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:44.142460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1267ba-35e8-41c4-9bf5-9df6d2a5c980 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:44.142948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1267ba-35e8-41c4-9bf5-9df6d2a5c980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:44.173572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-8c1267ba-35e8-41c4-9bf5-9df6d2a5c980 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:44.175556 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-8c1267ba-35e8-41c4-9bf5-9df6d2a5c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:44.216753 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:44.251309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 483/963] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:44.287162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3c45f-81e4-44de-8559-89c39346b20f None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:44.307387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 481/964] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:44.341504 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1267ba-35e8-41c4-9bf5-9df6d2a5c980 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:44.409475 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3c45f-81e4-44de-8559-89c39346b20f None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:44.449202 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:44.450268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:44.450557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:44.450841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:44.451613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:44.453084 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:44.465689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:44.470757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.sql.core [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:27:44.470757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:27:44.470757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [parameters: {'user_id': '3cf047583a174c49a550f7d61c7ccf2f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:27:44.470757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=207770) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:27:44.472732 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1267ba-35e8-41c4-9bf5-9df6d2a5c980 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result.read() Jul 28 08:27:44.481726 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:27:44.482852 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application yield resource Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise self.value Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application session.commit() Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:27:44.483982 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:27:44.484983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:27:44.485968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application result.read() Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:27:44.487004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [parameters: {'user_id': '3cf047583a174c49a550f7d61c7ccf2f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:44.487908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 08:27:44.488804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 08:27:44.489815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 08:27:44.489815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 08:27:44.489815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:27:44.489815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:27:44.489815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:27:44.489815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:44.492435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 484/965] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:27:44 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:44.502159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:44.549603 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2556dfd-e0fe-4ed6-a5da-708798c57d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:44.551026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2556dfd-e0fe-4ed6-a5da-708798c57d2a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:44.551663 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2556dfd-e0fe-4ed6-a5da-708798c57d2a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:44.552227 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2556dfd-e0fe-4ed6-a5da-708798c57d2a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:44.583100 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2556dfd-e0fe-4ed6-a5da-708798c57d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b2556dfd-e0fe-4ed6-a5da-708798c57d2a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:44.596889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:44.598997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:44.600896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 482/966] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:27:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:44.606854 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2556dfd-e0fe-4ed6-a5da-708798c57d2a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:44.633547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:44.712026 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968684-d292-4141-962f-91d7bdaa251b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:44.762709 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:44.763979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:44.764684 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:44.765091 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:44.767333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2556dfd-e0fe-4ed6-a5da-708798c57d2a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:44.776447 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:44.780322 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:44.802898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:44.804165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:27:44.805236 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:44.810207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 485/967] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:44.812740 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:44.823665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449e802-695f-4324-afbd-76d240de096f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:44.825292 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e802-695f-4324-afbd-76d240de096f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:44.825784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e802-695f-4324-afbd-76d240de096f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:44.826289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e802-695f-4324-afbd-76d240de096f None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:44.827754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449e802-695f-4324-afbd-76d240de096f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:44.828250 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449e802-695f-4324-afbd-76d240de096f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:44.830707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449e802-695f-4324-afbd-76d240de096f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:44.845142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 483/968] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:27:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:44.851746 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449e802-695f-4324-afbd-76d240de096f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:44.896162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:44.963136 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:44.963950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8bba5-4ab6-4403-a890-025e79dbdda3 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:44.991501 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449e802-695f-4324-afbd-76d240de096f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:45.020801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:45.022059 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.022668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.023120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:45.024420 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:45.024910 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:45.027630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:45.035287 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:45.036119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.036523 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.036912 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:45.051400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:45.053252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:45.073203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:45.074364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:45.075368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:45.075368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:45.075368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:45.075368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:45.076032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 484/969] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:45.084008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:45.098387 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 486/970] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:27:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:45.233675 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d38546-d009-4b6e-80e6-1b62637bb2f6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:45.283007 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:45.292486 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69588e7-f221-41d2-8a8f-6a1265c82f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:45.293654 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69588e7-f221-41d2-8a8f-6a1265c82f0e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.294096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69588e7-f221-41d2-8a8f-6a1265c82f0e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.294429 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69588e7-f221-41d2-8a8f-6a1265c82f0e None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:45.295169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69588e7-f221-41d2-8a8f-6a1265c82f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:45.295511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69588e7-f221-41d2-8a8f-6a1265c82f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:45.296978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69588e7-f221-41d2-8a8f-6a1265c82f0e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:45.316389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 485/971] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:45.322598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:45.448425 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3847e-0130-487e-8d50-3b9d122a3724 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:45.469318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:45.487697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:45.488364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.488697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.488970 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:45.497378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:45.498966 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:45.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:45.515469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:45.517204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 487/972] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:27:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:45.522895 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc384-1964-4905-9df2-9f030f10947e None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.523479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc384-1964-4905-9df2-9f030f10947e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.523479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc384-1964-4905-9df2-9f030f10947e None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:45.524890 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 486/973] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:45.529727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:45.530882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6c688-35e7-4d24-a63f-aa26981fbc3e None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.531326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6c688-35e7-4d24-a63f-aa26981fbc3e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.531694 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6c688-35e7-4d24-a63f-aa26981fbc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:45.551476 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-8ce6c688-35e7-4d24-a63f-aa26981fbc3e None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:45.553120 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-8ce6c688-35e7-4d24-a63f-aa26981fbc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:45.657465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 487/974] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:45.680772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6c688-35e7-4d24-a63f-aa26981fbc3e None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:45.684451 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529167-f5c2-487e-b20b-5c1b2feb9c4d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:45.734072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:45.734922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.735110 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.735405 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:45.736416 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:45.736797 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:45.739268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:45.760385 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 488/975] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:27:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:45.767423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:45.810708 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6c688-35e7-4d24-a63f-aa26981fbc3e None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:45.817502 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:45.836709 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:45.862574 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d215-80d6-4c71-b733-846e4e116b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:45.863328 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d215-80d6-4c71-b733-846e4e116b91 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.863625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d215-80d6-4c71-b733-846e4e116b91 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.865024 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d215-80d6-4c71-b733-846e4e116b91 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:45.865024 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d215-80d6-4c71-b733-846e4e116b91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:45.867097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d215-80d6-4c71-b733-846e4e116b91 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:45.893993 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 488/976] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:27:45 2022] POST /identity/v3/services => generated 242 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:45.918930 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaeb01-8b92-4950-92d2-82f13db90e7f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:45.976208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424ca99-93ef-441d-8141-048828884eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:45.976875 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424ca99-93ef-441d-8141-048828884eb4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:45.977159 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424ca99-93ef-441d-8141-048828884eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:45.977436 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424ca99-93ef-441d-8141-048828884eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.036036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c439-a555-4d7b-a51f-fa8aeffa2cb4 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.036265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c439-a555-4d7b-a51f-fa8aeffa2cb4 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.036567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c439-a555-4d7b-a51f-fa8aeffa2cb4 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.038095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 489/977] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:46.044869 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5524-8df8-406c-b0a0-6ea077ef7d5c None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.045243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5524-8df8-406c-b0a0-6ea077ef7d5c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.045630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5524-8df8-406c-b0a0-6ea077ef7d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.068484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-a03b5524-8df8-406c-b0a0-6ea077ef7d5c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:46.070433 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-a03b5524-8df8-406c-b0a0-6ea077ef7d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:46.087291 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424ca99-93ef-441d-8141-048828884eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:46.090206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424ca99-93ef-441d-8141-048828884eb4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:46.130414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:46.159882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 490/978] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:46.182935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b5524-8df8-406c-b0a0-6ea077ef7d5c None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.215892 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 489/979] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:45 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:27:46.222030 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424ca99-93ef-441d-8141-048828884eb4 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.304129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b5524-8df8-406c-b0a0-6ea077ef7d5c None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:46.318257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:46.341664 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424ca99-93ef-441d-8141-048828884eb4 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:46.349205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:46.349895 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.350205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.350483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.351178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:46.352574 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:46.363096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:46.364485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:46.365748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 491/980] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:46.370560 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.383361 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:46.390973 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:46.391741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.392027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.392298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.403955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:46.405470 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:46.425031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:46.426340 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:46.427365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:46.427365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:46.427365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:46.427365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:46.427365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 490/981] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:46.432432 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.518508 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9945517-008c-4df5-b866-f86f64818378 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:46.572027 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:46.572692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.573178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.573178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.573791 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:46.574097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:46.579410 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:46.580151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3207629-1c2f-4636-b3dd-618ad10beec0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:46.586885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 492/982] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:46.592653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.629371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:46.630017 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.630325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.630601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.631329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:46.631615 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:46.633054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:46.642242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:46.655076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 491/983] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:46.656310 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.712250 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:46.736367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530869df-a6c7-4f3c-aed5-41469e3048d6 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:46.783940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:46.784833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.785242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.785696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.786516 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:46.787092 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:46.789061 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:46.790093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00a80b-8bf0-4d6c-b79d-0fbc0cdeb25d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:46.795368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 493/984] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:46.801657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.839248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:46.839870 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.840263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.840463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.849292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:46.850828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:46.864321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:46.873493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 492/985] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:46.875535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.936630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7ec9a-391b-44da-9036-29ce358d3ee9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:46.938273 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:46.977878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:46.978549 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:46.978833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:46.979106 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:46.979867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] RBAC: Adding query filter params (interface=admin, service_id=e4d7eb37a40c4090806d0e2c5cceefe9, region_id=RegionOne) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:46.980180 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:46.981566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:46.987097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 494/986] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/endpoints?service_id=e4d7eb37a40c4090806d0e2c5cceefe9&interface=admin®ion_id=RegionOne => generated 193 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:46.991790 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce10804-af7f-400a-bf60-935b72a8f0f0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:46.992797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:46.995279 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:47.029638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:47.030487 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:47.030872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:47.031529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:47.032112 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:47.032557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:47.034298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:47.048482 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 493/987] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:47.054485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:47.118513 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587316-bbac-4bf1-873f-ec65d6520684 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:47.157219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e591b-d331-45d6-901e-29364b24b2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:47.157850 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e591b-d331-45d6-901e-29364b24b2e9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:47.158153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e591b-d331-45d6-901e-29364b24b2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:47.158428 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e591b-d331-45d6-901e-29364b24b2e9 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:47.159167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e591b-d331-45d6-901e-29364b24b2e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:47.160581 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e591b-d331-45d6-901e-29364b24b2e9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:47.167616 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 495/988] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:27:46 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:27:47.174172 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34df00-18ec-447b-af7e-ea96fd60f45a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:47.226118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153b889-f1f1-47b8-a9da-9e460fbdb700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:47.227387 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b889-f1f1-47b8-a9da-9e460fbdb700 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:47.227950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b889-f1f1-47b8-a9da-9e460fbdb700 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:47.228522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b889-f1f1-47b8-a9da-9e460fbdb700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:47.230243 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153b889-f1f1-47b8-a9da-9e460fbdb700 None admin] RBAC: Adding query filter params (role.id=678a59243ee345488ef8c869ef84f779, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=2569457ff8f6431daab71dd6083183a5) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:47.230858 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153b889-f1f1-47b8-a9da-9e460fbdb700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:47.233474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153b889-f1f1-47b8-a9da-9e460fbdb700 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:47.245302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 494/989] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:47 2022] GET /identity/v3/role_assignments?user.id=2569457ff8f6431daab71dd6083183a5&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=678a59243ee345488ef8c869ef84f779 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:48.027237 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ac030-6d25-40f7-a641-884b9a71da29 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:48.027885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ac030-6d25-40f7-a641-884b9a71da29 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:48.027885 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ac030-6d25-40f7-a641-884b9a71da29 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:48.029660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 496/990] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:48.035235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033c50-1902-4971-bd22-23162f466025 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:48.036003 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033c50-1902-4971-bd22-23162f466025 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:48.036003 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033c50-1902-4971-bd22-23162f466025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:48.055082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-94033c50-1902-4971-bd22-23162f466025 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:48.056673 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-94033c50-1902-4971-bd22-23162f466025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:48.082556 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:48.132766 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 495/991] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:48.155173 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ac030-6d25-40f7-a641-884b9a71da29 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:48.190094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:48.283012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ac030-6d25-40f7-a641-884b9a71da29 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:48.324784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:48.325460 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:48.325747 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:48.326025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] PATH_INFO: `/v3/services/compute` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:48.326695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:48.328104 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:48.337965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:48.339381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:48.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:48.342480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 497/992] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:48.345864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94033c50-1902-4971-bd22-23162f466025 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:48.455049 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94033c50-1902-4971-bd22-23162f466025 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:48.497266 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:48.498008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:48.498375 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:48.498718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:48.499493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] RBAC: Adding query filter params (name=compute) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:48.499888 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:48.501346 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:48.514211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 496/993] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:48.517881 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:48.644556 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:48.653307 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccfabb-9b95-4554-b5a6-3b7a2ce6b81a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:48.690882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:48.691527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:48.691813 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:48.692094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:48.692788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:48.693095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:48.694879 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:48.705177 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 498/994] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:48.706564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:48.738032 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:48.824885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00302f-9bd1-4ce2-bce9-9c9723ed7a72 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:48.861191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b6b85-c494-48ab-9649-daa128bbb39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:48.861972 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6b85-c494-48ab-9649-daa128bbb39a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:48.862280 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6b85-c494-48ab-9649-daa128bbb39a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:48.862565 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6b85-c494-48ab-9649-daa128bbb39a None admin] PATH_INFO: `/v3/endpoints` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:48.863329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6b85-c494-48ab-9649-daa128bbb39a None admin] RBAC: Adding query filter params (interface=public, service_id=85936d418fa34c1c97d395996e73fb2f, region_id=RegionOne) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:48.863772 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6b85-c494-48ab-9649-daa128bbb39a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:48.865194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6b85-c494-48ab-9649-daa128bbb39a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:48.876572 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 497/995] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:27:48 2022] GET /identity/v3/endpoints?service_id=85936d418fa34c1c97d395996e73fb2f&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:48.877905 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:48.994172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aad583-b9d0-419d-a177-32b84e4008c4 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:49.034920 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b776f4-8ad3-4081-8718-7789f8b9ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:49.035584 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b776f4-8ad3-4081-8718-7789f8b9ac60 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.035915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b776f4-8ad3-4081-8718-7789f8b9ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.036199 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b776f4-8ad3-4081-8718-7789f8b9ac60 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.036989 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b776f4-8ad3-4081-8718-7789f8b9ac60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:49.038423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b776f4-8ad3-4081-8718-7789f8b9ac60 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:49.054269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 499/996] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:27:48 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:27:49.221721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb453a-344a-4875-8275-763913fa3c85 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.221721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb453a-344a-4875-8275-763913fa3c85 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.222780 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb453a-344a-4875-8275-763913fa3c85 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.223749 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 498/997] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:49.229873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88705b-ccbc-47da-adc9-8c2f5298f2e8 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.230217 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88705b-ccbc-47da-adc9-8c2f5298f2e8 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.230454 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88705b-ccbc-47da-adc9-8c2f5298f2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.262440 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-dd88705b-ccbc-47da-adc9-8c2f5298f2e8 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:49.263595 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-dd88705b-ccbc-47da-adc9-8c2f5298f2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:49.285497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:49.346237 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 500/998] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:49.368707 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb453a-344a-4875-8275-763913fa3c85 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:49.374494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e35c7-c3a1-4551-8614-4b8e4dbaa754 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.374822 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e35c7-c3a1-4551-8614-4b8e4dbaa754 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.375068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e35c7-c3a1-4551-8614-4b8e4dbaa754 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.376505 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 501/999] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:49.383087 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f77e5-5390-4150-a171-440b34334162 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.383449 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f77e5-5390-4150-a171-440b34334162 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.383860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f77e5-5390-4150-a171-440b34334162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.404937 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-9f7f77e5-5390-4150-a171-440b34334162 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:49.406211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-9f7f77e5-5390-4150-a171-440b34334162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:49.470895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:49.490095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb453a-344a-4875-8275-763913fa3c85 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:49.491206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 502/1000] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:49.513061 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f77e5-5390-4150-a171-440b34334162 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:49.535946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:49.536646 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.536989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.537251 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.538004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:49.539541 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:49.551499 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:49.553248 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:49.554896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 499/1001] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:27:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:49.560189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:49.622648 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:49.645035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f77e5-5390-4150-a171-440b34334162 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:49.669674 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeb3b6-cb6c-4490-92ac-7f99b981fe4a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:49.684306 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:49.684969 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.685270 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.685539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] PATH_INFO: `/v3/services/image` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.686229 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:49.687853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:49.698337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:49.699522 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:49.701008 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 503/1002] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:27:49 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:49.706047 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:49.711465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:49.712111 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.712392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.712668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.713349 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:49.713660 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:49.715105 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:49.721095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 500/1003] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:49.726818 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:49.743724 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:49.821381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4d5e1-cdca-4e48-8c47-11654abacd7e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:49.844124 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9bd26-344f-4377-a6bf-0b42b28691cd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:49.862596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:49.863244 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.863524 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.863796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.864485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] RBAC: Adding query filter params (name=image) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:49.864789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:49.866206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:49.871657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 504/1004] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:49.877731 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:49.890889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:49.891129 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:49.891833 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:49.892516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:49.892843 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:49.893513 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:49.893782 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:49.895274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:49.905028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 501/1005] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:27:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:49.906267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:49.989681 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:49.993572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76d636-5981-4164-8da0-b62c6a1a7b27 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:50.013947 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7470a-13f6-4a43-ba25-af314b79f2c0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:50.029105 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a7554-92a0-4344-87c2-6edd5c4b33d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:50.029740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7554-92a0-4344-87c2-6edd5c4b33d8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:50.030021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7554-92a0-4344-87c2-6edd5c4b33d8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:50.030326 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7554-92a0-4344-87c2-6edd5c4b33d8 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:50.031002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7554-92a0-4344-87c2-6edd5c4b33d8 None admin] RBAC: Adding query filter params (type=image) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:50.031307 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7554-92a0-4344-87c2-6edd5c4b33d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:50.032692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7554-92a0-4344-87c2-6edd5c4b33d8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:50.037507 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 505/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:50.055395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d2954-96ea-4f2a-ae3d-780f39fddb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:50.056181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2954-96ea-4f2a-ae3d-780f39fddb61 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:50.056544 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2954-96ea-4f2a-ae3d-780f39fddb61 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:50.056887 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2954-96ea-4f2a-ae3d-780f39fddb61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:50.057698 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d2954-96ea-4f2a-ae3d-780f39fddb61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:50.059274 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d2954-96ea-4f2a-ae3d-780f39fddb61 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:50.080724 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 502/1007] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:27:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:51.088296 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6349b8-55ec-4b92-888d-13eeb687554f None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:51.088899 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6349b8-55ec-4b92-888d-13eeb687554f None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:51.088899 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6349b8-55ec-4b92-888d-13eeb687554f None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:51.090662 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 506/1008] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:51.095986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081205-cba9-46fa-98ee-7df66a4faa24 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:51.096441 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081205-cba9-46fa-98ee-7df66a4faa24 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:51.096550 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081205-cba9-46fa-98ee-7df66a4faa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:51.115953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-3e081205-cba9-46fa-98ee-7df66a4faa24 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:51.117139 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-3e081205-cba9-46fa-98ee-7df66a4faa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:51.197117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 503/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:51.219867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6349b8-55ec-4b92-888d-13eeb687554f None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:51.309609 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:51.339150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6349b8-55ec-4b92-888d-13eeb687554f None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:51.389688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:51.390811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:51.391297 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:51.391785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] PATH_INFO: `/v3/services/compute` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:51.392973 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:51.395566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:51.405488 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:51.412311 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:51.413241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 507/1010] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:51.414638 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e081205-cba9-46fa-98ee-7df66a4faa24 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:51.483275 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:51.538691 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e081205-cba9-46fa-98ee-7df66a4faa24 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:51.593014 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:51.594006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:51.594508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:51.595006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:51.595922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] RBAC: Adding query filter params (name=compute) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:51.596516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:51.598287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:51.604459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 504/1011] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:51.610547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:51.739210 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b81f-8ebd-4a03-993d-c07ab58780bd None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:51.790745 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7de79-bf93-4962-9637-11001b4a7ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:51.791385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7de79-bf93-4962-9637-11001b4a7ec6 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:51.791665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7de79-bf93-4962-9637-11001b4a7ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:51.791925 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7de79-bf93-4962-9637-11001b4a7ec6 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:51.792630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7de79-bf93-4962-9637-11001b4a7ec6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:51.792935 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7de79-bf93-4962-9637-11001b4a7ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:51.794435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7de79-bf93-4962-9637-11001b4a7ec6 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:51.800320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 508/1012] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:27:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:51.805946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:51.920180 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b6fbc-c22e-4997-989c-39323f14c227 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:51.950519 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:51.962204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567cdb2-01fb-4884-bc47-a670dd3dfbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:51.962859 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567cdb2-01fb-4884-bc47-a670dd3dfbd8 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:51.963120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567cdb2-01fb-4884-bc47-a670dd3dfbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:51.963416 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567cdb2-01fb-4884-bc47-a670dd3dfbd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:51.964147 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567cdb2-01fb-4884-bc47-a670dd3dfbd8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:51.965526 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567cdb2-01fb-4884-bc47-a670dd3dfbd8 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:51.998206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 505/1013] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:27:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:52.118915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f82210-e12a-41f1-a7ff-6decc9136792 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:52.119494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f82210-e12a-41f1-a7ff-6decc9136792 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:52.119494 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f82210-e12a-41f1-a7ff-6decc9136792 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:52.121190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 509/1014] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:52.126945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7ad8e-3fd7-4b1f-926d-2e104cc6cdd9 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:52.127620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7ad8e-3fd7-4b1f-926d-2e104cc6cdd9 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:52.127620 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7ad8e-3fd7-4b1f-926d-2e104cc6cdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:52.147334 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-d2d7ad8e-3fd7-4b1f-926d-2e104cc6cdd9 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:52.148581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-d2d7ad8e-3fd7-4b1f-926d-2e104cc6cdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:52.164054 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f39b30-79d5-406b-8f0b-9a94eda091b8 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:52.164412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f39b30-79d5-406b-8f0b-9a94eda091b8 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:52.164603 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f39b30-79d5-406b-8f0b-9a94eda091b8 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:52.166293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 510/1015] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:52.173102 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748fec1-6210-428b-83fc-2df100dbeae8 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:52.173386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748fec1-6210-428b-83fc-2df100dbeae8 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:52.173577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748fec1-6210-428b-83fc-2df100dbeae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:52.182404 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:52.197598 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-c748fec1-6210-428b-83fc-2df100dbeae8 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:52.199767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-c748fec1-6210-428b-83fc-2df100dbeae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:52.239261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 506/1016] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:52 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:52.262028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7ad8e-3fd7-4b1f-926d-2e104cc6cdd9 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:52.290279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 511/1017] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:52 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:52.315122 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748fec1-6210-428b-83fc-2df100dbeae8 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:52.391230 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7ad8e-3fd7-4b1f-926d-2e104cc6cdd9 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:52.404727 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:52.432660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748fec1-6210-428b-83fc-2df100dbeae8 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:52.433845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af890ed4-9961-4f15-b87e-7b3061c58768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:52.434526 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af890ed4-9961-4f15-b87e-7b3061c58768 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:52.434932 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af890ed4-9961-4f15-b87e-7b3061c58768 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:52.435206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af890ed4-9961-4f15-b87e-7b3061c58768 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:52.436038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af890ed4-9961-4f15-b87e-7b3061c58768 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:52.437457 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af890ed4-9961-4f15-b87e-7b3061c58768 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:52.463609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 507/1018] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:27:52 2022] POST /identity/v3/services => generated 242 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:27:52.469379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:52.484847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf16a3b-c562-43f8-aa85-478ee7e05dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:52.485530 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf16a3b-c562-43f8-aa85-478ee7e05dd5 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:52.485813 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf16a3b-c562-43f8-aa85-478ee7e05dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:52.486082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf16a3b-c562-43f8-aa85-478ee7e05dd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:52.487471 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf16a3b-c562-43f8-aa85-478ee7e05dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:52.488965 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf16a3b-c562-43f8-aa85-478ee7e05dd5 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:52.491654 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 512/1019] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:52.498193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af890ed4-9961-4f15-b87e-7b3061c58768 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:52.641538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af890ed4-9961-4f15-b87e-7b3061c58768 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:52.681961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e73081-4e5e-4c45-9715-5baa1077d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:52.682648 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e73081-4e5e-4c45-9715-5baa1077d688 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:52.682985 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e73081-4e5e-4c45-9715-5baa1077d688 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:52.683207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e73081-4e5e-4c45-9715-5baa1077d688 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:52.683924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e73081-4e5e-4c45-9715-5baa1077d688 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:52.685405 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e73081-4e5e-4c45-9715-5baa1077d688 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:52.717811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 508/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:27:52 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:54.164020 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556fcc4-fd7c-4755-85b9-5fc0029bfcb4 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.164605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556fcc4-fd7c-4755-85b9-5fc0029bfcb4 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.164605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556fcc4-fd7c-4755-85b9-5fc0029bfcb4 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.166092 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 513/1021] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:54.171571 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f104-078f-4e0e-94a7-0e3466a3966d None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.172027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f104-078f-4e0e-94a7-0e3466a3966d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.172111 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f104-078f-4e0e-94a7-0e3466a3966d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.191654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-9fb7f104-078f-4e0e-94a7-0e3466a3966d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:54.192874 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-9fb7f104-078f-4e0e-94a7-0e3466a3966d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:54.211698 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:54.266956 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 509/1022] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:54.290022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556fcc4-fd7c-4755-85b9-5fc0029bfcb4 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:54.399967 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556fcc4-fd7c-4755-85b9-5fc0029bfcb4 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:54.435818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c8021-7a4d-453e-9bfe-31ddb84ad712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:54.436397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c8021-7a4d-453e-9bfe-31ddb84ad712 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.436678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c8021-7a4d-453e-9bfe-31ddb84ad712 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.436950 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c8021-7a4d-453e-9bfe-31ddb84ad712 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.449046 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c8021-7a4d-453e-9bfe-31ddb84ad712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b95c8021-7a4d-453e-9bfe-31ddb84ad712 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:54.454068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:54.455141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:54.456480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:54.464373 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 514/1023] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:27:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:54.464593 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f104-078f-4e0e-94a7-0e3466a3966d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:54.496854 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c92fe-9fe2-4d69-b138-90d70a050052 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.497117 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c92fe-9fe2-4d69-b138-90d70a050052 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.498022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c92fe-9fe2-4d69-b138-90d70a050052 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.499623 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 515/1024] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:54.505071 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287a9a2-ae9d-469c-8ba5-a56c163afdd3 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.505579 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287a9a2-ae9d-469c-8ba5-a56c163afdd3 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.505831 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287a9a2-ae9d-469c-8ba5-a56c163afdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.530330 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-b287a9a2-ae9d-469c-8ba5-a56c163afdd3 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:54.532253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-b287a9a2-ae9d-469c-8ba5-a56c163afdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:54.592122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f104-078f-4e0e-94a7-0e3466a3966d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:54.596036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:54.616288 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 516/1025] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:54.630223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:54.630473 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.630820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.631164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.631960 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:54.632353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:54.633826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:54.638683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287a9a2-ae9d-469c-8ba5-a56c163afdd3 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:54.643444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 510/1026] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:27:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:54.648979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:54.665940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:54.757002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287a9a2-ae9d-469c-8ba5-a56c163afdd3 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:54.759547 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8012be2-db5c-4e6b-9c1b-c42a23dc72dd None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:54.792889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3593d-61f2-4286-8d63-966061c6e0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:54.794026 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3593d-61f2-4286-8d63-966061c6e0e9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.794515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3593d-61f2-4286-8d63-966061c6e0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.794970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3593d-61f2-4286-8d63-966061c6e0e9 None admin] PATH_INFO: `/v3/services/image` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.796059 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3593d-61f2-4286-8d63-966061c6e0e9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:54.796859 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd52bce-12c2-4204-968a-903ed4df83a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:54.797515 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd52bce-12c2-4204-968a-903ed4df83a4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.797799 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd52bce-12c2-4204-968a-903ed4df83a4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.798072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd52bce-12c2-4204-968a-903ed4df83a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.798346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3593d-61f2-4286-8d63-966061c6e0e9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-28d3593d-61f2-4286-8d63-966061c6e0e9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:54.808127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:54.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:54.810299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:54.811252 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd52bce-12c2-4204-968a-903ed4df83a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:54.811337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 517/1027] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:27:54 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:54.811413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd52bce-12c2-4204-968a-903ed4df83a4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:54.811496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526de35-befc-4a6e-b605-622a0a1a61dd None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.811496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526de35-befc-4a6e-b605-622a0a1a61dd None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.811496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526de35-befc-4a6e-b605-622a0a1a61dd None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.812448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 518/1028] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:54.815319 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526de35-befc-4a6e-b605-622a0a1a61dd None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-2dd52bce-12c2-4204-968a-903ed4df83a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:54.829865 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:54.831132 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:54.832138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:54.832138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:54.832138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:54.832138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:54.832138 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 511/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:54.832675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2135e4-0acb-4467-b750-a9558f16bb5b None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.832675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2135e4-0acb-4467-b750-a9558f16bb5b None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.832920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2135e4-0acb-4467-b750-a9558f16bb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.851054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-aa2135e4-0acb-4467-b750-a9558f16bb5b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:54.852286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-aa2135e4-0acb-4467-b750-a9558f16bb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:54.877276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:54.917851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:54.926169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 512/1030] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:54.926991 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2135e4-0acb-4467-b750-a9558f16bb5b None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:54.935115 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526de35-befc-4a6e-b605-622a0a1a61dd None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:54.976162 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:54.976961 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:54.977623 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:54.977912 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:54.978639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] RBAC: Adding query filter params (name=image) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:54.978952 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:54.980345 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:54.985245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 519/1031] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:54.986024 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:55.035368 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2135e4-0acb-4467-b750-a9558f16bb5b None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.074618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.075260 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.075580 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.075845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.076548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:55.076856 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.078288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.087838 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07865cc-867c-482d-a6d0-fa9361838229 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.090034 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 513/1032] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:55.090873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:55.107315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:55.129973 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e900cf-6fb0-4e1e-8a8a-7dfa705390ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.130661 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e900cf-6fb0-4e1e-8a8a-7dfa705390ef None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.130945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e900cf-6fb0-4e1e-8a8a-7dfa705390ef None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.131555 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e900cf-6fb0-4e1e-8a8a-7dfa705390ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.132368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e900cf-6fb0-4e1e-8a8a-7dfa705390ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.133820 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e900cf-6fb0-4e1e-8a8a-7dfa705390ef None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.135486 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 520/1033] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:55.136245 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e900cf-6fb0-4e1e-8a8a-7dfa705390ef None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:55.147409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:55.192185 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36edda7-8eee-4b76-9633-81f1fa26f233 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.227746 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.228374 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.228655 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.228929 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.229978 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] RBAC: Adding query filter params (type=image) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:55.229978 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.231589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.236653 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 514/1034] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:55.237431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:55.244013 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e900cf-6fb0-4e1e-8a8a-7dfa705390ef None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.278632 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.279289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.279570 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.279843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] PATH_INFO: `/v3/users/nova` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.288033 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.289426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:27:55.301142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:55.302258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 521/1035] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:27:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:55.303312 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:55.348125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b16137-16aa-4d68-bc96-feb6ae45c079 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.348385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:55.369711 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:55.384767 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bdea2-d539-4d8b-9994-f04a23c97968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.385404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bdea2-d539-4d8b-9994-f04a23c97968 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.385693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bdea2-d539-4d8b-9994-f04a23c97968 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.385971 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bdea2-d539-4d8b-9994-f04a23c97968 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.386708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bdea2-d539-4d8b-9994-f04a23c97968 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.388178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bdea2-d539-4d8b-9994-f04a23c97968 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.410581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 515/1036] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:27:55 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:55.410816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9835bc7-f6ff-43cb-a1df-3f7e6cc0367c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.411224 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bdea2-d539-4d8b-9994-f04a23c97968 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:55.469639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.470428 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.470721 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.471386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.472637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] RBAC: Adding query filter params (interface=public, service_id=c051ebc078bd41628730dd4777fce337, region_id=RegionOne) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:55.472962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.474576 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.481619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 522/1037] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:27:55 2022] GET /identity/v3/endpoints?service_id=c051ebc078bd41628730dd4777fce337&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:55.487284 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:55.540218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bdea2-d539-4d8b-9994-f04a23c97968 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.584380 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.585038 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.585313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.585586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.586441 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:55.586748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.588207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.609692 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 516/1038] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:27:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:55.610795 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6c07f-d6b8-4f91-bfff-44d341cf8272 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.616616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:55.618915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:55.653708 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb1c18-0286-4a83-90a3-d4b2efb88407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.654711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1c18-0286-4a83-90a3-d4b2efb88407 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.654997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1c18-0286-4a83-90a3-d4b2efb88407 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.655277 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1c18-0286-4a83-90a3-d4b2efb88407 None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.655954 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb1c18-0286-4a83-90a3-d4b2efb88407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.657345 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb1c18-0286-4a83-90a3-d4b2efb88407 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.660753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:55.664740 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 523/1039] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:27:55 2022] GET /identity/v3/services => generated 1957 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:27:55.740451 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88daee-c187-42e8-ade1-4f50d2674ae4 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:55.779853 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e64e7-902c-4796-a26a-6b6636b0ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:55.780510 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e64e7-902c-4796-a26a-6b6636b0ae44 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:55.780788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e64e7-902c-4796-a26a-6b6636b0ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:55.781062 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e64e7-902c-4796-a26a-6b6636b0ae44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:55.883037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e64e7-902c-4796-a26a-6b6636b0ae44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:55.884979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e64e7-902c-4796-a26a-6b6636b0ae44 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:55.925093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:56.003279 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 517/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:27:56.009970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb1c18-0286-4a83-90a3-d4b2efb88407 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:56.125135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb1c18-0286-4a83-90a3-d4b2efb88407 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:56.168219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:56.168767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:56.169058 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:56.169335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:56.181697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:56.183201 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:56.199799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:56.200921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:56.207035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:56.207035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:56.207035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:56.207035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:56.207035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 524/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:56.208925 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e64e7-902c-4796-a26a-6b6636b0ae44 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:56.324814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e64e7-902c-4796-a26a-6b6636b0ae44 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:56.365202 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:56.365849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:56.366141 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:56.366439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:56.367153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:56.367465 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:56.368876 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:56.370551 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:56.384752 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 518/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:56.391240 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:56.420619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:56.526738 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7af110-840e-4687-afc8-0f2ca08f522c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:56.587581 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:56.588753 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:56.589327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:56.589853 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:56.599573 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:56.601692 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:27:56.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:56.619642 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:56.621134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:27:56.621134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:27:56.621134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:27:56.621134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:27:56.621134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:27:56.621134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:27:56.621134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:27:56.621134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:56.626372 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 525/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:27:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:56.628290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:56.736301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09595986-dccf-4492-ac2d-fb4bbb2ce4fb None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:56.773966 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fdf22-ed57-4d11-898f-72d1362a88d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:56.774646 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fdf22-ed57-4d11-898f-72d1362a88d4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:56.774925 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fdf22-ed57-4d11-898f-72d1362a88d4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:56.775195 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fdf22-ed57-4d11-898f-72d1362a88d4 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:56.775956 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fdf22-ed57-4d11-898f-72d1362a88d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:56.776265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fdf22-ed57-4d11-898f-72d1362a88d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:56.777712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fdf22-ed57-4d11-898f-72d1362a88d4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:56.790072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 519/1044] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:27:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:56.796544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:56.887135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:56.909632 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7b957-61ca-4ba3-80ae-a2dca711aa41 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:56.946025 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067132c5-0eb7-4311-8366-f0d741f9b185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:56.947080 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067132c5-0eb7-4311-8366-f0d741f9b185 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:56.947555 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067132c5-0eb7-4311-8366-f0d741f9b185 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:56.948022 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067132c5-0eb7-4311-8366-f0d741f9b185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:56.949044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067132c5-0eb7-4311-8366-f0d741f9b185 None admin] RBAC: Adding query filter params (role.id=678a59243ee345488ef8c869ef84f779, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=c5ba9e7780f5437caac949931ca55b80) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:56.949547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067132c5-0eb7-4311-8366-f0d741f9b185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:56.951216 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067132c5-0eb7-4311-8366-f0d741f9b185 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:56.957447 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 526/1045] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:56 2022] GET /identity/v3/role_assignments?user.id=c5ba9e7780f5437caac949931ca55b80&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=678a59243ee345488ef8c869ef84f779 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:57.512440 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fdbd1-9980-4ed2-b4e2-0fa268a0042a None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:57.513028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fdbd1-9980-4ed2-b4e2-0fa268a0042a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:57.513358 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fdbd1-9980-4ed2-b4e2-0fa268a0042a None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:57.523658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af32f-3387-4f0f-b243-518306ef5adb None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:57.523658 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af32f-3387-4f0f-b243-518306ef5adb None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:57.524425 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af32f-3387-4f0f-b243-518306ef5adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:57.526244 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 520/1046] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:57 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:57.547257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-c26af32f-3387-4f0f-b243-518306ef5adb None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:57.548533 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-c26af32f-3387-4f0f-b243-518306ef5adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:57.629322 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 527/1047] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:57.652836 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fdbd1-9980-4ed2-b4e2-0fa268a0042a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:57.721064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:57.752333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe65d9b-d04c-4674-9024-e28136e7b723 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:57.752631 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe65d9b-d04c-4674-9024-e28136e7b723 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:57.752910 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe65d9b-d04c-4674-9024-e28136e7b723 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:57.754415 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 528/1048] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:27:57.759626 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db61fa2-daf4-4e87-81e4-714a631a7632 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:57.759920 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db61fa2-daf4-4e87-81e4-714a631a7632 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:57.760192 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db61fa2-daf4-4e87-81e4-714a631a7632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:57.771855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fdbd1-9980-4ed2-b4e2-0fa268a0042a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:57.779094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-5db61fa2-daf4-4e87-81e4-714a631a7632 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:57.780318 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-5db61fa2-daf4-4e87-81e4-714a631a7632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:57.809678 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ca009-bf83-4f8d-82e7-f9b24ac3647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:57.810372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca009-bf83-4f8d-82e7-f9b24ac3647b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:57.810662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca009-bf83-4f8d-82e7-f9b24ac3647b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:57.810929 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca009-bf83-4f8d-82e7-f9b24ac3647b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:57.822204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ca009-bf83-4f8d-82e7-f9b24ac3647b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-135ca009-bf83-4f8d-82e7-f9b24ac3647b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:57.827117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:57.828128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:57.829063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 521/1049] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:27:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:57.834714 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ca009-bf83-4f8d-82e7-f9b24ac3647b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:57.855672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 529/1050] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:27:57.878612 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db61fa2-daf4-4e87-81e4-714a631a7632 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:57.887496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:57.949154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ca009-bf83-4f8d-82e7-f9b24ac3647b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:57.965902 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:57.989862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:57.990704 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:57.990985 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:57.991261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:57.992127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:57.992728 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:57.993981 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:57.995500 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db61fa2-daf4-4e87-81e4-714a631a7632 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:58.003777 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 522/1051] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:27:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:58.009577 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:58.037332 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:58.037904 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.038281 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.038560 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] PATH_INFO: `/v3/services/image` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.039213 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:58.040659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:58.050435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:58.051600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:27:58.052921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 530/1052] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:27:57 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:27:58.057921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:58.126580 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59081f-94a0-4541-a707-672e608653a0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:58.134069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:58.164703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f50076-5bdc-4d23-b3d7-88c506c49fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:58.165372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f50076-5bdc-4d23-b3d7-88c506c49fad None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.165664 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f50076-5bdc-4d23-b3d7-88c506c49fad None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.165942 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f50076-5bdc-4d23-b3d7-88c506c49fad None admin] PATH_INFO: `/v3/users/a67950f0ed34442ba3723243c71a727f` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.170246 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82b2d6-5430-4fff-95d4-6a5e5622c589 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:58.180516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f50076-5bdc-4d23-b3d7-88c506c49fad None admin] RBAC: Authorizing `identity:get_user(user_id=a67950f0ed34442ba3723243c71a727f)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:58.182806 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f50076-5bdc-4d23-b3d7-88c506c49fad None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:58.184739 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 523/1053] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:27:58 2022] GET /identity/v3/users/a67950f0ed34442ba3723243c71a727f => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:58.190719 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f50076-5bdc-4d23-b3d7-88c506c49fad None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:58.208983 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:58.209628 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.209906 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.210203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.210889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] RBAC: Adding query filter params (name=image) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:58.211195 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:58.212569 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:58.217970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 531/1054] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:58.223847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:58.231056 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:58.308828 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f50076-5bdc-4d23-b3d7-88c506c49fad None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:58.333636 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a0b4-1169-4014-a09f-9254c298a2cc None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:58.351333 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90ee1e-9fd0-4325-89db-d3fcbe02b053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:58.351975 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90ee1e-9fd0-4325-89db-d3fcbe02b053 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.352246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90ee1e-9fd0-4325-89db-d3fcbe02b053 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.352521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90ee1e-9fd0-4325-89db-d3fcbe02b053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.371296 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:58.372092 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.372482 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.372866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.373639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] RBAC: Adding query filter params (type=image) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:58.374018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:58.375499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:58.377574 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:58.387515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 532/1055] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:27:58.388487 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:58.424734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90ee1e-9fd0-4325-89db-d3fcbe02b053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:58.427232 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90ee1e-9fd0-4325-89db-d3fcbe02b053 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:58.470436 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:58.510364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a393f-8f14-4983-8271-708d8d690cae None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:58.550461 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e45bc-ed67-45f0-9b94-3d1cdc9be168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:58.551153 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 524/1056] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:27:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:27:58.555945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e45bc-ed67-45f0-9b94-3d1cdc9be168 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.556309 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e45bc-ed67-45f0-9b94-3d1cdc9be168 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.556874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e45bc-ed67-45f0-9b94-3d1cdc9be168 None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.557526 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90ee1e-9fd0-4325-89db-d3fcbe02b053 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:58.557657 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e45bc-ed67-45f0-9b94-3d1cdc9be168 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:58.559114 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e45bc-ed67-45f0-9b94-3d1cdc9be168 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:58.584490 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 533/1057] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:27:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:27:58.701196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90ee1e-9fd0-4325-89db-d3fcbe02b053 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:58.744832 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:58.755850 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe534814-e3b9-407a-96e0-3ff6ca6d9cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:58.756517 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe534814-e3b9-407a-96e0-3ff6ca6d9cc2 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.756790 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe534814-e3b9-407a-96e0-3ff6ca6d9cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.757065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe534814-e3b9-407a-96e0-3ff6ca6d9cc2 None admin] PATH_INFO: `/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.758304 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe534814-e3b9-407a-96e0-3ff6ca6d9cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=9c22346d0e2b4c258c3e7dd986659aa9)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:58.759748 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe534814-e3b9-407a-96e0-3ff6ca6d9cc2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:58.761593 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 525/1058] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:27:58 2022] GET /identity/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9 => generated 311 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:58.767932 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e45bc-ed67-45f0-9b94-3d1cdc9be168 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:58.831950 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:58.895094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e45bc-ed67-45f0-9b94-3d1cdc9be168 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:58.933035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59398fb1-d1e2-457c-91e7-5e222c8d01b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:58.933683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59398fb1-d1e2-457c-91e7-5e222c8d01b9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.933963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59398fb1-d1e2-457c-91e7-5e222c8d01b9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.934273 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59398fb1-d1e2-457c-91e7-5e222c8d01b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.935094 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59398fb1-d1e2-457c-91e7-5e222c8d01b9 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=9c22346d0e2b4c258c3e7dd986659aa9, user.id=a67950f0ed34442ba3723243c71a727f) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:58.935401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59398fb1-d1e2-457c-91e7-5e222c8d01b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:58.936882 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59398fb1-d1e2-457c-91e7-5e222c8d01b9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:58.943732 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 534/1059] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:27:58 2022] GET /identity/v3/role_assignments?user.id=a67950f0ed34442ba3723243c71a727f&scope.project.id=9c22346d0e2b4c258c3e7dd986659aa9&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:58.981594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980707ec-471e-485a-9a94-5208d578bda5 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.981861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980707ec-471e-485a-9a94-5208d578bda5 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.982169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980707ec-471e-485a-9a94-5208d578bda5 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:58.983606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 526/1060] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:27:58.989654 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a295a8-1aa2-40c1-89b9-dd7067fe4d00 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:58.989943 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a295a8-1aa2-40c1-89b9-dd7067fe4d00 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:58.990261 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a295a8-1aa2-40c1-89b9-dd7067fe4d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:59.010544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-b6a295a8-1aa2-40c1-89b9-dd7067fe4d00 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:27:59.011834 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-b6a295a8-1aa2-40c1-89b9-dd7067fe4d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:27:59.090167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 535/1061] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:27:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:27:59.113761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980707ec-471e-485a-9a94-5208d578bda5 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:59.224313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980707ec-471e-485a-9a94-5208d578bda5 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:59.260231 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2523af-b03b-4f08-aade-4deb03d0d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:59.260914 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2523af-b03b-4f08-aade-4deb03d0d33f None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:59.261196 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2523af-b03b-4f08-aade-4deb03d0d33f None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:59.261475 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2523af-b03b-4f08-aade-4deb03d0d33f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:59.272560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2523af-b03b-4f08-aade-4deb03d0d33f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-7e2523af-b03b-4f08-aade-4deb03d0d33f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:27:59.277661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:59.278990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:27:59.280162 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:59.283972 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 527/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:27:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:59.285185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a295a8-1aa2-40c1-89b9-dd7067fe4d00 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:59.339098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:59.394866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a295a8-1aa2-40c1-89b9-dd7067fe4d00 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:59.432838 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:59.433484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:59.433755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:59.434045 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:59.434818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:59.435129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:59.436526 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:59.445920 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 536/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:27:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:59.451816 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2523af-b03b-4f08-aade-4deb03d0d33f None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:59.516527 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:59.583331 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2523af-b03b-4f08-aade-4deb03d0d33f None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:59.622832 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:59.623488 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:59.623765 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:59.624035 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:59.634937 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:59.636423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:27:59.651743 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:27:59.652798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:27:59.659214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:27:59.659214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:27:59.659214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:27:59.659214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:27:59.659214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 528/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:27:59.661143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:59.769122 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b63bb-ff0c-4512-acc8-601217272b16 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:59.780581 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:59.807932 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:59.808571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:59.808865 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:59.809129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:59.809835 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:27:59.810160 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:59.811562 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:27:59.824683 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 537/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:27:59.830594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:27:59.941652 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e030d-a52f-408f-81ee-081035eb9abb None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:27:59.953354 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:27:59.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:27:59.982312 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:27:59.982757 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:27:59.983188 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] PATH_INFO: `/v3/users/nova` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:27:59.991547 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:27:59.993225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:28:00.005317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:00.006609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:28:00.007826 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:00.010194 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 529/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:27:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:00.016192 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:00.125127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b5d96-1f31-4920-a753-bb1fe11c9cdb None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:00.160050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:00.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:00.160966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:00.161244 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:00.162037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:00.162366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:00.163805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:00.181794 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 538/1067] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:28:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:00.187404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:00.293237 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcaac2e-a6f9-46ef-bcd9-28ba9812a29a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:00.328415 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:00.329065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:00.329343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:00.329618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:00.340615 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:00.342076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:28:00.356598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:00.357769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:00.359154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:28:00.359154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:28:00.359154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:28:00.359154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:00.359741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 530/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:00.363695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:00.399368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:00.479909 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cc080-d8aa-41e3-9e41-e536e4df67f7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:00.525803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594217d0-2a25-4a0f-95fd-1d1a6c5a5b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:00.526498 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594217d0-2a25-4a0f-95fd-1d1a6c5a5b3f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:00.526763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594217d0-2a25-4a0f-95fd-1d1a6c5a5b3f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:00.527064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594217d0-2a25-4a0f-95fd-1d1a6c5a5b3f None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:00.527779 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594217d0-2a25-4a0f-95fd-1d1a6c5a5b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:00.528092 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594217d0-2a25-4a0f-95fd-1d1a6c5a5b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:00.529500 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594217d0-2a25-4a0f-95fd-1d1a6c5a5b3f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:00.547421 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 539/1069] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:00.548983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:00.581312 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:00.661752 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2343b-60d2-47b2-a1d5-80a8a5d2f8b5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:00.680577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1f0fe-fec0-49f1-9fd9-8716a708218d None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:00.681140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1f0fe-fec0-49f1-9fd9-8716a708218d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:00.681140 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1f0fe-fec0-49f1-9fd9-8716a708218d None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:00.682789 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 540/1070] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:28:00.687772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd59073-4b40-4f8e-89ce-578fff885055 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:00.688061 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd59073-4b40-4f8e-89ce-578fff885055 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:00.688344 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd59073-4b40-4f8e-89ce-578fff885055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:00.699435 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:00.700535 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:00.701020 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:00.701529 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:00.707379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-fcd59073-4b40-4f8e-89ce-578fff885055 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:00.708726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-fcd59073-4b40-4f8e-89ce-578fff885055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:00.709917 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:00.711639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:28:00.724701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:00.725803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:00.726885 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 531/1071] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:28:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:28:00.731822 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:00.789272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 541/1072] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:28:00.809629 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd59073-4b40-4f8e-89ce-578fff885055 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:00.829760 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:00.835736 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:00.856278 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b4548-dba0-4193-8d06-28457bd45c21 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:00.901230 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:00.902010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:00.902303 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:00.902581 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:00.903343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:00.903657 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:00.905245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:00.925419 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 532/1073] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:28:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:00.926738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:00.932711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd59073-4b40-4f8e-89ce-578fff885055 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:00.971513 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca46c5-98b9-4278-8a7d-0867325f0704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:00.972157 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca46c5-98b9-4278-8a7d-0867325f0704 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:00.972444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca46c5-98b9-4278-8a7d-0867325f0704 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:00.972706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca46c5-98b9-4278-8a7d-0867325f0704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.035715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501f355-1982-4a66-908d-09d9a7847455 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:01.046992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca46c5-98b9-4278-8a7d-0867325f0704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:01.048465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca46c5-98b9-4278-8a7d-0867325f0704 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:01.067383 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:01.074728 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aceed8-e190-4938-99d8-5e571a40d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:01.075911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aceed8-e190-4938-99d8-5e571a40d9d0 None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:01.076422 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aceed8-e190-4938-99d8-5e571a40d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:01.076884 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aceed8-e190-4938-99d8-5e571a40d9d0 None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/c5ba9e7780f5437caac949931ca55b80/roles/678a59243ee345488ef8c869ef84f779` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.080691 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aceed8-e190-4938-99d8-5e571a40d9d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=c5ba9e7780f5437caac949931ca55b80, role_id=678a59243ee345488ef8c869ef84f779)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:01.083539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aceed8-e190-4938-99d8-5e571a40d9d0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:01.101593 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 533/1074] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:28:00 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/c5ba9e7780f5437caac949931ca55b80/roles/678a59243ee345488ef8c869ef84f779 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:28:01.102820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f299c4b-3449-424c-9271-b2e2d987d153 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:01.103100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f299c4b-3449-424c-9271-b2e2d987d153 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:01.103373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f299c4b-3449-424c-9271-b2e2d987d153 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.104674 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 534/1075] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:01.110939 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d756e9-a967-40df-bef3-f7210216b75c None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:01.111388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d756e9-a967-40df-bef3-f7210216b75c None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:01.111651 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d756e9-a967-40df-bef3-f7210216b75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.118407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:01.133592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-24d756e9-a967-40df-bef3-f7210216b75c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:01.136223 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-24d756e9-a967-40df-bef3-f7210216b75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:01.190371 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 542/1076] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:28:01.191982 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca46c5-98b9-4278-8a7d-0867325f0704 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:01.240880 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 535/1077] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:01.279612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d756e9-a967-40df-bef3-f7210216b75c None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:01.328913 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca46c5-98b9-4278-8a7d-0867325f0704 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:01.353498 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:01.374284 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:01.375168 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:01.375446 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:01.375717 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.387015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:01.388475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:28:01.410252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:01.411534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:01.412748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:28:01.412748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:28:01.412748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:28:01.412748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:28:01.419223 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 543/1078] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:01.420306 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:01.423042 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d756e9-a967-40df-bef3-f7210216b75c None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:01.458325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:01.465327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708c596-8430-4af0-acc3-9d5022a7588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:01.465756 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708c596-8430-4af0-acc3-9d5022a7588e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:01.465991 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708c596-8430-4af0-acc3-9d5022a7588e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:01.466363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708c596-8430-4af0-acc3-9d5022a7588e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.477925 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708c596-8430-4af0-acc3-9d5022a7588e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-e708c596-8430-4af0-acc3-9d5022a7588e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:01.483063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:01.484263 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:01.485226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:01.489761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 536/1079] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:28:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:28:01.491089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708c596-8430-4af0-acc3-9d5022a7588e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:01.550396 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8b945-0fb6-4b97-bc4f-736cefe7e5e7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:01.593049 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:01.593733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:01.594848 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:01.594848 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.595122 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:01.595416 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:01.597014 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:01.611159 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 544/1080] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:28:01.636958 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708c596-8430-4af0-acc3-9d5022a7588e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:01.678521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:01.679249 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:01.679508 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:01.679688 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.680449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:01.680751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:01.682222 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:01.692767 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 537/1081] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:28:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:01.699289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:01.736930 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:01.823151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e75fd7-ec64-4b07-8115-3a07f5cc5284 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:01.866273 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd6235-7e6f-4131-8219-a63381c007a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:01.866945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd6235-7e6f-4131-8219-a63381c007a4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:01.867232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd6235-7e6f-4131-8219-a63381c007a4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:01.867507 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd6235-7e6f-4131-8219-a63381c007a4 None admin] PATH_INFO: `/v3/users/a67950f0ed34442ba3723243c71a727f` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:01.869032 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd6235-7e6f-4131-8219-a63381c007a4 None admin] RBAC: Authorizing `identity:get_user(user_id=a67950f0ed34442ba3723243c71a727f)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:01.870678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd6235-7e6f-4131-8219-a63381c007a4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:01.872565 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 545/1082] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:01 2022] GET /identity/v3/users/a67950f0ed34442ba3723243c71a727f => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:01.878611 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:01.948553 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:01.996268 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a9385-96ff-4616-ace9-2bd002e52dff None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:02.031313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b0105-1419-4c83-996b-c72bc9dd6cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:02.031959 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b0105-1419-4c83-996b-c72bc9dd6cab None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:02.032235 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b0105-1419-4c83-996b-c72bc9dd6cab None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:02.032478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b0105-1419-4c83-996b-c72bc9dd6cab None admin] PATH_INFO: `/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:02.033784 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b0105-1419-4c83-996b-c72bc9dd6cab None admin] RBAC: Authorizing `identity:get_project(project_id=9c22346d0e2b4c258c3e7dd986659aa9)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:02.035287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b0105-1419-4c83-996b-c72bc9dd6cab None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:02.036968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 538/1083] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:01 2022] GET /identity/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:02.043177 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd6235-7e6f-4131-8219-a63381c007a4 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:02.145031 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:02.149230 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd6235-7e6f-4131-8219-a63381c007a4 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:02.185666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06819aa2-1fa8-42de-bb94-150d88541beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:02.186386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06819aa2-1fa8-42de-bb94-150d88541beb None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:02.186666 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06819aa2-1fa8-42de-bb94-150d88541beb None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:02.186941 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06819aa2-1fa8-42de-bb94-150d88541beb None admin] PATH_INFO: `/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9/users/a67950f0ed34442ba3723243c71a727f/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:02.188941 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06819aa2-1fa8-42de-bb94-150d88541beb None admin] RBAC: Authorizing `identity:create_grant(project_id=9c22346d0e2b4c258c3e7dd986659aa9, user_id=a67950f0ed34442ba3723243c71a727f, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:02.190493 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06819aa2-1fa8-42de-bb94-150d88541beb None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:02.205877 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 546/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:28:02 2022] PUT /identity/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9/users/a67950f0ed34442ba3723243c71a727f/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:28:03.166133 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb10e33-c055-4d5b-bae8-31c898b8badd None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:03.166737 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb10e33-c055-4d5b-bae8-31c898b8badd None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:03.166737 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb10e33-c055-4d5b-bae8-31c898b8badd None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:03.168073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 539/1085] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:03.173646 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64abf8e-dca9-49c4-a6c6-8bfac838d167 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:03.174185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64abf8e-dca9-49c4-a6c6-8bfac838d167 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:03.174385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64abf8e-dca9-49c4-a6c6-8bfac838d167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:03.194347 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-f64abf8e-dca9-49c4-a6c6-8bfac838d167 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:03.195576 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-f64abf8e-dca9-49c4-a6c6-8bfac838d167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:03.270994 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 547/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:03.294636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb10e33-c055-4d5b-bae8-31c898b8badd None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:03.403472 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb10e33-c055-4d5b-bae8-31c898b8badd None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:03.427959 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:03.440671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62271-e4f8-4b0e-bb9e-144ace446872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:03.441330 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62271-e4f8-4b0e-bb9e-144ace446872 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:03.441612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62271-e4f8-4b0e-bb9e-144ace446872 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:03.441876 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62271-e4f8-4b0e-bb9e-144ace446872 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:03.453316 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b62271-e4f8-4b0e-bb9e-144ace446872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-97b62271-e4f8-4b0e-bb9e-144ace446872 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:03.458407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:03.459459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:03.460607 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 540/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:28:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:03.465860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64abf8e-dca9-49c4-a6c6-8bfac838d167 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:03.557534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:03.572325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64abf8e-dca9-49c4-a6c6-8bfac838d167 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:03.611456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:03.612095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:03.612382 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:03.612660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:03.613386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:03.613697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:03.615121 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:03.623791 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 548/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:28:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:03.629773 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62271-e4f8-4b0e-bb9e-144ace446872 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:03.735649 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62271-e4f8-4b0e-bb9e-144ace446872 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:03.770366 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:03.771018 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:03.771295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:03.771563 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:03.781652 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:03.783177 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:28:03.797968 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:03.799573 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:03.800831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:28:03.800831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:28:03.800831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:28:03.800831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:03.800831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 541/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:03.805111 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:03.914755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96617448-f671-42f4-9a6b-74c4713e3d58 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:03.954962 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:03.955762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:03.956042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:03.956323 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:03.957027 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:03.957333 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:03.958747 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:03.971406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 549/1090] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:28:03.977375 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.014352 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:04.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73467bf-7cbd-4dc5-87c5-88566159fc1c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:04.117860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51d4f-4c9c-4f22-947a-f922966e70c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:04.118553 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51d4f-4c9c-4f22-947a-f922966e70c6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.118837 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51d4f-4c9c-4f22-947a-f922966e70c6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.119112 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51d4f-4c9c-4f22-947a-f922966e70c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.126476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51d4f-4c9c-4f22-947a-f922966e70c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:04.128022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51d4f-4c9c-4f22-947a-f922966e70c6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-91d51d4f-4c9c-4f22-947a-f922966e70c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:28:04.140116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:04.141199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:04.142290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 542/1091] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:28:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:04.147608 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.163068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:04.207157 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d443a-9d5a-45a1-8859-d7cc673af35e None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.207745 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d443a-9d5a-45a1-8859-d7cc673af35e None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.207745 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d443a-9d5a-45a1-8859-d7cc673af35e None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.209079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 543/1092] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:28:04.214439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f417d-1bc5-4c15-89c7-9873d5b270a0 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.214731 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f417d-1bc5-4c15-89c7-9873d5b270a0 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.215009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f417d-1bc5-4c15-89c7-9873d5b270a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.233415 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-ed7f417d-1bc5-4c15-89c7-9873d5b270a0 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:04.234681 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-ed7f417d-1bc5-4c15-89c7-9873d5b270a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:04.266201 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38195a15-45b1-4f46-ba43-b6ee9e9f3166 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:04.304265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:04.304902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.305187 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.305468 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.306314 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:04.306626 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:04.307106 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 544/1093] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:28:04.308271 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:04.326848 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 550/1094] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:28:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:04.330471 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f417d-1bc5-4c15-89c7-9873d5b270a0 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.332738 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.355449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:04.404695 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:04.437548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f417d-1bc5-4c15-89c7-9873d5b270a0 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:04.448963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c23bf3-af7b-42eb-bd67-d91fdb997c32 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:04.479460 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4cb9c-c759-481e-8bb1-e705e21be99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:04.480126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4cb9c-c759-481e-8bb1-e705e21be99e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.480404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4cb9c-c759-481e-8bb1-e705e21be99e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.480679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4cb9c-c759-481e-8bb1-e705e21be99e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.489233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192d22a-9982-4cdc-8ad4-b8255b70508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:04.489870 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192d22a-9982-4cdc-8ad4-b8255b70508a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.490206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192d22a-9982-4cdc-8ad4-b8255b70508a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.490469 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192d22a-9982-4cdc-8ad4-b8255b70508a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.493459 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4cb9c-c759-481e-8bb1-e705e21be99e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-1de4cb9c-c759-481e-8bb1-e705e21be99e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:04.498662 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:04.499924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:04.500872 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 545/1095] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:28:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:04.506943 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4cb9c-c759-481e-8bb1-e705e21be99e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.574622 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192d22a-9982-4cdc-8ad4-b8255b70508a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:04.576153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192d22a-9982-4cdc-8ad4-b8255b70508a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:04.607747 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:04.625456 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4cb9c-c759-481e-8bb1-e705e21be99e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:04.651193 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:04.666409 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:04.667055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.667343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.667622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.668371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:04.668675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:04.670089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:04.680621 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 546/1096] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:28:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:28:04.686721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.693317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 551/1097] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:28:04.700206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192d22a-9982-4cdc-8ad4-b8255b70508a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.801091 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a91c9-c49a-4933-884e-8f7ce3240b6d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:04.808096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192d22a-9982-4cdc-8ad4-b8255b70508a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:04.836484 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8310da2-9017-43fb-9099-2005d2c0612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:04.837127 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8310da2-9017-43fb-9099-2005d2c0612e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.837406 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8310da2-9017-43fb-9099-2005d2c0612e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.837673 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8310da2-9017-43fb-9099-2005d2c0612e None admin] PATH_INFO: `/v3/users/a67950f0ed34442ba3723243c71a727f` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.838896 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8310da2-9017-43fb-9099-2005d2c0612e None admin] RBAC: Authorizing `identity:get_user(user_id=a67950f0ed34442ba3723243c71a727f)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:04.840363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8310da2-9017-43fb-9099-2005d2c0612e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:04.842211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 547/1098] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:04 2022] GET /identity/v3/users/a67950f0ed34442ba3723243c71a727f => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:04.848400 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8310da2-9017-43fb-9099-2005d2c0612e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.852001 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:04.852676 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:04.852936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:04.853224 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:04.865211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:04.866754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:04.867126 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:28:04.884113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:04.885963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:04.887097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:28:04.887097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:28:04.887097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:28:04.887097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:28:04.887097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 552/1099] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:28:04.891350 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:04.916040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:04.971387 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8310da2-9017-43fb-9099-2005d2c0612e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:05.004464 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fd793-275b-47cb-b492-9ed0cb1f747b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:05.009017 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac657-3ea5-4a09-8743-8b83dc43fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:05.009664 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac657-3ea5-4a09-8743-8b83dc43fb69 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:05.009948 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac657-3ea5-4a09-8743-8b83dc43fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:05.010259 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac657-3ea5-4a09-8743-8b83dc43fb69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:05.047164 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:05.047799 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:05.048082 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:05.048361 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:05.049064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:05.049369 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:05.050786 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:05.070423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 553/1100] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:28:05.072402 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:05.092516 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac657-3ea5-4a09-8743-8b83dc43fb69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:05.094007 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac657-3ea5-4a09-8743-8b83dc43fb69 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:05.114164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:05.167295 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:05.191784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf2925-a837-4c72-b5f4-a997edc21fc1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:05.223386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 548/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:28:05.229644 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac657-3ea5-4a09-8743-8b83dc43fb69 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:05.235562 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:05.236232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:05.236510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:05.236762 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:05.244771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:05.246268 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:28:05.258975 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:05.260243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:28:05.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 554/1102] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:28:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:28:05.266234 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:05.348831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac657-3ea5-4a09-8743-8b83dc43fb69 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:05.365805 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:05.379258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c62eb-b9c4-4e98-b30f-3fcb81143096 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:05.388207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59929a1-65df-44f6-9736-b7471bfe66ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:05.388882 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59929a1-65df-44f6-9736-b7471bfe66ab None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:05.389169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59929a1-65df-44f6-9736-b7471bfe66ab None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:05.389450 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59929a1-65df-44f6-9736-b7471bfe66ab None admin] PATH_INFO: `/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:05.390682 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59929a1-65df-44f6-9736-b7471bfe66ab None admin] RBAC: Authorizing `identity:get_project(project_id=9c22346d0e2b4c258c3e7dd986659aa9)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:05.392164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59929a1-65df-44f6-9736-b7471bfe66ab None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:05.399420 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 549/1103] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:05 2022] GET /identity/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:05.400383 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59929a1-65df-44f6-9736-b7471bfe66ab None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:05.420876 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:05.421690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:05.422069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:05.422469 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:05.423364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:05.423774 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:05.426396 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:05.437951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:05.442591 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 555/1104] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:28:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:05.448738 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:05.523286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59929a1-65df-44f6-9736-b7471bfe66ab None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:05.563300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ccf1-0dab-465a-b90d-92b9c963aa4d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:05.564022 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca0cf3-48cb-4d06-893d-77d701825d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:05.564676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0cf3-48cb-4d06-893d-77d701825d20 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:05.564956 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0cf3-48cb-4d06-893d-77d701825d20 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:05.565221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0cf3-48cb-4d06-893d-77d701825d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:05.566020 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca0cf3-48cb-4d06-893d-77d701825d20 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=9c22346d0e2b4c258c3e7dd986659aa9, user.id=a67950f0ed34442ba3723243c71a727f) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:05.566360 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca0cf3-48cb-4d06-893d-77d701825d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:05.567879 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca0cf3-48cb-4d06-893d-77d701825d20 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:05.575186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 550/1105] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:28:05 2022] GET /identity/v3/role_assignments?user.id=a67950f0ed34442ba3723243c71a727f&scope.project.id=9c22346d0e2b4c258c3e7dd986659aa9&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:28:05.620408 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ec6b8-6fb4-49e0-9911-3c976f7855fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:05.621147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ec6b8-6fb4-49e0-9911-3c976f7855fe None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:05.621588 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ec6b8-6fb4-49e0-9911-3c976f7855fe None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:05.622011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ec6b8-6fb4-49e0-9911-3c976f7855fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:05.623317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ec6b8-6fb4-49e0-9911-3c976f7855fe None admin] RBAC: Adding query filter params (role.id=678a59243ee345488ef8c869ef84f779, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=c5ba9e7780f5437caac949931ca55b80) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:05.623718 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ec6b8-6fb4-49e0-9911-3c976f7855fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:05.625771 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ec6b8-6fb4-49e0-9911-3c976f7855fe None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:05.637610 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 556/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:28:05 2022] GET /identity/v3/role_assignments?user.id=c5ba9e7780f5437caac949931ca55b80&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=678a59243ee345488ef8c869ef84f779 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:28:07.697668 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1bfef-a50b-477b-afe0-d0742f139e99 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:07.698323 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1bfef-a50b-477b-afe0-d0742f139e99 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:07.698323 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1bfef-a50b-477b-afe0-d0742f139e99 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:07.699601 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 551/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:07.705215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dfb37-b600-4beb-ae93-feb1d3898fc2 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:07.705721 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dfb37-b600-4beb-ae93-feb1d3898fc2 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:07.705721 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dfb37-b600-4beb-ae93-feb1d3898fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:07.726925 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-9c5dfb37-b600-4beb-ae93-feb1d3898fc2 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:07.728231 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-9c5dfb37-b600-4beb-ae93-feb1d3898fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:07.793845 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:07.804196 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 557/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:07.829522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1bfef-a50b-477b-afe0-d0742f139e99 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:07.883983 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:07.947533 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1bfef-a50b-477b-afe0-d0742f139e99 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:07.985065 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebb248-8176-4c0c-9a40-67981e8c1dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:07.985730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebb248-8176-4c0c-9a40-67981e8c1dea None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:07.986011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebb248-8176-4c0c-9a40-67981e8c1dea None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:07.986353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebb248-8176-4c0c-9a40-67981e8c1dea None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:07.987519 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebb248-8176-4c0c-9a40-67981e8c1dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:07.988991 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebb248-8176-4c0c-9a40-67981e8c1dea None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:07.990690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 552/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:28:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:07.997058 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dfb37-b600-4beb-ae93-feb1d3898fc2 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:08.113837 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dfb37-b600-4beb-ae93-feb1d3898fc2 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:08.152830 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499ed0-b8f8-4678-9fef-e68e2ffbfcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:08.153465 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499ed0-b8f8-4678-9fef-e68e2ffbfcf4 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:08.153744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499ed0-b8f8-4678-9fef-e68e2ffbfcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:08.154019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499ed0-b8f8-4678-9fef-e68e2ffbfcf4 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:08.154758 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f499ed0-b8f8-4678-9fef-e68e2ffbfcf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:08.156214 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f499ed0-b8f8-4678-9fef-e68e2ffbfcf4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:08.180978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 558/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:28:07 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:28:10.285496 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc696191-897d-4894-80d5-3a6a20cba142 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:10.286165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc696191-897d-4894-80d5-3a6a20cba142 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:10.286165 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc696191-897d-4894-80d5-3a6a20cba142 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:10.287427 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 553/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:10.293557 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da6caa-e7d3-4602-9ba8-97a1aace2d2b None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:10.294125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da6caa-e7d3-4602-9ba8-97a1aace2d2b None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:10.294125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da6caa-e7d3-4602-9ba8-97a1aace2d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:10.314072 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-78da6caa-e7d3-4602-9ba8-97a1aace2d2b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:10.315581 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-78da6caa-e7d3-4602-9ba8-97a1aace2d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:10.373960 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:10.399448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 559/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:28:10.423221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc696191-897d-4894-80d5-3a6a20cba142 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:10.534954 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc696191-897d-4894-80d5-3a6a20cba142 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:10.552735 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:10.576363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c706d1-13fb-4840-873b-6c43a22fd8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:10.577034 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c706d1-13fb-4840-873b-6c43a22fd8ee None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:10.577317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c706d1-13fb-4840-873b-6c43a22fd8ee None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:10.577594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c706d1-13fb-4840-873b-6c43a22fd8ee None admin] PATH_INFO: `/v3/roles/d883529ef5284c35b27b759c57353383` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:10.581538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c706d1-13fb-4840-873b-6c43a22fd8ee None admin] RBAC: Authorizing `identity:get_role(role_id=d883529ef5284c35b27b759c57353383)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:10.583003 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c706d1-13fb-4840-873b-6c43a22fd8ee None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:10.584453 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 554/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:10 2022] GET /identity/v3/roles/d883529ef5284c35b27b759c57353383 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:10.590873 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da6caa-e7d3-4602-9ba8-97a1aace2d2b None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:10.716555 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da6caa-e7d3-4602-9ba8-97a1aace2d2b None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:10.755325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8c09-b65d-407e-a908-e613c200f237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:10.755986 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8c09-b65d-407e-a908-e613c200f237 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:10.756272 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8c09-b65d-407e-a908-e613c200f237 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:10.756550 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8c09-b65d-407e-a908-e613c200f237 None admin] PATH_INFO: `/v3/users/cc3b2e706a264fd6ae57d071e55a49b9` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:10.772559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e8c09-b65d-407e-a908-e613c200f237 None admin] RBAC: Authorizing `identity:get_user(user_id=cc3b2e706a264fd6ae57d071e55a49b9)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:10.774099 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e8c09-b65d-407e-a908-e613c200f237 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:10.775996 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 560/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:10 2022] GET /identity/v3/users/cc3b2e706a264fd6ae57d071e55a49b9 => generated 286 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:10.782364 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c706d1-13fb-4840-873b-6c43a22fd8ee None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:10.892895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c706d1-13fb-4840-873b-6c43a22fd8ee None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:10.932934 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89735-73b8-41aa-a776-4b6972f91ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:10.933632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89735-73b8-41aa-a776-4b6972f91ba7 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:10.933933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89735-73b8-41aa-a776-4b6972f91ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:10.934246 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89735-73b8-41aa-a776-4b6972f91ba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:11.013288 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:11.042479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d89735-73b8-41aa-a776-4b6972f91ba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:11.044670 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d89735-73b8-41aa-a776-4b6972f91ba7 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:11.158526 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 555/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:28:11.165150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8c09-b65d-407e-a908-e613c200f237 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:11.209350 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:11.293071 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8c09-b65d-407e-a908-e613c200f237 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:11.335969 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82382e25-7a6c-4d2f-b563-3da4cfc0c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:11.336689 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82382e25-7a6c-4d2f-b563-3da4cfc0c850 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:11.336909 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82382e25-7a6c-4d2f-b563-3da4cfc0c850 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:11.337219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82382e25-7a6c-4d2f-b563-3da4cfc0c850 None admin] PATH_INFO: `/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:11.338629 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82382e25-7a6c-4d2f-b563-3da4cfc0c850 None admin] RBAC: Authorizing `identity:get_project(project_id=9c22346d0e2b4c258c3e7dd986659aa9)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:11.340097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82382e25-7a6c-4d2f-b563-3da4cfc0c850 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:11.342159 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 561/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:11 2022] GET /identity/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:11.349882 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89735-73b8-41aa-a776-4b6972f91ba7 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:11.440087 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:11.460519 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89735-73b8-41aa-a776-4b6972f91ba7 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:11.496987 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f19f0-7590-46f6-9590-1172a216390e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:11.497622 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f19f0-7590-46f6-9590-1172a216390e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:11.497904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f19f0-7590-46f6-9590-1172a216390e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:11.498204 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f19f0-7590-46f6-9590-1172a216390e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:11.499010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f19f0-7590-46f6-9590-1172a216390e None admin] RBAC: Adding query filter params (role.id=d883529ef5284c35b27b759c57353383, scope.project.id=9c22346d0e2b4c258c3e7dd986659aa9, user.id=cc3b2e706a264fd6ae57d071e55a49b9) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:11.499319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f19f0-7590-46f6-9590-1172a216390e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:11.500860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f19f0-7590-46f6-9590-1172a216390e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:11.506854 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 556/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:28:11 2022] GET /identity/v3/role_assignments?user.id=cc3b2e706a264fd6ae57d071e55a49b9&scope.project.id=9c22346d0e2b4c258c3e7dd986659aa9&role.id=d883529ef5284c35b27b759c57353383 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:28:13.566444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbaf20-56fd-44eb-9904-f26bc6570802 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:13.567050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbaf20-56fd-44eb-9904-f26bc6570802 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:13.567050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbaf20-56fd-44eb-9904-f26bc6570802 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:13.568423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 562/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:13.574991 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829a905-a937-4e6f-878b-ba8e937072a5 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:13.575713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829a905-a937-4e6f-878b-ba8e937072a5 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:13.575713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829a905-a937-4e6f-878b-ba8e937072a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:13.596971 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-5829a905-a937-4e6f-878b-ba8e937072a5 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:13.598398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-5829a905-a937-4e6f-878b-ba8e937072a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:13.688072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 557/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:13.718889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbaf20-56fd-44eb-9904-f26bc6570802 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:13.839892 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbaf20-56fd-44eb-9904-f26bc6570802 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:13.851846 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:13.884087 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115a40-bf9c-4f9d-a421-8b8b1eb4d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:13.884833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115a40-bf9c-4f9d-a421-8b8b1eb4d239 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:13.885135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115a40-bf9c-4f9d-a421-8b8b1eb4d239 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:13.885433 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115a40-bf9c-4f9d-a421-8b8b1eb4d239 None admin] PATH_INFO: `/v3/roles/d883529ef5284c35b27b759c57353383` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:13.887128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48115a40-bf9c-4f9d-a421-8b8b1eb4d239 None admin] RBAC: Authorizing `identity:get_role(role_id=d883529ef5284c35b27b759c57353383)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:13.888640 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48115a40-bf9c-4f9d-a421-8b8b1eb4d239 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:13.890182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 563/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:13 2022] GET /identity/v3/roles/d883529ef5284c35b27b759c57353383 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:13.897115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829a905-a937-4e6f-878b-ba8e937072a5 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:13.983947 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:14.012245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829a905-a937-4e6f-878b-ba8e937072a5 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:14.051438 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b41bb-599d-4498-9d41-09d43ed68507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:14.052210 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b41bb-599d-4498-9d41-09d43ed68507 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:14.052504 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b41bb-599d-4498-9d41-09d43ed68507 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:14.052905 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b41bb-599d-4498-9d41-09d43ed68507 None admin] PATH_INFO: `/v3/users/cc3b2e706a264fd6ae57d071e55a49b9` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:14.054362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b41bb-599d-4498-9d41-09d43ed68507 None admin] RBAC: Authorizing `identity:get_user(user_id=cc3b2e706a264fd6ae57d071e55a49b9)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:14.056006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b41bb-599d-4498-9d41-09d43ed68507 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:14.057895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 558/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:13 2022] GET /identity/v3/users/cc3b2e706a264fd6ae57d071e55a49b9 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:14.064838 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115a40-bf9c-4f9d-a421-8b8b1eb4d239 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:14.181340 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115a40-bf9c-4f9d-a421-8b8b1eb4d239 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:14.222518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca149b34-f259-4a2b-9ea9-16a80643620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:14.223314 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149b34-f259-4a2b-9ea9-16a80643620b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:14.223639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149b34-f259-4a2b-9ea9-16a80643620b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:14.224000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149b34-f259-4a2b-9ea9-16a80643620b None admin] PATH_INFO: `/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:14.225399 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca149b34-f259-4a2b-9ea9-16a80643620b None admin] RBAC: Authorizing `identity:get_project(project_id=9c22346d0e2b4c258c3e7dd986659aa9)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:14.227349 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca149b34-f259-4a2b-9ea9-16a80643620b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:14.229454 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 564/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:14 2022] GET /identity/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:14.236287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b41bb-599d-4498-9d41-09d43ed68507 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:14.356371 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b41bb-599d-4498-9d41-09d43ed68507 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:14.399078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5612c-038e-4ecb-8075-3e36c22241e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:14.399815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5612c-038e-4ecb-8075-3e36c22241e6 None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:14.400140 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5612c-038e-4ecb-8075-3e36c22241e6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:14.400423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5612c-038e-4ecb-8075-3e36c22241e6 None admin] PATH_INFO: `/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9/users/cc3b2e706a264fd6ae57d071e55a49b9/roles/d883529ef5284c35b27b759c57353383` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:14.403078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5612c-038e-4ecb-8075-3e36c22241e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c22346d0e2b4c258c3e7dd986659aa9, user_id=cc3b2e706a264fd6ae57d071e55a49b9, role_id=d883529ef5284c35b27b759c57353383)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:14.404559 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5612c-038e-4ecb-8075-3e36c22241e6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:14.421463 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 559/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:28:14 2022] PUT /identity/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9/users/cc3b2e706a264fd6ae57d071e55a49b9/roles/d883529ef5284c35b27b759c57353383 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:28:16.654613 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b0039-ee4f-4fd9-94ed-a88411ec7ca5 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:16.655234 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b0039-ee4f-4fd9-94ed-a88411ec7ca5 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:16.655234 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b0039-ee4f-4fd9-94ed-a88411ec7ca5 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:16.656703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 565/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:16.664616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b02c1-bf0b-4cab-9e67-45ad73e242f0 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:16.665218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b02c1-bf0b-4cab-9e67-45ad73e242f0 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:16.665218 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b02c1-bf0b-4cab-9e67-45ad73e242f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:16.673057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:16.687171 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-0e2b02c1-bf0b-4cab-9e67-45ad73e242f0 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:16.688690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-0e2b02c1-bf0b-4cab-9e67-45ad73e242f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:16.769082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 560/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:16.796610 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b0039-ee4f-4fd9-94ed-a88411ec7ca5 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:16.853361 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:16.948219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b0039-ee4f-4fd9-94ed-a88411ec7ca5 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:16.988919 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976181f8-1667-4025-9b1f-d77a041fa5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:16.989902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976181f8-1667-4025-9b1f-d77a041fa5d3 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:16.990242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976181f8-1667-4025-9b1f-d77a041fa5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:16.990524 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976181f8-1667-4025-9b1f-d77a041fa5d3 None admin] PATH_INFO: `/v3/roles/d883529ef5284c35b27b759c57353383` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:16.992158 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976181f8-1667-4025-9b1f-d77a041fa5d3 None admin] RBAC: Authorizing `identity:get_role(role_id=d883529ef5284c35b27b759c57353383)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:16.993619 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976181f8-1667-4025-9b1f-d77a041fa5d3 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:16.995215 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 566/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:16 2022] GET /identity/v3/roles/d883529ef5284c35b27b759c57353383 => generated 223 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:17.001545 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b02c1-bf0b-4cab-9e67-45ad73e242f0 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:17.115219 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b02c1-bf0b-4cab-9e67-45ad73e242f0 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:17.155557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e9ed0-6667-48bc-8f9d-41a75aa574e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:17.156294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e9ed0-6667-48bc-8f9d-41a75aa574e2 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:17.156512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e9ed0-6667-48bc-8f9d-41a75aa574e2 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:17.156894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e9ed0-6667-48bc-8f9d-41a75aa574e2 None admin] PATH_INFO: `/v3/users/cc3b2e706a264fd6ae57d071e55a49b9` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:17.158351 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e9ed0-6667-48bc-8f9d-41a75aa574e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cc3b2e706a264fd6ae57d071e55a49b9)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:17.159931 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e9ed0-6667-48bc-8f9d-41a75aa574e2 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:17.161811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 561/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:17 2022] GET /identity/v3/users/cc3b2e706a264fd6ae57d071e55a49b9 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:17.170247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976181f8-1667-4025-9b1f-d77a041fa5d3 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:17.302135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976181f8-1667-4025-9b1f-d77a041fa5d3 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:17.314629 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:17.341289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da938c-3acb-4427-a1f4-9bfc20e72a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:17.342014 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da938c-3acb-4427-a1f4-9bfc20e72a56 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:17.342319 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da938c-3acb-4427-a1f4-9bfc20e72a56 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:17.342601 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da938c-3acb-4427-a1f4-9bfc20e72a56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:17.433773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da938c-3acb-4427-a1f4-9bfc20e72a56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:17.435401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da938c-3acb-4427-a1f4-9bfc20e72a56 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:17.573613 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 567/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:28:17.580114 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e9ed0-6667-48bc-8f9d-41a75aa574e2 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:17.583761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:17.697667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e9ed0-6667-48bc-8f9d-41a75aa574e2 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:17.735765 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab262e-c550-4cba-9581-cd726fa9088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:17.736467 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab262e-c550-4cba-9581-cd726fa9088e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:17.736751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab262e-c550-4cba-9581-cd726fa9088e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:17.737027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab262e-c550-4cba-9581-cd726fa9088e None admin] PATH_INFO: `/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:17.738267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab262e-c550-4cba-9581-cd726fa9088e None admin] RBAC: Authorizing `identity:get_project(project_id=9c22346d0e2b4c258c3e7dd986659aa9)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:17.739814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab262e-c550-4cba-9581-cd726fa9088e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:17.741721 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 562/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:17 2022] GET /identity/v3/projects/9c22346d0e2b4c258c3e7dd986659aa9 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:17.748877 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da938c-3acb-4427-a1f4-9bfc20e72a56 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:17.761950 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:17.882874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da938c-3acb-4427-a1f4-9bfc20e72a56 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:17.924362 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2501c-82bf-47e1-89fa-edc2aa05c529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:17.925010 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2501c-82bf-47e1-89fa-edc2aa05c529 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:17.925286 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2501c-82bf-47e1-89fa-edc2aa05c529 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:17.925557 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2501c-82bf-47e1-89fa-edc2aa05c529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:17.926404 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2501c-82bf-47e1-89fa-edc2aa05c529 None admin] RBAC: Adding query filter params (role.id=d883529ef5284c35b27b759c57353383, scope.project.id=9c22346d0e2b4c258c3e7dd986659aa9, user.id=cc3b2e706a264fd6ae57d071e55a49b9) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:17.926716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2501c-82bf-47e1-89fa-edc2aa05c529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:17.928251 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2501c-82bf-47e1-89fa-edc2aa05c529 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:17.935459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 568/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:28:17 2022] GET /identity/v3/role_assignments?user.id=cc3b2e706a264fd6ae57d071e55a49b9&scope.project.id=9c22346d0e2b4c258c3e7dd986659aa9&role.id=d883529ef5284c35b27b759c57353383 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:20.039209 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add42ee-fe94-4d09-ae86-6a51c81da6c1 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:20.039821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add42ee-fe94-4d09-ae86-6a51c81da6c1 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:20.039821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add42ee-fe94-4d09-ae86-6a51c81da6c1 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:20.041152 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 563/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:20.046596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592c387-ecaf-401f-aa84-c176f78bec84 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:20.047149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592c387-ecaf-401f-aa84-c176f78bec84 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:20.047149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592c387-ecaf-401f-aa84-c176f78bec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:20.067661 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-e592c387-ecaf-401f-aa84-c176f78bec84 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:20.069244 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-e592c387-ecaf-401f-aa84-c176f78bec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:20.145641 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 569/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:20.169096 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add42ee-fe94-4d09-ae86-6a51c81da6c1 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:20.245010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:20.280606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add42ee-fe94-4d09-ae86-6a51c81da6c1 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:20.317612 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8264c-5205-4b39-9ec0-d9814ae6dff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:20.318189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8264c-5205-4b39-9ec0-d9814ae6dff3 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:20.318482 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8264c-5205-4b39-9ec0-d9814ae6dff3 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:20.318753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8264c-5205-4b39-9ec0-d9814ae6dff3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:20.320334 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8264c-5205-4b39-9ec0-d9814ae6dff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:20.321830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8264c-5205-4b39-9ec0-d9814ae6dff3 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:20.323782 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 564/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:28:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:20.330380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592c387-ecaf-401f-aa84-c176f78bec84 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:20.333820 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:20.436632 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592c387-ecaf-401f-aa84-c176f78bec84 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:20.473862 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d57f5c-0f4f-49dc-8bb9-bd0f089f882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:20.474525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d57f5c-0f4f-49dc-8bb9-bd0f089f882e None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:20.474811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d57f5c-0f4f-49dc-8bb9-bd0f089f882e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:20.475089 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d57f5c-0f4f-49dc-8bb9-bd0f089f882e None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:20.475781 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d57f5c-0f4f-49dc-8bb9-bd0f089f882e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:20.477243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d57f5c-0f4f-49dc-8bb9-bd0f089f882e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:20.506535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 570/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:28:20 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:28:22.547074 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850b72-8593-4d7f-a0c3-1efff5154adc None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:22.547805 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850b72-8593-4d7f-a0c3-1efff5154adc None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:22.547934 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850b72-8593-4d7f-a0c3-1efff5154adc None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:22.550186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 565/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:22.556432 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15ae7-6bf5-41d8-b112-770098c8966b None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:22.556998 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15ae7-6bf5-41d8-b112-770098c8966b None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:22.557265 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15ae7-6bf5-41d8-b112-770098c8966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:22.581917 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-09d15ae7-6bf5-41d8-b112-770098c8966b None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:22.583442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-09d15ae7-6bf5-41d8-b112-770098c8966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:22.647617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:22.672336 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 571/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:22.697225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01850b72-8593-4d7f-a0c3-1efff5154adc None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:22.836189 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01850b72-8593-4d7f-a0c3-1efff5154adc None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:22.894097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f656a49-f3ad-403b-972e-32897f2767eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:22.894796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f656a49-f3ad-403b-972e-32897f2767eb None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:22.895077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f656a49-f3ad-403b-972e-32897f2767eb None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:22.895351 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f656a49-f3ad-403b-972e-32897f2767eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:22.896664 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f656a49-f3ad-403b-972e-32897f2767eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:22.898154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f656a49-f3ad-403b-972e-32897f2767eb None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:22.899820 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 566/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:28:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:22.906135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d15ae7-6bf5-41d8-b112-770098c8966b None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:23.039589 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d15ae7-6bf5-41d8-b112-770098c8966b None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:23.081176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573aabe-5255-4aaf-9352-df040a981b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:23.081832 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573aabe-5255-4aaf-9352-df040a981b90 None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:23.082149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573aabe-5255-4aaf-9352-df040a981b90 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:23.082431 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573aabe-5255-4aaf-9352-df040a981b90 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:23.083154 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573aabe-5255-4aaf-9352-df040a981b90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:23.084646 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573aabe-5255-4aaf-9352-df040a981b90 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:23.109942 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 572/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:28:22 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:28:25.213676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24680a3f-b9a1-4c76-bcda-dc9da854f1a6 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:25.214324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24680a3f-b9a1-4c76-bcda-dc9da854f1a6 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:25.214324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24680a3f-b9a1-4c76-bcda-dc9da854f1a6 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:25.215625 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 567/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:25.221081 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bad6b1-f7ca-45f8-99eb-fbd5b8212ebf None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:25.221510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bad6b1-f7ca-45f8-99eb-fbd5b8212ebf None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:25.221625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bad6b1-f7ca-45f8-99eb-fbd5b8212ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:25.240977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-c5bad6b1-f7ca-45f8-99eb-fbd5b8212ebf None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:25.242235 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-c5bad6b1-f7ca-45f8-99eb-fbd5b8212ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:25.280211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:25.318559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 573/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:25.342146 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24680a3f-b9a1-4c76-bcda-dc9da854f1a6 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:25.351883 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:25.461168 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24680a3f-b9a1-4c76-bcda-dc9da854f1a6 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:25.498671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfe62f-8893-4150-ab2a-df59ba048576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:25.499240 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe62f-8893-4150-ab2a-df59ba048576 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:25.499523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe62f-8893-4150-ab2a-df59ba048576 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:25.499801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe62f-8893-4150-ab2a-df59ba048576 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:25.510732 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfe62f-8893-4150-ab2a-df59ba048576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-06bfe62f-8893-4150-ab2a-df59ba048576 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:25.516350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:25.519324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:25.521324 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 568/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:28:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:25.527015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bad6b1-f7ca-45f8-99eb-fbd5b8212ebf None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:25.635944 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bad6b1-f7ca-45f8-99eb-fbd5b8212ebf None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:25.672270 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:25.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:25.673186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:25.673515 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:25.674236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:25.674572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:25.675968 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:25.684705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 574/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:28:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:25.690899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfe62f-8893-4150-ab2a-df59ba048576 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:25.763439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:25.801715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfe62f-8893-4150-ab2a-df59ba048576 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:25.838623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833e63-2c9b-4a97-8216-0ed0c4e85fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:25.839277 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833e63-2c9b-4a97-8216-0ed0c4e85fca None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:25.839560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833e63-2c9b-4a97-8216-0ed0c4e85fca None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:25.839840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833e63-2c9b-4a97-8216-0ed0c4e85fca None admin] PATH_INFO: `/v3/users/8ca56ce70eb44e9d829ffda957fa5314` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:25.855140 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5833e63-2c9b-4a97-8216-0ed0c4e85fca None admin] RBAC: Authorizing `identity:get_user(user_id=8ca56ce70eb44e9d829ffda957fa5314)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:25.856628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5833e63-2c9b-4a97-8216-0ed0c4e85fca None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:25.858385 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 569/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:25 2022] GET /identity/v3/users/8ca56ce70eb44e9d829ffda957fa5314 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:25.864840 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:25.905793 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:25.977759 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520905-2890-45c3-9394-353fb0990a65 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:26.014908 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e40e2-e234-4ee1-9572-32c25901b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:26.015450 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e40e2-e234-4ee1-9572-32c25901b67c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:26.015731 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e40e2-e234-4ee1-9572-32c25901b67c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:26.016019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e40e2-e234-4ee1-9572-32c25901b67c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:26.093920 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e40e2-e234-4ee1-9572-32c25901b67c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:26.095566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e40e2-e234-4ee1-9572-32c25901b67c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:26.132151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:26.204817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 575/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:28:26.211305 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833e63-2c9b-4a97-8216-0ed0c4e85fca None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:26.315025 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833e63-2c9b-4a97-8216-0ed0c4e85fca None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:26.339507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:26.351548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb952e-5a10-4bca-a10e-9ed60034dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:26.352214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb952e-5a10-4bca-a10e-9ed60034dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:26.352497 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb952e-5a10-4bca-a10e-9ed60034dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:26.352773 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb952e-5a10-4bca-a10e-9ed60034dfa0 None admin] PATH_INFO: `/v3/projects/d4b91945558e4c66ba2fb4e8db38341a` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:26.353997 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb952e-5a10-4bca-a10e-9ed60034dfa0 None admin] RBAC: Authorizing `identity:get_project(project_id=d4b91945558e4c66ba2fb4e8db38341a)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:26.355495 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb952e-5a10-4bca-a10e-9ed60034dfa0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:26.357221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 570/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:26 2022] GET /identity/v3/projects/d4b91945558e4c66ba2fb4e8db38341a => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:26.363841 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e40e2-e234-4ee1-9572-32c25901b67c None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:26.471567 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e40e2-e234-4ee1-9572-32c25901b67c None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:26.508335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1262c0-9cc5-49b4-859f-cb93b8245b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:26.509092 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1262c0-9cc5-49b4-859f-cb93b8245b78 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:26.509520 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1262c0-9cc5-49b4-859f-cb93b8245b78 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:26.509896 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1262c0-9cc5-49b4-859f-cb93b8245b78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:26.511065 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1262c0-9cc5-49b4-859f-cb93b8245b78 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=d4b91945558e4c66ba2fb4e8db38341a, user.id=8ca56ce70eb44e9d829ffda957fa5314) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:26.511295 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1262c0-9cc5-49b4-859f-cb93b8245b78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:26.513243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1262c0-9cc5-49b4-859f-cb93b8245b78 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:26.516186 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:26.522203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 576/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:28:26 2022] GET /identity/v3/role_assignments?user.id=8ca56ce70eb44e9d829ffda957fa5314&scope.project.id=d4b91945558e4c66ba2fb4e8db38341a&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:28.527861 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ba02d-b85a-4368-af40-033188b73362 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:28.528403 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ba02d-b85a-4368-af40-033188b73362 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:28.528717 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ba02d-b85a-4368-af40-033188b73362 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:28.531067 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 571/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:28.537040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43782215-d6b7-4363-944f-c8f470991cc7 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:28.537527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43782215-d6b7-4363-944f-c8f470991cc7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:28.537633 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43782215-d6b7-4363-944f-c8f470991cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:28.556012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-43782215-d6b7-4363-944f-c8f470991cc7 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:28.557227 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-43782215-d6b7-4363-944f-c8f470991cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:28.633653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 577/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:28.656424 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ba02d-b85a-4368-af40-033188b73362 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:28.767866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ba02d-b85a-4368-af40-033188b73362 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:28.804993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943d575-e773-4233-b8b5-607738aa5086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:28.805563 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943d575-e773-4233-b8b5-607738aa5086 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:28.805842 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943d575-e773-4233-b8b5-607738aa5086 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:28.806431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943d575-e773-4233-b8b5-607738aa5086 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:28.823506 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943d575-e773-4233-b8b5-607738aa5086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-a943d575-e773-4233-b8b5-607738aa5086 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:28.828113 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:28.830321 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:28.831480 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 572/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:28:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:28.836085 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43782215-d6b7-4363-944f-c8f470991cc7 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:28.947349 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43782215-d6b7-4363-944f-c8f470991cc7 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:28.977742 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:28.990436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:28.990436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:28.990436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:28.990436 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:28.991137 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:28.991137 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:28.992533 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:29.001542 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 578/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:28:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:29.007923 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943d575-e773-4233-b8b5-607738aa5086 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:29.065460 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:29.113783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943d575-e773-4233-b8b5-607738aa5086 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:29.150174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ccfab-8ccf-49a8-b517-2f1a15aec106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:29.150840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ccfab-8ccf-49a8-b517-2f1a15aec106 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:29.151115 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ccfab-8ccf-49a8-b517-2f1a15aec106 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:29.151396 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ccfab-8ccf-49a8-b517-2f1a15aec106 None admin] PATH_INFO: `/v3/users/8ca56ce70eb44e9d829ffda957fa5314` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:29.152722 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ccfab-8ccf-49a8-b517-2f1a15aec106 None admin] RBAC: Authorizing `identity:get_user(user_id=8ca56ce70eb44e9d829ffda957fa5314)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:29.154212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ccfab-8ccf-49a8-b517-2f1a15aec106 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:29.155873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 573/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:29 2022] GET /identity/v3/users/8ca56ce70eb44e9d829ffda957fa5314 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:29.162300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:29.277527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03535811-e623-43ce-957f-40eefadabfc7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:29.316270 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc8919-fb52-481c-bf9f-cb185299f2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:29.316826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8919-fb52-481c-bf9f-cb185299f2da None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:29.317103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8919-fb52-481c-bf9f-cb185299f2da None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:29.317381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8919-fb52-481c-bf9f-cb185299f2da None admin] PATH_INFO: `/v3/projects/d4b91945558e4c66ba2fb4e8db38341a` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:29.318602 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc8919-fb52-481c-bf9f-cb185299f2da None admin] RBAC: Authorizing `identity:get_project(project_id=d4b91945558e4c66ba2fb4e8db38341a)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:29.320430 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc8919-fb52-481c-bf9f-cb185299f2da None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:29.322135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 579/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:29 2022] GET /identity/v3/projects/d4b91945558e4c66ba2fb4e8db38341a => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:29.328609 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ccfab-8ccf-49a8-b517-2f1a15aec106 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:29.450937 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ccfab-8ccf-49a8-b517-2f1a15aec106 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:29.472873 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:29.489921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8baf0-8f13-48ce-b6fd-27e3340b9b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:29.490644 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8baf0-8f13-48ce-b6fd-27e3340b9b9d None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:29.490933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8baf0-8f13-48ce-b6fd-27e3340b9b9d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:29.491212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8baf0-8f13-48ce-b6fd-27e3340b9b9d None admin] PATH_INFO: `/v3/projects/d4b91945558e4c66ba2fb4e8db38341a/users/8ca56ce70eb44e9d829ffda957fa5314/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:29.493451 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8baf0-8f13-48ce-b6fd-27e3340b9b9d None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b91945558e4c66ba2fb4e8db38341a, user_id=8ca56ce70eb44e9d829ffda957fa5314, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:29.494953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8baf0-8f13-48ce-b6fd-27e3340b9b9d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:29.510741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 574/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:28:29 2022] PUT /identity/v3/projects/d4b91945558e4c66ba2fb4e8db38341a/users/8ca56ce70eb44e9d829ffda957fa5314/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:28:31.562445 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc13f6-11ed-4752-9cc2-81d113b45f25 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:31.563048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc13f6-11ed-4752-9cc2-81d113b45f25 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:31.563048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc13f6-11ed-4752-9cc2-81d113b45f25 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:31.564378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 580/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:28:31.572675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53a30-6a0a-457f-b2f2-9aa4bf64388a None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:31.573264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53a30-6a0a-457f-b2f2-9aa4bf64388a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:31.573264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53a30-6a0a-457f-b2f2-9aa4bf64388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:31.600092 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-08c53a30-6a0a-457f-b2f2-9aa4bf64388a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:31.601715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-08c53a30-6a0a-457f-b2f2-9aa4bf64388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:31.682908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 575/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:31.706554 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc13f6-11ed-4752-9cc2-81d113b45f25 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:31.774231 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:31.828243 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc13f6-11ed-4752-9cc2-81d113b45f25 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:31.865121 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec772e98-df39-45a8-a676-f666ed6a213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:31.865788 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec772e98-df39-45a8-a676-f666ed6a213f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:31.866075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec772e98-df39-45a8-a676-f666ed6a213f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:31.866389 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec772e98-df39-45a8-a676-f666ed6a213f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:31.879668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec772e98-df39-45a8-a676-f666ed6a213f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-ec772e98-df39-45a8-a676-f666ed6a213f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:31.884750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:31.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:28:31.887630 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 581/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:31.893421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c53a30-6a0a-457f-b2f2-9aa4bf64388a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:32.002319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c53a30-6a0a-457f-b2f2-9aa4bf64388a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:32.014891 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:32.048830 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:32.049712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:32.050147 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:32.050564 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:32.051841 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:32.052354 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:32.054700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:32.067849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 576/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:32.074518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec772e98-df39-45a8-a676-f666ed6a213f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:32.186377 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec772e98-df39-45a8-a676-f666ed6a213f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:32.204654 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:32.223532 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cde5b1-833e-4508-985a-667b5a648699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:32.224194 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cde5b1-833e-4508-985a-667b5a648699 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:32.224472 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cde5b1-833e-4508-985a-667b5a648699 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:32.224751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cde5b1-833e-4508-985a-667b5a648699 None admin] PATH_INFO: `/v3/users/8ca56ce70eb44e9d829ffda957fa5314` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:32.226018 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cde5b1-833e-4508-985a-667b5a648699 None admin] RBAC: Authorizing `identity:get_user(user_id=8ca56ce70eb44e9d829ffda957fa5314)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:32.227509 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cde5b1-833e-4508-985a-667b5a648699 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:32.229144 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 582/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:32 2022] GET /identity/v3/users/8ca56ce70eb44e9d829ffda957fa5314 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:32.235520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:32.347232 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910f8bd-bde7-4cc9-94d6-3ed63f4fc991 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:32.383079 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448ed19-243c-4a8b-855a-25fd79899a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:32.383709 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448ed19-243c-4a8b-855a-25fd79899a60 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:32.383989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448ed19-243c-4a8b-855a-25fd79899a60 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:32.384259 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448ed19-243c-4a8b-855a-25fd79899a60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:32.427939 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:32.456632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448ed19-243c-4a8b-855a-25fd79899a60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:32.458156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448ed19-243c-4a8b-855a-25fd79899a60 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:32.579616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 577/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:28:32.585905 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cde5b1-833e-4508-985a-667b5a648699 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:32.692118 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cde5b1-833e-4508-985a-667b5a648699 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:32.728727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f936509-2577-4bd6-82f8-548effa098e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:32.729391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f936509-2577-4bd6-82f8-548effa098e9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:32.729673 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f936509-2577-4bd6-82f8-548effa098e9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:32.729948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f936509-2577-4bd6-82f8-548effa098e9 None admin] PATH_INFO: `/v3/projects/d4b91945558e4c66ba2fb4e8db38341a` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:32.731206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f936509-2577-4bd6-82f8-548effa098e9 None admin] RBAC: Authorizing `identity:get_project(project_id=d4b91945558e4c66ba2fb4e8db38341a)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:32.732668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f936509-2577-4bd6-82f8-548effa098e9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:32.734403 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 583/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:32 2022] GET /identity/v3/projects/d4b91945558e4c66ba2fb4e8db38341a => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:32.741071 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448ed19-243c-4a8b-855a-25fd79899a60 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:32.817395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:32.848497 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448ed19-243c-4a8b-855a-25fd79899a60 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:32.883404 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ddc43-884e-41e6-a493-c0aea934fd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:32.884033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ddc43-884e-41e6-a493-c0aea934fd86 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:32.884311 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ddc43-884e-41e6-a493-c0aea934fd86 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:32.884579 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ddc43-884e-41e6-a493-c0aea934fd86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:32.885373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ddc43-884e-41e6-a493-c0aea934fd86 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=d4b91945558e4c66ba2fb4e8db38341a, user.id=8ca56ce70eb44e9d829ffda957fa5314) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:32.885660 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ddc43-884e-41e6-a493-c0aea934fd86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:32.887174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ddc43-884e-41e6-a493-c0aea934fd86 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:32.893714 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 578/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:28:32 2022] GET /identity/v3/role_assignments?user.id=8ca56ce70eb44e9d829ffda957fa5314&scope.project.id=d4b91945558e4c66ba2fb4e8db38341a&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:34.961966 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338033f-46ae-4063-bb32-2cfaffd7c497 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:34.962637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338033f-46ae-4063-bb32-2cfaffd7c497 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:34.962637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338033f-46ae-4063-bb32-2cfaffd7c497 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:34.963926 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 584/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:28:34.970298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77835-94cb-479e-80f8-2711e715c2e9 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:34.970940 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77835-94cb-479e-80f8-2711e715c2e9 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:34.971064 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77835-94cb-479e-80f8-2711e715c2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:34.990988 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-eed77835-94cb-479e-80f8-2711e715c2e9 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:34.992238 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-eed77835-94cb-479e-80f8-2711e715c2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:35.070265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 579/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:28:35.092951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338033f-46ae-4063-bb32-2cfaffd7c497 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:35.154154 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:35.203187 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338033f-46ae-4063-bb32-2cfaffd7c497 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:35.240299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fd456-44d2-411a-b876-a9855dddb3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:35.240839 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fd456-44d2-411a-b876-a9855dddb3b9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:35.241124 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fd456-44d2-411a-b876-a9855dddb3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:35.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fd456-44d2-411a-b876-a9855dddb3b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:35.317576 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fd456-44d2-411a-b876-a9855dddb3b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:35.319480 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fd456-44d2-411a-b876-a9855dddb3b9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:35.377299 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:35.434564 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 585/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:28:35.443275 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77835-94cb-479e-80f8-2711e715c2e9 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:35.529985 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:35.566191 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77835-94cb-479e-80f8-2711e715c2e9 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:35.602398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1166a7c-6ff4-49ef-ae7a-f188fb7abf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:35.603055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166a7c-6ff4-49ef-ae7a-f188fb7abf2a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:35.603339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166a7c-6ff4-49ef-ae7a-f188fb7abf2a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:35.603613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166a7c-6ff4-49ef-ae7a-f188fb7abf2a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:35.610339 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1166a7c-6ff4-49ef-ae7a-f188fb7abf2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:35.611862 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1166a7c-6ff4-49ef-ae7a-f188fb7abf2a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-f1166a7c-6ff4-49ef-ae7a-f188fb7abf2a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:35.623446 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:35.624829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:35.625788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 580/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:28:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:35.630947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fd456-44d2-411a-b876-a9855dddb3b9 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:35.736614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fd456-44d2-411a-b876-a9855dddb3b9 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:35.774493 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f302b-176a-43d6-8650-9baa7e834a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:35.775062 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f302b-176a-43d6-8650-9baa7e834a2f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:35.775343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f302b-176a-43d6-8650-9baa7e834a2f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:35.775623 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f302b-176a-43d6-8650-9baa7e834a2f None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:35.776340 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f302b-176a-43d6-8650-9baa7e834a2f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:35.776651 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f302b-176a-43d6-8650-9baa7e834a2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:35.778057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f302b-176a-43d6-8650-9baa7e834a2f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:35.786744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 586/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:28:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:37.803517 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a88e7-e30d-4806-b56c-9003dd977bdc None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:37.804077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a88e7-e30d-4806-b56c-9003dd977bdc None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:37.804077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a88e7-e30d-4806-b56c-9003dd977bdc None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:37.805317 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 581/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:37.810863 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9eb33-4792-4fa2-9b73-22d68380aa7d None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:37.811327 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9eb33-4792-4fa2-9b73-22d68380aa7d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:37.811432 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9eb33-4792-4fa2-9b73-22d68380aa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:37.831456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-67d9eb33-4792-4fa2-9b73-22d68380aa7d None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:37.832669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-67d9eb33-4792-4fa2-9b73-22d68380aa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:37.846702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:37.906011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 587/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:28:37.929036 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a88e7-e30d-4806-b56c-9003dd977bdc None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:38.039683 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a88e7-e30d-4806-b56c-9003dd977bdc None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:38.066415 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:38.076708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9520d-e932-4fea-baef-2a8b85da2a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:38.077343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9520d-e932-4fea-baef-2a8b85da2a85 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:38.077626 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9520d-e932-4fea-baef-2a8b85da2a85 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:38.077903 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9520d-e932-4fea-baef-2a8b85da2a85 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:38.078657 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb9520d-e932-4fea-baef-2a8b85da2a85 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:38.080075 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb9520d-e932-4fea-baef-2a8b85da2a85 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:38.098221 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 582/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jul 28 08:28:37 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:28:40.134851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2aa02-1ba6-4a7f-b33d-585d945a656e None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:40.135490 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2aa02-1ba6-4a7f-b33d-585d945a656e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:40.135490 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2aa02-1ba6-4a7f-b33d-585d945a656e None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:40.136787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 588/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:40.143305 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ba94f-79d3-434f-a7a4-1f97245d7ec4 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:40.144060 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ba94f-79d3-434f-a7a4-1f97245d7ec4 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:40.144255 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ba94f-79d3-434f-a7a4-1f97245d7ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:40.169368 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-2f2ba94f-79d3-434f-a7a4-1f97245d7ec4 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:40.170587 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-2f2ba94f-79d3-434f-a7a4-1f97245d7ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:40.242838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 583/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:40.266558 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2aa02-1ba6-4a7f-b33d-585d945a656e None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:40.384365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2aa02-1ba6-4a7f-b33d-585d945a656e None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:40.425614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ab01-70c5-40e6-b7ad-78b252d09bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:40.426219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ab01-70c5-40e6-b7ad-78b252d09bb9 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:40.426507 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ab01-70c5-40e6-b7ad-78b252d09bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:40.426785 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ab01-70c5-40e6-b7ad-78b252d09bb9 None admin] PATH_INFO: `/v3/domains/5301e56624ac48d288c4ed489de46daa` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:40.438359 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453ab01-70c5-40e6-b7ad-78b252d09bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5301e56624ac48d288c4ed489de46daa)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:40.439886 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453ab01-70c5-40e6-b7ad-78b252d09bb9 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:40.441556 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 589/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:28:40 2022] GET /identity/v3/domains/5301e56624ac48d288c4ed489de46daa => generated 267 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:40.448230 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ba94f-79d3-434f-a7a4-1f97245d7ec4 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:40.555730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ba94f-79d3-434f-a7a4-1f97245d7ec4 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:40.574700 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:40.597989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268436e-e216-4914-9e93-3023d2fdfb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:40.598889 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268436e-e216-4914-9e93-3023d2fdfb50 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:40.599154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268436e-e216-4914-9e93-3023d2fdfb50 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:40.599439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268436e-e216-4914-9e93-3023d2fdfb50 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:40.600167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268436e-e216-4914-9e93-3023d2fdfb50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:40.601633 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268436e-e216-4914-9e93-3023d2fdfb50 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:40.631362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 584/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:28:40 2022] POST /identity/v3/projects => generated 361 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:28:42.701225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534d3c-66ec-45ad-9eeb-980b6a36a8a7 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:42.701830 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534d3c-66ec-45ad-9eeb-980b6a36a8a7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:42.701830 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534d3c-66ec-45ad-9eeb-980b6a36a8a7 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:42.703228 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 590/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:42.708869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebeca-fb2a-424b-88ce-41316a7240e4 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:42.709361 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebeca-fb2a-424b-88ce-41316a7240e4 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:42.709534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebeca-fb2a-424b-88ce-41316a7240e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:42.728908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-b6eebeca-fb2a-424b-88ce-41316a7240e4 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:42.730167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-b6eebeca-fb2a-424b-88ce-41316a7240e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:42.802586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 585/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:42.827857 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af534d3c-66ec-45ad-9eeb-980b6a36a8a7 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:42.840453 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:42.937952 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af534d3c-66ec-45ad-9eeb-980b6a36a8a7 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:42.973401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a68fe-8a87-40d7-9ca1-f2361cb0af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:42.974060 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a68fe-8a87-40d7-9ca1-f2361cb0af1e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:42.974378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a68fe-8a87-40d7-9ca1-f2361cb0af1e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:42.974653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a68fe-8a87-40d7-9ca1-f2361cb0af1e None admin] PATH_INFO: `/v3/domains/5301e56624ac48d288c4ed489de46daa` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:42.975864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a68fe-8a87-40d7-9ca1-f2361cb0af1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=5301e56624ac48d288c4ed489de46daa)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:42.977341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a68fe-8a87-40d7-9ca1-f2361cb0af1e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:42.979004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 591/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:28:42 2022] GET /identity/v3/domains/5301e56624ac48d288c4ed489de46daa => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:42.985120 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eebeca-fb2a-424b-88ce-41316a7240e4 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:43.088187 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:43.092585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eebeca-fb2a-424b-88ce-41316a7240e4 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:43.132756 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08ddf0-1855-49cb-94e2-6e71aa4a5afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:43.133378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08ddf0-1855-49cb-94e2-6e71aa4a5afe None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:43.133654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08ddf0-1855-49cb-94e2-6e71aa4a5afe None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:43.133924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08ddf0-1855-49cb-94e2-6e71aa4a5afe None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:43.134635 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08ddf0-1855-49cb-94e2-6e71aa4a5afe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:43.136092 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08ddf0-1855-49cb-94e2-6e71aa4a5afe None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:43.158042 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 586/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:28:42 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:28:45.217412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0bc1-8fe4-473a-8f0c-ad1bf28abe98 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:45.218001 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0bc1-8fe4-473a-8f0c-ad1bf28abe98 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:45.218001 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0bc1-8fe4-473a-8f0c-ad1bf28abe98 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:45.219385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 592/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:45.225047 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e547b9b-8ea7-4b38-af3b-9c5fdfe0cabf None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:45.225603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e547b9b-8ea7-4b38-af3b-9c5fdfe0cabf None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:45.225603 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e547b9b-8ea7-4b38-af3b-9c5fdfe0cabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:45.245372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-3e547b9b-8ea7-4b38-af3b-9c5fdfe0cabf None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:45.246623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-3e547b9b-8ea7-4b38-af3b-9c5fdfe0cabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:45.318095 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 587/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:45.341600 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb0bc1-8fe4-473a-8f0c-ad1bf28abe98 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:45.430483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:45.465678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb0bc1-8fe4-473a-8f0c-ad1bf28abe98 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:45.511719 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a0876-f2c5-41ea-9b2f-23004f7f7a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:45.512280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a0876-f2c5-41ea-9b2f-23004f7f7a53 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:45.512557 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a0876-f2c5-41ea-9b2f-23004f7f7a53 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:45.512829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a0876-f2c5-41ea-9b2f-23004f7f7a53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:45.527370 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a0876-f2c5-41ea-9b2f-23004f7f7a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-209a0876-f2c5-41ea-9b2f-23004f7f7a53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:45.533452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:45.534667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:28:45.535670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 593/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:28:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:45.541086 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e547b9b-8ea7-4b38-af3b-9c5fdfe0cabf None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:45.628388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:45.645671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e547b9b-8ea7-4b38-af3b-9c5fdfe0cabf None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:45.681319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:45.681949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:45.682261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:45.682521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:45.683242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:45.683548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:45.684944 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:45.693251 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 588/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:28:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:45.699787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a0876-f2c5-41ea-9b2f-23004f7f7a53 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:45.806993 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a0876-f2c5-41ea-9b2f-23004f7f7a53 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:45.842059 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698226-9668-4e91-8a1d-8147546da758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:45.842726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698226-9668-4e91-8a1d-8147546da758 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:45.843005 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698226-9668-4e91-8a1d-8147546da758 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:45.843278 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698226-9668-4e91-8a1d-8147546da758 None admin] PATH_INFO: `/v3/users/71a4d739d35d4f6fb374c4725f34c87c` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:45.846406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:45.858947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02698226-9668-4e91-8a1d-8147546da758 None admin] RBAC: Authorizing `identity:get_user(user_id=71a4d739d35d4f6fb374c4725f34c87c)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:45.860518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02698226-9668-4e91-8a1d-8147546da758 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:45.862858 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 594/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:45 2022] GET /identity/v3/users/71a4d739d35d4f6fb374c4725f34c87c => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:45.869206 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:45.977150 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edfd41-a206-4561-8c11-a4989f0e5d3d None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:46.011676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c730de-9f56-442a-9163-2f9eeea1f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:46.012294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c730de-9f56-442a-9163-2f9eeea1f008 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:46.012575 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c730de-9f56-442a-9163-2f9eeea1f008 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:46.012853 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c730de-9f56-442a-9163-2f9eeea1f008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:46.028424 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:46.083478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c730de-9f56-442a-9163-2f9eeea1f008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:46.084901 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c730de-9f56-442a-9163-2f9eeea1f008 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:46.184675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 589/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:28:46.191095 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698226-9668-4e91-8a1d-8147546da758 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:46.308423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698226-9668-4e91-8a1d-8147546da758 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:46.346705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f940e-a412-4f8c-8fc4-f2f94501310b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:46.347269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f940e-a412-4f8c-8fc4-f2f94501310b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:46.347551 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f940e-a412-4f8c-8fc4-f2f94501310b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:46.347821 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f940e-a412-4f8c-8fc4-f2f94501310b None admin] PATH_INFO: `/v3/projects/5492db1864e2481a9ba613b8b68d557b` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:46.349107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f940e-a412-4f8c-8fc4-f2f94501310b None admin] RBAC: Authorizing `identity:get_project(project_id=5492db1864e2481a9ba613b8b68d557b)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:46.350596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f940e-a412-4f8c-8fc4-f2f94501310b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:46.352300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 595/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:46 2022] GET /identity/v3/projects/5492db1864e2481a9ba613b8b68d557b => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:46.358908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c730de-9f56-442a-9163-2f9eeea1f008 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:46.408895 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:46.464390 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c730de-9f56-442a-9163-2f9eeea1f008 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:46.500746 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fefb5-eab0-4081-a7b1-46ad741e7a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:46.501647 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fefb5-eab0-4081-a7b1-46ad741e7a35 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:46.502181 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fefb5-eab0-4081-a7b1-46ad741e7a35 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:46.502560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fefb5-eab0-4081-a7b1-46ad741e7a35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:46.503783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fefb5-eab0-4081-a7b1-46ad741e7a35 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=5492db1864e2481a9ba613b8b68d557b, user.id=71a4d739d35d4f6fb374c4725f34c87c) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:46.504220 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fefb5-eab0-4081-a7b1-46ad741e7a35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:46.505718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fefb5-eab0-4081-a7b1-46ad741e7a35 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:46.512082 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 590/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:28:46 2022] GET /identity/v3/role_assignments?user.id=71a4d739d35d4f6fb374c4725f34c87c&scope.project.id=5492db1864e2481a9ba613b8b68d557b&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:48.508551 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd101-4dbd-4f15-9ac5-029c4ea8d243 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:48.509137 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd101-4dbd-4f15-9ac5-029c4ea8d243 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:48.509137 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd101-4dbd-4f15-9ac5-029c4ea8d243 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:48.510720 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 596/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:48.517264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1e0cd-c2f9-496f-aaab-2770c666971c None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:48.517723 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1e0cd-c2f9-496f-aaab-2770c666971c None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:48.517957 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1e0cd-c2f9-496f-aaab-2770c666971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:48.540880 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-5ba1e0cd-c2f9-496f-aaab-2770c666971c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:48.542134 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-5ba1e0cd-c2f9-496f-aaab-2770c666971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:48.617008 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 591/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:48.639613 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dd101-4dbd-4f15-9ac5-029c4ea8d243 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:48.713179 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:48.754323 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dd101-4dbd-4f15-9ac5-029c4ea8d243 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:48.792139 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772beca6-0f9d-4cac-b9e0-2c095fb36ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:48.792706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772beca6-0f9d-4cac-b9e0-2c095fb36ab5 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:48.792984 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772beca6-0f9d-4cac-b9e0-2c095fb36ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:48.793255 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772beca6-0f9d-4cac-b9e0-2c095fb36ab5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:48.804755 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772beca6-0f9d-4cac-b9e0-2c095fb36ab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-772beca6-0f9d-4cac-b9e0-2c095fb36ab5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:48.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:48.810970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:28:48.812276 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 597/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:28:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:48.817483 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1e0cd-c2f9-496f-aaab-2770c666971c None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:48.861430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:48.926472 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1e0cd-c2f9-496f-aaab-2770c666971c None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:48.968677 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:48.969315 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:48.969600 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:48.969876 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:48.970640 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:48.970945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:48.972337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:48.986584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 592/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:28:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:48.993090 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772beca6-0f9d-4cac-b9e0-2c095fb36ab5 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:49.105869 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772beca6-0f9d-4cac-b9e0-2c095fb36ab5 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:49.131132 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:49.147053 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66ad4f-1680-479b-9825-c274d6942359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:49.147879 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ad4f-1680-479b-9825-c274d6942359 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:49.148233 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ad4f-1680-479b-9825-c274d6942359 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:49.148578 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ad4f-1680-479b-9825-c274d6942359 None admin] PATH_INFO: `/v3/users/71a4d739d35d4f6fb374c4725f34c87c` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:49.150069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66ad4f-1680-479b-9825-c274d6942359 None admin] RBAC: Authorizing `identity:get_user(user_id=71a4d739d35d4f6fb374c4725f34c87c)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:49.151958 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66ad4f-1680-479b-9825-c274d6942359 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:49.153843 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 598/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:48 2022] GET /identity/v3/users/71a4d739d35d4f6fb374c4725f34c87c => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:49.161265 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:49.277979 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307971-8344-4a7e-adfc-2c646fc1d4b6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:49.279543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:49.314851 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb50b28-6afc-4476-81d0-7ac8ccb98786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:49.315738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb50b28-6afc-4476-81d0-7ac8ccb98786 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:49.316232 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb50b28-6afc-4476-81d0-7ac8ccb98786 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:49.316741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb50b28-6afc-4476-81d0-7ac8ccb98786 None admin] PATH_INFO: `/v3/projects/5492db1864e2481a9ba613b8b68d557b` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:49.318842 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb50b28-6afc-4476-81d0-7ac8ccb98786 None admin] RBAC: Authorizing `identity:get_project(project_id=5492db1864e2481a9ba613b8b68d557b)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:49.321486 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb50b28-6afc-4476-81d0-7ac8ccb98786 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:49.323319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 593/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:49 2022] GET /identity/v3/projects/5492db1864e2481a9ba613b8b68d557b => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:28:49.330147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66ad4f-1680-479b-9825-c274d6942359 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:49.433042 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66ad4f-1680-479b-9825-c274d6942359 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:49.469135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7175477-ea84-4f05-a566-afb7caed1ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:49.469856 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175477-ea84-4f05-a566-afb7caed1ff0 None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:49.470153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175477-ea84-4f05-a566-afb7caed1ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:49.470448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175477-ea84-4f05-a566-afb7caed1ff0 None admin] PATH_INFO: `/v3/projects/5492db1864e2481a9ba613b8b68d557b/users/71a4d739d35d4f6fb374c4725f34c87c/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:49.472521 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7175477-ea84-4f05-a566-afb7caed1ff0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5492db1864e2481a9ba613b8b68d557b, user_id=71a4d739d35d4f6fb374c4725f34c87c, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:49.473963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7175477-ea84-4f05-a566-afb7caed1ff0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:49.490532 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 599/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:28:49 2022] PUT /identity/v3/projects/5492db1864e2481a9ba613b8b68d557b/users/71a4d739d35d4f6fb374c4725f34c87c/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:28:51.508582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453e5f5-2c67-4aa7-a241-319286960f7d None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:51.509167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453e5f5-2c67-4aa7-a241-319286960f7d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:51.509167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453e5f5-2c67-4aa7-a241-319286960f7d None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:51.510548 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 594/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:28:51.515912 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ba11d-dcc5-4461-aa97-23e1c114014e None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:51.516359 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ba11d-dcc5-4461-aa97-23e1c114014e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:51.516457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ba11d-dcc5-4461-aa97-23e1c114014e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:51.535289 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-b81ba11d-dcc5-4461-aa97-23e1c114014e None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:28:51.536511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-b81ba11d-dcc5-4461-aa97-23e1c114014e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:28:51.590805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:51.614763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 600/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:28:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:28:51.638089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453e5f5-2c67-4aa7-a241-319286960f7d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:51.746681 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453e5f5-2c67-4aa7-a241-319286960f7d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:51.783511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb819e57-376c-4bd8-9d30-c8e62a028d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:51.784077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb819e57-376c-4bd8-9d30-c8e62a028d47 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:51.784356 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb819e57-376c-4bd8-9d30-c8e62a028d47 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:51.784629 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb819e57-376c-4bd8-9d30-c8e62a028d47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:51.795171 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb819e57-376c-4bd8-9d30-c8e62a028d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-cb819e57-376c-4bd8-9d30-c8e62a028d47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:28:51.800190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:28:51.801353 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:28:51.802566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 595/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:28:51.807810 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ba11d-dcc5-4461-aa97-23e1c114014e None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:51.919241 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ba11d-dcc5-4461-aa97-23e1c114014e None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:51.957475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:51.958145 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:51.958412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:51.958685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:51.959400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:51.959705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:51.961159 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:51.969511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 601/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:51.977309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb819e57-376c-4bd8-9d30-c8e62a028d47 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:52.010479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:52.088430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb819e57-376c-4bd8-9d30-c8e62a028d47 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:52.124125 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376f75-6f41-49ac-bd90-78b0b2f18207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:52.124779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376f75-6f41-49ac-bd90-78b0b2f18207 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:52.125150 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376f75-6f41-49ac-bd90-78b0b2f18207 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:52.125534 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376f75-6f41-49ac-bd90-78b0b2f18207 None admin] PATH_INFO: `/v3/users/71a4d739d35d4f6fb374c4725f34c87c` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:52.127645 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376f75-6f41-49ac-bd90-78b0b2f18207 None admin] RBAC: Authorizing `identity:get_user(user_id=71a4d739d35d4f6fb374c4725f34c87c)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:52.130386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376f75-6f41-49ac-bd90-78b0b2f18207 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:52.132907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 596/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:28:51 2022] GET /identity/v3/users/71a4d739d35d4f6fb374c4725f34c87c => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:52.139113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:52.184232 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:52.245219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc97a01-81a0-407a-be8c-b3cd04d686ae None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:52.280496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000cb54-4b20-44fb-81f4-8c40406094d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:52.281131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000cb54-4b20-44fb-81f4-8c40406094d8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:52.281412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000cb54-4b20-44fb-81f4-8c40406094d8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:52.281686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000cb54-4b20-44fb-81f4-8c40406094d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:52.353303 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000cb54-4b20-44fb-81f4-8c40406094d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:52.354712 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000cb54-4b20-44fb-81f4-8c40406094d8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:52.395397 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:52.454833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 602/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:28:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:28:52.461143 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376f75-6f41-49ac-bd90-78b0b2f18207 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:52.565868 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:28:52.570380 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376f75-6f41-49ac-bd90-78b0b2f18207 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:52.606599 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d966-7ab2-43b5-a83f-c0c665f7c7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:52.608069 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d966-7ab2-43b5-a83f-c0c665f7c7fc None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:52.608069 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d966-7ab2-43b5-a83f-c0c665f7c7fc None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:52.608069 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d966-7ab2-43b5-a83f-c0c665f7c7fc None admin] PATH_INFO: `/v3/projects/5492db1864e2481a9ba613b8b68d557b` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:52.608679 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d966-7ab2-43b5-a83f-c0c665f7c7fc None admin] RBAC: Authorizing `identity:get_project(project_id=5492db1864e2481a9ba613b8b68d557b)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:52.610217 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d966-7ab2-43b5-a83f-c0c665f7c7fc None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:52.612025 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 597/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:28:52 2022] GET /identity/v3/projects/5492db1864e2481a9ba613b8b68d557b => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:28:52.619338 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000cb54-4b20-44fb-81f4-8c40406094d8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:28:52.724222 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000cb54-4b20-44fb-81f4-8c40406094d8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:28:52.761109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4fe06-acdd-43e8-b33a-2a82863aeef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:28:52.761734 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4fe06-acdd-43e8-b33a-2a82863aeef2 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:28:52.762016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4fe06-acdd-43e8-b33a-2a82863aeef2 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:28:52.762331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4fe06-acdd-43e8-b33a-2a82863aeef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:28:52.763134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4fe06-acdd-43e8-b33a-2a82863aeef2 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=5492db1864e2481a9ba613b8b68d557b, user.id=71a4d739d35d4f6fb374c4725f34c87c) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:28:52.763446 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4fe06-acdd-43e8-b33a-2a82863aeef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:28:52.764894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4fe06-acdd-43e8-b33a-2a82863aeef2 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:28:52.771108 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 603/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:28:52 2022] GET /identity/v3/role_assignments?user.id=71a4d739d35d4f6fb374c4725f34c87c&scope.project.id=5492db1864e2481a9ba613b8b68d557b&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:29:02.601570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=17 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:02.768391 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=97 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:48.801962 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2a1a9-1963-4b16-8a2d-949a01db0faf None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:48.802588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2a1a9-1963-4b16-8a2d-949a01db0faf None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:48.802588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2a1a9-1963-4b16-8a2d-949a01db0faf None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:48.804087 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 598/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:29:48.811105 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b298a-49a2-43a1-9d7f-f9d143a12321 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:48.811660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b298a-49a2-43a1-9d7f-f9d143a12321 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:48.811660 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b298a-49a2-43a1-9d7f-f9d143a12321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:48.840340 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-f86b298a-49a2-43a1-9d7f-f9d143a12321 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:29:48.842012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-f86b298a-49a2-43a1-9d7f-f9d143a12321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:29:48.934527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 604/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:29:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:29:48.959075 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2a1a9-1963-4b16-8a2d-949a01db0faf None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:49.074053 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2a1a9-1963-4b16-8a2d-949a01db0faf None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:49.112536 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:49.113198 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:49.113521 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:49.113788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:49.129904 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:49.131445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:29:49.147967 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:49.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:49.150329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:29:49.150329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:29:49.150329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:49.150329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:29:49.150329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 599/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:49.155493 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b298a-49a2-43a1-9d7f-f9d143a12321 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:49.308727 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b298a-49a2-43a1-9d7f-f9d143a12321 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:49.344115 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:49.355970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa274f-5d39-448f-adfc-ed728ed5bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:49.356668 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa274f-5d39-448f-adfc-ed728ed5bc1e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:49.356977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa274f-5d39-448f-adfc-ed728ed5bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:49.357324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa274f-5d39-448f-adfc-ed728ed5bc1e None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:49.358077 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa274f-5d39-448f-adfc-ed728ed5bc1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:49.358514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa274f-5d39-448f-adfc-ed728ed5bc1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:49.360267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa274f-5d39-448f-adfc-ed728ed5bc1e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:49.380978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 605/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:49.383898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:49.466421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:49.547511 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdbe67-48b5-41ae-bd84-72777db40fa5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:49.611149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1c3a7-14a7-4694-88fd-840c4e80d776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:49.611716 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1c3a7-14a7-4694-88fd-840c4e80d776 None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:49.612015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1c3a7-14a7-4694-88fd-840c4e80d776 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:49.612295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1c3a7-14a7-4694-88fd-840c4e80d776 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:49.612295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1c3a7-14a7-4694-88fd-840c4e80d776 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:49.614585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1c3a7-14a7-4694-88fd-840c4e80d776 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:49.650774 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 600/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:29:49 2022] POST /identity/v3/users => generated 251 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:29:51.973987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fe960-329a-4f49-b97e-41f998cd4c54 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:51.974627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fe960-329a-4f49-b97e-41f998cd4c54 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:51.974627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fe960-329a-4f49-b97e-41f998cd4c54 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:51.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 606/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:29:51.982890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ccf3-aae8-4e6f-8b9e-6a727ff3a25a None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:51.983514 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ccf3-aae8-4e6f-8b9e-6a727ff3a25a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:51.983514 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ccf3-aae8-4e6f-8b9e-6a727ff3a25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:52.011485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-3fd8ccf3-aae8-4e6f-8b9e-6a727ff3a25a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:29:52.013073 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-3fd8ccf3-aae8-4e6f-8b9e-6a727ff3a25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:29:52.145458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 601/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:29:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:29:52.173991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fe960-329a-4f49-b97e-41f998cd4c54 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:52.346475 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fe960-329a-4f49-b97e-41f998cd4c54 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:52.399621 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394c915-1bad-453f-95a7-276668502a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:52.400308 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c915-1bad-453f-95a7-276668502a0d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:52.400592 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c915-1bad-453f-95a7-276668502a0d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:52.400871 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c915-1bad-453f-95a7-276668502a0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:52.424011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394c915-1bad-453f-95a7-276668502a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-2394c915-1bad-453f-95a7-276668502a0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:29:52.431379 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:52.432874 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:29:52.434055 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 607/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:29:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:52.439709 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8ccf3-aae8-4e6f-8b9e-6a727ff3a25a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:52.511423 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:52.606756 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8ccf3-aae8-4e6f-8b9e-6a727ff3a25a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:52.655814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:52.656553 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:52.656834 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:52.657416 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:52.658287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:52.658769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:52.660933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:52.671538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 602/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:29:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:52.678686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394c915-1bad-453f-95a7-276668502a0d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:52.772107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:52.871820 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394c915-1bad-453f-95a7-276668502a0d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:52.934343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:52.935352 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:52.935776 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:52.936251 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:52.955618 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:52.958238 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:29:52.977426 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:52.982253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:52.983725 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:29:52.983725 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:29:52.983725 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:52.983725 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:29:52.983725 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 608/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:52.984792 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:53.125784 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:53.131810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18eebd-676f-4263-97ca-22965ea17a8e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:53.183450 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:53.184495 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:53.184978 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:53.185437 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:53.186350 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:53.186590 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:53.188193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:53.203055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 603/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:29:53.209831 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:53.356368 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8d37-20b6-4d67-9b06-080553fbe312 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:53.358422 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:53.399238 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:53.399854 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:53.400128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:53.400526 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] PATH_INFO: `/v3/users/placement` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:53.410786 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:53.412502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:29:53.426787 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:53.429040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:29:53.431267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 609/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:29:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:53.434627 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:53.561449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47794b1-517a-40b5-8825-c660f8e2b8c9 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:53.619313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:53.620916 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:53.621363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:53.621697 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:53.623158 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:53.623701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:53.626028 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:53.659990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 604/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:29:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:53.670715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:53.800704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6943-37f5-4d76-a157-3e2cd1572517 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:53.840699 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e9e19-d4d5-48ee-a955-2541c095a11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:53.841547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e9e19-d4d5-48ee-a955-2541c095a11e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:53.841900 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e9e19-d4d5-48ee-a955-2541c095a11e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:53.842275 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e9e19-d4d5-48ee-a955-2541c095a11e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:53.892345 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:53.942953 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e9e19-d4d5-48ee-a955-2541c095a11e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:53.944867 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e9e19-d4d5-48ee-a955-2541c095a11e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:54.092772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 610/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:29:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:29:54.094961 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:54.128689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:54.250495 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d5912-60ca-4bb9-a255-645c560b60e0 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:54.289741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:54.290478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:54.290934 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:54.291063 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:54.304442 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:54.306479 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:29:54.324301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:54.325462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:54.326512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:29:54.326512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:29:54.326512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:54.326512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:29:54.326512 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 605/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:54.331889 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e9e19-d4d5-48ee-a955-2541c095a11e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:54.410569 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:54.471381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e9e19-d4d5-48ee-a955-2541c095a11e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:54.531678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:54.533003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:54.533644 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:54.534221 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:54.535599 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:54.536483 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:54.539628 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:54.561802 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 611/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:29:54.567801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:54.676169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:54.715705 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee65cd-af44-4dbc-80fb-6476f95ec870 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:54.765608 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:54.766327 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:54.766593 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:54.766864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:54.776309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:54.776309 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:29:54.791015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:54.794636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:29:54.796899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 606/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:54.799358 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:54.940182 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1d3a-4093-4858-979c-3aaf7fdda2b8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:54.965713 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:54.982063 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb211559-01ba-48f0-8c3b-aa6500ae4697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:54.982855 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb211559-01ba-48f0-8c3b-aa6500ae4697 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:54.983148 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb211559-01ba-48f0-8c3b-aa6500ae4697 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:54.983477 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb211559-01ba-48f0-8c3b-aa6500ae4697 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:54.984678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb211559-01ba-48f0-8c3b-aa6500ae4697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:54.985035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb211559-01ba-48f0-8c3b-aa6500ae4697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:54.986672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb211559-01ba-48f0-8c3b-aa6500ae4697 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:55.006144 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 612/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:29:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:55.011078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:55.132922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15c036-0752-4da5-8078-b355e6d31455 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:55.170369 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:55.178734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28333f37-0575-465f-aedc-0683643ec893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:55.179623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28333f37-0575-465f-aedc-0683643ec893 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:55.180009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28333f37-0575-465f-aedc-0683643ec893 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:55.180252 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28333f37-0575-465f-aedc-0683643ec893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:55.181200 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28333f37-0575-465f-aedc-0683643ec893 None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=b83a030752874fb2b7290691762ab9c2) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:55.181690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28333f37-0575-465f-aedc-0683643ec893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:55.183561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28333f37-0575-465f-aedc-0683643ec893 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:55.191326 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 607/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:29:55 2022] GET /identity/v3/role_assignments?user.id=b83a030752874fb2b7290691762ab9c2&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:57.413005 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb01e0-29ba-42fd-91dc-7c57932730be None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:57.413951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb01e0-29ba-42fd-91dc-7c57932730be None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:57.413951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb01e0-29ba-42fd-91dc-7c57932730be None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:57.418177 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 613/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:29:57.424804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64027e17-ed53-46a8-b601-92a52080405a None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:57.425457 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64027e17-ed53-46a8-b601-92a52080405a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:57.425720 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64027e17-ed53-46a8-b601-92a52080405a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:57.455552 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-64027e17-ed53-46a8-b601-92a52080405a None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:29:57.456807 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-64027e17-ed53-46a8-b601-92a52080405a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:29:57.539804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 608/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:29:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:29:57.565710 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb01e0-29ba-42fd-91dc-7c57932730be None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:57.728044 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb01e0-29ba-42fd-91dc-7c57932730be None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:57.787048 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da0bb7-5e29-44e2-963c-f3813fc8b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:57.788113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0bb7-5e29-44e2-963c-f3813fc8b108 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:57.788860 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0bb7-5e29-44e2-963c-f3813fc8b108 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:57.789067 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0bb7-5e29-44e2-963c-f3813fc8b108 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:57.807559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da0bb7-5e29-44e2-963c-f3813fc8b108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-b9da0bb7-5e29-44e2-963c-f3813fc8b108 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:29:57.812690 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:57.814773 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:29:57.816749 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:29:57.819510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 614/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:29:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:57.820606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64027e17-ed53-46a8-b601-92a52080405a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:57.969108 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64027e17-ed53-46a8-b601-92a52080405a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:58.012242 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:58.030045 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:58.031116 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:58.031597 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:58.032045 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:58.033130 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] RBAC: Adding query filter params (name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:58.033689 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:58.035958 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:58.052174 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 609/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:29:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:29:58.060165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da0bb7-5e29-44e2-963c-f3813fc8b108 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:58.133157 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:58.223201 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da0bb7-5e29-44e2-963c-f3813fc8b108 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:58.271910 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:58.272689 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:58.273012 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:58.273363 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:58.289987 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:58.291814 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:29:58.312086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:58.314015 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:58.316019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:29:58.316019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:29:58.316019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:58.316019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:29:58.316019 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 615/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:58.321156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:58.462458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae571e-ce6b-454b-9178-a71e0631dcad None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:58.511723 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:58.512449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:58.512766 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:58.513091 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:58.513937 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:58.514445 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:58.516155 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:58.533838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 610/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:58.544553 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:58.669674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:58.678444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5701-e3ec-43e3-8819-e7c208c49330 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:58.719821 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:58.720669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:58.721086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:58.721479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:58.730878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:58.732615 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:29:58.746829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:58.749036 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:29:58.751257 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 616/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:29:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:58.758586 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:58.826220 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:58.916117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9f30-f64d-40c0-88ec-26a29f3b7210 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:58.970788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:58.971737 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:58.972198 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:58.972672 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:58.973699 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:58.974211 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:58.975989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:59.008500 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 611/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:29:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:59.010178 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:59.187524 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f651525-b5fc-4f12-9709-6b1719e346d4 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:59.234253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:59.235088 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:59.235484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:59.235772 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:59.249979 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:59.251527 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:29:59.270716 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:59.272411 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:59.274806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:29:59.274806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:29:59.274806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:29:59.274806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:29:59.274806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 617/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:29:59.280932 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:59.432763 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa23e-7b37-4552-a6d6-bd64361bc18b None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:59.436286 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:59.484294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:59.485034 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:59.485203 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:59.485396 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:59.486133 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:59.486450 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:59.487878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:59.501061 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 612/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:59.507616 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:59.519921 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:29:59.671803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a62da-8427-4802-8027-f31b8e32da22 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:59.721531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:59.722505 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:59.722826 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:59.723280 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:59.735400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:59.737566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:29:59.755356 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:29:59.756634 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:29:59.757751 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 618/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:29:59.767492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:29:59.918590 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed91778-083d-451a-8e85-a95336626978 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:29:59.962789 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac73b-81ab-4845-9bfa-b590f9379a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:29:59.963618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac73b-81ab-4845-9bfa-b590f9379a21 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:29:59.963941 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac73b-81ab-4845-9bfa-b590f9379a21 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:29:59.964318 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac73b-81ab-4845-9bfa-b590f9379a21 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:29:59.965238 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dac73b-81ab-4845-9bfa-b590f9379a21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:29:59.965561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dac73b-81ab-4845-9bfa-b590f9379a21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:29:59.967226 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dac73b-81ab-4845-9bfa-b590f9379a21 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:29:59.983469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 613/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:29:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:29:59.992726 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:00.102501 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:00.162129 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbde476-5cc4-40cf-9c08-2bda8382b09b None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:00.200959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b717c0-2190-45ee-8951-f9d1514fcb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:00.201761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b717c0-2190-45ee-8951-f9d1514fcb3e None admin] REQUEST_METHOD: `PUT` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:00.202205 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b717c0-2190-45ee-8951-f9d1514fcb3e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:00.202401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b717c0-2190-45ee-8951-f9d1514fcb3e None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/b83a030752874fb2b7290691762ab9c2/roles/070199642db14984a418795ea5af54cb` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:00.221984 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b717c0-2190-45ee-8951-f9d1514fcb3e None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=b83a030752874fb2b7290691762ab9c2, role_id=070199642db14984a418795ea5af54cb)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:00.223618 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b717c0-2190-45ee-8951-f9d1514fcb3e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:00.238605 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 619/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:29:59 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/b83a030752874fb2b7290691762ab9c2/roles/070199642db14984a418795ea5af54cb => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:30:02.636774 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce12e66-57b7-4516-8437-076e162c4c3a None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:02.637589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce12e66-57b7-4516-8437-076e162c4c3a None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:02.637821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce12e66-57b7-4516-8437-076e162c4c3a None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:02.640953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 614/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:30:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:30:02.647470 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b73363-1d58-43e9-8e04-703108ccb481 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:02.648127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b73363-1d58-43e9-8e04-703108ccb481 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:02.648127 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b73363-1d58-43e9-8e04-703108ccb481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:02.671559 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-71b73363-1d58-43e9-8e04-703108ccb481 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:30:02.673682 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-71b73363-1d58-43e9-8e04-703108ccb481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:30:02.762238 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 620/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:30:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:30:02.794381 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce12e66-57b7-4516-8437-076e162c4c3a None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:02.830344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:02.933775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce12e66-57b7-4516-8437-076e162c4c3a None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:02.971762 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5a38-e21f-4f5e-b98b-1536b105ff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:02.972613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5a38-e21f-4f5e-b98b-1536b105ff48 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:02.972899 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5a38-e21f-4f5e-b98b-1536b105ff48 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:02.973177 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5a38-e21f-4f5e-b98b-1536b105ff48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:02.985084 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad5a38-e21f-4f5e-b98b-1536b105ff48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-9bad5a38-e21f-4f5e-b98b-1536b105ff48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:30:02.990122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:03.001413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:03.003821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 615/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:30:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:30:03.005502 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b73363-1d58-43e9-8e04-703108ccb481 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:03.053207 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:03.113833 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b73363-1d58-43e9-8e04-703108ccb481 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:03.154400 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56459244-8928-4793-b748-beffb183bb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:03.155073 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56459244-8928-4793-b748-beffb183bb1f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:03.155442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56459244-8928-4793-b748-beffb183bb1f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:03.155684 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56459244-8928-4793-b748-beffb183bb1f None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:03.156453 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56459244-8928-4793-b748-beffb183bb1f None admin] RBAC: Adding query filter params (name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:03.156820 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56459244-8928-4793-b748-beffb183bb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:03.158535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56459244-8928-4793-b748-beffb183bb1f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:03.169171 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 621/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:30:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:30:03.175152 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5a38-e21f-4f5e-b98b-1536b105ff48 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:03.289491 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:03.311882 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5a38-e21f-4f5e-b98b-1536b105ff48 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:03.369867 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:03.370902 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:03.371541 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:03.372119 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:03.386678 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:03.388785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:30:03.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:03.412037 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:03.413476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:30:03.413476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:30:03.413476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:03.413476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:03.413476 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 616/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:03.424725 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56459244-8928-4793-b748-beffb183bb1f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:03.569887 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56459244-8928-4793-b748-beffb183bb1f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:03.583638 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:03.633231 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:03.634298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:03.634796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:03.635268 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:03.636560 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:03.637185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:03.639669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:03.669209 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 622/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:30:03.676130 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:03.812477 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf272af-6772-4706-a23f-5b7de858ae30 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:03.873431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:03.873431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:03.873431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:03.873431 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:03.877344 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:03.885821 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:03.887981 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:30:03.903372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:03.904890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:30:03.907308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:03.911195 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 617/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:30:03.911412 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:04.036980 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d83f75-27b2-499e-9714-548b781e7e0d None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:04.094919 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:04.095694 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:04.096053 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:04.096518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:04.098319 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:04.098884 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:04.101933 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:04.134037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 623/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:30:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:04.141724 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:04.299659 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6cac0-e0d2-4aa6-8760-527aa5234b42 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:04.345294 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcb059-c32e-4dc0-96b3-f9dc219cedc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:04.345937 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcb059-c32e-4dc0-96b3-f9dc219cedc1 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:04.346259 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcb059-c32e-4dc0-96b3-f9dc219cedc1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:04.346527 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcb059-c32e-4dc0-96b3-f9dc219cedc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:04.422628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:04.431424 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcb059-c32e-4dc0-96b3-f9dc219cedc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:04.433271 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcb059-c32e-4dc0-96b3-f9dc219cedc1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:04.567202 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 618/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:30:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:30:04.573924 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:04.616711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:04.708797 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700328f-1192-4e26-8187-c8d8c00d4109 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:04.753667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:04.754386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:04.754672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:04.754983 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:04.768162 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:04.769675 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:30:04.789365 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:04.790974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:04.792153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:30:04.792153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:30:04.792153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:04.792153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:04.792153 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 624/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:04.797962 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcb059-c32e-4dc0-96b3-f9dc219cedc1 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:04.908703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:04.915616 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcb059-c32e-4dc0-96b3-f9dc219cedc1 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:04.959674 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:04.960577 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:04.961013 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:04.961468 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:04.962367 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:04.962829 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:04.964389 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:04.977637 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 619/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:04.984017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:05.079875 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:05.114568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b1716-191f-4d4e-aa4d-7bf744eae3a0 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:05.154761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:05.155514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:05.155703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:05.156069 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:05.163342 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:05.164914 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:30:05.179457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:05.180817 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:05.182002 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 625/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:05.188118 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:05.310882 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013c653-e5a7-4cd7-9e55-db162aabfe71 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:05.350357 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fc69f-fe36-4a2f-953a-8fbf113a3a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:05.351015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fc69f-fe36-4a2f-953a-8fbf113a3a72 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:05.351295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fc69f-fe36-4a2f-953a-8fbf113a3a72 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:05.351578 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fc69f-fe36-4a2f-953a-8fbf113a3a72 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:05.352352 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fc69f-fe36-4a2f-953a-8fbf113a3a72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:05.352802 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fc69f-fe36-4a2f-953a-8fbf113a3a72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:05.354179 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fc69f-fe36-4a2f-953a-8fbf113a3a72 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:05.367701 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 620/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:30:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:30:05.374138 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:05.515339 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc37b2-b3f8-4329-9278-8d592e26e5bb None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:05.584947 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89142dc5-9562-43ad-b600-8feabaac50ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:05.585938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142dc5-9562-43ad-b600-8feabaac50ca None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:05.586458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142dc5-9562-43ad-b600-8feabaac50ca None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:05.586991 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142dc5-9562-43ad-b600-8feabaac50ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:05.588386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89142dc5-9562-43ad-b600-8feabaac50ca None admin] RBAC: Adding query filter params (role.id=070199642db14984a418795ea5af54cb, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=b83a030752874fb2b7290691762ab9c2) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:05.588920 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89142dc5-9562-43ad-b600-8feabaac50ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:05.591722 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89142dc5-9562-43ad-b600-8feabaac50ca None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:05.606922 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 626/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:30:05 2022] GET /identity/v3/role_assignments?user.id=b83a030752874fb2b7290691762ab9c2&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=070199642db14984a418795ea5af54cb => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:07.772421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e19e-6cdd-413d-b4ab-4ed31c735296 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:07.773329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e19e-6cdd-413d-b4ab-4ed31c735296 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:07.773329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e19e-6cdd-413d-b4ab-4ed31c735296 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:07.774753 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 621/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:30:07.784401 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234790db-0414-470a-91d9-eafaa6d734f8 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:07.784945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234790db-0414-470a-91d9-eafaa6d734f8 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:07.785247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234790db-0414-470a-91d9-eafaa6d734f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:07.795503 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:07.813774 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-234790db-0414-470a-91d9-eafaa6d734f8 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:30:07.815064 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-234790db-0414-470a-91d9-eafaa6d734f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:30:07.912034 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 627/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:30:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:30:07.935588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e19e-6cdd-413d-b4ab-4ed31c735296 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:07.955306 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:08.057746 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e19e-6cdd-413d-b4ab-4ed31c735296 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:08.105585 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598d63-e9ba-45ad-b67e-f914a56569b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:08.106489 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598d63-e9ba-45ad-b67e-f914a56569b5 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:08.106947 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598d63-e9ba-45ad-b67e-f914a56569b5 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:08.107328 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598d63-e9ba-45ad-b67e-f914a56569b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:08.121972 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a598d63-e9ba-45ad-b67e-f914a56569b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-4a598d63-e9ba-45ad-b67e-f914a56569b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:30:08.130151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:08.131783 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:08.132993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 622/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:30:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:30:08.139729 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234790db-0414-470a-91d9-eafaa6d734f8 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:08.270760 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234790db-0414-470a-91d9-eafaa6d734f8 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:08.303932 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:08.311351 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:08.311964 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:08.312534 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:08.313113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:08.314761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:08.315572 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:08.317732 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:08.331337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 628/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:30:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:08.339307 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598d63-e9ba-45ad-b67e-f914a56569b5 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:08.439909 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:08.471468 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598d63-e9ba-45ad-b67e-f914a56569b5 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:08.526537 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:08.527215 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:08.527500 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:08.527774 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:08.542921 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:08.544552 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:30:08.561474 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:08.562993 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:08.564175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:30:08.564175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:30:08.564175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:08.564175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:08.564175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 623/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:08.571011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:08.722212 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff4d3d-0aa8-4272-96ac-b38043ee4250 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:08.780051 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:08.780981 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:08.781343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:08.781705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:08.782622 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:08.783035 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:08.784878 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:08.801730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 629/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:08.809192 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:08.953054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22268225-4bcf-42cb-b262-324be3db6a3c None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:09.000183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:09.013522 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:09.016031 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:09.016382 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:09.016684 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:09.035740 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:09.038966 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:30:09.052520 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:09.053570 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:09.055373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 624/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:09.060980 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:09.162057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:09.229864 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff06507-5492-4416-a716-e7c73ea9e096 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:09.287920 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:09.289017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:09.289437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:09.289922 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:09.290974 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:09.291413 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:09.293165 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:09.325239 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 630/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:30:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:09.331402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:09.496699 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfc40a-4eeb-47df-ae8a-8cd99fa47eb6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:09.559329 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d911b-3893-4adf-a127-43c7f7116135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:09.560414 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d911b-3893-4adf-a127-43c7f7116135 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:09.560860 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d911b-3893-4adf-a127-43c7f7116135 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:09.561333 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d911b-3893-4adf-a127-43c7f7116135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:09.664849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:09.693068 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d911b-3893-4adf-a127-43c7f7116135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:09.696421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d911b-3893-4adf-a127-43c7f7116135 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:09.875654 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 625/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:30:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:30:09.882747 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:10.009731 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f87a4-d979-4c48-be34-418cb2c67b5e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:10.027761 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:10.053291 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:10.054017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:10.054369 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:10.054675 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:10.066686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:10.068185 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:30:10.086617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:10.088963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:10.091313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:30:10.091313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:30:10.091313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:10.091313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:10.091313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 631/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:10.095494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d911b-3893-4adf-a127-43c7f7116135 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:10.225055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:10.243698 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d911b-3893-4adf-a127-43c7f7116135 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:10.284374 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:10.285048 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:10.285386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:10.285635 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:10.286430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:10.286769 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:10.288308 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:10.301651 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 626/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:10.307705 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:10.456748 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5cff6-fef7-4923-b6ad-d3c680d786c1 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:10.522880 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:10.523640 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:10.524017 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:10.524317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] PATH_INFO: `/v3/projects/service` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:10.534142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:10.535835 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:30:10.552614 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:10.554437 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:10.555715 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 632/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:30:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:10.560619 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:10.716012 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143d3c7-1096-41d2-95c1-261f942feb56 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:10.772596 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257beeef-5708-4934-87c2-1a364b1e5948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:10.773599 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257beeef-5708-4934-87c2-1a364b1e5948 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:10.774313 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257beeef-5708-4934-87c2-1a364b1e5948 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:10.774756 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257beeef-5708-4934-87c2-1a364b1e5948 None admin] PATH_INFO: `/v3/projects` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:10.775933 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257beeef-5708-4934-87c2-1a364b1e5948 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:10.776484 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257beeef-5708-4934-87c2-1a364b1e5948 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:10.778560 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257beeef-5708-4934-87c2-1a364b1e5948 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:10.804898 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 627/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:30:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:10.807029 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:10.848300 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:10.968948 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a94e1-ffc8-4cd9-adef-55cca8d36def None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:11.016132 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b882dd3-9f88-4976-bf82-657adad767ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:11.017057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882dd3-9f88-4976-bf82-657adad767ca None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:11.017733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882dd3-9f88-4976-bf82-657adad767ca None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:11.017733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882dd3-9f88-4976-bf82-657adad767ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:11.018452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b882dd3-9f88-4976-bf82-657adad767ca None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=b83a030752874fb2b7290691762ab9c2) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:11.018769 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b882dd3-9f88-4976-bf82-657adad767ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:11.020458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b882dd3-9f88-4976-bf82-657adad767ca None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:11.028902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 633/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:30:10 2022] GET /identity/v3/role_assignments?user.id=b83a030752874fb2b7290691762ab9c2&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:13.194991 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1278dd-5b40-4894-bf0b-48a57582decc None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:13.195855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1278dd-5b40-4894-bf0b-48a57582decc None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:13.195855 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1278dd-5b40-4894-bf0b-48a57582decc None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:13.197927 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 628/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:30:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:30:13.204511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9f355-6f0d-4ce4-b5eb-1f51774182a5 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:13.205317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9f355-6f0d-4ce4-b5eb-1f51774182a5 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:13.205317 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9f355-6f0d-4ce4-b5eb-1f51774182a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:13.228627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-b3d9f355-6f0d-4ce4-b5eb-1f51774182a5 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:30:13.230167 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-b3d9f355-6f0d-4ce4-b5eb-1f51774182a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:30:13.334195 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 634/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:30:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:30:13.373137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1278dd-5b40-4894-bf0b-48a57582decc None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:13.388169 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:13.529421 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1278dd-5b40-4894-bf0b-48a57582decc None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:13.579610 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14389f4-fd1f-4cc2-9839-d046241fe96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:13.580411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14389f4-fd1f-4cc2-9839-d046241fe96e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:13.580765 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14389f4-fd1f-4cc2-9839-d046241fe96e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:13.581137 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14389f4-fd1f-4cc2-9839-d046241fe96e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:13.594897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14389f4-fd1f-4cc2-9839-d046241fe96e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-a14389f4-fd1f-4cc2-9839-d046241fe96e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:30:13.600775 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:13.602949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:13.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 629/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:30:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:13.614452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9f355-6f0d-4ce4-b5eb-1f51774182a5 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:13.647645 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:13.768859 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9f355-6f0d-4ce4-b5eb-1f51774182a5 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:13.834928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:13.835652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:13.835990 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:13.836427 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] PATH_INFO: `/v3/roles` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:13.837764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:13.838381 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:13.840442 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:13.855166 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 635/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:30:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:30:13.861566 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14389f4-fd1f-4cc2-9839-d046241fe96e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:13.979869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:14.027998 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14389f4-fd1f-4cc2-9839-d046241fe96e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:14.086915 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:14.087877 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:14.088264 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:14.088708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:14.110539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:14.113505 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:30:14.143945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:14.145054 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:14.146089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:30:14.146089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:30:14.146089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:14.146089 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:14.146640 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 630/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:14.156227 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:14.275639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:14.310704 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769ae10-ea85-41f4-98d8-0338c8f3bd0a None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:14.377763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:14.378607 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:14.378898 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:14.379184 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:14.379928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:14.380236 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:14.381729 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:14.396818 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 636/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:14.403176 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:14.566653 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fd712-d42d-48d1-a3d0-1545b83e9c0a None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:14.600594 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:14.629523 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:14.630322 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:14.630749 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:14.631164 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] PATH_INFO: `/v3/users/placement` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:14.645370 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:14.648101 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:30:14.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:14.675136 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:14.676225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 631/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:14.684999 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:14.848760 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b217b-d3f9-48f0-bad7-289d1d8b3de4 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:14.905994 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:14.907135 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:14.907707 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:14.908639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] PATH_INFO: `/v3/users` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:14.909724 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:14.910366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:14.913049 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:14.925661 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:14.937387 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 637/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:30:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:14.947470 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:15.103702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e52a-be60-45cd-a712-2161ba7dea76 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:15.143796 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:15.144458 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:15.144738 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:15.145009 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:15.158399 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:15.159804 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:30:15.174413 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:15.176466 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:15.178730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:30:15.178730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:30:15.178730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:15.178730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:15.178730 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 632/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:15.183103 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:15.306242 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e9001-f163-4c9c-add4-71fe87538b55 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:15.345918 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:15.346592 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:15.346872 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:15.347147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:15.347836 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:15.348142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:15.349531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:15.364815 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 638/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:15.372908 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:15.409882 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:15.484930 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eb7aa-414a-4cb0-81a9-d8e6d6eee606 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:15.532887 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:15.533964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:15.534436 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:15.534890 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:15.542539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:15.544011 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:30:15.555532 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:15.556538 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:15.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 633/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:15.562988 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:15.628763 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:15.674489 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ed8a3-9849-4dbc-b09c-2be7f845f908 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:15.725816 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284503e-0484-45d6-809a-a9ad718724cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:15.726617 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284503e-0484-45d6-809a-a9ad718724cf None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:15.726902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284503e-0484-45d6-809a-a9ad718724cf None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:15.727177 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284503e-0484-45d6-809a-a9ad718724cf None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:15.727956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284503e-0484-45d6-809a-a9ad718724cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:15.728267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284503e-0484-45d6-809a-a9ad718724cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:15.729738 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284503e-0484-45d6-809a-a9ad718724cf None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:15.749247 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 639/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:30:15.756635 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:15.897815 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:15.912444 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf91905-f24c-4ecd-adff-756a2c04a077 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:15.952953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b30ab6-bd96-4e63-bca1-17a344ce7ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:15.954245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b30ab6-bd96-4e63-bca1-17a344ce7ef1 None admin] REQUEST_METHOD: `PUT` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:15.954528 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b30ab6-bd96-4e63-bca1-17a344ce7ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:15.954840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b30ab6-bd96-4e63-bca1-17a344ce7ef1 None admin] PATH_INFO: `/v3/projects/d354364f219049a7ae56aa2de34993d8/users/b83a030752874fb2b7290691762ab9c2/roles/d10f749332ad4fd58e4378ef32d1bc2d` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:15.957299 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b30ab6-bd96-4e63-bca1-17a344ce7ef1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d354364f219049a7ae56aa2de34993d8, user_id=b83a030752874fb2b7290691762ab9c2, role_id=d10f749332ad4fd58e4378ef32d1bc2d)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:15.959178 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b30ab6-bd96-4e63-bca1-17a344ce7ef1 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:15.979764 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 634/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:30:15 2022] PUT /identity/v3/projects/d354364f219049a7ae56aa2de34993d8/users/b83a030752874fb2b7290691762ab9c2/roles/d10f749332ad4fd58e4378ef32d1bc2d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:30:18.017113 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e8ab-f6f9-4bcf-baa9-a107545ee79f None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:18.017989 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e8ab-f6f9-4bcf-baa9-a107545ee79f None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:18.018183 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e8ab-f6f9-4bcf-baa9-a107545ee79f None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:18.020149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 640/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:30:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:30:18.027542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249e85f-8089-4175-ba1f-f0701a177215 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:18.028175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249e85f-8089-4175-ba1f-f0701a177215 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:18.028175 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249e85f-8089-4175-ba1f-f0701a177215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:18.048543 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-5249e85f-8089-4175-ba1f-f0701a177215 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:30:18.049810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-5249e85f-8089-4175-ba1f-f0701a177215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:30:18.122507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 635/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:30:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:30:18.147883 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40e8ab-f6f9-4bcf-baa9-a107545ee79f None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:18.272406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40e8ab-f6f9-4bcf-baa9-a107545ee79f None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:18.291836 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:18.311335 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b07a78-db58-469e-84a6-ec9a624e00c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:18.312038 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b07a78-db58-469e-84a6-ec9a624e00c7 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:18.312328 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b07a78-db58-469e-84a6-ec9a624e00c7 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:18.312623 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b07a78-db58-469e-84a6-ec9a624e00c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:18.324444 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b07a78-db58-469e-84a6-ec9a624e00c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-d7b07a78-db58-469e-84a6-ec9a624e00c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:30:18.329653 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:18.331678 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:18.333385 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 641/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:30:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:18.337608 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249e85f-8089-4175-ba1f-f0701a177215 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:18.430801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:18.452002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249e85f-8089-4175-ba1f-f0701a177215 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:18.489150 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:18.489795 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:18.490078 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:18.490379 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] PATH_INFO: `/v3/roles` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:18.491111 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] RBAC: Adding query filter params (name=admin) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:18.491430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:18.492835 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:18.504734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 636/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:30:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:18.513004 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b07a78-db58-469e-84a6-ec9a624e00c7 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:18.643183 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b07a78-db58-469e-84a6-ec9a624e00c7 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:18.689671 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:18.690981 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:18.690981 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:18.690981 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:18.703744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:18.705260 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:30:18.723225 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:18.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:18.727484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:30:18.727484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:30:18.727484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:18.727484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:18.727484 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 642/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:18.732368 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:18.877485 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35a81e-95c0-4da7-b87a-a01c0344a255 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:18.916811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:18.917355 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:18.917636 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:18.917920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] PATH_INFO: `/v3/domains` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:18.918650 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:18.918970 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:18.920395 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:18.934072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 637/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:18.940386 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:18.977547 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:19.072448 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80fa77-81fa-4bc9-9650-0a432bfd43fe None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:19.116452 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:19.117134 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:19.117422 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:19.117697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:19.128811 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:19.130419 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:30:19.145021 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:19.147274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:19.149395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 643/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:19.154949 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:19.159642 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:19.276618 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c716-8a28-46cf-a16f-cf8a188a7407 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:19.332528 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:19.333598 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:19.334043 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:19.334540 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] PATH_INFO: `/v3/users` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:19.335878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:19.336386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:19.337986 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:19.359250 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 638/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:30:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:19.365625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:19.499169 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:19.510919 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad42d66-f51f-46bd-b011-733e5ff0d8a8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:19.549119 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab9bd2-1ec8-40ba-b669-f66de5c2a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:19.549672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab9bd2-1ec8-40ba-b669-f66de5c2a698 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:19.549951 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab9bd2-1ec8-40ba-b669-f66de5c2a698 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:19.550252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab9bd2-1ec8-40ba-b669-f66de5c2a698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:19.631937 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab9bd2-1ec8-40ba-b669-f66de5c2a698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:19.633417 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab9bd2-1ec8-40ba-b669-f66de5c2a698 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:19.752805 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:19.765957 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 644/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:30:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:30:19.773725 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:19.845373 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:19.886343 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f632-67a5-4692-9d17-85d778e9e24e None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:19.924058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:19.924740 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:19.925023 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:19.925301 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:19.936663 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:19.938388 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:30:19.955392 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:19.956676 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:19.957715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:30:19.957715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:30:19.957715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:30:19.957715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:19.957715 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 639/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:19.962782 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab9bd2-1ec8-40ba-b669-f66de5c2a698 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:20.087970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab9bd2-1ec8-40ba-b669-f66de5c2a698 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:20.135246 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:20.136342 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:20.136767 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:20.137261 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] PATH_INFO: `/v3/domains` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:20.138685 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:20.139252 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:20.141894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:20.166181 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 645/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:20.172693 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:20.290866 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156dc74-9431-4395-bab2-9322cfe67824 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:20.306509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:20.330234 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:20.330920 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:20.331199 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:20.331469 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:20.339145 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:20.340606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:30:20.352386 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:20.354156 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: ERROR keystone.server.flask.application Jul 28 08:30:20.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 640/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:30:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:20.361474 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:20.415085 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:20.468697 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09df7-3101-4082-8657-03d34341a414 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:20.502593 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9af5b-9105-4014-873f-74d5b5605d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:20.503234 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9af5b-9105-4014-873f-74d5b5605d11 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:20.503513 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9af5b-9105-4014-873f-74d5b5605d11 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:20.503784 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9af5b-9105-4014-873f-74d5b5605d11 None admin] PATH_INFO: `/v3/projects` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:20.504541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9af5b-9105-4014-873f-74d5b5605d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:20.504847 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9af5b-9105-4014-873f-74d5b5605d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:20.506315 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9af5b-9105-4014-873f-74d5b5605d11 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:20.519866 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 646/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:20.529646 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:20.640848 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd2848-7508-4d76-89e8-aa40a3108ad6 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:20.676167 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95194884-f202-45e6-83b6-04f04e162090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:20.676699 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95194884-f202-45e6-83b6-04f04e162090 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:20.676977 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95194884-f202-45e6-83b6-04f04e162090 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:20.677257 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95194884-f202-45e6-83b6-04f04e162090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:20.678047 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95194884-f202-45e6-83b6-04f04e162090 None admin] RBAC: Adding query filter params (role.id=d10f749332ad4fd58e4378ef32d1bc2d, scope.project.id=d354364f219049a7ae56aa2de34993d8, user.id=b83a030752874fb2b7290691762ab9c2) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:20.678384 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95194884-f202-45e6-83b6-04f04e162090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:20.679840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95194884-f202-45e6-83b6-04f04e162090 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:20.685889 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 641/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:30:20 2022] GET /identity/v3/role_assignments?user.id=b83a030752874fb2b7290691762ab9c2&scope.project.id=d354364f219049a7ae56aa2de34993d8&role.id=d10f749332ad4fd58e4378ef32d1bc2d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:22.736250 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95acf7c-35d1-44a2-90ac-30afe30dc9a7 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:22.736796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95acf7c-35d1-44a2-90ac-30afe30dc9a7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:22.736796 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95acf7c-35d1-44a2-90ac-30afe30dc9a7 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:22.738337 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 647/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:30:22.745493 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc209b9d-2f43-40f0-a155-09c2cfb9a4bf None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:22.746004 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc209b9d-2f43-40f0-a155-09c2cfb9a4bf None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:22.746220 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc209b9d-2f43-40f0-a155-09c2cfb9a4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:22.778563 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-dc209b9d-2f43-40f0-a155-09c2cfb9a4bf None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:30:22.780184 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-dc209b9d-2f43-40f0-a155-09c2cfb9a4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:30:22.818099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:22.859541 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 642/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:30:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:30:22.881970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95acf7c-35d1-44a2-90ac-30afe30dc9a7 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:23.000778 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95acf7c-35d1-44a2-90ac-30afe30dc9a7 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:23.013086 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:23.042541 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:23.043126 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:23.043439 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:23.043713 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:23.044353 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:23.045775 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:23.055163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:23.056367 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:23.057496 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 648/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:30:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:23.062814 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc209b9d-2f43-40f0-a155-09c2cfb9a4bf None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:23.168635 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc209b9d-2f43-40f0-a155-09c2cfb9a4bf None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:23.204368 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c695b14-ba12-454b-b3b9-f1665c9d1fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:23.205006 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c695b14-ba12-454b-b3b9-f1665c9d1fb4 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:23.205289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c695b14-ba12-454b-b3b9-f1665c9d1fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:23.205569 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c695b14-ba12-454b-b3b9-f1665c9d1fb4 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:23.206281 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c695b14-ba12-454b-b3b9-f1665c9d1fb4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:23.206589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c695b14-ba12-454b-b3b9-f1665c9d1fb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:23.208002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c695b14-ba12-454b-b3b9-f1665c9d1fb4 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:23.213420 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 643/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:30:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:23.219639 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:23.334956 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937fb5-b8a3-4908-a42c-ee7ea99f66d8 None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:23.371959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad18c85-79d1-47bb-b864-6643108055db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:23.372588 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad18c85-79d1-47bb-b864-6643108055db None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:23.372877 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad18c85-79d1-47bb-b864-6643108055db None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:23.373154 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad18c85-79d1-47bb-b864-6643108055db None admin] PATH_INFO: `/v3/services` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:23.373829 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad18c85-79d1-47bb-b864-6643108055db None admin] RBAC: Adding query filter params (type=placement) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:23.374141 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad18c85-79d1-47bb-b864-6643108055db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:23.375544 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad18c85-79d1-47bb-b864-6643108055db None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:23.380568 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 649/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:30:23 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:25.408076 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadc238-ea25-40f7-830f-0a3667a4632d None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:25.408645 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadc238-ea25-40f7-830f-0a3667a4632d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:25.408645 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadc238-ea25-40f7-830f-0a3667a4632d None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:25.410015 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 644/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:30:25.418197 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18073-814f-48e9-b5fc-c57bf596f0a4 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:25.418687 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18073-814f-48e9-b5fc-c57bf596f0a4 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:25.418801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18073-814f-48e9-b5fc-c57bf596f0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:25.438838 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-b3b18073-814f-48e9-b5fc-c57bf596f0a4 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:30:25.440060 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-b3b18073-814f-48e9-b5fc-c57bf596f0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:30:25.473499 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:25.518294 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 650/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:30:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:30:25.541924 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadc238-ea25-40f7-830f-0a3667a4632d None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:25.577703 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:25.656411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadc238-ea25-40f7-830f-0a3667a4632d None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:25.694539 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55407bb5-b1f8-4e8b-961c-d8745897783b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:25.695099 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407bb5-b1f8-4e8b-961c-d8745897783b None admin] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:25.695381 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407bb5-b1f8-4e8b-961c-d8745897783b None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:25.695657 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407bb5-b1f8-4e8b-961c-d8745897783b None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:25.696357 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55407bb5-b1f8-4e8b-961c-d8745897783b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:25.697751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55407bb5-b1f8-4e8b-961c-d8745897783b None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:25.713501 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 645/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:30:25 2022] POST /identity/v3/services => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:30:27.753742 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d7f0-530e-41ef-95c5-d3151b10def2 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:27.754380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d7f0-530e-41ef-95c5-d3151b10def2 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:27.754380 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d7f0-530e-41ef-95c5-d3151b10def2 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:27.755774 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 651/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:30:27.762287 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48832130-749e-4356-9766-5a1d37ab33f7 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:27.762975 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48832130-749e-4356-9766-5a1d37ab33f7 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:27.762975 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48832130-749e-4356-9766-5a1d37ab33f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:27.787270 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-48832130-749e-4356-9766-5a1d37ab33f7 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:30:27.788883 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-48832130-749e-4356-9766-5a1d37ab33f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:30:27.875629 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 646/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:30:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:30:27.901571 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d7f0-530e-41ef-95c5-d3151b10def2 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:28.027415 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d7f0-530e-41ef-95c5-d3151b10def2 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:28.067490 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:28.068091 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:28.068395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:28.068643 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] PATH_INFO: `/v3/services/placement` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:28.069665 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:28.071110 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:28.083143 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:28.089506 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:28.090733 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 652/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:30:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:28.091844 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48832130-749e-4356-9766-5a1d37ab33f7 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:28.119777 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:28.209613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48832130-749e-4356-9766-5a1d37ab33f7 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:28.247100 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:28.247739 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:28.248020 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:28.248298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:28.248990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] RBAC: Adding query filter params (name=placement) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:28.249335 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:28.250788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:28.256579 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 647/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:30:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:28.263664 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:28.304211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:28.379667 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ab3e-0afb-4774-9087-9ef3e63a460f None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:28.416267 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0013ec7-3250-457f-8666-e535d078be0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:28.416911 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0013ec7-3250-457f-8666-e535d078be0c None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:28.417191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0013ec7-3250-457f-8666-e535d078be0c None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:28.417467 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0013ec7-3250-457f-8666-e535d078be0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:28.418281 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0013ec7-3250-457f-8666-e535d078be0c None admin] RBAC: Adding query filter params (interface=public, service_id=b4b56cf6240b407fb0f56146938f1d62, region_id=RegionOne) {{(pid=207769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:28.418597 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0013ec7-3250-457f-8666-e535d078be0c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:28.419997 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0013ec7-3250-457f-8666-e535d078be0c None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:28.431574 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 653/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:30:28 2022] GET /identity/v3/endpoints?service_id=b4b56cf6240b407fb0f56146938f1d62&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:28.438040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:28.531193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:28.556027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11995021-958e-4fa1-9d30-0044c2526670 None admin] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:28.593985 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa2150-6d36-4c34-9cc9-4ddd56b22746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:28.594741 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa2150-6d36-4c34-9cc9-4ddd56b22746 None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:28.595027 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa2150-6d36-4c34-9cc9-4ddd56b22746 None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:28.595302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa2150-6d36-4c34-9cc9-4ddd56b22746 None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:28.595964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa2150-6d36-4c34-9cc9-4ddd56b22746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:28.597378 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa2150-6d36-4c34-9cc9-4ddd56b22746 None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:28.604735 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 648/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:30:28 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:30:30.629507 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc2065-2b7c-4b5d-8e62-b265de5fa621 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:30.630331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc2065-2b7c-4b5d-8e62-b265de5fa621 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:30.630331 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc2065-2b7c-4b5d-8e62-b265de5fa621 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:30.631540 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 654/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:30:30.638718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f1643-01df-40a0-b64f-392e93d97799 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:30.639261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f1643-01df-40a0-b64f-392e93d97799 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:30.639261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f1643-01df-40a0-b64f-392e93d97799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:30.663850 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-db3f1643-01df-40a0-b64f-392e93d97799 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:30:30.665805 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-db3f1643-01df-40a0-b64f-392e93d97799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:30:30.768180 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 649/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:30:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:30:30.797737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc2065-2b7c-4b5d-8e62-b265de5fa621 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:30.934696 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:30.945661 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc2065-2b7c-4b5d-8e62-b265de5fa621 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:30.998681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:30.999423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:30.999706 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:30.999978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] PATH_INFO: `/v3/services/placement` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:31.000632 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:31.002073 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:30:31.012254 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:30:31.013809 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application with Lock( Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: ERROR keystone.server.flask.application Jul 28 08:30:31.015378 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 655/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:30:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:30:31.021058 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f1643-01df-40a0-b64f-392e93d97799 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:31.143208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:31.175781 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f1643-01df-40a0-b64f-392e93d97799 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:31.221785 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd50426-4da8-4072-898b-593874d0c8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:31.222771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd50426-4da8-4072-898b-593874d0c8ff None admin] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:31.223245 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd50426-4da8-4072-898b-593874d0c8ff None admin] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:31.223626 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd50426-4da8-4072-898b-593874d0c8ff None admin] PATH_INFO: `/v3/services` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:31.224597 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd50426-4da8-4072-898b-593874d0c8ff None admin] RBAC: Adding query filter params (name=placement) {{(pid=207770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:30:31.225081 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd50426-4da8-4072-898b-593874d0c8ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:31.226907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd50426-4da8-4072-898b-593874d0c8ff None admin] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:31.234102 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 650/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:30:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:30:31.240566 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:30:31.364406 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00df000-186d-4b00-a1ac-adf1c5be980e None admin] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:30:31.405923 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689ac51-eabf-4236-b533-e8720b3d566d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8bf4bf753c41b2a872093710161103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:30:31.407050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689ac51-eabf-4236-b533-e8720b3d566d None admin] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:30:31.407050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689ac51-eabf-4236-b533-e8720b3d566d None admin] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:30:31.407279 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689ac51-eabf-4236-b533-e8720b3d566d None admin] PATH_INFO: `/v3/endpoints` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:30:31.407821 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689ac51-eabf-4236-b533-e8720b3d566d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:30:31.409219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689ac51-eabf-4236-b533-e8720b3d566d None admin] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:30:31.441517 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 656/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:30:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:30:41.229999 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=30 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:30:41.429163 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:39.416794 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd948a-5f3e-4796-98b2-b88a436688fd None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:39.417407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd948a-5f3e-4796-98b2-b88a436688fd None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:39.417407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd948a-5f3e-4796-98b2-b88a436688fd None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:39.418870 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 651/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:39.425112 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74122-370a-404c-acd2-a71a47b5e5b3 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:39.425688 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74122-370a-404c-acd2-a71a47b5e5b3 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:39.425884 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74122-370a-404c-acd2-a71a47b5e5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:39.521945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-def74122-370a-404c-acd2-a71a47b5e5b3 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:39.524003 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-def74122-370a-404c-acd2-a71a47b5e5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:39.716895 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 657/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:31:39.773111 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef7a2-7102-4d22-8993-79969bbac7e6 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:39.773407 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef7a2-7102-4d22-8993-79969bbac7e6 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:39.773606 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef7a2-7102-4d22-8993-79969bbac7e6 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:39.775279 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 652/1309] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 08:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:39.787849 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e4ec7-0ade-40ec-a310-dc67c2e16e54 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:39.788207 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e4ec7-0ade-40ec-a310-dc67c2e16e54 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:39.788497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e4ec7-0ade-40ec-a310-dc67c2e16e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:39.947151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-544e4ec7-0ade-40ec-a310-dc67c2e16e54 None None] MFA Rules not processed for user `78394014f5904021823a96d12902b2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:39.951016 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-544e4ec7-0ade-40ec-a310-dc67c2e16e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:39.993047 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:40.074970 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 658/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 08:31:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:31:40.084561 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ef7a2-7102-4d22-8993-79969bbac7e6 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:31:40.086781 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-4b6ef7a2-7102-4d22-8993-79969bbac7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:40.214091 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ef7a2-7102-4d22-8993-79969bbac7e6 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:31:40.313097 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4bc0b-e54d-4a84-80d1-8b5078983277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:31:40.313711 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4bc0b-e54d-4a84-80d1-8b5078983277 service neutron] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:40.313995 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4bc0b-e54d-4a84-80d1-8b5078983277 service neutron] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:40.314295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4bc0b-e54d-4a84-80d1-8b5078983277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:40.316103 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-37a4bc0b-e54d-4a84-80d1-8b5078983277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:40.386303 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:40.464713 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4bc0b-e54d-4a84-80d1-8b5078983277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:31:40.466398 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4bc0b-e54d-4a84-80d1-8b5078983277 service neutron] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:31:40.487977 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 653/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:31:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:31:42.872489 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86e92-61eb-4876-8e04-ec2624fa4d2a None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:42.873037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86e92-61eb-4876-8e04-ec2624fa4d2a None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:42.873037 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86e92-61eb-4876-8e04-ec2624fa4d2a None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:42.874596 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 659/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:42.879779 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb6dd8-c265-4ba2-aabc-e5965ffdebed None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:42.880214 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb6dd8-c265-4ba2-aabc-e5965ffdebed None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:42.880298 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb6dd8-c265-4ba2-aabc-e5965ffdebed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:42.964955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-dbcb6dd8-c265-4ba2-aabc-e5965ffdebed None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:42.966292 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-dbcb6dd8-c265-4ba2-aabc-e5965ffdebed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:43.082402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 654/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:31:43.120914 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d86e92-61eb-4876-8e04-ec2624fa4d2a None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:31:43.122655 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-58d86e92-61eb-4876-8e04-ec2624fa4d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:43.245792 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d86e92-61eb-4876-8e04-ec2624fa4d2a None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:31:43.332672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:43.336551 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839c7b1-e9ea-4c1f-8382-ac9ca5a2c767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:31:43.337244 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839c7b1-e9ea-4c1f-8382-ac9ca5a2c767 service neutron] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:43.337602 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839c7b1-e9ea-4c1f-8382-ac9ca5a2c767 service neutron] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:43.337828 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839c7b1-e9ea-4c1f-8382-ac9ca5a2c767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:43.339669 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-9839c7b1-e9ea-4c1f-8382-ac9ca5a2c767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:43.466166 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839c7b1-e9ea-4c1f-8382-ac9ca5a2c767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:31:43.467831 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839c7b1-e9ea-4c1f-8382-ac9ca5a2c767 service neutron] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:31:43.489051 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 660/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:31:45.863280 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0f8c4-5a7e-43f8-a113-6e5bc80c449c None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:45.863849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0f8c4-5a7e-43f8-a113-6e5bc80c449c None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:45.863849 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0f8c4-5a7e-43f8-a113-6e5bc80c449c None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:45.865148 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 655/1315] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:45.870510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22102ce-7f04-4956-92ea-629521cfdfc4 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:45.870984 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22102ce-7f04-4956-92ea-629521cfdfc4 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:45.871096 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22102ce-7f04-4956-92ea-629521cfdfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:45.942945 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-c22102ce-7f04-4956-92ea-629521cfdfc4 None None] MFA Rules not processed for user `169f2ee6fbe84afe8edc653401e01357`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:45.944217 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-c22102ce-7f04-4956-92ea-629521cfdfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:46.005107 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:46.054744 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 661/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:31:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:31:46.097834 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0f8c4-5a7e-43f8-a113-6e5bc80c449c None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:31:46.099503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-21c0f8c4-5a7e-43f8-a113-6e5bc80c449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:46.111256 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:46.221549 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0f8c4-5a7e-43f8-a113-6e5bc80c449c None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:31:46.302201 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d017dee-7348-46e0-b5e1-7b18d41d8f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:31:46.302840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d017dee-7348-46e0-b5e1-7b18d41d8f90 service neutron] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:46.303117 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d017dee-7348-46e0-b5e1-7b18d41d8f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:46.303402 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d017dee-7348-46e0-b5e1-7b18d41d8f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:46.305043 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-7d017dee-7348-46e0-b5e1-7b18d41d8f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:46.369284 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:46.420362 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d017dee-7348-46e0-b5e1-7b18d41d8f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:31:46.421840 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d017dee-7348-46e0-b5e1-7b18d41d8f90 service neutron] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:31:46.441193 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 656/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:31:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:31:49.079043 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0bb2-5b63-41b5-b7eb-23ec494ee153 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:49.079672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0bb2-5b63-41b5-b7eb-23ec494ee153 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:49.079672 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0bb2-5b63-41b5-b7eb-23ec494ee153 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:49.081098 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 662/1318] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:49.087412 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b8cc-6677-4749-bc33-b06240f6f60c None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:49.088001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b8cc-6677-4749-bc33-b06240f6f60c None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:49.088001 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b8cc-6677-4749-bc33-b06240f6f60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:49.165950 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-9c55b8cc-6677-4749-bc33-b06240f6f60c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:49.167233 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-9c55b8cc-6677-4749-bc33-b06240f6f60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:49.267005 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 657/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:31:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:31:49.305940 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a0bb2-5b63-41b5-b7eb-23ec494ee153 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:31:49.307543 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-4b0a0bb2-5b63-41b5-b7eb-23ec494ee153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:49.419652 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a0bb2-5b63-41b5-b7eb-23ec494ee153 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:31:49.501200 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a2ba3-29c4-4505-8533-2a351b898308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:31:49.501851 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a2ba3-29c4-4505-8533-2a351b898308 service neutron] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:49.502147 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a2ba3-29c4-4505-8533-2a351b898308 service neutron] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:49.502435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a2ba3-29c4-4505-8533-2a351b898308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:49.504151 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-016a2ba3-29c4-4505-8533-2a351b898308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:49.512750 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:49.625992 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a2ba3-29c4-4505-8533-2a351b898308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:31:49.627510 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a2ba3-29c4-4505-8533-2a351b898308 service neutron] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:31:49.648999 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 663/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:31:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 08:31:52.416186 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72873aff-070a-4c1a-ace6-9726c3eee1a7 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:52.416838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72873aff-070a-4c1a-ace6-9726c3eee1a7 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:52.416838 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72873aff-070a-4c1a-ace6-9726c3eee1a7 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:52.418144 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 658/1321] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:52.423659 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8be31-2875-40b2-9ee0-8b3d170de765 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:52.424128 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8be31-2875-40b2-9ee0-8b3d170de765 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:52.424355 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8be31-2875-40b2-9ee0-8b3d170de765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:52.498176 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-d6c8be31-2875-40b2-9ee0-8b3d170de765 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:52.499702 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-d6c8be31-2875-40b2-9ee0-8b3d170de765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:52.595324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:52.645745 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 664/1322] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:31:52.699244 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902df09-a9cb-4a37-8ad6-5b7e469fd076 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:52.699492 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902df09-a9cb-4a37-8ad6-5b7e469fd076 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:52.699798 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902df09-a9cb-4a37-8ad6-5b7e469fd076 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:52.701209 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 659/1323] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 08:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:52.708100 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d3877-fc2a-4b9c-bf48-65bee54dd40c None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:52.708489 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d3877-fc2a-4b9c-bf48-65bee54dd40c None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:52.708836 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d3877-fc2a-4b9c-bf48-65bee54dd40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:52.802190 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-998d3877-fc2a-4b9c-bf48-65bee54dd40c None None] MFA Rules not processed for user `78394014f5904021823a96d12902b2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:52.803457 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-998d3877-fc2a-4b9c-bf48-65bee54dd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:52.892813 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 665/1324] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 08:31:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:31:52.902381 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902df09-a9cb-4a37-8ad6-5b7e469fd076 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:31:52.904069 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-f902df09-a9cb-4a37-8ad6-5b7e469fd076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:52.925786 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:53.042788 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902df09-a9cb-4a37-8ad6-5b7e469fd076 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:31:53.126363 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a03bf2-9095-4c1f-97d1-7c5fc4755721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:31:53.127003 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a03bf2-9095-4c1f-97d1-7c5fc4755721 service neutron] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:53.127293 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a03bf2-9095-4c1f-97d1-7c5fc4755721 service neutron] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:53.127589 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a03bf2-9095-4c1f-97d1-7c5fc4755721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:53.129228 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-72a03bf2-9095-4c1f-97d1-7c5fc4755721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:53.209955 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:53.250628 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a03bf2-9095-4c1f-97d1-7c5fc4755721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:31:53.252088 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a03bf2-9095-4c1f-97d1-7c5fc4755721 service neutron] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:31:53.271293 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 660/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:31:55.585341 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06f478-c626-4ced-bf5c-b04e04aba6ca None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:55.585915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06f478-c626-4ced-bf5c-b04e04aba6ca None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:55.585915 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06f478-c626-4ced-bf5c-b04e04aba6ca None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:55.587325 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 666/1326] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:55.592910 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6aecc-26c9-413d-a345-c46d6748d363 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:55.593372 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6aecc-26c9-413d-a345-c46d6748d363 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:55.593472 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6aecc-26c9-413d-a345-c46d6748d363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:55.673751 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-66f6aecc-26c9-413d-a345-c46d6748d363 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:55.675087 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-66f6aecc-26c9-413d-a345-c46d6748d363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:55.776047 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 661/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:31:55.812757 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06f478-c626-4ced-bf5c-b04e04aba6ca None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:31:55.814362 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-6a06f478-c626-4ced-bf5c-b04e04aba6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:55.879253 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:55.923825 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06f478-c626-4ced-bf5c-b04e04aba6ca None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:31:56.008978 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac7d8d-eb53-4d2a-b635-cd02632a5808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:31:56.009533 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac7d8d-eb53-4d2a-b635-cd02632a5808 service neutron] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:56.009814 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac7d8d-eb53-4d2a-b635-cd02632a5808 service neutron] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:56.010087 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac7d8d-eb53-4d2a-b635-cd02632a5808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:56.011806 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-90ac7d8d-eb53-4d2a-b635-cd02632a5808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:56.128256 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:56.131020 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac7d8d-eb53-4d2a-b635-cd02632a5808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:31:56.132531 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac7d8d-eb53-4d2a-b635-cd02632a5808 service neutron] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:31:56.151830 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 667/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:31:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:31:58.199934 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37065f-552e-4697-95ac-4fd1f0801127 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:58.200530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37065f-552e-4697-95ac-4fd1f0801127 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:58.200530 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37065f-552e-4697-95ac-4fd1f0801127 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:58.201894 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 662/1329] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:31:58.207518 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85456ca-4b34-4f68-9ea6-835a86a1286a None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:58.207913 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85456ca-4b34-4f68-9ea6-835a86a1286a None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:58.208068 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85456ca-4b34-4f68-9ea6-835a86a1286a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:58.277274 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-e85456ca-4b34-4f68-9ea6-835a86a1286a None None] MFA Rules not processed for user `169f2ee6fbe84afe8edc653401e01357`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:31:58.278545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-e85456ca-4b34-4f68-9ea6-835a86a1286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:58.384131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 668/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:31:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:31:58.422057 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37065f-552e-4697-95ac-4fd1f0801127 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:31:58.423675 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-ee37065f-552e-4697-95ac-4fd1f0801127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:58.456509 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:58.533411 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37065f-552e-4697-95ac-4fd1f0801127 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:31:58.611953 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bcbe0-8ff2-4c4f-a05c-b00e8f8451fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:31:58.612583 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bcbe0-8ff2-4c4f-a05c-b00e8f8451fc service neutron] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:31:58.612864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bcbe0-8ff2-4c4f-a05c-b00e8f8451fc service neutron] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:31:58.613151 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bcbe0-8ff2-4c4f-a05c-b00e8f8451fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:31:58.614846 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-d41bcbe0-8ff2-4c4f-a05c-b00e8f8451fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:31:58.693869 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:31:58.729183 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41bcbe0-8ff2-4c4f-a05c-b00e8f8451fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:31:58.730623 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41bcbe0-8ff2-4c4f-a05c-b00e8f8451fc service neutron] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:31:58.749289 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 663/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:31:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:01.166625 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1bc3-510c-4c59-96a1-912c39a0211e None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:01.167212 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1bc3-510c-4c59-96a1-912c39a0211e None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:01.167212 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1bc3-510c-4c59-96a1-912c39a0211e None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:01.168577 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 669/1332] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:01.174083 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d31e32-cd26-4125-a928-bad4b84e5124 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:01.174580 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d31e32-cd26-4125-a928-bad4b84e5124 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:01.174696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d31e32-cd26-4125-a928-bad4b84e5124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:01.246055 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-b9d31e32-cd26-4125-a928-bad4b84e5124 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:01.247365 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-b9d31e32-cd26-4125-a928-bad4b84e5124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:01.352667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 664/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:01.388938 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb1bc3-510c-4c59-96a1-912c39a0211e None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:01.390638 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-2ecb1bc3-510c-4c59-96a1-912c39a0211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:01.468203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:01.509790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb1bc3-510c-4c59-96a1-912c39a0211e None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:01.611790 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724ab84-4444-4200-a105-6a03604e75eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:01.612407 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724ab84-4444-4200-a105-6a03604e75eb service neutron] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:01.612681 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724ab84-4444-4200-a105-6a03604e75eb service neutron] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:01.612980 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724ab84-4444-4200-a105-6a03604e75eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:01.614723 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-0724ab84-4444-4200-a105-6a03604e75eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:01.735539 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724ab84-4444-4200-a105-6a03604e75eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:01.736895 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724ab84-4444-4200-a105-6a03604e75eb service neutron] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:01.747364 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:01.756513 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 670/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:03.834878 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385d508-cc66-4686-827e-d36724876cda None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:03.835460 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385d508-cc66-4686-827e-d36724876cda None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:03.835460 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385d508-cc66-4686-827e-d36724876cda None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:03.836847 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 665/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:03.842324 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ab1b1-bbeb-4c99-ba28-b1b35ec1eef5 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:03.842993 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ab1b1-bbeb-4c99-ba28-b1b35ec1eef5 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:03.842993 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ab1b1-bbeb-4c99-ba28-b1b35ec1eef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:03.917011 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-333ab1b1-bbeb-4c99-ba28-b1b35ec1eef5 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:03.918142 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-333ab1b1-bbeb-4c99-ba28-b1b35ec1eef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:04.020701 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 671/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:04.061945 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385d508-cc66-4686-827e-d36724876cda None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:04.063642 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-1385d508-cc66-4686-827e-d36724876cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:04.109093 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:04.207225 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385d508-cc66-4686-827e-d36724876cda None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:04.325066 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757948-1735-4ea2-8665-c843fad8bdc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:04.325709 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757948-1735-4ea2-8665-c843fad8bdc1 service neutron] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:04.325988 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757948-1735-4ea2-8665-c843fad8bdc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:04.326295 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757948-1735-4ea2-8665-c843fad8bdc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:04.328261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-51757948-1735-4ea2-8665-c843fad8bdc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:04.419630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:04.447797 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51757948-1735-4ea2-8665-c843fad8bdc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:04.449273 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51757948-1735-4ea2-8665-c843fad8bdc1 service neutron] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:04.468314 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 666/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:06.836628 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f03221-06cf-4f6a-9c0f-e1dba3ae6c83 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:06.837219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f03221-06cf-4f6a-9c0f-e1dba3ae6c83 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:06.837219 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f03221-06cf-4f6a-9c0f-e1dba3ae6c83 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:06.838485 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 672/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:06.843911 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5726b-4c12-4c02-a974-de5296b0f74f None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:06.844379 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5726b-4c12-4c02-a974-de5296b0f74f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:06.844494 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5726b-4c12-4c02-a974-de5296b0f74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:06.912014 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-5fd5726b-4c12-4c02-a974-de5296b0f74f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:06.913267 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-5fd5726b-4c12-4c02-a974-de5296b0f74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:07.015201 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 667/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:07.052150 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f03221-06cf-4f6a-9c0f-e1dba3ae6c83 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:07.053741 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-17f03221-06cf-4f6a-9c0f-e1dba3ae6c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:07.137497 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:07.168093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f03221-06cf-4f6a-9c0f-e1dba3ae6c83 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:07.244125 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32e3ff-b464-49d1-96d7-38519d424e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:07.244770 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32e3ff-b464-49d1-96d7-38519d424e4e service neutron] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:07.245057 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32e3ff-b464-49d1-96d7-38519d424e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:07.245334 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32e3ff-b464-49d1-96d7-38519d424e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:07.247043 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-5b32e3ff-b464-49d1-96d7-38519d424e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:07.363880 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32e3ff-b464-49d1-96d7-38519d424e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:07.365343 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32e3ff-b464-49d1-96d7-38519d424e4e service neutron] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:07.384109 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 673/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:11.573230 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a156f8-93cc-4509-a6a7-cd7b38322c97 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:11.573811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a156f8-93cc-4509-a6a7-cd7b38322c97 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:11.573811 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a156f8-93cc-4509-a6a7-cd7b38322c97 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:11.575285 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 668/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:11.581859 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc8b6-8b0f-4391-98cc-0fdffa662077 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:11.582458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc8b6-8b0f-4391-98cc-0fdffa662077 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:11.582458 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc8b6-8b0f-4391-98cc-0fdffa662077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:11.589923 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:11.678188 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-822cc8b6-8b0f-4391-98cc-0fdffa662077 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:11.678916 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-822cc8b6-8b0f-4391-98cc-0fdffa662077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:11.808345 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 674/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:11.849870 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a156f8-93cc-4509-a6a7-cd7b38322c97 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:11.851613 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-28a156f8-93cc-4509-a6a7-cd7b38322c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:11.915989 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:11.986922 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a156f8-93cc-4509-a6a7-cd7b38322c97 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:12.081481 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc7080-4b1a-41ff-a4f9-45099ea52314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:12.082149 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7080-4b1a-41ff-a4f9-45099ea52314 service neutron] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:12.082449 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7080-4b1a-41ff-a4f9-45099ea52314 service neutron] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:12.082718 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7080-4b1a-41ff-a4f9-45099ea52314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:12.084442 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-a0cc7080-4b1a-41ff-a4f9-45099ea52314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:12.199803 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:12.227962 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc7080-4b1a-41ff-a4f9-45099ea52314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:12.229464 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc7080-4b1a-41ff-a4f9-45099ea52314 service neutron] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:12.251825 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 669/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:15.075423 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34698b9-ec8d-41f8-8bce-795565917872 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:15.076007 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34698b9-ec8d-41f8-8bce-795565917872 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:15.076131 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34698b9-ec8d-41f8-8bce-795565917872 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:15.077689 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 675/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:15.084077 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e925265-c3a2-4a16-963f-f3e169ef3a0f None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:15.084712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e925265-c3a2-4a16-963f-f3e169ef3a0f None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:15.084712 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e925265-c3a2-4a16-963f-f3e169ef3a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:15.161337 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-8e925265-c3a2-4a16-963f-f3e169ef3a0f None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:15.162690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-8e925265-c3a2-4a16-963f-f3e169ef3a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:15.275519 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 670/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:15.316269 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34698b9-ec8d-41f8-8bce-795565917872 None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:15.317902 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-f34698b9-ec8d-41f8-8bce-795565917872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:15.414545 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:15.432211 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34698b9-ec8d-41f8-8bce-795565917872 None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:15.517395 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c32d59-cfb5-40ae-a21d-4b6e68379b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:15.518040 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c32d59-cfb5-40ae-a21d-4b6e68379b69 service neutron] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:15.518340 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c32d59-cfb5-40ae-a21d-4b6e68379b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:15.518615 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c32d59-cfb5-40ae-a21d-4b6e68379b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:15.520361 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-c5c32d59-cfb5-40ae-a21d-4b6e68379b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:15.654459 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c32d59-cfb5-40ae-a21d-4b6e68379b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:15.656000 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c32d59-cfb5-40ae-a21d-4b6e68379b69 service neutron] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:15.675928 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 676/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:19.883052 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494557f-9b75-46a6-bdbb-20aa536eec13 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:19.883519 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494557f-9b75-46a6-bdbb-20aa536eec13 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:19.883630 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494557f-9b75-46a6-bdbb-20aa536eec13 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:19.884987 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 671/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:19.890995 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf811-f7a3-4475-bd3c-6812c33e78e7 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:19.891514 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf811-f7a3-4475-bd3c-6812c33e78e7 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:19.891608 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf811-f7a3-4475-bd3c-6812c33e78e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:19.908435 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:19.970374 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-e2cdf811-f7a3-4475-bd3c-6812c33e78e7 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:19.971650 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-e2cdf811-f7a3-4475-bd3c-6812c33e78e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:20.080526 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 677/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:20.118771 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494557f-9b75-46a6-bdbb-20aa536eec13 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:20.120696 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-3494557f-9b75-46a6-bdbb-20aa536eec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:20.185447 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:20.239946 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494557f-9b75-46a6-bdbb-20aa536eec13 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:20.317810 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f963156-31ee-4c55-8166-fee5914db304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78394014f5904021823a96d12902b2ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:20.319000 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f963156-31ee-4c55-8166-fee5914db304 service neutron] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:20.319831 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f963156-31ee-4c55-8166-fee5914db304 service neutron] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:20.320319 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f963156-31ee-4c55-8166-fee5914db304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:20.322661 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-6f963156-31ee-4c55-8166-fee5914db304 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:20.444900 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:20.449845 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f963156-31ee-4c55-8166-fee5914db304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:20.451806 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f963156-31ee-4c55-8166-fee5914db304 service neutron] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:20.478284 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 672/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:25.556674 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f14b27-7f12-40d9-a9d2-c4baf4d80027 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:25.557104 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f14b27-7f12-40d9-a9d2-c4baf4d80027 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:25.557298 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f14b27-7f12-40d9-a9d2-c4baf4d80027 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:25.559097 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 678/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:25.564040 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8523c-b64c-4fd1-baac-c8667ad3a029 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:25.564497 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8523c-b64c-4fd1-baac-c8667ad3a029 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:25.564592 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8523c-b64c-4fd1-baac-c8667ad3a029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:25.685261 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-e5f8523c-b64c-4fd1-baac-c8667ad3a029 None None] MFA Rules not processed for user `b83a030752874fb2b7290691762ab9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:25.686542 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-e5f8523c-b64c-4fd1-baac-c8667ad3a029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:25.870462 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 673/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:32:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:30.078293 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=59 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:30.170861 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980508f-f700-421a-b180-f2704d6eb213 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:30.171606 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980508f-f700-421a-b180-f2704d6eb213 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:30.172050 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980508f-f700-421a-b180-f2704d6eb213 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:30.173451 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 679/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:30.178907 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb9201-7d36-4f02-a1d9-3d01ef919c69 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:30.179400 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb9201-7d36-4f02-a1d9-3d01ef919c69 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:30.179507 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb9201-7d36-4f02-a1d9-3d01ef919c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:30.238889 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:30.266690 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-87eb9201-7d36-4f02-a1d9-3d01ef919c69 None None] MFA Rules not processed for user `b83a030752874fb2b7290691762ab9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:30.268050 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-87eb9201-7d36-4f02-a1d9-3d01ef919c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:30.396393 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 674/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:32:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:31.628093 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7319030-7579-4cfc-b31a-1b8649b87ca1 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:31.628691 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7319030-7579-4cfc-b31a-1b8649b87ca1 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:31.628691 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7319030-7579-4cfc-b31a-1b8649b87ca1 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:31.630014 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 680/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:31.635160 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe23-dad2-47ff-b7c3-8ddac48caa64 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:31.635600 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe23-dad2-47ff-b7c3-8ddac48caa64 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:31.635881 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe23-dad2-47ff-b7c3-8ddac48caa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:31.719430 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-564bbe23-dad2-47ff-b7c3-8ddac48caa64 None None] MFA Rules not processed for user `b83a030752874fb2b7290691762ab9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:31.720667 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-564bbe23-dad2-47ff-b7c3-8ddac48caa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:31.766154 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:31.834990 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 675/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:32:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:35.316538 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f7145-ef77-4577-85ed-171166208027 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:35.317100 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f7145-ef77-4577-85ed-171166208027 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:35.317100 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f7145-ef77-4577-85ed-171166208027 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:35.318479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 681/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:35.324720 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d1ce9-49b1-4738-a409-a4bbf63360e3 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:35.325212 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d1ce9-49b1-4738-a409-a4bbf63360e3 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:35.325562 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d1ce9-49b1-4738-a409-a4bbf63360e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:35.410285 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-1a2d1ce9-49b1-4738-a409-a4bbf63360e3 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:35.411557 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-1a2d1ce9-49b1-4738-a409-a4bbf63360e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:35.527588 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 676/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:35.585493 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4610a63-0aec-4af6-90b6-a6eb76b111ab None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:35.586730 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4610a63-0aec-4af6-90b6-a6eb76b111ab None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:35.587206 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4610a63-0aec-4af6-90b6-a6eb76b111ab None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:35.589056 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 682/1358] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 08:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:35.597551 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36595c4-09e1-43a7-a0b8-6aece44bd50d None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:35.598671 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36595c4-09e1-43a7-a0b8-6aece44bd50d None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:35.599128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36595c4-09e1-43a7-a0b8-6aece44bd50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:35.618302 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:35.689881 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-a36595c4-09e1-43a7-a0b8-6aece44bd50d None None] MFA Rules not processed for user `c5ba9e7780f5437caac949931ca55b80`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:35.691510 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-a36595c4-09e1-43a7-a0b8-6aece44bd50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:35.821109 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 677/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 08:32:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:35.829172 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4610a63-0aec-4af6-90b6-a6eb76b111ab None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:35.830972 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-e4610a63-0aec-4af6-90b6-a6eb76b111ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:35.992637 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4610a63-0aec-4af6-90b6-a6eb76b111ab None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:36.078456 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:36.116139 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da500df-57e9-4588-9244-032d37f8e92b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ba9e7780f5437caac949931ca55b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:36.116913 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da500df-57e9-4588-9244-032d37f8e92b service nova] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:36.117290 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da500df-57e9-4588-9244-032d37f8e92b service nova] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:36.117661 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da500df-57e9-4588-9244-032d37f8e92b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:36.120872 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-8da500df-57e9-4588-9244-032d37f8e92b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:36.216747 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da500df-57e9-4588-9244-032d37f8e92b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:36.218632 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da500df-57e9-4588-9244-032d37f8e92b service nova] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:36.245366 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 683/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:38.479584 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4d3a0-8a95-4e6e-a5ee-dafd8b1aa6d6 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:38.480122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4d3a0-8a95-4e6e-a5ee-dafd8b1aa6d6 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:38.480122 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4d3a0-8a95-4e6e-a5ee-dafd8b1aa6d6 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:38.481777 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 678/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:38.487511 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e24adf-5fd1-4d7e-9ad6-63164cf66034 None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:38.487977 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e24adf-5fd1-4d7e-9ad6-63164cf66034 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:38.488258 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e24adf-5fd1-4d7e-9ad6-63164cf66034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:38.569894 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-a4e24adf-5fd1-4d7e-9ad6-63164cf66034 None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:38.571191 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-a4e24adf-5fd1-4d7e-9ad6-63164cf66034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:38.614731 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:38.682027 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 684/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:38.726734 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4d3a0-8a95-4e6e-a5ee-dafd8b1aa6d6 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:38.728439 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-b4e4d3a0-8a95-4e6e-a5ee-dafd8b1aa6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:38.807657 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:38.913010 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4d3a0-8a95-4e6e-a5ee-dafd8b1aa6d6 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:39.075128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92bb7e-f49a-49d3-a804-3ec385f69671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ba9e7780f5437caac949931ca55b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:39.075863 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92bb7e-f49a-49d3-a804-3ec385f69671 service nova] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:39.076208 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92bb7e-f49a-49d3-a804-3ec385f69671 service nova] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:39.076547 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92bb7e-f49a-49d3-a804-3ec385f69671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:39.078506 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-0d92bb7e-f49a-49d3-a804-3ec385f69671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:39.104002 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:39.182964 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92bb7e-f49a-49d3-a804-3ec385f69671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:39.184819 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92bb7e-f49a-49d3-a804-3ec385f69671 service nova] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:39.204864 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 679/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:41.590936 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ec3b-844e-48f0-b3ad-4fc421ea5087 None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:41.591493 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ec3b-844e-48f0-b3ad-4fc421ea5087 None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:41.591493 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ec3b-844e-48f0-b3ad-4fc421ea5087 None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:41.592963 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 685/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:41.598729 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c1d91-ad80-4192-9a91-26215f6ab16c None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:41.599304 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c1d91-ad80-4192-9a91-26215f6ab16c None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:41.599304 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c1d91-ad80-4192-9a91-26215f6ab16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:41.685228 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-9c1c1d91-ad80-4192-9a91-26215f6ab16c None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:41.686760 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-9c1c1d91-ad80-4192-9a91-26215f6ab16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:41.793708 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:41.802686 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 680/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:41.848535 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d3dfe-fc12-498d-835b-4673d932697d None None] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:41.848803 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d3dfe-fc12-498d-835b-4673d932697d None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:41.849078 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d3dfe-fc12-498d-835b-4673d932697d None None] PATH_INFO: `/` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:41.850654 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 686/1366] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 08:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:41.857034 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59063-3492-4c63-b549-97288e1a5341 None None] REQUEST_METHOD: `POST` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:41.857325 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59063-3492-4c63-b549-97288e1a5341 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:41.857761 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59063-3492-4c63-b549-97288e1a5341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:41.948632 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.auth.core [None req-61e59063-3492-4c63-b549-97288e1a5341 None None] MFA Rules not processed for user `c5ba9e7780f5437caac949931ca55b80`. Rule list: `[]` (Enabled: `True`). {{(pid=207770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:41.949938 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-61e59063-3492-4c63-b549-97288e1a5341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:42.078201 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 681/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 08:32:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:42.086933 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d3dfe-fc12-498d-835b-4673d932697d None None] Authenticating user token {{(pid=207769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:42.088525 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-d30d3dfe-fc12-498d-835b-4673d932697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:42.264075 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d3dfe-fc12-498d-835b-4673d932697d None None] Validating token access rules against request {{(pid=207769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:42.288552 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:42.407764 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d87867-2c53-4fea-b64e-4f64bb14e9bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ba9e7780f5437caac949931ca55b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:42.408407 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87867-2c53-4fea-b64e-4f64bb14e9bc service nova] REQUEST_METHOD: `GET` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:42.408686 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87867-2c53-4fea-b64e-4f64bb14e9bc service nova] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:42.408959 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87867-2c53-4fea-b64e-4f64bb14e9bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:42.410656 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-c5d87867-2c53-4fea-b64e-4f64bb14e9bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:42.512703 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d87867-2c53-4fea-b64e-4f64bb14e9bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:42.514627 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d87867-2c53-4fea-b64e-4f64bb14e9bc service nova] RBAC: Authorization granted {{(pid=207769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:42.538623 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 687/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:43.366579 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883ec6-fcfb-4de5-8283-d7133e702974 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:43.367190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883ec6-fcfb-4de5-8283-d7133e702974 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:43.367190 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883ec6-fcfb-4de5-8283-d7133e702974 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:43.368702 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 682/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:43.374218 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459eb4be-759e-45dd-a4f8-adddfea03afc None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:43.374711 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459eb4be-759e-45dd-a4f8-adddfea03afc None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:43.374821 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459eb4be-759e-45dd-a4f8-adddfea03afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:43.410076 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:43.451670 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-459eb4be-759e-45dd-a4f8-adddfea03afc None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:43.453735 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-459eb4be-759e-45dd-a4f8-adddfea03afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:43.577203 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 688/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:32:44.667197 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61921a1b-cbab-453c-b22a-5acfd8f0bcf4 None None] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:44.667784 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61921a1b-cbab-453c-b22a-5acfd8f0bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:44.667973 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61921a1b-cbab-453c-b22a-5acfd8f0bcf4 None None] PATH_INFO: `/` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:44.669801 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 683/1371] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:32:44.675737 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb684a4-161b-448a-98ee-456a994e53cd None None] REQUEST_METHOD: `POST` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:44.676313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb684a4-161b-448a-98ee-456a994e53cd None None] SCRIPT_NAME: `/identity` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:44.676313 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb684a4-161b-448a-98ee-456a994e53cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=207769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:44.756479 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.auth.core [None req-cdb684a4-161b-448a-98ee-456a994e53cd None None] MFA Rules not processed for user `5a8bf4bf753c41b2a872093710161103`. Rule list: `[]` (Enabled: `True`). {{(pid=207769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:32:44.757714 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG keystone.common.fernet_utils [None req-cdb684a4-161b-448a-98ee-456a994e53cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:44.825346 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=100 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:44.874010 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: [pid: 207769|app: 0|req: 689/1372] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:32:44.918582 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61921a1b-cbab-453c-b22a-5acfd8f0bcf4 None None] Authenticating user token {{(pid=207770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:32:44.920072 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-61921a1b-cbab-453c-b22a-5acfd8f0bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:44.973639 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:45.072128 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61921a1b-cbab-453c-b22a-5acfd8f0bcf4 None None] Validating token access rules against request {{(pid=207770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:32:45.192503 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63235f-94d8-4b8d-a4a3-d115cb4c477f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ba9e7780f5437caac949931ca55b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd354364f219049a7ae56aa2de34993d8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=207770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:32:45.193239 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63235f-94d8-4b8d-a4a3-d115cb4c477f service nova] REQUEST_METHOD: `GET` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:32:45.193617 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63235f-94d8-4b8d-a4a3-d115cb4c477f service nova] SCRIPT_NAME: `/identity` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:32:45.194013 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63235f-94d8-4b8d-a4a3-d115cb4c477f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=207770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:32:45.196201 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.fernet_utils [None req-dc63235f-94d8-4b8d-a4a3-d115cb4c477f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=207770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:32:45.206637 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=100 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:45.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63235f-94d8-4b8d-a4a3-d115cb4c477f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:32:45.409897 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63235f-94d8-4b8d-a4a3-d115cb4c477f service nova] RBAC: Authorization granted {{(pid=207770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:32:45.471326 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: [pid: 207770|app: 0|req: 684/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:32:54.872224 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: DEBUG dbcounter [-] [207769] Writing DB stats keystone:SELECT=19 {{(pid=207769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:32:55.448207 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: DEBUG dbcounter [-] [207770] Writing DB stats keystone:SELECT=41 {{(pid=207770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:33:05.861917 dv-cinder-851351-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Jul 28 08:33:05.862810 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: Thu Jul 28 08:33:05 2022 - graceful shutdown triggered... Jul 28 08:33:05.863349 dv-cinder-851351-1-iscsi devstack@keystone.service[207770]: Gracefully killing worker 2 (pid: 207770)... Jul 28 08:33:05.863801 dv-cinder-851351-1-iscsi devstack@keystone.service[207769]: Gracefully killing worker 1 (pid: 207769)... Jul 28 08:33:06.863919 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: worker 1 buried after 1 seconds Jul 28 08:33:06.863919 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: worker 2 buried after 1 seconds Jul 28 08:33:06.863919 dv-cinder-851351-1-iscsi devstack@keystone.service[207757]: goodbye to uWSGI. Jul 28 08:33:06.865604 dv-cinder-851351-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Jul 28 08:33:06.866229 dv-cinder-851351-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. Jul 28 08:37:54.674684 dv-cinder-851351-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 08:37:54.688910 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 08:37:54.690009 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 08:37:54 2022] *** Jul 28 08:37:54.690009 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 08:37:54.690009 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 08:37:54.690009 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: nodename: dv-cinder-851351-1-iscsi Jul 28 08:37:54.690009 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: machine: x86_64 Jul 28 08:37:54.690009 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: clock source: unix Jul 28 08:37:54.690009 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: pcre jit disabled Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: detected number of CPU cores: 4 Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: current working directory: / Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: detected binary path: /usr/bin/uwsgi-core Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: your processes number limit is 47680 Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: your memory page size is 4096 bytes Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: detected max file descriptor number: 2048 Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: lock engine: pthread robust mutexes Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: thunder lock: enabled Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 08:37:54.691039 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 08:37:54.730639 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: Python main interpreter initialized at 0x559cbefcf9c0 Jul 28 08:37:54.730639 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: python threads support enabled Jul 28 08:37:54.730639 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: your server socket listen backlog is limited to 100 connections Jul 28 08:37:54.730639 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: your mercy for graceful operations on workers is 90 seconds Jul 28 08:37:54.731136 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 08:37:54.731136 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: *** Operational MODE: preforking *** Jul 28 08:37:54.731379 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: *** uWSGI is running in multiple interpreter mode *** Jul 28 08:37:54.731379 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: spawned uWSGI master process (pid: 277701) Jul 28 08:37:54.731647 dv-cinder-851351-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 08:37:54.732260 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: spawned uWSGI worker 1 (pid: 277712, cores: 1) Jul 28 08:37:54.732794 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: spawned uWSGI worker 2 (pid: 277713, cores: 1) Jul 28 08:37:54.733106 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 08:38:00.328032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:38:00.328032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: from cryptography.utils import int_from_bytes Jul 28 08:38:00.329168 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:38:00.329168 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: from cryptography.utils import int_from_bytes Jul 28 08:38:00.329443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:38:00.329443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: from cryptography.utils import int_from_bytes Jul 28 08:38:00.330045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:38:00.330045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: from cryptography.utils import int_from_bytes Jul 28 08:38:00.379460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:38:00.379769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:38:00.379922 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] command line args: [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:38:00.380073 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:38:00.380073 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:38:00.380365 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:38:00.380502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] command line args: [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:38:00.380649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ================================================================================ {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:38:00.380649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] admin_token = **** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.380928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:38:00.380928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ================================================================================ {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:38:00.381206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] config_dir = [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.381206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.381475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] admin_token = **** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.381890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] config_source = [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.382097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] config_dir = [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.382336 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] debug = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.382533 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.382723 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.384289 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] config_source = [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.384289 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] debug = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.384605 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.384767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.384942 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] insecure_debug = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.385092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.385382 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.385555 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] insecure_debug = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.385723 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.385876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.386031 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] list_limit = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.386239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.386389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] log_config_append = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.386591 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] list_limit = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.386746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.386911 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] log_config_append = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.387063 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] log_dir = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.387220 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.387374 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] log_file = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.387528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] log_dir = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.387694 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.387843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] log_file = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.387999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.388164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.388331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.388558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.388726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.388876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.389031 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.389213 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.389406 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.389562 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.389701 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.389850 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.389994 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.390157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.390311 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.390457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.390598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.390750 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.390884 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.391031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.391031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.391294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.391440 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.391579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.391721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.391859 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] notification_format = cadf {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.392005 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] notification_format = cadf {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.392145 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.392301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.392301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.392569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.392716 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] publish_errors = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.392854 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] publish_errors = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.392992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.393143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.393274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.393414 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.393552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.393690 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.393827 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.393991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.394158 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.394302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.394446 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] standard_threads = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.394584 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] standard_threads = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.394739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] strict_password_check = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.394884 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] strict_password_check = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.395046 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.395185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.395331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] use_eventlog = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.395470 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] use_eventlog = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.395623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] use_journal = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.395754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] use_journal = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.395897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] use_json = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.396038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] use_json = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.396182 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] use_stderr = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.396326 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] use_stderr = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.396466 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] use_syslog = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.396602 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] use_syslog = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.396740 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] watch_log_file = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.396882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] watch_log_file = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:38:00.397021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.397164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.397302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.397439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.397595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.397868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.398033 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.398211 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.398364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.398504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.398644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.398789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.398928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.399068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.399212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.399344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.399493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.399639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.399776 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.399915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.400063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.400202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.400341 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.400486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.400624 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.400773 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.400913 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.401055 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.401194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.401335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.401474 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.401625 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.401763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.401906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.402045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.402270 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.402404 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.402549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.402684 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.402836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.402971 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.403123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.403265 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.403407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.403550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.403686 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.403831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.404025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.404162 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.404306 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.404448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.404590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.404734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.404884 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.405034 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.405167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.405310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.405452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.405604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.405746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.405892 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.406032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.406219 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.406360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.406515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.406654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.406795 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.406948 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.407090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.407232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.407456 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.407600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.407741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.407883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.408020 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.408170 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.408325 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.408461 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.408602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.408743 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.408884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.408884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.409153 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.409295 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.409435 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.409570 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.409570 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.409838 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.409985 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.410150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.410290 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.410577 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.410722 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.410880 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.411040 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.411040 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.411326 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.411474 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.411618 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.411774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.411774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.412057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.412193 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.412341 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.412485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.412633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.412778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.412932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.413077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.413212 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.413370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.413535 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.413693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.413840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.413986 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.414194 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.414343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.414489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.414629 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.414770 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.414919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.415060 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.415202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.415356 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.415498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.415640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.415796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.415938 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.416089 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.416230 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.416373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.416533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.416669 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.416816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.416816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.417098 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.417232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.417375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.417517 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.417661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.417979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.418148 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.418316 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.418453 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.enabled = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.418593 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.418741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.418894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.419039 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.419175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.419337 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.419472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.419642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.419788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.419934 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.420072 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.enabled = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.420221 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.420366 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.420520 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.420655 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.420807 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.420943 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.421092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.421237 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.421399 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.421542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.421689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.421830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.421978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.422149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.422314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.422450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.422607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.422761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.422905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.423049 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.423196 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.423331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.423478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.423615 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.423759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.423903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.424059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.424205 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.424348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.424484 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.424641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.424777 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.424927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.425071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.425216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.425364 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.425509 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.425663 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.425663 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.425946 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.426090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.426268 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.426411 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.426555 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.426700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.426834 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.426978 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.427125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] auth.external = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.427266 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.427408 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] auth.mapped = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.427553 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.427695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.427839 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.427973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.428122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.428264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] auth.password = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.428419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] auth.external = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.428574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] auth.token = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.428726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] auth.mapped = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.428902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.428902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] catalog.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.429214 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.429356 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.429507 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.429659 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.429796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] auth.password = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.429796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] auth.token = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.430059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.430238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.430391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.430553 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] catalog.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.430710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.430860 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.431006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] credential.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.431152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.431306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] credential.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.431451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.431598 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.431747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.431901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.432047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.432275 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.432421 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] credential.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.432568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.432712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] credential.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.432857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.433001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.433154 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.433310 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.433454 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.433599 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.433744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.433896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.434048 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.434265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.434407 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.434550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.434693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.434833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.434979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.435121 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.435257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.435389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.435538 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.435685 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.435837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] federation.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.435978 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.436120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.436261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.436403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] federation.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.436544 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.436694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.436835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.436969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.437104 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] federation.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.437247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.437381 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.437530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.437665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] federation.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.437805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.437949 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.438091 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.438278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.438422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.438565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.438701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.438846 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.439004 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.439156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.439293 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.439429 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.439576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.439714 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.439856 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.440008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.440152 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.440296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.440439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.440582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.440736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.440971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.441443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.441595 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.441734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.441875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.442030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.442275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.442419 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.442562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.442705 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.442847 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.442990 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.443141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.443296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.443438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.443580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.443725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.443860 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.444019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.444617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.444775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.444924 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.445082 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.445229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.445376 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.445519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.445653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.445796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.445945 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.446085 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.446264 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.446410 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.446549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.446693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.446836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.446979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.446979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.447260 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.447260 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.447526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.447668 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.447810 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.447959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.448102 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.448247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.448377 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.448513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.448686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.448845 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.448990 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.449127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.449275 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.449437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.449590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.449727 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.449866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.450017 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.450182 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.450319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.450479 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.450615 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.450757 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.password = **** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.450897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.451040 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.451175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.451323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.451478 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.451626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.451769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.451911 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.452053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.password = **** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.452203 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.452337 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.452480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.452615 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.452752 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.452897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.453042 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.453184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.453326 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.453460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.453603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.453766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.453926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.454070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.454245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.454393 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.454550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.454685 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.454847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.454983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.455122 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.455258 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.455406 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.455550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.455700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.455837 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.455984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.456120 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.456267 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.456407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.456571 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.456716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.456859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.457003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.457140 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.457286 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.457433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.457567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.457734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.457888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.458031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.458200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.458352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.458497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.458642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.458779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.458927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.459069 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.459217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.459352 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.459493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.459646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.459782 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.459924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.460097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.460296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.460433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.460572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.460715 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.460858 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.461080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.461080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.461352 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.461495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.461639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.461639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.461944 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.462123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.462288 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.462433 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.462574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.462726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.462862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.463007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.463151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] policy.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.463293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.463435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.463598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.463742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.463885 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.464022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.464165 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.464300 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] receipt.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.464462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] policy.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.464652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.464796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.464934 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.465090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.465247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.465400 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.465536 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.465691 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] receipt.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.465841 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.465978 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.466164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] resource.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.466323 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.466468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.466606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.466741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] resource.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.466882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.467035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.467183 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.467326 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.467461 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] resource.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.467603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.467745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.467887 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] revoke.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.468038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] resource.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.468175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.468318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.468466 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.468610 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.468747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] role.cache_time = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.468898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.469034 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] role.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.469169 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] revoke.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.469304 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] role.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.469456 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.469606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] role.list_limit = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.469757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.469893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.470037 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] role.cache_time = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.470248 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.470408 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] role.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.470545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.470694 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] role.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.470840 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.470987 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] role.list_limit = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.471123 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.471271 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.471415 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.471563 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.471846 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.471982 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.472119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.472267 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.472423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.472567 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.472705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.472841 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.472841 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.473109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.473264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.473408 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.473408 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.473673 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.473809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.473953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.474135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.474315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.474460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.474605 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.474749 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.474897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.474897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.475165 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.475310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.475458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.475616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.475807 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.475964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.476107 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.476261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.476414 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.476563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.476711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.476848 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.476992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.477148 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.477283 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.477431 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.477579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.477731 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.477867 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.478023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.478225 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.478377 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.478579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.478718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.478860 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.479004 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] token.cache_time = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.479139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.479297 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] token.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.479447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.479590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.479728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.479870 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] token.provider = fernet {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.480005 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.480145 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.480302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.480442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.480589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] token.cache_time = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.480733 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.480875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] token.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.481022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.481166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.481309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.481463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] token.provider = fernet {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.481598 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.481741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.481891 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] trust.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.482036 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.482190 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.482352 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.482488 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.482630 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.482764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.482914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.483057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.483191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.483334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.483468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] trust.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.483611 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.483755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.483898 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.484041 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.484187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.484333 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.484492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.connection = **** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.484628 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.484778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.484921 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.485079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.485225 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.485425 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.485581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.485718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.485857 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.486009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.486249 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.connection = **** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.486388 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.486538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.486775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.486918 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.487061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.487196 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.487331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.487465 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.487611 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.487763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.487897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.488031 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.488173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.488317 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.488470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.488605 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.488748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.488883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.489017 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.489171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.489320 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.489467 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.489612 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.489747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.489881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.490016 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.490194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:38:00.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:38:00.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:38:00.496396 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.496632 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.496863 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.497220 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.497449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.505581 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.506043 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.538866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.539181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.540286 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.540526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.592709 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.592962 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.643959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.644291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.644845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.645059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.710587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.710998 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.710998 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.711380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.711580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.711764 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.711764 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.712112 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.712112 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.712112 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.712607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.712607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.738509 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.738880 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.739078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.739276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.739501 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.739688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.739883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.739883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.740206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.740373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.740373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.740717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.740717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.740717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.751527 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.753684 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.764512 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.764807 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.765018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.765196 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.765397 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.765607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.765772 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.766520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.766748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.766954 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.767148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.767348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.767550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.767758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:38:00.782888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.783264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:38:00.783516 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.783731 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.783922 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:38:00.784105 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:38:00.784308 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.784493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.784692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.784894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:38:00.785121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.785121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.785607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.785607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.785607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.786219 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.794560 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.794829 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.795066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.795379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.795821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.796116 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.796351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.796642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.802369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.803890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.807531 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.807817 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.808116 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.808377 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.808688 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.808916 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.809204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.809456 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.809641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.809822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.810001 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.810221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.810394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.810581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.810769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.810948 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.810948 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.810948 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.811458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.811458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.811786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.811972 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.833992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.835449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.835678 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.835892 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.836975 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.837211 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.840495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.840741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.841051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.841857 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.842154 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.842448 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.847919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.848164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.849289 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.849665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.853007 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.853495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.853675 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.853874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.854196 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.854433 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.854622 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.854820 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.854996 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.855187 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.855368 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.855538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.855538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.855904 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.856074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.856359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.871253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.871649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.872592 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.872834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.875549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.876850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.878755 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.879025 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.880129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.880404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.883570 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.884940 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.886812 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.888155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.889960 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.890313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.890617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.890877 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.891182 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.891404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.891589 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.891802 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.891989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.892247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.892469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.892799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.907591 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.907919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.908110 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.908401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.908900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.909174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.909468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.909705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.915611 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.916888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.917113 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.917431 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.918459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.918753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.920310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.920547 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.920884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.921121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.921438 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.921659 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.921844 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.922141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.922423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.922729 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.932977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.933243 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.933542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.933789 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.934095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.934344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.934531 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.934720 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.934912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.935123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.935123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.935478 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.935648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.935860 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.950743 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.951932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.953781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.954986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.956799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.957930 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.959485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.960689 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.961032 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.961335 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.961656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.962309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.962555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.962863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.968215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.969300 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.969841 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.971088 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.972802 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.973068 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.973390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.973685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.974332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.974673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.975004 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.975305 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.981877 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.982178 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.982472 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.983563 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.983837 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.984142 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.989371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=277713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.989637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.989933 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.990206 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.990533 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.990766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.991057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=277712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:38:00.991279 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.991499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.991672 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.991848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.992033 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.992216 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.992385 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.992568 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.992734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.992918 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.993095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.993453 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.993629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.993812 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=277713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.993979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.993979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.993979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.994542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.994542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:00.994911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=277712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:38:01.026810 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559cbefcf9c0 pid: 277713 (default app) Jul 28 08:38:01.029381 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559cbefcf9c0 pid: 277712 (default app) Jul 28 08:38:01.030477 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7c43e-1506-47aa-b033-d05f4365de3a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:01.030770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7c43e-1506-47aa-b033-d05f4365de3a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:01.031055 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7c43e-1506-47aa-b033-d05f4365de3a None None] PATH_INFO: `/v3` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:01.035470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jul 28 08:38:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:08.809718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb307ac-c8df-47e2-863a-bd9951e9deff None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:08.810439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb307ac-c8df-47e2-863a-bd9951e9deff None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:08.810439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb307ac-c8df-47e2-863a-bd9951e9deff None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:08.815643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:08.823236 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:08.823780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:08.823780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:08.894618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: INFO dbcounter [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] Registered counter for database keystone Jul 28 08:38:08.917190 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG oslo_db.sqlalchemy.engines [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=277713) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:38:08.924643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writer thread running {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:38:09.114268 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG passlib.handlers.bcrypt [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=277713) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:38:09.114604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG passlib.handlers.bcrypt [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=277713) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:38:09.170348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:09.172037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:09.306240 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:38:09.330270 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb307ac-c8df-47e2-863a-bd9951e9deff None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:09.402535 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: INFO dbcounter [None req-bbb307ac-c8df-47e2-863a-bd9951e9deff None None] Registered counter for database keystone Jul 28 08:38:09.422457 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG oslo_db.sqlalchemy.engines [None req-bbb307ac-c8df-47e2-863a-bd9951e9deff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=277712) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:38:09.430234 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writer thread running {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:38:09.553212 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb307ac-c8df-47e2-863a-bd9951e9deff None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:09.559590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383509e6-de0c-4d25-93f4-b4948767e32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c33208ac31c49d19dce8ea7c7b40c7a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:09.560275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383509e6-de0c-4d25-93f4-b4948767e32b admin admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:09.560557 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383509e6-de0c-4d25-93f4-b4948767e32b admin admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:09.560828 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383509e6-de0c-4d25-93f4-b4948767e32b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:09.570515 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383509e6-de0c-4d25-93f4-b4948767e32b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:09.624022 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: WARNING keystone.common.rbac_enforcer.enforcer [None req-383509e6-de0c-4d25-93f4-b4948767e32b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:38:09.624371 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383509e6-de0c-4d25-93f4-b4948767e32b admin admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:09.635362 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:38:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:38:09.641544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:09.656531 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fc4da-7400-4d8a-8345-252ff2a00eeb None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:09.661624 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740d014-3e90-4db8-b056-14bb3b9703ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c33208ac31c49d19dce8ea7c7b40c7a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:09.662804 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740d014-3e90-4db8-b056-14bb3b9703ce admin admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:09.663137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740d014-3e90-4db8-b056-14bb3b9703ce admin admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:09.663414 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740d014-3e90-4db8-b056-14bb3b9703ce admin admin] PATH_INFO: `/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:09.664984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740d014-3e90-4db8-b056-14bb3b9703ce admin admin] RBAC: Authorizing `identity:get_project(project_id=0c33208ac31c49d19dce8ea7c7b40c7a)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:09.715076 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b740d014-3e90-4db8-b056-14bb3b9703ce admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:38:09.715422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740d014-3e90-4db8-b056-14bb3b9703ce admin admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:09.720282 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jul 28 08:38:09 2022] GET /identity/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:11.713286 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d126-cbeb-4e13-b7c2-5a5470a557a8 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:11.713878 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d126-cbeb-4e13-b7c2-5a5470a557a8 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:11.714061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d126-cbeb-4e13-b7c2-5a5470a557a8 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:11.719424 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:11.724925 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e3f3-cf69-49cb-9eb9-66b01eef039a None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:11.725425 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e3f3-cf69-49cb-9eb9-66b01eef039a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:11.725532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e3f3-cf69-49cb-9eb9-66b01eef039a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:11.747749 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-bbd9e3f3-cf69-49cb-9eb9-66b01eef039a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:11.749349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-bbd9e3f3-cf69-49cb-9eb9-66b01eef039a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:11.758709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:38:11.781452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62d126-cbeb-4e13-b7c2-5a5470a557a8 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:11.794740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62d126-cbeb-4e13-b7c2-5a5470a557a8 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:11.799288 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de1ca7-b64e-416a-99c1-791488a3d2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c33208ac31c49d19dce8ea7c7b40c7a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:11.799959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1ca7-b64e-416a-99c1-791488a3d2bc admin admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:11.800235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1ca7-b64e-416a-99c1-791488a3d2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:11.800515 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1ca7-b64e-416a-99c1-791488a3d2bc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:11.809316 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de1ca7-b64e-416a-99c1-791488a3d2bc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:11.811165 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de1ca7-b64e-416a-99c1-791488a3d2bc admin admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:11.821040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:38:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:38:11.827294 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9e3f3-cf69-49cb-9eb9-66b01eef039a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:11.837423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9e3f3-cf69-49cb-9eb9-66b01eef039a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:11.843381 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6315-4571-4561-adb0-5fb9d8715dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c33208ac31c49d19dce8ea7c7b40c7a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:11.844044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6315-4571-4561-adb0-5fb9d8715dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:11.844330 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6315-4571-4561-adb0-5fb9d8715dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:11.844604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6315-4571-4561-adb0-5fb9d8715dd1 admin admin] PATH_INFO: `/v3/users/b56a4c63e0f64f7182c22666f3fab6a1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:11.846199 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6315-4571-4561-adb0-5fb9d8715dd1 admin admin] RBAC: Authorizing `identity:get_user(user_id=b56a4c63e0f64f7182c22666f3fab6a1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:11.848097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6315-4571-4561-adb0-5fb9d8715dd1 admin admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:11.849704 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jul 28 08:38:11 2022] GET /identity/v3/users/b56a4c63e0f64f7182c22666f3fab6a1 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:13.932753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3566c2-780d-4e55-9ef1-327cf99e240e None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:13.933353 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3566c2-780d-4e55-9ef1-327cf99e240e None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:13.933353 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3566c2-780d-4e55-9ef1-327cf99e240e None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:13.934692 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:13.940355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0f346-a8a8-4f68-9f7b-2870044d27dc None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:13.940815 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0f346-a8a8-4f68-9f7b-2870044d27dc None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:13.940922 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0f346-a8a8-4f68-9f7b-2870044d27dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:13.961488 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-65c0f346-a8a8-4f68-9f7b-2870044d27dc None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:13.962754 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-65c0f346-a8a8-4f68-9f7b-2870044d27dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:13.970380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32652433-48c5-4557-9503-00503ec34b9d None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:13.971140 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32652433-48c5-4557-9503-00503ec34b9d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:13.971430 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32652433-48c5-4557-9503-00503ec34b9d None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:13.973378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:13.978419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:13.978709 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:13.978993 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:14.022125 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG passlib.handlers.bcrypt [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=277712) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:38:14.022680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG passlib.handlers.bcrypt [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=277712) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:38:14.051785 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:14.071869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:14.073548 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:14.075427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0f346-a8a8-4f68-9f7b-2870044d27dc None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:14.101081 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:14.179574 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:14.188121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0f346-a8a8-4f68-9f7b-2870044d27dc None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:14.202233 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:14.224725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3dcfc-b690-4670-ae42-1cc9a7313837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:14.225429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3dcfc-b690-4670-ae42-1cc9a7313837 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:14.225743 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3dcfc-b690-4670-ae42-1cc9a7313837 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:14.226024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3dcfc-b690-4670-ae42-1cc9a7313837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:14.238862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3dcfc-b690-4670-ae42-1cc9a7313837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a1a3dcfc-b690-4670-ae42-1cc9a7313837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:14.248260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:14.249702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:14.251207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:38:14.256115 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3dcfc-b690-4670-ae42-1cc9a7313837 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:14.315357 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec167-a71b-4505-9bf6-f0ffbba75cb2 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:14.318335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:14.341206 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:14.358297 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38cd4-9d3b-4c30-a24e-8736144a4917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:14.359210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38cd4-9d3b-4c30-a24e-8736144a4917 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:14.359494 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38cd4-9d3b-4c30-a24e-8736144a4917 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:14.359771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38cd4-9d3b-4c30-a24e-8736144a4917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:14.360758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3dcfc-b690-4670-ae42-1cc9a7313837 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:14.396746 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:14.397507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:14.397701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:14.397972 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:14.398745 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:14.399046 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:14.400486 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:14.412837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:14.419700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:14.436585 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af38cd4-9d3b-4c30-a24e-8736144a4917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:14.438415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af38cd4-9d3b-4c30-a24e-8736144a4917 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:14.535629 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997d820-b937-44ab-83b3-0d5f64b974a5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:14.547441 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:14.553929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38cd4-9d3b-4c30-a24e-8736144a4917 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:14.557375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:14.673208 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38cd4-9d3b-4c30-a24e-8736144a4917 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:14.691784 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06a74d-d013-45ab-a12e-15c9aeb576f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:14.692468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a74d-d013-45ab-a12e-15c9aeb576f1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:14.692751 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a74d-d013-45ab-a12e-15c9aeb576f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:14.693028 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a74d-d013-45ab-a12e-15c9aeb576f1 None admin] PATH_INFO: `/v3/users/b56a4c63e0f64f7182c22666f3fab6a1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:14.694652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06a74d-d013-45ab-a12e-15c9aeb576f1 None admin] RBAC: Authorizing `identity:get_user(user_id=b56a4c63e0f64f7182c22666f3fab6a1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:14.696164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06a74d-d013-45ab-a12e-15c9aeb576f1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:14.697796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:14 2022] GET /identity/v3/users/b56a4c63e0f64f7182c22666f3fab6a1 => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:14.703500 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06a74d-d013-45ab-a12e-15c9aeb576f1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:14.709908 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:14.710840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:14.711130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:14.711411 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:14.719904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:14.731794 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:14.733298 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:38:14.752309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:14.753666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:38:14.755221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:38:14.755221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:38:14.755221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:14.755221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:14.759572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:14.811577 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06a74d-d013-45ab-a12e-15c9aeb576f1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:14.825172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:14.855873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de16d8a-3683-4c8f-9ed3-883d09410657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:14.856574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de16d8a-3683-4c8f-9ed3-883d09410657 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:14.856865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de16d8a-3683-4c8f-9ed3-883d09410657 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:14.857279 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de16d8a-3683-4c8f-9ed3-883d09410657 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:14.859182 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de16d8a-3683-4c8f-9ed3-883d09410657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:14.862599 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de16d8a-3683-4c8f-9ed3-883d09410657 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:14.866031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:14.872339 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de16d8a-3683-4c8f-9ed3-883d09410657 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:14.883767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728a6ac-6da1-4ef4-b37f-9766569bfa6b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:14.920691 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979ff52-7536-490e-a393-d39d9b58525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:14.921354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979ff52-7536-490e-a393-d39d9b58525f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:14.921637 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979ff52-7536-490e-a393-d39d9b58525f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:14.921910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979ff52-7536-490e-a393-d39d9b58525f None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:14.922682 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979ff52-7536-490e-a393-d39d9b58525f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:14.922992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979ff52-7536-490e-a393-d39d9b58525f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:14.924453 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979ff52-7536-490e-a393-d39d9b58525f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:14.944084 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:14.969697 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:14.994559 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de16d8a-3683-4c8f-9ed3-883d09410657 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:15.036551 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866173-2114-4ecf-8516-27c8982d3730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:15.037199 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866173-2114-4ecf-8516-27c8982d3730 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:15.037485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866173-2114-4ecf-8516-27c8982d3730 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:15.037766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866173-2114-4ecf-8516-27c8982d3730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:15.038780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866173-2114-4ecf-8516-27c8982d3730 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.domain.id=default, user.id=b56a4c63e0f64f7182c22666f3fab6a1) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:15.039069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866173-2114-4ecf-8516-27c8982d3730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:15.040657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866173-2114-4ecf-8516-27c8982d3730 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:15.048874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:38:14 2022] GET /identity/v3/role_assignments?user.id=b56a4c63e0f64f7182c22666f3fab6a1&scope.domain.id=default&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 233 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:17.244562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570482f-1405-44cc-8522-50f3432d8d8c None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:17.245386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570482f-1405-44cc-8522-50f3432d8d8c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:17.245386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570482f-1405-44cc-8522-50f3432d8d8c None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:17.246833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:38:17.252654 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fb3bc-a5c7-41a8-ba24-6786ee8a00fe None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:17.253327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fb3bc-a5c7-41a8-ba24-6786ee8a00fe None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:17.253327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fb3bc-a5c7-41a8-ba24-6786ee8a00fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:17.284820 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-1f5fb3bc-a5c7-41a8-ba24-6786ee8a00fe None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:17.286304 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-1f5fb3bc-a5c7-41a8-ba24-6786ee8a00fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:17.350524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e309a-d362-4a6b-86fb-53bd35320aa6 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:17.350966 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e309a-d362-4a6b-86fb-53bd35320aa6 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:17.351232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e309a-d362-4a6b-86fb-53bd35320aa6 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:17.352698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:38:17.362971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d78f8-168c-4c7f-8346-4c56e4cf46c4 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:17.363269 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d78f8-168c-4c7f-8346-4c56e4cf46c4 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:17.363546 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d78f8-168c-4c7f-8346-4c56e4cf46c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:17.377507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:17.399297 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-e79d78f8-168c-4c7f-8346-4c56e4cf46c4 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:17.400558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-e79d78f8-168c-4c7f-8346-4c56e4cf46c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:17.420727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fb3bc-a5c7-41a8-ba24-6786ee8a00fe None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:17.539651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 13/25] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:17.590164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:17.590380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d78f8-168c-4c7f-8346-4c56e4cf46c4 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:17.642196 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fb3bc-a5c7-41a8-ba24-6786ee8a00fe None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:17.679131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc299f-3728-4f87-a41a-9eae487506ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:17.680253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc299f-3728-4f87-a41a-9eae487506ae None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:17.680799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc299f-3728-4f87-a41a-9eae487506ae None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:17.681307 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc299f-3728-4f87-a41a-9eae487506ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:17.695023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc299f-3728-4f87-a41a-9eae487506ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-1fcc299f-3728-4f87-a41a-9eae487506ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:17.700691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:17.701855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:17.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:17.705526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 13/26] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:38:17.707637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba49ea-1192-4135-8739-eaf97872b324 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:17.708166 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba49ea-1192-4135-8739-eaf97872b324 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:17.708665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba49ea-1192-4135-8739-eaf97872b324 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:17.710291 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 14/27] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:17.718352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70e1a9-657e-4cfb-b9e6-9725af47fede None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:17.718352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70e1a9-657e-4cfb-b9e6-9725af47fede None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:17.718352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70e1a9-657e-4cfb-b9e6-9725af47fede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:17.719182 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:17.752898 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-1a70e1a9-657e-4cfb-b9e6-9725af47fede None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:17.754166 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-1a70e1a9-657e-4cfb-b9e6-9725af47fede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:17.801379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d78f8-168c-4c7f-8346-4c56e4cf46c4 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:17.842610 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:17.843296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:17.843583 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:17.843862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:17.905927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 15/28] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:17.906929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70e1a9-657e-4cfb-b9e6-9725af47fede None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:18.022183 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:18.023726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:18.041103 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:38:18.049394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:18.050578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:38:18.051657 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:38:18.051657 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:38:18.051657 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:18.051657 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 14/29] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:18.051657 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:18.107117 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70e1a9-657e-4cfb-b9e6-9725af47fede None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:18.152322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:18.153101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.153390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.153674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.154475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:18.154796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:18.156371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:18.165573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 16/30] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:18.166465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:18.213588 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:18.222395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11a55c-81e6-43ba-bca4-13f547257dce None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:18.282713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d634d-f4df-48d7-a40d-02cd35a8fc72 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:18.300425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c91fbd-4fbf-4528-b868-0b2370715b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:18.302151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91fbd-4fbf-4528-b868-0b2370715b4c None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.302611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91fbd-4fbf-4528-b868-0b2370715b4c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.303258 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91fbd-4fbf-4528-b868-0b2370715b4c None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.304740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c91fbd-4fbf-4528-b868-0b2370715b4c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:18.306396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c91fbd-4fbf-4528-b868-0b2370715b4c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:18.321679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:18.329632 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ce056-c963-488d-ab2f-6e91dc91c222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:18.330518 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ce056-c963-488d-ab2f-6e91dc91c222 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.330872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ce056-c963-488d-ab2f-6e91dc91c222 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.331229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ce056-c963-488d-ab2f-6e91dc91c222 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.332410 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ce056-c963-488d-ab2f-6e91dc91c222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:18.332822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ce056-c963-488d-ab2f-6e91dc91c222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:18.334943 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ce056-c963-488d-ab2f-6e91dc91c222 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:18.360053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 17/31] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:18.360053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2db1-098c-4cef-b36c-d349b7774447 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.360053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2db1-098c-4cef-b36c-d349b7774447 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.360053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2db1-098c-4cef-b36c-d349b7774447 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.360053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 18/32] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:18.360053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d2db1-098c-4cef-b36c-d349b7774447 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:18.465214 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 15/33] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:38:18 2022] POST /identity/v3/roles => generated 219 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:18.466418 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c91fbd-4fbf-4528-b868-0b2370715b4c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:18.483360 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d2db1-098c-4cef-b36c-d349b7774447 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:18.523118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a4876-64a3-4941-9a17-dd838d6f8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:18.524156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a4876-64a3-4941-9a17-dd838d6f8deb None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.524606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a4876-64a3-4941-9a17-dd838d6f8deb None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.525048 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a4876-64a3-4941-9a17-dd838d6f8deb None admin] PATH_INFO: `/v3/users/b56a4c63e0f64f7182c22666f3fab6a1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.526629 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a4876-64a3-4941-9a17-dd838d6f8deb None admin] RBAC: Authorizing `identity:get_user(user_id=b56a4c63e0f64f7182c22666f3fab6a1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:18.528384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a4876-64a3-4941-9a17-dd838d6f8deb None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:18.530492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 19/34] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:18 2022] GET /identity/v3/users/b56a4c63e0f64f7182c22666f3fab6a1 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:18.532219 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badeaa9-f994-4881-a00b-a5862330f5c3 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.532698 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badeaa9-f994-4881-a00b-a5862330f5c3 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.533168 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badeaa9-f994-4881-a00b-a5862330f5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.562173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-3badeaa9-f994-4881-a00b-a5862330f5c3 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:18.563846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-3badeaa9-f994-4881-a00b-a5862330f5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:18.640790 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c91fbd-4fbf-4528-b868-0b2370715b4c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:18.681971 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:18.708755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca03544-4785-417c-be57-ac2ca7f4a3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:18.709918 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca03544-4785-417c-be57-ac2ca7f4a3f1 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.710485 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca03544-4785-417c-be57-ac2ca7f4a3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.711009 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca03544-4785-417c-be57-ac2ca7f4a3f1 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.712279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca03544-4785-417c-be57-ac2ca7f4a3f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:18.714309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca03544-4785-417c-be57-ac2ca7f4a3f1 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:18.718732 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 20/35] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:18.718732 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badeaa9-f994-4881-a00b-a5862330f5c3 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:18.761315 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 16/36] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:38:18 2022] POST /identity/v3/projects => generated 301 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:18.767090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca03544-4785-417c-be57-ac2ca7f4a3f1 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:18.796468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:18.881275 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badeaa9-f994-4881-a00b-a5862330f5c3 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:18.928924 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792765b-94fb-440e-bc15-854e7b2940e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:18.930592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792765b-94fb-440e-bc15-854e7b2940e3 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.931117 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792765b-94fb-440e-bc15-854e7b2940e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.931619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792765b-94fb-440e-bc15-854e7b2940e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.933212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792765b-94fb-440e-bc15-854e7b2940e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:18.935057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792765b-94fb-440e-bc15-854e7b2940e3 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:18.937964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 21/37] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:18.939961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4512-14d5-40b8-8ad4-2f3d91716a5f None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:18.940479 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4512-14d5-40b8-8ad4-2f3d91716a5f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:18.940991 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4512-14d5-40b8-8ad4-2f3d91716a5f None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:18.942872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 22/38] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:18.945447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca4512-14d5-40b8-8ad4-2f3d91716a5f None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:19.013399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca03544-4785-417c-be57-ac2ca7f4a3f1 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:19.015651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:19.057614 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb50bf-183f-466c-b370-656380bf2190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:19.058318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb50bf-183f-466c-b370-656380bf2190 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:19.058605 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb50bf-183f-466c-b370-656380bf2190 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:19.058883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb50bf-183f-466c-b370-656380bf2190 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:19.059623 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb50bf-183f-466c-b370-656380bf2190 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:19.061027 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb50bf-183f-466c-b370-656380bf2190 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:19.077188 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca4512-14d5-40b8-8ad4-2f3d91716a5f None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:19.080220 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 17/39] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:38:18 2022] POST /identity/v3/roles => generated 225 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:19.081653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621b51-03c9-4518-8b87-198a75791394 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:19.081950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621b51-03c9-4518-8b87-198a75791394 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:19.082309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621b51-03c9-4518-8b87-198a75791394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:19.102874 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-fa621b51-03c9-4518-8b87-198a75791394 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:19.104531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-fa621b51-03c9-4518-8b87-198a75791394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:19.120943 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5e22-d66a-4bd3-9cae-b1860b4ebf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:19.121790 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5e22-d66a-4bd3-9cae-b1860b4ebf6f None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:19.122194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5e22-d66a-4bd3-9cae-b1860b4ebf6f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:19.122458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5e22-d66a-4bd3-9cae-b1860b4ebf6f None admin] PATH_INFO: `/v3/domains/default/users/b56a4c63e0f64f7182c22666f3fab6a1/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:19.124636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee5e22-d66a-4bd3-9cae-b1860b4ebf6f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b56a4c63e0f64f7182c22666f3fab6a1, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:19.126237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee5e22-d66a-4bd3-9cae-b1860b4ebf6f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:19.181725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:19.190474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 18/40] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:19.192417 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4396-bb99-43b7-9715-ca645d23146d None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:19.192960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4396-bb99-43b7-9715-ca645d23146d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:19.193447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4396-bb99-43b7-9715-ca645d23146d None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:19.195916 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 19/41] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:19.203795 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fea9-3f0a-45f4-9e98-785a66e16dce None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:19.204138 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fea9-3f0a-45f4-9e98-785a66e16dce None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:19.204372 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fea9-3f0a-45f4-9e98-785a66e16dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:19.224475 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-7dd3fea9-3f0a-45f4-9e98-785a66e16dce None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:19.225813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-7dd3fea9-3f0a-45f4-9e98-785a66e16dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:19.254634 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 23/42] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jul 28 08:38:18 2022] PUT /identity/v3/domains/default/users/b56a4c63e0f64f7182c22666f3fab6a1/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:38:19.256088 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5e22-d66a-4bd3-9cae-b1860b4ebf6f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:19.316215 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 20/43] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:19.342857 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fea9-3f0a-45f4-9e98-785a66e16dce None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:19.410976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5e22-d66a-4bd3-9cae-b1860b4ebf6f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:19.428962 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:19.459422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc5c8e-0ea4-483c-a4f1-8c742b8fdd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:19.460075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc5c8e-0ea4-483c-a4f1-8c742b8fdd6e None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:19.460371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc5c8e-0ea4-483c-a4f1-8c742b8fdd6e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:19.460653 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc5c8e-0ea4-483c-a4f1-8c742b8fdd6e None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:19.461392 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc5c8e-0ea4-483c-a4f1-8c742b8fdd6e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:19.462001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fea9-3f0a-45f4-9e98-785a66e16dce None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:19.463052 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc5c8e-0ea4-483c-a4f1-8c742b8fdd6e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:19.483362 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 24/44] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:38:19 2022] POST /identity/v3/roles => generated 223 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:19.494548 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:19.502132 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d9fbf-53c4-4735-a695-bc65c9e13c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:19.502849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9fbf-53c4-4735-a695-bc65c9e13c38 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:19.503144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9fbf-53c4-4735-a695-bc65c9e13c38 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:19.503477 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9fbf-53c4-4735-a695-bc65c9e13c38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:19.505102 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d9fbf-53c4-4735-a695-bc65c9e13c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:19.506740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d9fbf-53c4-4735-a695-bc65c9e13c38 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:19.513916 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 21/45] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:19.515161 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc5c8e-0ea4-483c-a4f1-8c742b8fdd6e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:19.639481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc5c8e-0ea4-483c-a4f1-8c742b8fdd6e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:19.677706 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9e6eb-458a-41bd-9774-853f106eb3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:19.678422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e6eb-458a-41bd-9774-853f106eb3c8 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:19.678719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e6eb-458a-41bd-9774-853f106eb3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:19.679002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e6eb-458a-41bd-9774-853f106eb3c8 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:19.679732 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9e6eb-458a-41bd-9774-853f106eb3c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:19.681245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9e6eb-458a-41bd-9774-853f106eb3c8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:19.715857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 25/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:38:19 2022] POST /identity/v3/projects => generated 312 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:21.275577 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734c18f-cd80-4d2d-a4b5-50e5083d09d5 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:21.276187 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734c18f-cd80-4d2d-a4b5-50e5083d09d5 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:21.276187 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734c18f-cd80-4d2d-a4b5-50e5083d09d5 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:21.277519 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 22/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:21.283198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e7b8-940b-473e-bf5a-84b5102c088f None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:21.283698 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e7b8-940b-473e-bf5a-84b5102c088f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:21.283799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e7b8-940b-473e-bf5a-84b5102c088f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:21.303686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-b280e7b8-940b-473e-bf5a-84b5102c088f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:21.305267 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-b280e7b8-940b-473e-bf5a-84b5102c088f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:21.325743 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:21.378666 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 26/48] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:21.401650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734c18f-cd80-4d2d-a4b5-50e5083d09d5 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:21.514093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734c18f-cd80-4d2d-a4b5-50e5083d09d5 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:21.549976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43679ae4-f421-428e-a958-1aadec022bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:21.550576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679ae4-f421-428e-a958-1aadec022bfe None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:21.550860 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679ae4-f421-428e-a958-1aadec022bfe None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:21.551139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679ae4-f421-428e-a958-1aadec022bfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:21.562664 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43679ae4-f421-428e-a958-1aadec022bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-43679ae4-f421-428e-a958-1aadec022bfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:21.569556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:21.570755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:21.571830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 23/49] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:21.579128 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280e7b8-940b-473e-bf5a-84b5102c088f None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:21.685302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280e7b8-940b-473e-bf5a-84b5102c088f None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:21.729703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:21.730402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:21.730699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:21.730984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:21.731734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:21.732048 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:21.733499 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:21.742285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 27/50] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:21.743150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b14f9-76d9-4eb5-b759-039ddf804149 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:21.743558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b14f9-76d9-4eb5-b759-039ddf804149 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:21.743870 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b14f9-76d9-4eb5-b759-039ddf804149 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:21.745232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 24/51] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:21.748220 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:21.750440 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfb1d5-63ba-46f0-aef6-e152cc960f18 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:21.750724 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfb1d5-63ba-46f0-aef6-e152cc960f18 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:21.751014 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfb1d5-63ba-46f0-aef6-e152cc960f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:21.767141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:21.769087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-31dfb1d5-63ba-46f0-aef6-e152cc960f18 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:21.770355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-31dfb1d5-63ba-46f0-aef6-e152cc960f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:21.826210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:21.841627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 25/52] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:21.853128 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209ba09-2e38-474f-8982-d714fac61150 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:21.864601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfb1d5-63ba-46f0-aef6-e152cc960f18 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:21.890484 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e3a15-09ff-41d2-be9c-bda89d24e9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:21.891184 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e3a15-09ff-41d2-be9c-bda89d24e9b2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:21.891476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e3a15-09ff-41d2-be9c-bda89d24e9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:21.891763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e3a15-09ff-41d2-be9c-bda89d24e9b2 None admin] PATH_INFO: `/v3/users/b56a4c63e0f64f7182c22666f3fab6a1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:21.893074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e3a15-09ff-41d2-be9c-bda89d24e9b2 None admin] RBAC: Authorizing `identity:get_user(user_id=b56a4c63e0f64f7182c22666f3fab6a1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:21.894715 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e3a15-09ff-41d2-be9c-bda89d24e9b2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:21.896387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 28/53] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:21 2022] GET /identity/v3/users/b56a4c63e0f64f7182c22666f3fab6a1 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:21.901970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e3a15-09ff-41d2-be9c-bda89d24e9b2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:21.975523 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfb1d5-63ba-46f0-aef6-e152cc960f18 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:21.988462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:22.004628 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e3a15-09ff-41d2-be9c-bda89d24e9b2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:22.011822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a96be-283d-4312-a2a6-736c215a71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:22.012507 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a96be-283d-4312-a2a6-736c215a71db None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:22.012821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a96be-283d-4312-a2a6-736c215a71db None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:22.013110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a96be-283d-4312-a2a6-736c215a71db None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:22.014360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a96be-283d-4312-a2a6-736c215a71db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:22.015854 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a96be-283d-4312-a2a6-736c215a71db None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:22.017492 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 26/54] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:22.023337 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a96be-283d-4312-a2a6-736c215a71db None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:22.039752 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3952b3-2531-415b-bf6e-1c532c7f9f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:22.040429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3952b3-2531-415b-bf6e-1c532c7f9f32 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:22.040719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3952b3-2531-415b-bf6e-1c532c7f9f32 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:22.041005 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3952b3-2531-415b-bf6e-1c532c7f9f32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:22.042226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3952b3-2531-415b-bf6e-1c532c7f9f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:22.043734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3952b3-2531-415b-bf6e-1c532c7f9f32 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:22.045375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 29/55] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:22.051145 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3952b3-2531-415b-bf6e-1c532c7f9f32 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:22.081009 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:22.126367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a96be-283d-4312-a2a6-736c215a71db None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:22.154877 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3952b3-2531-415b-bf6e-1c532c7f9f32 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:22.160804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2140105-8c06-45f5-8e22-7a541d28cfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:22.161449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2140105-8c06-45f5-8e22-7a541d28cfdb None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:22.161762 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2140105-8c06-45f5-8e22-7a541d28cfdb None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:22.162038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2140105-8c06-45f5-8e22-7a541d28cfdb None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:22.162778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2140105-8c06-45f5-8e22-7a541d28cfdb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:22.164256 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2140105-8c06-45f5-8e22-7a541d28cfdb None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:22.190352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513bc2d-6f82-445b-b366-ead5dae1e532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:22.191016 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bc2d-6f82-445b-b366-ead5dae1e532 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:22.191305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bc2d-6f82-445b-b366-ead5dae1e532 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:22.191592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bc2d-6f82-445b-b366-ead5dae1e532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:22.192411 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513bc2d-6f82-445b-b366-ead5dae1e532 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.domain.id=default, user.id=b56a4c63e0f64f7182c22666f3fab6a1) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:22.192771 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513bc2d-6f82-445b-b366-ead5dae1e532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:22.194739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513bc2d-6f82-445b-b366-ead5dae1e532 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:22.196444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 27/56] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:38:22 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:38:22.203373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 30/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:38:22 2022] GET /identity/v3/role_assignments?user.id=b56a4c63e0f64f7182c22666f3fab6a1&scope.domain.id=default&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:24.254182 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f377b8-3e22-452e-95e7-9e883a797521 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:24.254771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f377b8-3e22-452e-95e7-9e883a797521 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:24.254771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f377b8-3e22-452e-95e7-9e883a797521 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:24.256217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 28/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:24.261909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87711216-5c06-42c3-9a41-6ae0cf2978a5 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:24.262384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87711216-5c06-42c3-9a41-6ae0cf2978a5 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:24.262483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87711216-5c06-42c3-9a41-6ae0cf2978a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:24.281649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-87711216-5c06-42c3-9a41-6ae0cf2978a5 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:24.282922 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-87711216-5c06-42c3-9a41-6ae0cf2978a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:24.298139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:24.370556 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 31/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:24.395208 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f377b8-3e22-452e-95e7-9e883a797521 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:24.511740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f377b8-3e22-452e-95e7-9e883a797521 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:24.522603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:24.549830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421c2c8-5a9f-4e38-a79a-18f56e3e3ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:24.550527 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c2c8-5a9f-4e38-a79a-18f56e3e3ab0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:24.550818 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c2c8-5a9f-4e38-a79a-18f56e3e3ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:24.551093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c2c8-5a9f-4e38-a79a-18f56e3e3ab0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:24.552285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421c2c8-5a9f-4e38-a79a-18f56e3e3ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:24.553874 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421c2c8-5a9f-4e38-a79a-18f56e3e3ab0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:24.555530 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 29/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:24.563569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87711216-5c06-42c3-9a41-6ae0cf2978a5 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:24.677734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87711216-5c06-42c3-9a41-6ae0cf2978a5 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:24.715830 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e1de-6495-4095-bfdb-a781a8a86cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:24.716524 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e1de-6495-4095-bfdb-a781a8a86cab None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:24.716818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e1de-6495-4095-bfdb-a781a8a86cab None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:24.717095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e1de-6495-4095-bfdb-a781a8a86cab None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:24.717790 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79e1de-6495-4095-bfdb-a781a8a86cab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:24.719325 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79e1de-6495-4095-bfdb-a781a8a86cab None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:24.747597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 32/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:38:24 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:27.923016 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965f32f-06eb-43c5-b060-5a0bd76f5c87 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:27.924151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965f32f-06eb-43c5-b060-5a0bd76f5c87 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:27.924669 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965f32f-06eb-43c5-b060-5a0bd76f5c87 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:27.926344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 30/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:27.932010 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92585-42b7-4164-b3b4-27d7ec5f39f7 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:27.932462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92585-42b7-4164-b3b4-27d7ec5f39f7 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:27.932568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92585-42b7-4164-b3b4-27d7ec5f39f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:27.942172 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:27.958388 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-77b92585-42b7-4164-b3b4-27d7ec5f39f7 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:27.959685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-77b92585-42b7-4164-b3b4-27d7ec5f39f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:28.063701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 33/63] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:28.105466 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b92585-42b7-4164-b3b4-27d7ec5f39f7 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:28.105906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcd966-7310-4446-87e3-267391f9e889 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.106486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcd966-7310-4446-87e3-267391f9e889 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.107000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcd966-7310-4446-87e3-267391f9e889 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.108606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 31/64] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:28.114001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c088060-3502-4530-80bb-85d8fdf62105 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.114639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c088060-3502-4530-80bb-85d8fdf62105 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.115157 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c088060-3502-4530-80bb-85d8fdf62105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.149511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-4c088060-3502-4530-80bb-85d8fdf62105 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:28.151276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-4c088060-3502-4530-80bb-85d8fdf62105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:28.221385 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b92585-42b7-4164-b3b4-27d7ec5f39f7 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:28.228025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 32/65] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:28.229814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd662-1f1b-4c6f-8f72-b07bc6c71190 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.230203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd662-1f1b-4c6f-8f72-b07bc6c71190 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.230727 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd662-1f1b-4c6f-8f72-b07bc6c71190 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.232381 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 33/66] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:28.241921 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d922-86fc-4adb-abe0-d033a246621c None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.242560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d922-86fc-4adb-abe0-d033a246621c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.243023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d922-86fc-4adb-abe0-d033a246621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.257942 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:28.264422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e356277-4fff-4e5b-9e19-50a4884f733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:28.265414 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e356277-4fff-4e5b-9e19-50a4884f733c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.265858 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-75f0d922-86fc-4adb-abe0-d033a246621c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:28.266144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e356277-4fff-4e5b-9e19-50a4884f733c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.266515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e356277-4fff-4e5b-9e19-50a4884f733c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.267819 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-75f0d922-86fc-4adb-abe0-d033a246621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:28.278371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e356277-4fff-4e5b-9e19-50a4884f733c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-4e356277-4fff-4e5b-9e19-50a4884f733c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:28.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:28.285369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:28.286710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:28.288570 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 34/67] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:28.289463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e356277-4fff-4e5b-9e19-50a4884f733c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:28.349325 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 34/68] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:28.350445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0d922-86fc-4adb-abe0-d033a246621c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:28.405346 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:28.430197 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e356277-4fff-4e5b-9e19-50a4884f733c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:28.468845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e5f41-cd17-4a26-b603-c26b73f1f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:28.469611 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e5f41-cd17-4a26-b603-c26b73f1f9be None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.469964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e5f41-cd17-4a26-b603-c26b73f1f9be None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.470341 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e5f41-cd17-4a26-b603-c26b73f1f9be None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.481994 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e5f41-cd17-4a26-b603-c26b73f1f9be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a90e5f41-cd17-4a26-b603-c26b73f1f9be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:28.487313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:28.488412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:28.489434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 35/69] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:28.490228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0d922-86fc-4adb-abe0-d033a246621c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:28.490323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e5f41-cd17-4a26-b603-c26b73f1f9be None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:28.532882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:28.533536 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.533830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.534121 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.534866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:28.535178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:28.536582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:28.553473 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 35/70] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:28.554075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:28.554303 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:28.612048 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e5f41-cd17-4a26-b603-c26b73f1f9be None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:28.647434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac3124-10a5-479f-ac0a-f77de0d052d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:28.648100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3124-10a5-479f-ac0a-f77de0d052d7 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.648380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3124-10a5-479f-ac0a-f77de0d052d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.648661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3124-10a5-479f-ac0a-f77de0d052d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.657559 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:28.660026 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac3124-10a5-479f-ac0a-f77de0d052d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:28.662235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26db8-0aa8-42a2-9c97-4ec1ac1afd33 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-fcac3124-10a5-479f-ac0a-f77de0d052d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:28.665106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:28.666378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:28.667714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 36/71] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:38:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:28.668491 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a04a56-61fb-489c-9b97-0e924e7e73a1 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.668491 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a04a56-61fb-489c-9b97-0e924e7e73a1 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.668491 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a04a56-61fb-489c-9b97-0e924e7e73a1 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.669877 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 37/72] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:28.670426 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a04a56-61fb-489c-9b97-0e924e7e73a1 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:28.703562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd74245-d3c4-416f-bc3d-a6cf2d60bbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:28.704218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd74245-d3c4-416f-bc3d-a6cf2d60bbac None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.704513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd74245-d3c4-416f-bc3d-a6cf2d60bbac None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.704780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd74245-d3c4-416f-bc3d-a6cf2d60bbac None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.705512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd74245-d3c4-416f-bc3d-a6cf2d60bbac None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:28.705847 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd74245-d3c4-416f-bc3d-a6cf2d60bbac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:28.707314 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd74245-d3c4-416f-bc3d-a6cf2d60bbac None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:28.715889 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 36/73] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:28.718312 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a859556-86e1-4154-9b86-52d3b81636e5 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.718312 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a859556-86e1-4154-9b86-52d3b81636e5 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.718312 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a859556-86e1-4154-9b86-52d3b81636e5 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.719008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 37/74] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:28.719678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a859556-86e1-4154-9b86-52d3b81636e5 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:28.791990 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a04a56-61fb-489c-9b97-0e924e7e73a1 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:28.809974 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:28.834090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe9cad-c128-4f44-96fc-900ecc9af870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:28.834900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe9cad-c128-4f44-96fc-900ecc9af870 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.835189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe9cad-c128-4f44-96fc-900ecc9af870 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.835465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe9cad-c128-4f44-96fc-900ecc9af870 None admin] PATH_INFO: `/v3/users/b56a4c63e0f64f7182c22666f3fab6a1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.836078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a859556-86e1-4154-9b86-52d3b81636e5 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:28.836753 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe9cad-c128-4f44-96fc-900ecc9af870 None admin] RBAC: Authorizing `identity:get_user(user_id=b56a4c63e0f64f7182c22666f3fab6a1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:28.838332 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe9cad-c128-4f44-96fc-900ecc9af870 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:28.840051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 38/75] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:28 2022] GET /identity/v3/users/b56a4c63e0f64f7182c22666f3fab6a1 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:28.841216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ff4f0-866a-42b6-bfe8-ce014b12f2a7 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.841501 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ff4f0-866a-42b6-bfe8-ce014b12f2a7 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.841830 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ff4f0-866a-42b6-bfe8-ce014b12f2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.864662 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-ca6ff4f0-866a-42b6-bfe8-ce014b12f2a7 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:28.866044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-ca6ff4f0-866a-42b6-bfe8-ce014b12f2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:28.874483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:28.875144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.875423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.875710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.876457 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:28.876770 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:28.878247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:28.887299 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 38/76] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:38:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:28.892339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:28.913461 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:28.938237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 39/77] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:28.939744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eee88-f7ff-486d-9b89-545666c73863 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:28.940047 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eee88-f7ff-486d-9b89-545666c73863 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:28.940333 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eee88-f7ff-486d-9b89-545666c73863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:28.959403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-df6eee88-f7ff-486d-9b89-545666c73863 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:28.960685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-df6eee88-f7ff-486d-9b89-545666c73863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:29.001587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a29652-9a69-4540-86ee-18b1265bcf1e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:29.031195 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 40/78] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:29.035156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6eee88-f7ff-486d-9b89-545666c73863 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:29.043897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8da7e-445f-468a-b74d-c4fd1f42759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:29.044509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8da7e-445f-468a-b74d-c4fd1f42759d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:29.044800 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8da7e-445f-468a-b74d-c4fd1f42759d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:29.045085 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8da7e-445f-468a-b74d-c4fd1f42759d None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:29.063342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8da7e-445f-468a-b74d-c4fd1f42759d None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:29.064865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8da7e-445f-468a-b74d-c4fd1f42759d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:29.066702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 39/79] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:28 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:29.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8da7e-445f-468a-b74d-c4fd1f42759d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:29.098337 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:29.156093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:29.161632 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6eee88-f7ff-486d-9b89-545666c73863 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:29.175634 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8da7e-445f-468a-b74d-c4fd1f42759d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:29.199639 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e0e9-eb94-41cc-bb74-5c331f50e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:29.200578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e0e9-eb94-41cc-bb74-5c331f50e497 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:29.200870 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e0e9-eb94-41cc-bb74-5c331f50e497 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:29.201153 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e0e9-eb94-41cc-bb74-5c331f50e497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:29.215735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e3489-f29b-4825-a219-8e77c8e59f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:29.216416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3489-f29b-4825-a219-8e77c8e59f20 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:29.216683 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3489-f29b-4825-a219-8e77c8e59f20 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:29.217024 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3489-f29b-4825-a219-8e77c8e59f20 None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:29.218814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e3489-f29b-4825-a219-8e77c8e59f20 None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:29.221164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e3489-f29b-4825-a219-8e77c8e59f20 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:29.223474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 40/80] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:29 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:29.224455 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e3489-f29b-4825-a219-8e77c8e59f20 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:29.278032 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5e0e9-eb94-41cc-bb74-5c331f50e497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:29.279606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5e0e9-eb94-41cc-bb74-5c331f50e497 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:29.322475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:29.328101 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e3489-f29b-4825-a219-8e77c8e59f20 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:29.363590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31f1f1-f127-4b01-b7ac-b795973aedbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:29.364276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31f1f1-f127-4b01-b7ac-b795973aedbe None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:29.364568 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31f1f1-f127-4b01-b7ac-b795973aedbe None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:29.364855 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31f1f1-f127-4b01-b7ac-b795973aedbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:29.366127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31f1f1-f127-4b01-b7ac-b795973aedbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:29.367651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31f1f1-f127-4b01-b7ac-b795973aedbe None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:29.369317 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 41/81] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:29.374238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31f1f1-f127-4b01-b7ac-b795973aedbe None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:29.386514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 41/82] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:29.387399 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e0e9-eb94-41cc-bb74-5c331f50e497 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:29.396721 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:29.497982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31f1f1-f127-4b01-b7ac-b795973aedbe None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:29.498528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e0e9-eb94-41cc-bb74-5c331f50e497 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:29.536386 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d880-35e5-4333-baa1-73b4c729d70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:29.536672 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607c912-80f2-4b83-99d0-53ed42969696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:29.537041 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d880-35e5-4333-baa1-73b4c729d70c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:29.537156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c912-80f2-4b83-99d0-53ed42969696 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:29.537329 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d880-35e5-4333-baa1-73b4c729d70c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:29.537433 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c912-80f2-4b83-99d0-53ed42969696 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:29.537711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c912-80f2-4b83-99d0-53ed42969696 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:29.537811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d880-35e5-4333-baa1-73b4c729d70c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:29.549106 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607c912-80f2-4b83-99d0-53ed42969696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:29.549343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-f607c912-80f2-4b83-99d0-53ed42969696 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:29.555136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:29.556275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 42/83] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:29.557782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607c912-80f2-4b83-99d0-53ed42969696 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:29.617804 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86d880-35e5-4333-baa1-73b4c729d70c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:29.619292 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86d880-35e5-4333-baa1-73b4c729d70c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:29.645536 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:29.670047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607c912-80f2-4b83-99d0-53ed42969696 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:29.717811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46e51d-6271-4f45-8a2d-a6cbb933b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:29.718509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e51d-6271-4f45-8a2d-a6cbb933b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:29.718795 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e51d-6271-4f45-8a2d-a6cbb933b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:29.719079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e51d-6271-4f45-8a2d-a6cbb933b2e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:29.735970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 42/84] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:29.736901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d880-35e5-4333-baa1-73b4c729d70c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:29.782061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:29.798405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46e51d-6271-4f45-8a2d-a6cbb933b2e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:29.799917 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46e51d-6271-4f45-8a2d-a6cbb933b2e5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:29.857055 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d880-35e5-4333-baa1-73b4c729d70c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:29.894467 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0b3d-305b-4b94-a24e-1904f896fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:29.895122 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0b3d-305b-4b94-a24e-1904f896fa4d None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:29.895409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0b3d-305b-4b94-a24e-1904f896fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:29.895696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0b3d-305b-4b94-a24e-1904f896fa4d None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:29.896413 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c0b3d-305b-4b94-a24e-1904f896fa4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:29.898035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c0b3d-305b-4b94-a24e-1904f896fa4d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:29.903842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:29.919853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 43/85] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:38:29 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:29.920719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0b3d-305b-4b94-a24e-1904f896fa4d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:29.928897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 43/86] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:29.928897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46e51d-6271-4f45-8a2d-a6cbb933b2e5 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:30.045237 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46e51d-6271-4f45-8a2d-a6cbb933b2e5 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.046844 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:30.054793 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0b3d-305b-4b94-a24e-1904f896fa4d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.091533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.092678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.092678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.092877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.093488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:30.093805 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.095253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c66b10-de78-4ed8-b12e-a54be4f7df40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.095387 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.095935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66b10-de78-4ed8-b12e-a54be4f7df40 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.096227 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66b10-de78-4ed8-b12e-a54be4f7df40 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.096513 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66b10-de78-4ed8-b12e-a54be4f7df40 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.098353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c66b10-de78-4ed8-b12e-a54be4f7df40 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.099875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c66b10-de78-4ed8-b12e-a54be4f7df40 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.102251 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 44/87] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:29 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:30.103076 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c66b10-de78-4ed8-b12e-a54be4f7df40 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:30.108443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 44/88] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:30.109344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:30.153676 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:30.229690 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c66b10-de78-4ed8-b12e-a54be4f7df40 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.233491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fe7f4-b8c3-4e5a-b8b9-187609493cc1 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.272546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36442-6260-49ca-b220-6caa1faedb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.273224 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36442-6260-49ca-b220-6caa1faedb8b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.273504 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36442-6260-49ca-b220-6caa1faedb8b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.273784 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36442-6260-49ca-b220-6caa1faedb8b None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.275208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36442-6260-49ca-b220-6caa1faedb8b None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.276648 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36442-6260-49ca-b220-6caa1faedb8b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.277638 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080bd45-573e-4a51-b86f-45e0e9e5cb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.278621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080bd45-573e-4a51-b86f-45e0e9e5cb9b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.278773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 45/89] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:30 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:30.279205 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36442-6260-49ca-b220-6caa1faedb8b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:30.283433 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080bd45-573e-4a51-b86f-45e0e9e5cb9b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.284113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080bd45-573e-4a51-b86f-45e0e9e5cb9b None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.285236 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080bd45-573e-4a51-b86f-45e0e9e5cb9b None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.286798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080bd45-573e-4a51-b86f-45e0e9e5cb9b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.289053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 45/90] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:30 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:30.289862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080bd45-573e-4a51-b86f-45e0e9e5cb9b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:30.333374 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:30.398815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080bd45-573e-4a51-b86f-45e0e9e5cb9b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.410580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:30.430135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36442-6260-49ca-b220-6caa1faedb8b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.438909 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f02a2-a7ae-404e-9645-d913851cacd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.440537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f02a2-a7ae-404e-9645-d913851cacd4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.440992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f02a2-a7ae-404e-9645-d913851cacd4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.441436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f02a2-a7ae-404e-9645-d913851cacd4 None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.443761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f02a2-a7ae-404e-9645-d913851cacd4 None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.446558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f02a2-a7ae-404e-9645-d913851cacd4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.449247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 46/91] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:30 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:30.450588 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f02a2-a7ae-404e-9645-d913851cacd4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:30.471446 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.472302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.472685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.473187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.474456 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=cfd603839dbc47efa27c6534b49c18c1, user.id=b56a4c63e0f64f7182c22666f3fab6a1) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:30.474642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.476345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.483216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 46/92] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:30 2022] GET /identity/v3/role_assignments?user.id=b56a4c63e0f64f7182c22666f3fab6a1&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:30.484241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:30.600514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:30.602500 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f02a2-a7ae-404e-9645-d913851cacd4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.613093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dde35-af19-475f-be97-07b8c7f507c7 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.642963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.643841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.644232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.645060 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.645543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] RBAC: Adding query filter params (role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=cfd603839dbc47efa27c6534b49c18c1, user.id=20f19629679940f0b5f79bf0614b5da6) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:30.645972 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.647773 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.659726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 47/93] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:30 2022] GET /identity/v3/role_assignments?user.id=20f19629679940f0b5f79bf0614b5da6&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:30.661012 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:30.673214 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.673978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.674394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.674685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.675530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] RBAC: Adding query filter params (role.id=8d072b58656b442f922c3287a5430603, scope.project.id=cfd603839dbc47efa27c6534b49c18c1, user.id=20f19629679940f0b5f79bf0614b5da6) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:30.675908 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.677459 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.692270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 47/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:30 2022] GET /identity/v3/role_assignments?user.id=20f19629679940f0b5f79bf0614b5da6&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=8d072b58656b442f922c3287a5430603 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:30.715811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:30.794317 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05427e70-6aaf-4d13-88c9-faaa5b23c53f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:30.839919 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a7031-33b7-4840-b44c-24992215c964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:30.841127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7031-33b7-4840-b44c-24992215c964 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:30.841647 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7031-33b7-4840-b44c-24992215c964 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:30.842183 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7031-33b7-4840-b44c-24992215c964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:30.922540 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a7031-33b7-4840-b44c-24992215c964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:30.924420 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a7031-33b7-4840-b44c-24992215c964 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:30.969783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:31.040961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 48/95] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:31.046870 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:31.195253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51880852-cd84-49f5-981c-becd26d360a3 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:31.253806 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1b10a-5c9b-459d-921e-3d72790ddb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:31.254532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b10a-5c9b-459d-921e-3d72790ddb19 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:31.254822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b10a-5c9b-459d-921e-3d72790ddb19 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:31.255107 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b10a-5c9b-459d-921e-3d72790ddb19 None admin] PATH_INFO: `/v3/projects/c1863fa57878420392969f0e6a8fe6b2` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:31.256392 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1b10a-5c9b-459d-921e-3d72790ddb19 None admin] RBAC: Authorizing `identity:get_project(project_id=c1863fa57878420392969f0e6a8fe6b2)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:31.257889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1b10a-5c9b-459d-921e-3d72790ddb19 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:31.260757 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 48/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:31 2022] GET /identity/v3/projects/c1863fa57878420392969f0e6a8fe6b2 => generated 312 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:31.265695 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a7031-33b7-4840-b44c-24992215c964 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:31.421275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a7031-33b7-4840-b44c-24992215c964 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:31.467207 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed3461-4f06-43c5-8bde-1a1ca33689dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:31.468254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed3461-4f06-43c5-8bde-1a1ca33689dd None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:31.468772 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed3461-4f06-43c5-8bde-1a1ca33689dd None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:31.469280 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed3461-4f06-43c5-8bde-1a1ca33689dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:31.470358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed3461-4f06-43c5-8bde-1a1ca33689dd None admin] RBAC: Adding query filter params (role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=c1863fa57878420392969f0e6a8fe6b2, user.id=20f19629679940f0b5f79bf0614b5da6) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:31.470909 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed3461-4f06-43c5-8bde-1a1ca33689dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:31.472608 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed3461-4f06-43c5-8bde-1a1ca33689dd None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:31.482378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 49/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:31 2022] GET /identity/v3/role_assignments?user.id=20f19629679940f0b5f79bf0614b5da6&scope.project.id=c1863fa57878420392969f0e6a8fe6b2&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:32.457819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b45d41-2501-4886-b1f5-f47602e37817 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:32.458442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b45d41-2501-4886-b1f5-f47602e37817 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:32.458539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b45d41-2501-4886-b1f5-f47602e37817 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:32.460386 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 49/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:32.472353 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24ed31-4692-4cf9-bcb2-2a555593a95f None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:32.472902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24ed31-4692-4cf9-bcb2-2a555593a95f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:32.472902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24ed31-4692-4cf9-bcb2-2a555593a95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:32.481124 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:32.496217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-9e24ed31-4692-4cf9-bcb2-2a555593a95f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:32.497483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-9e24ed31-4692-4cf9-bcb2-2a555593a95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:32.589277 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 50/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:32.614838 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b45d41-2501-4886-b1f5-f47602e37817 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:32.654836 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:32.762095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b45d41-2501-4886-b1f5-f47602e37817 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:32.810332 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadc25b-2deb-4fbd-8527-78d5d8165812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:32.811021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc25b-2deb-4fbd-8527-78d5d8165812 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:32.811319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc25b-2deb-4fbd-8527-78d5d8165812 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:32.811583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc25b-2deb-4fbd-8527-78d5d8165812 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:32.829324 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadc25b-2deb-4fbd-8527-78d5d8165812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-5aadc25b-2deb-4fbd-8527-78d5d8165812 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:32.835173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:32.837155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:32.840695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 50/100] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:32.844033 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24ed31-4692-4cf9-bcb2-2a555593a95f None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:32.984744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24ed31-4692-4cf9-bcb2-2a555593a95f None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:33.016160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:33.026620 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:33.027469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:33.027857 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:33.028155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:33.029485 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:33.030308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:33.032602 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:33.045595 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 51/101] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:33.051392 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadc25b-2deb-4fbd-8527-78d5d8165812 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:33.171006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:33.185892 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadc25b-2deb-4fbd-8527-78d5d8165812 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:33.231200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233696a-ef97-4c7d-8164-7b91e915f7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:33.231881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233696a-ef97-4c7d-8164-7b91e915f7f5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:33.232170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233696a-ef97-4c7d-8164-7b91e915f7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:33.232455 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233696a-ef97-4c7d-8164-7b91e915f7f5 None admin] PATH_INFO: `/v3/users/37a45369145140c180c0a012ee3a84bb` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:33.259881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233696a-ef97-4c7d-8164-7b91e915f7f5 None admin] RBAC: Authorizing `identity:get_user(user_id=37a45369145140c180c0a012ee3a84bb)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:33.261444 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233696a-ef97-4c7d-8164-7b91e915f7f5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:33.263347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 51/102] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:33 2022] GET /identity/v3/users/37a45369145140c180c0a012ee3a84bb => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:33.269441 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:33.435434 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f281de-dbde-4197-953e-ae05b2bcf9e4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:33.480797 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372249b2-b08c-4926-94e1-bf8d628dc719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:33.482384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372249b2-b08c-4926-94e1-bf8d628dc719 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:33.482929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372249b2-b08c-4926-94e1-bf8d628dc719 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:33.483447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372249b2-b08c-4926-94e1-bf8d628dc719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:33.560646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:33.567168 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3d1f1-6583-457b-892c-9a13785787a9 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:33.568170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3d1f1-6583-457b-892c-9a13785787a9 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:33.568721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3d1f1-6583-457b-892c-9a13785787a9 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:33.571006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 52/103] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:33.574611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372249b2-b08c-4926-94e1-bf8d628dc719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:33.576430 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372249b2-b08c-4926-94e1-bf8d628dc719 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:33.578001 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14d66-eee8-494d-ade6-14a0fc04bb5c None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:33.578535 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14d66-eee8-494d-ade6-14a0fc04bb5c None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:33.579188 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14d66-eee8-494d-ade6-14a0fc04bb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:33.609841 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-71d14d66-eee8-494d-ade6-14a0fc04bb5c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:33.611187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-71d14d66-eee8-494d-ade6-14a0fc04bb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:33.760667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 53/104] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:33.763134 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3bc2e-ad82-49ad-95cf-b66bf1306b1b None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:33.763433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3bc2e-ad82-49ad-95cf-b66bf1306b1b None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:33.764493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3bc2e-ad82-49ad-95cf-b66bf1306b1b None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:33.766747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 54/105] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:33.773090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 52/106] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:33.779675 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3bc2e-ad82-49ad-95cf-b66bf1306b1b None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:33.794611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f988-9961-4a94-8495-207c27f66c73 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:33.795197 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f988-9961-4a94-8495-207c27f66c73 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:33.795484 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f988-9961-4a94-8495-207c27f66c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:33.817641 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-a2b5f988-9961-4a94-8495-207c27f66c73 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:33.819630 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-a2b5f988-9961-4a94-8495-207c27f66c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:33.897426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 53/107] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:33.898827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5f988-9961-4a94-8495-207c27f66c73 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:33.929167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:33.954365 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:34.009195 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3bc2e-ad82-49ad-95cf-b66bf1306b1b None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.055164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5f988-9961-4a94-8495-207c27f66c73 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.088899 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801cb21-b6f4-4c63-aa3f-868babb953fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.089532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801cb21-b6f4-4c63-aa3f-868babb953fe None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.089777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801cb21-b6f4-4c63-aa3f-868babb953fe None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.090057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801cb21-b6f4-4c63-aa3f-868babb953fe None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.091846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801cb21-b6f4-4c63-aa3f-868babb953fe None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:34.093659 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801cb21-b6f4-4c63-aa3f-868babb953fe None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:34.096319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 55/108] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:33 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:34.098405 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801cb21-b6f4-4c63-aa3f-868babb953fe None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:34.110313 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271669be-2d68-4bd5-8d94-0677db65bf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.110991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271669be-2d68-4bd5-8d94-0677db65bf66 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.111278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271669be-2d68-4bd5-8d94-0677db65bf66 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.111559 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271669be-2d68-4bd5-8d94-0677db65bf66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.123270 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271669be-2d68-4bd5-8d94-0677db65bf66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-271669be-2d68-4bd5-8d94-0677db65bf66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:34.128329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:34.129524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:34.131384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 54/109] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:34.132196 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b977e4-c544-4fa5-8c48-27c5e14188ca None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.132291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b977e4-c544-4fa5-8c48-27c5e14188ca None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.132643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b977e4-c544-4fa5-8c48-27c5e14188ca None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.134177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 55/110] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:34.135153 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b977e4-c544-4fa5-8c48-27c5e14188ca None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:34.212930 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801cb21-b6f4-4c63-aa3f-868babb953fe None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.236475 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:34.249515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd367c-5b8b-4e1b-98b5-fe3ff41cf607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.250316 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd367c-5b8b-4e1b-98b5-fe3ff41cf607 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.250600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd367c-5b8b-4e1b-98b5-fe3ff41cf607 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.250890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd367c-5b8b-4e1b-98b5-fe3ff41cf607 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.252762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd367c-5b8b-4e1b-98b5-fe3ff41cf607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:34.254285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd367c-5b8b-4e1b-98b5-fe3ff41cf607 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:34.260641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 56/111] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:34.260641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc44cd-3e6e-44b0-877c-ed1f1e0be20f None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.260641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc44cd-3e6e-44b0-877c-ed1f1e0be20f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.260641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc44cd-3e6e-44b0-877c-ed1f1e0be20f None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.260641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 57/112] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:34.260641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc44cd-3e6e-44b0-877c-ed1f1e0be20f None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:34.264345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b977e4-c544-4fa5-8c48-27c5e14188ca None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.276085 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:34.304939 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06842982-f70a-4435-8398-dfe39686e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.305590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842982-f70a-4435-8398-dfe39686e46c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.305880 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842982-f70a-4435-8398-dfe39686e46c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.306188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842982-f70a-4435-8398-dfe39686e46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.307001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06842982-f70a-4435-8398-dfe39686e46c None admin] RBAC: Adding query filter params (role.id=9fc42581f1f84e28b6d65cdc76126fdc, scope.project.id=cfd603839dbc47efa27c6534b49c18c1, user.id=37a45369145140c180c0a012ee3a84bb) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:34.307320 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06842982-f70a-4435-8398-dfe39686e46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:34.308809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06842982-f70a-4435-8398-dfe39686e46c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:34.322144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 56/113] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:34 2022] GET /identity/v3/role_assignments?user.id=37a45369145140c180c0a012ee3a84bb&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=9fc42581f1f84e28b6d65cdc76126fdc => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:34.322144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6444a98-e2ed-4cd5-aa2c-521540ef42cc None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.322144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6444a98-e2ed-4cd5-aa2c-521540ef42cc None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.322144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6444a98-e2ed-4cd5-aa2c-521540ef42cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.336670 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-a6444a98-e2ed-4cd5-aa2c-521540ef42cc None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:34.337942 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-a6444a98-e2ed-4cd5-aa2c-521540ef42cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:34.402933 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc44cd-3e6e-44b0-877c-ed1f1e0be20f None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.433186 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 57/114] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:34.434148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6444a98-e2ed-4cd5-aa2c-521540ef42cc None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:34.453775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd761a5-d439-4e39-8873-26e98e70c86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.454636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd761a5-d439-4e39-8873-26e98e70c86f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.454928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd761a5-d439-4e39-8873-26e98e70c86f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.455209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd761a5-d439-4e39-8873-26e98e70c86f None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.456171 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd761a5-d439-4e39-8873-26e98e70c86f None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:34.456438 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd761a5-d439-4e39-8873-26e98e70c86f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:34.457919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd761a5-d439-4e39-8873-26e98e70c86f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:34.470028 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 58/115] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:34.471554 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c3179-1757-4f7f-88dd-212497964039 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.471892 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c3179-1757-4f7f-88dd-212497964039 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.472176 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c3179-1757-4f7f-88dd-212497964039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.496996 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-033c3179-1757-4f7f-88dd-212497964039 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:34.498778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-033c3179-1757-4f7f-88dd-212497964039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:34.536960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:34.563051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6444a98-e2ed-4cd5-aa2c-521540ef42cc None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.586503 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 59/116] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:34.595690 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c3179-1757-4f7f-88dd-212497964039 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:34.600811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:34.604960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8951a43-e1a3-4935-be48-6f95ef813319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.605998 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8951a43-e1a3-4935-be48-6f95ef813319 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.606583 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8951a43-e1a3-4935-be48-6f95ef813319 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.607074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8951a43-e1a3-4935-be48-6f95ef813319 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8951a43-e1a3-4935-be48-6f95ef813319 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:34.609646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8951a43-e1a3-4935-be48-6f95ef813319 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:34.638446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 58/117] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:38:34 2022] POST /identity/v3/projects => generated 302 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:34.639576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8951a43-e1a3-4935-be48-6f95ef813319 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:34.741040 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c3179-1757-4f7f-88dd-212497964039 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.783172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8951a43-e1a3-4935-be48-6f95ef813319 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.801697 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4a803-13e7-416e-a57f-08c8bc1cb210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.802448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4a803-13e7-416e-a57f-08c8bc1cb210 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.802735 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4a803-13e7-416e-a57f-08c8bc1cb210 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.803079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4a803-13e7-416e-a57f-08c8bc1cb210 None admin] PATH_INFO: `/v3/users/b56a4c63e0f64f7182c22666f3fab6a1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.804465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4a803-13e7-416e-a57f-08c8bc1cb210 None admin] RBAC: Authorizing `identity:get_user(user_id=b56a4c63e0f64f7182c22666f3fab6a1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:34.806045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4a803-13e7-416e-a57f-08c8bc1cb210 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:34.808077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 60/118] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:34 2022] GET /identity/v3/users/b56a4c63e0f64f7182c22666f3fab6a1 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:34.810126 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0c897-38ff-40f4-9419-ff8939884165 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.810420 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0c897-38ff-40f4-9419-ff8939884165 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.810700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0c897-38ff-40f4-9419-ff8939884165 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.812256 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 61/119] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:34.812883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0c897-38ff-40f4-9419-ff8939884165 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:34.831752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:34.841932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff0043-6482-4ece-8ff9-eb2d2d030c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.843108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0043-6482-4ece-8ff9-eb2d2d030c65 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.843395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0043-6482-4ece-8ff9-eb2d2d030c65 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.843674 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0043-6482-4ece-8ff9-eb2d2d030c65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.861823 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff0043-6482-4ece-8ff9-eb2d2d030c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-93ff0043-6482-4ece-8ff9-eb2d2d030c65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:34.867813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:34.869150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 59/120] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:34.870426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff0043-6482-4ece-8ff9-eb2d2d030c65 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:34.905256 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:34.941228 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0c897-38ff-40f4-9419-ff8939884165 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.981798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364c9f8-2437-4852-acb4-3c2e9a28db65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:34.982509 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364c9f8-2437-4852-acb4-3c2e9a28db65 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:34.982794 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364c9f8-2437-4852-acb4-3c2e9a28db65 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:34.983077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364c9f8-2437-4852-acb4-3c2e9a28db65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:34.990418 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff0043-6482-4ece-8ff9-eb2d2d030c65 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:34.995421 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364c9f8-2437-4852-acb4-3c2e9a28db65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-e364c9f8-2437-4852-acb4-3c2e9a28db65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:35.001044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:35.002694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 62/121] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:38:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:35.003734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebe44-1863-4bd9-b0f3-010d5fb14f6f None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.004541 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebe44-1863-4bd9-b0f3-010d5fb14f6f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.004541 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebe44-1863-4bd9-b0f3-010d5fb14f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.023777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-6c0ebe44-1863-4bd9-b0f3-010d5fb14f6f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:35.025130 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-6c0ebe44-1863-4bd9-b0f3-010d5fb14f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:35.037118 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7586cb-8edd-41b5-8b94-c2882537939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.038318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7586cb-8edd-41b5-8b94-c2882537939d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.038841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7586cb-8edd-41b5-8b94-c2882537939d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.039347 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7586cb-8edd-41b5-8b94-c2882537939d None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.040866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7586cb-8edd-41b5-8b94-c2882537939d None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.042717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7586cb-8edd-41b5-8b94-c2882537939d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.045166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 60/122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:34 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.046199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7586cb-8edd-41b5-8b94-c2882537939d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.102724 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 63/123] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:35.103615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebe44-1863-4bd9-b0f3-010d5fb14f6f None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.123432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:35.156814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7586cb-8edd-41b5-8b94-c2882537939d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.167884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:35.200393 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.204238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.206963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.207481 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.210569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebe44-1863-4bd9-b0f3-010d5fb14f6f None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.210955 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:35.211484 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.213127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.222605 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 61/124] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.223774 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.253305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.253975 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.254297 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.254586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.255334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:35.255647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.257156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.266188 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 64/125] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.267129 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.339941 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5bec9-3453-4c01-aab6-e5d7cba935c7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.363578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:35.382636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ca5fb-dc06-4a26-9952-d9284ea4c1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.383958 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca5fb-dc06-4a26-9952-d9284ea4c1cd None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.384534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca5fb-dc06-4a26-9952-d9284ea4c1cd None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.385102 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca5fb-dc06-4a26-9952-d9284ea4c1cd None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1/users/b56a4c63e0f64f7182c22666f3fab6a1/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.387869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ca5fb-dc06-4a26-9952-d9284ea4c1cd None admin] RBAC: Authorizing `identity:create_grant(project_id=cfd603839dbc47efa27c6534b49c18c1, user_id=b56a4c63e0f64f7182c22666f3fab6a1, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.389605 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ca5fb-dc06-4a26-9952-d9284ea4c1cd None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.392765 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f153-60ce-4ecc-b9a0-1f448a96ea9e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.406217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 62/126] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:35 2022] PUT /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1/users/b56a4c63e0f64f7182c22666f3fab6a1/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:35.407549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ca5fb-dc06-4a26-9952-d9284ea4c1cd None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.421165 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:35.435485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14098adf-e19e-488c-a301-8fe4dec9ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.436500 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14098adf-e19e-488c-a301-8fe4dec9ba94 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.436500 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14098adf-e19e-488c-a301-8fe4dec9ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.436825 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14098adf-e19e-488c-a301-8fe4dec9ba94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.450632 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14098adf-e19e-488c-a301-8fe4dec9ba94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-14098adf-e19e-488c-a301-8fe4dec9ba94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:35.456603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:35.457872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:35.458973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:35.462988 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 65/127] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:35.464006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14098adf-e19e-488c-a301-8fe4dec9ba94 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.537449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ca5fb-dc06-4a26-9952-d9284ea4c1cd None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.577076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb235042-e9c7-4e55-ac90-07b7faf2dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.578300 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb235042-e9c7-4e55-ac90-07b7faf2dc0b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.578828 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb235042-e9c7-4e55-ac90-07b7faf2dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.579369 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb235042-e9c7-4e55-ac90-07b7faf2dc0b None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.580973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb235042-e9c7-4e55-ac90-07b7faf2dc0b None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.582837 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb235042-e9c7-4e55-ac90-07b7faf2dc0b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.584738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14098adf-e19e-488c-a301-8fe4dec9ba94 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.585987 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 63/128] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.587287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb235042-e9c7-4e55-ac90-07b7faf2dc0b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.625587 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282619d-6092-478d-92cc-7df75ea33128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.626386 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282619d-6092-478d-92cc-7df75ea33128 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.626606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282619d-6092-478d-92cc-7df75ea33128 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.626904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282619d-6092-478d-92cc-7df75ea33128 None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.628219 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282619d-6092-478d-92cc-7df75ea33128 None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.629912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282619d-6092-478d-92cc-7df75ea33128 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.632233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 66/129] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.633165 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282619d-6092-478d-92cc-7df75ea33128 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.648124 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:35.699751 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:35.714195 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb235042-e9c7-4e55-ac90-07b7faf2dc0b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.753884 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.754795 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.755193 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.755646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.756563 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:35.756980 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.757701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282619d-6092-478d-92cc-7df75ea33128 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.758641 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.773314 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 64/130] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.773314 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.804863 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ed87-0a79-4c85-b30c-2b9119a9cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.805867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ed87-0a79-4c85-b30c-2b9119a9cc06 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.806436 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ed87-0a79-4c85-b30c-2b9119a9cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.806994 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ed87-0a79-4c85-b30c-2b9119a9cc06 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.808727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3ed87-0a79-4c85-b30c-2b9119a9cc06 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.810534 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3ed87-0a79-4c85-b30c-2b9119a9cc06 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.812736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 67/131] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.813862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ed87-0a79-4c85-b30c-2b9119a9cc06 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.884643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c80876-fc6d-4323-b86a-b5d1e409e381 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.896590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:35.927697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed704-eac3-4446-a297-8ef96938fa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.928432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed704-eac3-4446-a297-8ef96938fa13 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.928967 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed704-eac3-4446-a297-8ef96938fa13 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.929095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed704-eac3-4446-a297-8ef96938fa13 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.930305 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eed704-eac3-4446-a297-8ef96938fa13 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.931865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eed704-eac3-4446-a297-8ef96938fa13 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.932608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ed87-0a79-4c85-b30c-2b9119a9cc06 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:35.934037 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 65/132] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.934953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed704-eac3-4446-a297-8ef96938fa13 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:35.957054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:35.975858 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecb594-95bd-4511-a2a1-59676c00ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:35.976998 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb594-95bd-4511-a2a1-59676c00ce98 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:35.977687 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb594-95bd-4511-a2a1-59676c00ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:35.978232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb594-95bd-4511-a2a1-59676c00ce98 None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:35.979879 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecb594-95bd-4511-a2a1-59676c00ce98 None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:35.982237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecb594-95bd-4511-a2a1-59676c00ce98 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:35.984524 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 68/133] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:35 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:35.991318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecb594-95bd-4511-a2a1-59676c00ce98 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:36.045433 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed704-eac3-4446-a297-8ef96938fa13 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:36.083159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2278b-3c6a-4e50-bc94-901afce14a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:36.084070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2278b-3c6a-4e50-bc94-901afce14a82 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:36.084240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2278b-3c6a-4e50-bc94-901afce14a82 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:36.084482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2278b-3c6a-4e50-bc94-901afce14a82 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1/users/20f19629679940f0b5f79bf0614b5da6/roles/914dcb9114e54c75a514ce6af60ffbb3` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:36.086902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2278b-3c6a-4e50-bc94-901afce14a82 None admin] RBAC: Authorizing `identity:create_grant(project_id=cfd603839dbc47efa27c6534b49c18c1, user_id=20f19629679940f0b5f79bf0614b5da6, role_id=914dcb9114e54c75a514ce6af60ffbb3)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:36.088645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2278b-3c6a-4e50-bc94-901afce14a82 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:36.104524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 66/134] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:35 2022] PUT /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1/users/20f19629679940f0b5f79bf0614b5da6/roles/914dcb9114e54c75a514ce6af60ffbb3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:36.105260 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2278b-3c6a-4e50-bc94-901afce14a82 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:36.136095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecb594-95bd-4511-a2a1-59676c00ce98 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:36.159904 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:36.227827 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729f5bd-4adf-4380-a546-ca9fe472f5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:36.228517 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f5bd-4adf-4380-a546-ca9fe472f5f6 None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:36.228821 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f5bd-4adf-4380-a546-ca9fe472f5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:36.229107 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f5bd-4adf-4380-a546-ca9fe472f5f6 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1/users/20f19629679940f0b5f79bf0614b5da6/roles/8d072b58656b442f922c3287a5430603` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:36.232666 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729f5bd-4adf-4380-a546-ca9fe472f5f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cfd603839dbc47efa27c6534b49c18c1, user_id=20f19629679940f0b5f79bf0614b5da6, role_id=8d072b58656b442f922c3287a5430603)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:36.234505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729f5bd-4adf-4380-a546-ca9fe472f5f6 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:36.259201 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 69/135] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:35 2022] PUT /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1/users/20f19629679940f0b5f79bf0614b5da6/roles/8d072b58656b442f922c3287a5430603 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:36.280143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2278b-3c6a-4e50-bc94-901afce14a82 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:36.325356 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d0b89-1109-4ccf-9363-5a3416565af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:36.326076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d0b89-1109-4ccf-9363-5a3416565af5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:36.326488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d0b89-1109-4ccf-9363-5a3416565af5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:36.326776 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d0b89-1109-4ccf-9363-5a3416565af5 None admin] PATH_INFO: `/v3/projects/c1863fa57878420392969f0e6a8fe6b2` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:36.328334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d0b89-1109-4ccf-9363-5a3416565af5 None admin] RBAC: Authorizing `identity:get_project(project_id=c1863fa57878420392969f0e6a8fe6b2)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:36.330077 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d0b89-1109-4ccf-9363-5a3416565af5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:36.332262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 67/136] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:36 2022] GET /identity/v3/projects/c1863fa57878420392969f0e6a8fe6b2 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:36.338867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729f5bd-4adf-4380-a546-ca9fe472f5f6 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:36.401587 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:36.464247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729f5bd-4adf-4380-a546-ca9fe472f5f6 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:36.503441 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b2698-6a09-4bbf-91d4-df55a6761506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:36.504080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b2698-6a09-4bbf-91d4-df55a6761506 None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:36.504375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b2698-6a09-4bbf-91d4-df55a6761506 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:36.504662 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b2698-6a09-4bbf-91d4-df55a6761506 None admin] PATH_INFO: `/v3/projects/c1863fa57878420392969f0e6a8fe6b2/users/20f19629679940f0b5f79bf0614b5da6/roles/914dcb9114e54c75a514ce6af60ffbb3` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:36.506802 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b2698-6a09-4bbf-91d4-df55a6761506 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1863fa57878420392969f0e6a8fe6b2, user_id=20f19629679940f0b5f79bf0614b5da6, role_id=914dcb9114e54c75a514ce6af60ffbb3)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:36.508440 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b2698-6a09-4bbf-91d4-df55a6761506 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:36.525639 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 70/137] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:36 2022] PUT /identity/v3/projects/c1863fa57878420392969f0e6a8fe6b2/users/20f19629679940f0b5f79bf0614b5da6/roles/914dcb9114e54c75a514ce6af60ffbb3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:36.847530 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715233-75d9-4b7a-9e7a-cdcb03e98319 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:36.848075 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715233-75d9-4b7a-9e7a-cdcb03e98319 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:36.848075 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715233-75d9-4b7a-9e7a-cdcb03e98319 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:36.849526 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 68/138] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:36.855346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fb60a-b797-4ca4-a667-049388bc18d3 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:36.855865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fb60a-b797-4ca4-a667-049388bc18d3 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:36.855960 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fb60a-b797-4ca4-a667-049388bc18d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:36.879331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-531fb60a-b797-4ca4-a667-049388bc18d3 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:36.880582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-531fb60a-b797-4ca4-a667-049388bc18d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:36.966423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 71/139] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:36.992595 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7715233-75d9-4b7a-9e7a-cdcb03e98319 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:37.100282 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7715233-75d9-4b7a-9e7a-cdcb03e98319 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:37.110466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:37.136809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f9ebe-5f16-4851-ad71-786d143cddb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:37.137488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9ebe-5f16-4851-ad71-786d143cddb2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:37.137775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9ebe-5f16-4851-ad71-786d143cddb2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:37.138057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9ebe-5f16-4851-ad71-786d143cddb2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:37.149656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f9ebe-5f16-4851-ad71-786d143cddb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-4f3f9ebe-5f16-4851-ad71-786d143cddb2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:37.154863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:37.155928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:37.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 69/140] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:37.162191 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fb60a-b797-4ca4-a667-049388bc18d3 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:37.222600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:37.253580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe535a2-43c3-47f6-bc5c-3fd8a1947348 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:37.253939 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe535a2-43c3-47f6-bc5c-3fd8a1947348 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:37.254319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe535a2-43c3-47f6-bc5c-3fd8a1947348 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:37.255816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 70/141] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:37.271566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b652b-170f-4903-8252-f59c843b9171 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:37.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b652b-170f-4903-8252-f59c843b9171 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:37.272201 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b652b-170f-4903-8252-f59c843b9171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:37.292759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fb60a-b797-4ca4-a667-049388bc18d3 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:37.306287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-856b652b-170f-4903-8252-f59c843b9171 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:37.307538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-856b652b-170f-4903-8252-f59c843b9171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:37.370550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:37.371125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:37.371409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:37.371688 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:37.374927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:37.375245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:37.380206 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:37.400122 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 72/142] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:37.406946 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:37.468411 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 71/143] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:37.504284 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b652b-170f-4903-8252-f59c843b9171 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:37.615073 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b652b-170f-4903-8252-f59c843b9171 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:37.634493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:37.653214 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2580e6e-c897-4b7b-aa57-515ee00544b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:37.653890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:37.654119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2580e6e-c897-4b7b-aa57-515ee00544b6 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:37.654422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2580e6e-c897-4b7b-aa57-515ee00544b6 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:37.654707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2580e6e-c897-4b7b-aa57-515ee00544b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:37.656104 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2580e6e-c897-4b7b-aa57-515ee00544b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:37.657874 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2580e6e-c897-4b7b-aa57-515ee00544b6 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:37.660509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 72/144] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:37.666640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2580e6e-c897-4b7b-aa57-515ee00544b6 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:37.668118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7ab9b-38c6-44c9-b21c-1ab350a9dd6b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:37.808404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2580e6e-c897-4b7b-aa57-515ee00544b6 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:37.845508 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce5978-7a65-4508-9203-74399c0e5f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:37.846207 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce5978-7a65-4508-9203-74399c0e5f92 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:37.846503 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce5978-7a65-4508-9203-74399c0e5f92 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:37.846855 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce5978-7a65-4508-9203-74399c0e5f92 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:37.847582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce5978-7a65-4508-9203-74399c0e5f92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:37.849140 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce5978-7a65-4508-9203-74399c0e5f92 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:37.855485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bac3e-ac67-40c3-8347-7a6b03c2ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:37.856183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bac3e-ac67-40c3-8347-7a6b03c2ff26 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:37.856471 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bac3e-ac67-40c3-8347-7a6b03c2ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:37.856754 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bac3e-ac67-40c3-8347-7a6b03c2ff26 None admin] PATH_INFO: `/v3/users/37a45369145140c180c0a012ee3a84bb` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:37.857993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bac3e-ac67-40c3-8347-7a6b03c2ff26 None admin] RBAC: Authorizing `identity:get_user(user_id=37a45369145140c180c0a012ee3a84bb)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:37.859531 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bac3e-ac67-40c3-8347-7a6b03c2ff26 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:37.861531 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 73/145] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:37 2022] GET /identity/v3/users/37a45369145140c180c0a012ee3a84bb => generated 289 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:37.874257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bac3e-ac67-40c3-8347-7a6b03c2ff26 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:37.886689 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 73/146] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:38:37 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:38.013272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bac3e-ac67-40c3-8347-7a6b03c2ff26 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:38.057494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278fb04-3ec3-4997-b223-a96798a2808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:38.058519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278fb04-3ec3-4997-b223-a96798a2808d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:38.058995 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278fb04-3ec3-4997-b223-a96798a2808d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:38.059439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278fb04-3ec3-4997-b223-a96798a2808d None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:38.060840 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278fb04-3ec3-4997-b223-a96798a2808d None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:38.062592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278fb04-3ec3-4997-b223-a96798a2808d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:38.064539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 74/147] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:37 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:38.071235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce5978-7a65-4508-9203-74399c0e5f92 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:38.103281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:38.221041 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce5978-7a65-4508-9203-74399c0e5f92 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:38.268214 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0fa46-34b4-4bbc-bb40-b7747cc3ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:38.268930 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0fa46-34b4-4bbc-bb40-b7747cc3ce1a None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:38.269214 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0fa46-34b4-4bbc-bb40-b7747cc3ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:38.269495 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0fa46-34b4-4bbc-bb40-b7747cc3ce1a None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1/users/37a45369145140c180c0a012ee3a84bb/roles/9fc42581f1f84e28b6d65cdc76126fdc` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:38.271553 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0fa46-34b4-4bbc-bb40-b7747cc3ce1a None admin] RBAC: Authorizing `identity:create_grant(project_id=cfd603839dbc47efa27c6534b49c18c1, user_id=37a45369145140c180c0a012ee3a84bb, role_id=9fc42581f1f84e28b6d65cdc76126fdc)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:38.273091 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0fa46-34b4-4bbc-bb40-b7747cc3ce1a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:38.300775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 74/148] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:38 2022] PUT /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1/users/37a45369145140c180c0a012ee3a84bb/roles/9fc42581f1f84e28b6d65cdc76126fdc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:38.313746 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffb83-5094-4ef8-9d83-43324866355f None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:38.314123 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffb83-5094-4ef8-9d83-43324866355f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:38.314388 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffb83-5094-4ef8-9d83-43324866355f None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:38.315895 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 75/149] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:38.322232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e0cfb-54eb-4269-908e-998800ab93ab None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:38.322580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e0cfb-54eb-4269-908e-998800ab93ab None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:38.322803 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e0cfb-54eb-4269-908e-998800ab93ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:38.344087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-d94e0cfb-54eb-4269-908e-998800ab93ab None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:38.345368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-d94e0cfb-54eb-4269-908e-998800ab93ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:38.434618 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 75/150] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:38.462646 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfffb83-5094-4ef8-9d83-43324866355f None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:38.525042 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:38.619079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfffb83-5094-4ef8-9d83-43324866355f None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:38.667407 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97ab84-baf1-4526-a37f-2411c6908901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:38.668101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97ab84-baf1-4526-a37f-2411c6908901 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:38.668385 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97ab84-baf1-4526-a37f-2411c6908901 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:38.668664 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97ab84-baf1-4526-a37f-2411c6908901 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:38.683127 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97ab84-baf1-4526-a37f-2411c6908901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-7a97ab84-baf1-4526-a37f-2411c6908901 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:38.688963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:38.690155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:38.691714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 76/151] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:38.696992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e0cfb-54eb-4269-908e-998800ab93ab None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:38.716502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:38.837721 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e0cfb-54eb-4269-908e-998800ab93ab None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:38.882057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:38.882757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:38.883035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:38.883266 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:38.884017 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:38.884317 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:38.885812 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:38.896928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 76/152] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:38.902717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97ab84-baf1-4526-a37f-2411c6908901 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:39.018997 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:39.026715 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97ab84-baf1-4526-a37f-2411c6908901 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:39.068232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594384c7-22a1-45c7-aa5a-d59a44053322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:39.069054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594384c7-22a1-45c7-aa5a-d59a44053322 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:39.069730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594384c7-22a1-45c7-aa5a-d59a44053322 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:39.070012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594384c7-22a1-45c7-aa5a-d59a44053322 None admin] PATH_INFO: `/v3/users/b56a4c63e0f64f7182c22666f3fab6a1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:39.071307 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594384c7-22a1-45c7-aa5a-d59a44053322 None admin] RBAC: Authorizing `identity:get_user(user_id=b56a4c63e0f64f7182c22666f3fab6a1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:39.072785 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594384c7-22a1-45c7-aa5a-d59a44053322 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:39.074470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 77/153] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:38 2022] GET /identity/v3/users/b56a4c63e0f64f7182c22666f3fab6a1 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:39.080120 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:39.172300 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:39.213625 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18554fc-da38-4e00-9b2c-1342b7b25a86 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:39.253383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0d2e2-9dd9-45f5-a75a-578aa4eb71eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:39.254049 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d2e2-9dd9-45f5-a75a-578aa4eb71eb None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:39.254360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d2e2-9dd9-45f5-a75a-578aa4eb71eb None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:39.254641 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d2e2-9dd9-45f5-a75a-578aa4eb71eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:39.337160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0d2e2-9dd9-45f5-a75a-578aa4eb71eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:39.338719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0d2e2-9dd9-45f5-a75a-578aa4eb71eb None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:39.422602 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:39.466352 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 77/154] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:39.471791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594384c7-22a1-45c7-aa5a-d59a44053322 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:39.606010 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594384c7-22a1-45c7-aa5a-d59a44053322 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:39.645258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22e16d-0ee9-4d23-a569-065f1cca1b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:39.646000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22e16d-0ee9-4d23-a569-065f1cca1b37 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:39.646357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22e16d-0ee9-4d23-a569-065f1cca1b37 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:39.646748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22e16d-0ee9-4d23-a569-065f1cca1b37 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:39.648007 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22e16d-0ee9-4d23-a569-065f1cca1b37 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:39.649511 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22e16d-0ee9-4d23-a569-065f1cca1b37 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:39.651275 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 78/155] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:39 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:39.657476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0d2e2-9dd9-45f5-a75a-578aa4eb71eb None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:39.775312 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0d2e2-9dd9-45f5-a75a-578aa4eb71eb None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:39.792177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce28c-a245-4fa7-8483-7faa644f6735 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:39.792546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce28c-a245-4fa7-8483-7faa644f6735 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:39.792751 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce28c-a245-4fa7-8483-7faa644f6735 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:39.794324 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 79/156] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:38:39.799978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d8f26-4f78-4d6c-94ec-fa7b14fee22c None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:39.800287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d8f26-4f78-4d6c-94ec-fa7b14fee22c None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:39.800547 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d8f26-4f78-4d6c-94ec-fa7b14fee22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:39.848068 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-781d8f26-4f78-4d6c-94ec-fa7b14fee22c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:39.849305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-781d8f26-4f78-4d6c-94ec-fa7b14fee22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:39.863164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54e53-b4e4-49a5-87c9-4ec1ab3d4e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:39.863819 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54e53-b4e4-49a5-87c9-4ec1ab3d4e2a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:39.864103 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54e53-b4e4-49a5-87c9-4ec1ab3d4e2a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:39.864399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54e53-b4e4-49a5-87c9-4ec1ab3d4e2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:39.865190 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad54e53-b4e4-49a5-87c9-4ec1ab3d4e2a None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=cfd603839dbc47efa27c6534b49c18c1, user.id=b56a4c63e0f64f7182c22666f3fab6a1) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:39.865548 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad54e53-b4e4-49a5-87c9-4ec1ab3d4e2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:39.867113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad54e53-b4e4-49a5-87c9-4ec1ab3d4e2a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:39.878997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 78/157] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:39 2022] GET /identity/v3/role_assignments?user.id=b56a4c63e0f64f7182c22666f3fab6a1&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:39.910481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:39.940439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124ab0-7f24-4b81-9a03-ccf193a84b00 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:39.941072 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124ab0-7f24-4b81-9a03-ccf193a84b00 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:39.941072 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124ab0-7f24-4b81-9a03-ccf193a84b00 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:39.942594 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 79/158] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:38:39.955758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ddb5-5926-4d72-a774-9078eb243e8b None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:39.956106 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ddb5-5926-4d72-a774-9078eb243e8b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:39.956322 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ddb5-5926-4d72-a774-9078eb243e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:39.983531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-2f57ddb5-5926-4d72-a774-9078eb243e8b None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:39.984993 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-2f57ddb5-5926-4d72-a774-9078eb243e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:39.999974 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 80/159] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:40.029361 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:40.043662 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d8f26-4f78-4d6c-94ec-fa7b14fee22c None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:40.083591 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 80/160] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:40.133791 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ddb5-5926-4d72-a774-9078eb243e8b None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:40.291769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ddb5-5926-4d72-a774-9078eb243e8b None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:40.346747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368fc91-22e4-408a-9249-b5eacc958e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:40.347603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368fc91-22e4-408a-9249-b5eacc958e4e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:40.348011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368fc91-22e4-408a-9249-b5eacc958e4e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:40.348390 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368fc91-22e4-408a-9249-b5eacc958e4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:40.364989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368fc91-22e4-408a-9249-b5eacc958e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8368fc91-22e4-408a-9249-b5eacc958e4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:40.371039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:40.373199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:40.375115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:40.378252 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 81/161] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:38:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:40.379981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368fc91-22e4-408a-9249-b5eacc958e4e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:40.404510 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d8f26-4f78-4d6c-94ec-fa7b14fee22c None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:40.426591 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:40.482818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cab6dc-25ea-42d3-b949-4edbdeb06f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:40.483516 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cab6dc-25ea-42d3-b949-4edbdeb06f5f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:40.484145 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cab6dc-25ea-42d3-b949-4edbdeb06f5f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:40.484437 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cab6dc-25ea-42d3-b949-4edbdeb06f5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:40.505210 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cab6dc-25ea-42d3-b949-4edbdeb06f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-08cab6dc-25ea-42d3-b949-4edbdeb06f5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:40.511435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:40.512544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:40.514059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 81/162] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 470 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:38:40.519454 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d709f4-86fd-41bd-bee1-d71ebdd68d9a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:40.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d709f4-86fd-41bd-bee1-d71ebdd68d9a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:40.520036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d709f4-86fd-41bd-bee1-d71ebdd68d9a None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:40.521749 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 82/163] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:40.522335 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d709f4-86fd-41bd-bee1-d71ebdd68d9a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:40.558346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:40.601562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368fc91-22e4-408a-9249-b5eacc958e4e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:40.667543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d709f4-86fd-41bd-bee1-d71ebdd68d9a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:40.696468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba9f67-5f5f-4b19-95f1-a5ef93876987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:40.696468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba9f67-5f5f-4b19-95f1-a5ef93876987 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:40.696468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba9f67-5f5f-4b19-95f1-a5ef93876987 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:40.696468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba9f67-5f5f-4b19-95f1-a5ef93876987 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:40.696924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba9f67-5f5f-4b19-95f1-a5ef93876987 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:40.696924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba9f67-5f5f-4b19-95f1-a5ef93876987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:40.696924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba9f67-5f5f-4b19-95f1-a5ef93876987 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:40.714842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 82/164] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:38:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:40.716183 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf774f1-4ef9-41dc-aabd-baf79d194522 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:40.716475 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf774f1-4ef9-41dc-aabd-baf79d194522 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:40.716757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf774f1-4ef9-41dc-aabd-baf79d194522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:40.739965 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:40.740654 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:40.740941 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:40.741210 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:40.741956 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] RBAC: Adding query filter params (name=member) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:40.742298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:40.743867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:40.754266 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-5cf774f1-4ef9-41dc-aabd-baf79d194522 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:40.755501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-5cf774f1-4ef9-41dc-aabd-baf79d194522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:40.764418 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 83/165] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:40.769754 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:40.894603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 83/166] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:40.896848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf774f1-4ef9-41dc-aabd-baf79d194522 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:40.939970 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:40.942524 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:40.965142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c60c2-12ed-4d58-9bbc-e76d8e0c0c85 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:41.012631 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e97ead-503f-452b-aee9-b7a153762107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:41.013365 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e97ead-503f-452b-aee9-b7a153762107 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:41.013654 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e97ead-503f-452b-aee9-b7a153762107 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:41.013931 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e97ead-503f-452b-aee9-b7a153762107 None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:41.015257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e97ead-503f-452b-aee9-b7a153762107 None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:41.016764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e97ead-503f-452b-aee9-b7a153762107 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:41.018676 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 84/167] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:40 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:41.019389 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e97ead-503f-452b-aee9-b7a153762107 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:41.157234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e97ead-503f-452b-aee9-b7a153762107 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:41.197043 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf774f1-4ef9-41dc-aabd-baf79d194522 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:41.226522 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984a5c8-c4d7-4d42-a92b-283f0a7db600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:41.227195 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984a5c8-c4d7-4d42-a92b-283f0a7db600 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:41.227476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984a5c8-c4d7-4d42-a92b-283f0a7db600 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:41.227750 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984a5c8-c4d7-4d42-a92b-283f0a7db600 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:41.251231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984a5c8-c4d7-4d42-a92b-283f0a7db600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-d984a5c8-c4d7-4d42-a92b-283f0a7db600 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:41.257231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:41.258457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 85/168] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:41.259506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984a5c8-c4d7-4d42-a92b-283f0a7db600 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:41.274276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb759f5-51c1-40ba-a543-514c0a989bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:41.274975 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb759f5-51c1-40ba-a543-514c0a989bb2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:41.275255 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb759f5-51c1-40ba-a543-514c0a989bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:41.275532 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb759f5-51c1-40ba-a543-514c0a989bb2 None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:41.277803 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb759f5-51c1-40ba-a543-514c0a989bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:41.279354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb759f5-51c1-40ba-a543-514c0a989bb2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:41.281329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 84/169] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:40 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:41.286238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb759f5-51c1-40ba-a543-514c0a989bb2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:41.305606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:41.448431 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:41.473840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb759f5-51c1-40ba-a543-514c0a989bb2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:41.478981 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984a5c8-c4d7-4d42-a92b-283f0a7db600 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:41.538371 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:41.539399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:41.539683 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:41.540516 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:41.541262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:41.541565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:41.542914 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecee8d-7ce0-49bd-9750-22363dbdd6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:41.543064 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:41.543582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecee8d-7ce0-49bd-9750-22363dbdd6a2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:41.543865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecee8d-7ce0-49bd-9750-22363dbdd6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:41.544142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecee8d-7ce0-49bd-9750-22363dbdd6a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:41.552110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 85/170] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:41.553043 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:41.669581 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecee8d-7ce0-49bd-9750-22363dbdd6a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:41.671147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecee8d-7ce0-49bd-9750-22363dbdd6a2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:41.705039 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:41.712943 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f54dc-563d-48e1-b35a-dfda960e3153 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:41.773766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5b720-b3ce-4d09-a174-d61b6e954aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:41.774452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5b720-b3ce-4d09-a174-d61b6e954aa7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:41.775014 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5b720-b3ce-4d09-a174-d61b6e954aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:41.775407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5b720-b3ce-4d09-a174-d61b6e954aa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:41.797469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:41.858495 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5b720-b3ce-4d09-a174-d61b6e954aa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:41.860614 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5b720-b3ce-4d09-a174-d61b6e954aa7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:41.864519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 86/171] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:41.865675 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecee8d-7ce0-49bd-9750-22363dbdd6a2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:42.002267 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 86/172] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:42.007304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5b720-b3ce-4d09-a174-d61b6e954aa7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:42.071295 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:42.121528 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5b720-b3ce-4d09-a174-d61b6e954aa7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:42.157697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb9955-5d9d-438a-84af-4b2317324ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:42.158327 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb9955-5d9d-438a-84af-4b2317324ce8 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.158608 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb9955-5d9d-438a-84af-4b2317324ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.158879 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb9955-5d9d-438a-84af-4b2317324ce8 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.160291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb9955-5d9d-438a-84af-4b2317324ce8 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:42.162845 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb9955-5d9d-438a-84af-4b2317324ce8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:42.165161 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 87/173] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:42 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:42.166903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663de46-4ffc-44ed-94de-607b471c78d9 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.167190 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663de46-4ffc-44ed-94de-607b471c78d9 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.167463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663de46-4ffc-44ed-94de-607b471c78d9 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.168685 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 88/174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:42.169444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663de46-4ffc-44ed-94de-607b471c78d9 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:42.176809 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecee8d-7ce0-49bd-9750-22363dbdd6a2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:42.191019 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:42.214100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bb289-73b7-47c8-b819-4ac29b8ac242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:42.214903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bb289-73b7-47c8-b819-4ac29b8ac242 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.215246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bb289-73b7-47c8-b819-4ac29b8ac242 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.215590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bb289-73b7-47c8-b819-4ac29b8ac242 None admin] PATH_INFO: `/v3/users/20f19629679940f0b5f79bf0614b5da6` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.216859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bb289-73b7-47c8-b819-4ac29b8ac242 None admin] RBAC: Authorizing `identity:get_user(user_id=20f19629679940f0b5f79bf0614b5da6)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:42.218435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bb289-73b7-47c8-b819-4ac29b8ac242 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:42.220557 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 87/175] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:41 2022] GET /identity/v3/users/20f19629679940f0b5f79bf0614b5da6 => generated 275 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:42.221856 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e066-f6bb-4fc3-81b1-88a5fcbc41b3 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.222227 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e066-f6bb-4fc3-81b1-88a5fcbc41b3 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.222570 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e066-f6bb-4fc3-81b1-88a5fcbc41b3 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.223872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 88/176] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:42.228677 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984e066-f6bb-4fc3-81b1-88a5fcbc41b3 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:42.303786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663de46-4ffc-44ed-94de-607b471c78d9 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:42.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:42.344871 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff0bfe-16cb-4cf3-94f9-554312be9d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:42.345750 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff0bfe-16cb-4cf3-94f9-554312be9d05 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.346149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff0bfe-16cb-4cf3-94f9-554312be9d05 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.346362 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984e066-f6bb-4fc3-81b1-88a5fcbc41b3 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:42.346613 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff0bfe-16cb-4cf3-94f9-554312be9d05 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.348092 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff0bfe-16cb-4cf3-94f9-554312be9d05 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:42.350799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff0bfe-16cb-4cf3-94f9-554312be9d05 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:42.352931 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 89/177] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:42 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:42.354390 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4e54-4904-40f2-8644-50a616f575d4 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.354793 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4e54-4904-40f2-8644-50a616f575d4 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.355164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4e54-4904-40f2-8644-50a616f575d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.377406 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-726d4e54-4904-40f2-8644-50a616f575d4 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:42.379015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-726d4e54-4904-40f2-8644-50a616f575d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:42.399832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:42.400617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.400903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.401446 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.402705 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] RBAC: Adding query filter params (role.id=8d072b58656b442f922c3287a5430603, scope.project.id=cfd603839dbc47efa27c6534b49c18c1, user.id=20f19629679940f0b5f79bf0614b5da6) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:42.403021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:42.404521 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:42.414183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 89/178] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:42 2022] GET /identity/v3/role_assignments?user.id=20f19629679940f0b5f79bf0614b5da6&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=8d072b58656b442f922c3287a5430603 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:42.415202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:42.477189 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 90/179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:42.478713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667ea6f-e7a5-46b7-b993-88a8ddea227d None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.479073 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667ea6f-e7a5-46b7-b993-88a8ddea227d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.479413 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667ea6f-e7a5-46b7-b993-88a8ddea227d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.484563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:42.510156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-2667ea6f-e7a5-46b7-b993-88a8ddea227d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:42.511849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-2667ea6f-e7a5-46b7-b993-88a8ddea227d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:42.571241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0118ea5-d634-4d16-85ac-5a02bd888d56 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:42.590367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 91/180] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:42.592503 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667ea6f-e7a5-46b7-b993-88a8ddea227d None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:42.616777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9d73-a64f-47af-9486-ea81813a083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:42.617337 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9d73-a64f-47af-9486-ea81813a083b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.617666 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9d73-a64f-47af-9486-ea81813a083b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.617946 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9d73-a64f-47af-9486-ea81813a083b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.657644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:42.723622 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9d73-a64f-47af-9486-ea81813a083b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:42.725244 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9d73-a64f-47af-9486-ea81813a083b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:42.736219 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667ea6f-e7a5-46b7-b993-88a8ddea227d None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:42.772916 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:42.778056 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92fd85-c16a-4369-b6b5-9f6dca413073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:42.778746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fd85-c16a-4369-b6b5-9f6dca413073 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.779030 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fd85-c16a-4369-b6b5-9f6dca413073 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.779309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fd85-c16a-4369-b6b5-9f6dca413073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.780138 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92fd85-c16a-4369-b6b5-9f6dca413073 None admin] RBAC: Adding query filter params (role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=cfd603839dbc47efa27c6534b49c18c1, user.id=20f19629679940f0b5f79bf0614b5da6) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:42.780452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92fd85-c16a-4369-b6b5-9f6dca413073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:42.781932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92fd85-c16a-4369-b6b5-9f6dca413073 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:42.788579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 92/181] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:42 2022] GET /identity/v3/role_assignments?user.id=20f19629679940f0b5f79bf0614b5da6&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:42.788579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195aeea-cc45-42ce-a259-f9ec8e3c0328 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.788579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195aeea-cc45-42ce-a259-f9ec8e3c0328 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.788579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195aeea-cc45-42ce-a259-f9ec8e3c0328 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.788579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 93/182] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:42.792462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195aeea-cc45-42ce-a259-f9ec8e3c0328 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:42.848719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 90/183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:42.849984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9d73-a64f-47af-9486-ea81813a083b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:42.935636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195aeea-cc45-42ce-a259-f9ec8e3c0328 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:42.947851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:42.976224 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f59fd5-802f-4cbb-b758-39096bb3f6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:42.976906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f59fd5-802f-4cbb-b758-39096bb3f6fb None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.977188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f59fd5-802f-4cbb-b758-39096bb3f6fb None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.977468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f59fd5-802f-4cbb-b758-39096bb3f6fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:42.983073 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9d73-a64f-47af-9486-ea81813a083b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:42.990642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f59fd5-802f-4cbb-b758-39096bb3f6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-38f59fd5-802f-4cbb-b758-39096bb3f6fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:42.995756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:42.996850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:42.997872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 94/184] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:42.999477 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676a149-9316-4877-91e2-9ad54b464fbf None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:42.999477 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676a149-9316-4877-91e2-9ad54b464fbf None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:42.999689 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676a149-9316-4877-91e2-9ad54b464fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.020238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-d676a149-9316-4877-91e2-9ad54b464fbf None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:43.021059 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-d676a149-9316-4877-91e2-9ad54b464fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:43.023987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfc787-62ce-422f-943f-9add01bca00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.024657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc787-62ce-422f-943f-9add01bca00e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.024937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc787-62ce-422f-943f-9add01bca00e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.025213 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc787-62ce-422f-943f-9add01bca00e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.038917 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfc787-62ce-422f-943f-9add01bca00e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-47cfc787-62ce-422f-943f-9add01bca00e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:43.044031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:43.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 91/185] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:43.046142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfc787-62ce-422f-943f-9add01bca00e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.056305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:43.094501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 95/186] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:43.095804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a6309-b74d-40ac-a879-d4328c979a67 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.096092 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a6309-b74d-40ac-a879-d4328c979a67 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.096363 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a6309-b74d-40ac-a879-d4328c979a67 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.097681 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 96/187] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:43.102654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a6309-b74d-40ac-a879-d4328c979a67 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.155738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfc787-62ce-422f-943f-9add01bca00e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.190736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e4064-85f4-4942-bc76-5119ff6fd6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.191406 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e4064-85f4-4942-bc76-5119ff6fd6e6 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.191659 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e4064-85f4-4942-bc76-5119ff6fd6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.191935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e4064-85f4-4942-bc76-5119ff6fd6e6 None admin] PATH_INFO: `/v3/projects/c1863fa57878420392969f0e6a8fe6b2` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.193137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e4064-85f4-4942-bc76-5119ff6fd6e6 None admin] RBAC: Authorizing `identity:get_project(project_id=c1863fa57878420392969f0e6a8fe6b2)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:43.195012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e4064-85f4-4942-bc76-5119ff6fd6e6 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:43.196719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 92/188] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/projects/c1863fa57878420392969f0e6a8fe6b2 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:43.197490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e4064-85f4-4942-bc76-5119ff6fd6e6 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.205313 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:43.210695 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a6309-b74d-40ac-a879-d4328c979a67 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.245170 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5637c-2ae6-49ab-a478-977bb324be83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.245816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5637c-2ae6-49ab-a478-977bb324be83 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.246097 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5637c-2ae6-49ab-a478-977bb324be83 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.246406 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5637c-2ae6-49ab-a478-977bb324be83 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.247136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5637c-2ae6-49ab-a478-977bb324be83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:43.247445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5637c-2ae6-49ab-a478-977bb324be83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:43.248856 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5637c-2ae6-49ab-a478-977bb324be83 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:43.257463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 97/189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:43.258729 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350ab8a-249a-475e-b082-4b2ec2707b0c None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.259018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350ab8a-249a-475e-b082-4b2ec2707b0c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.259299 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350ab8a-249a-475e-b082-4b2ec2707b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.276869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-0350ab8a-249a-475e-b082-4b2ec2707b0c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:43.278084 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-0350ab8a-249a-475e-b082-4b2ec2707b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:43.278269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:43.309446 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e4064-85f4-4942-bc76-5119ff6fd6e6 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.343346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.343978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.344274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.344539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.345258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] RBAC: Adding query filter params (name=reader) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:43.345558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:43.346906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 98/190] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:43.347357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:43.347782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350ab8a-249a-475e-b082-4b2ec2707b0c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.356128 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 93/191] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:43.358369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.454308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:43.465097 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350ab8a-249a-475e-b082-4b2ec2707b0c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.468501 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dd65c-c001-4015-8885-49022eb2396e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.500349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35540f-6eac-4de7-aa79-283db34cd822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.501019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35540f-6eac-4de7-aa79-283db34cd822 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.501305 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35540f-6eac-4de7-aa79-283db34cd822 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.501579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35540f-6eac-4de7-aa79-283db34cd822 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.505311 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.505597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.505874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.506792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] RBAC: Adding query filter params (role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=c1863fa57878420392969f0e6a8fe6b2, user.id=20f19629679940f0b5f79bf0614b5da6) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:43.507111 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:43.508602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:43.510540 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:43.513769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35540f-6eac-4de7-aa79-283db34cd822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-7a35540f-6eac-4de7-aa79-283db34cd822 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:43.519399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:43.520711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:43.521778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:43.522504 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 94/192] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/role_assignments?user.id=20f19629679940f0b5f79bf0614b5da6&scope.project.id=c1863fa57878420392969f0e6a8fe6b2&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:43.522592 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 99/193] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:43.522676 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.522754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35540f-6eac-4de7-aa79-283db34cd822 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.655994 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35540f-6eac-4de7-aa79-283db34cd822 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.676433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3ba09-246b-4298-810e-6fda5e0e1371 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.703414 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910e321-a801-49c4-ae66-24ad529afbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.704809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910e321-a801-49c4-ae66-24ad529afbbc None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.705092 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910e321-a801-49c4-ae66-24ad529afbbc None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.705368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910e321-a801-49c4-ae66-24ad529afbbc None admin] PATH_INFO: `/v3/users/37a45369145140c180c0a012ee3a84bb` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.707960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910e321-a801-49c4-ae66-24ad529afbbc None admin] RBAC: Authorizing `identity:get_user(user_id=37a45369145140c180c0a012ee3a84bb)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:43.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910e321-a801-49c4-ae66-24ad529afbbc None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:43.710296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 100/194] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/users/37a45369145140c180c0a012ee3a84bb => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:43.715151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910e321-a801-49c4-ae66-24ad529afbbc None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.723692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e274ff-a859-4a18-96eb-328fd12dbcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.724340 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e274ff-a859-4a18-96eb-328fd12dbcf3 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.724619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e274ff-a859-4a18-96eb-328fd12dbcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.724975 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e274ff-a859-4a18-96eb-328fd12dbcf3 None admin] PATH_INFO: `/v3/users/e5b4629902714fb3b29af6f40b73e825` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.741074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:43.742957 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e274ff-a859-4a18-96eb-328fd12dbcf3 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b4629902714fb3b29af6f40b73e825)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:43.744538 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e274ff-a859-4a18-96eb-328fd12dbcf3 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:43.747099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 95/195] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/users/e5b4629902714fb3b29af6f40b73e825 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:43.750174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e274ff-a859-4a18-96eb-328fd12dbcf3 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.817538 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:43.842255 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910e321-a801-49c4-ae66-24ad529afbbc None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.866642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e274ff-a859-4a18-96eb-328fd12dbcf3 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:43.880625 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a04f0-63ce-4e23-a9eb-00b61088a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.881369 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a04f0-63ce-4e23-a9eb-00b61088a329 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.881654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a04f0-63ce-4e23-a9eb-00b61088a329 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.881934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a04f0-63ce-4e23-a9eb-00b61088a329 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.883250 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a04f0-63ce-4e23-a9eb-00b61088a329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:43.884775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a04f0-63ce-4e23-a9eb-00b61088a329 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:43.886524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 101/196] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:43.887328 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a04f0-63ce-4e23-a9eb-00b61088a329 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.908783 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:43.909347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:43.909862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:43.909862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:43.910656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:43.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:43.912445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:43.921741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 96/197] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:43.923277 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:43.981821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:43.998135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a04f0-63ce-4e23-a9eb-00b61088a329 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.035942 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bef7b3-86f4-458a-ba1d-34b75b87e468 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.040748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e7858-4e58-44b8-a743-c7b04e8acb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.040924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e7858-4e58-44b8-a743-c7b04e8acb36 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.041063 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e7858-4e58-44b8-a743-c7b04e8acb36 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.041340 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e7858-4e58-44b8-a743-c7b04e8acb36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.054867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:44.075539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a5f69-2cc2-4332-bc1d-f48e043f6c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.076158 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a5f69-2cc2-4332-bc1d-f48e043f6c5b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.076460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a5f69-2cc2-4332-bc1d-f48e043f6c5b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.076727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a5f69-2cc2-4332-bc1d-f48e043f6c5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.115337 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e7858-4e58-44b8-a743-c7b04e8acb36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.116846 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e7858-4e58-44b8-a743-c7b04e8acb36 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.147978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a5f69-2cc2-4332-bc1d-f48e043f6c5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.149788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a5f69-2cc2-4332-bc1d-f48e043f6c5b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.201871 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:44.218333 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 102/198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:44.219173 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e7858-4e58-44b8-a743-c7b04e8acb36 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:44.251536 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 97/199] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:44.252413 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a5f69-2cc2-4332-bc1d-f48e043f6c5b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:44.274330 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:44.327295 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e7858-4e58-44b8-a743-c7b04e8acb36 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.358004 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a5f69-2cc2-4332-bc1d-f48e043f6c5b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.362163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4baa76-e009-4ef3-a0ae-68cb89b1bbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.362834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4baa76-e009-4ef3-a0ae-68cb89b1bbcc None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.363115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4baa76-e009-4ef3-a0ae-68cb89b1bbcc None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.363396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4baa76-e009-4ef3-a0ae-68cb89b1bbcc None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.364104 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4baa76-e009-4ef3-a0ae-68cb89b1bbcc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.365580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4baa76-e009-4ef3-a0ae-68cb89b1bbcc None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.388340 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 103/200] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:38:44 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:44.389195 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4baa76-e009-4ef3-a0ae-68cb89b1bbcc None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:44.397901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9384f-34c2-4932-9056-8e92d4081b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.398625 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9384f-34c2-4932-9056-8e92d4081b52 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.398911 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9384f-34c2-4932-9056-8e92d4081b52 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.399206 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9384f-34c2-4932-9056-8e92d4081b52 None admin] PATH_INFO: `/v3/users/e5b4629902714fb3b29af6f40b73e825` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.400493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9384f-34c2-4932-9056-8e92d4081b52 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b4629902714fb3b29af6f40b73e825)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.402019 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9384f-34c2-4932-9056-8e92d4081b52 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.403970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 98/201] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:44 2022] GET /identity/v3/users/e5b4629902714fb3b29af6f40b73e825 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:44.404830 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9384f-34c2-4932-9056-8e92d4081b52 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:44.462828 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:44.504762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:44.524494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9384f-34c2-4932-9056-8e92d4081b52 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.531000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4baa76-e009-4ef3-a0ae-68cb89b1bbcc None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.565622 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4b9b6-156b-45c6-8028-89376f54dbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.566361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b9b6-156b-45c6-8028-89376f54dbb5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.566651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b9b6-156b-45c6-8028-89376f54dbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.566933 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b9b6-156b-45c6-8028-89376f54dbb5 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.568181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4b9b6-156b-45c6-8028-89376f54dbb5 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.570078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4b9b6-156b-45c6-8028-89376f54dbb5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.573574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 99/202] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:44 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:44.573574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4b9b6-156b-45c6-8028-89376f54dbb5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:44.575617 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcabae-dec7-4be9-9ba0-861edbcb7865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.575617 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcabae-dec7-4be9-9ba0-861edbcb7865 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.575914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcabae-dec7-4be9-9ba0-861edbcb7865 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.576105 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcabae-dec7-4be9-9ba0-861edbcb7865 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.577282 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcabae-dec7-4be9-9ba0-861edbcb7865 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.579943 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcabae-dec7-4be9-9ba0-861edbcb7865 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.580509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 104/203] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:44 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:44.583348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcabae-dec7-4be9-9ba0-861edbcb7865 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:44.710213 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4b9b6-156b-45c6-8028-89376f54dbb5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.714198 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcabae-dec7-4be9-9ba0-861edbcb7865 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.734537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:44.752725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b93a3-5a53-42ff-9939-37741246c5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.753383 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b93a3-5a53-42ff-9939-37741246c5e5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.753653 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b93a3-5a53-42ff-9939-37741246c5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.753928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b93a3-5a53-42ff-9939-37741246c5e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.756795 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.756795 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.756795 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.757117 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.758790 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=713f2c71f07c4deeb316712af011e233, user.id=e5b4629902714fb3b29af6f40b73e825) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:44.758790 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.760529 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.773416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 105/204] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:44 2022] GET /identity/v3/role_assignments?user.id=e5b4629902714fb3b29af6f40b73e825&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:44.773416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:44.785752 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:44.867027 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b93a3-5a53-42ff-9939-37741246c5e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.868788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b93a3-5a53-42ff-9939-37741246c5e5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.930294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683434a5-31b6-48bb-bd04-f29698475543 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:44.970734 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:44.971337 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:44.971614 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:44.971896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:44.972896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] RBAC: Adding query filter params (role.id=9fc42581f1f84e28b6d65cdc76126fdc, scope.project.id=cfd603839dbc47efa27c6534b49c18c1, user.id=37a45369145140c180c0a012ee3a84bb) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:44.973275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:44.976162 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:44.983777 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 106/205] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:44 2022] GET /identity/v3/role_assignments?user.id=37a45369145140c180c0a012ee3a84bb&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=9fc42581f1f84e28b6d65cdc76126fdc => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:45.106509 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 100/206] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:45.112173 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:45.188344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:45.258560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49693c2-f45f-43cd-9ca5-851370c3cc3f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:45.304426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7073e-e209-43c9-b211-e1603aa8f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:45.305095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7073e-e209-43c9-b211-e1603aa8f086 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:45.305374 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7073e-e209-43c9-b211-e1603aa8f086 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:45.305647 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7073e-e209-43c9-b211-e1603aa8f086 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:45.306824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7073e-e209-43c9-b211-e1603aa8f086 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:45.308271 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7073e-e209-43c9-b211-e1603aa8f086 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:45.310294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 107/207] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:45 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:45.316355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b93a3-5a53-42ff-9939-37741246c5e5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:45.382059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:45.431676 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b93a3-5a53-42ff-9939-37741246c5e5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:45.471374 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e4cc5-48ea-40ff-84a3-a460771aa0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:45.472054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4cc5-48ea-40ff-84a3-a460771aa0f9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:45.472334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4cc5-48ea-40ff-84a3-a460771aa0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:45.472611 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4cc5-48ea-40ff-84a3-a460771aa0f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:45.473425 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e4cc5-48ea-40ff-84a3-a460771aa0f9 None admin] RBAC: Adding query filter params (role.id=8d072b58656b442f922c3287a5430603, scope.project.id=713f2c71f07c4deeb316712af011e233, user.id=e5b4629902714fb3b29af6f40b73e825) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:45.473750 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e4cc5-48ea-40ff-84a3-a460771aa0f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:45.475308 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e4cc5-48ea-40ff-84a3-a460771aa0f9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:45.481670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 101/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:45 2022] GET /identity/v3/role_assignments?user.id=e5b4629902714fb3b29af6f40b73e825&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=8d072b58656b442f922c3287a5430603 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:46.727464 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddeea5-ecc3-4807-b1c1-a85ce0322727 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:46.728482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddeea5-ecc3-4807-b1c1-a85ce0322727 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:46.729007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddeea5-ecc3-4807-b1c1-a85ce0322727 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:46.730986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 108/209] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:46.736909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11c06-eaee-4216-a2ad-16cd89d94638 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:46.737383 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11c06-eaee-4216-a2ad-16cd89d94638 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:46.737469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11c06-eaee-4216-a2ad-16cd89d94638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:46.758977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-aea11c06-eaee-4216-a2ad-16cd89d94638 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:46.760580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-aea11c06-eaee-4216-a2ad-16cd89d94638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:46.838328 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e939726-0cda-4654-a24f-3ce2b3fa6b3f None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:46.838706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e939726-0cda-4654-a24f-3ce2b3fa6b3f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:46.838985 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e939726-0cda-4654-a24f-3ce2b3fa6b3f None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:46.839393 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 102/210] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:46.840598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 109/211] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:46.845797 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f633fb-b0e0-4eab-9d0a-651d5dbe327d None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:46.846080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f633fb-b0e0-4eab-9d0a-651d5dbe327d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:46.846391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f633fb-b0e0-4eab-9d0a-651d5dbe327d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:46.865231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ce5fd-90c8-41f7-b432-f72b7344589e None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:46.865596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ce5fd-90c8-41f7-b432-f72b7344589e None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:46.865718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-a3f633fb-b0e0-4eab-9d0a-651d5dbe327d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:46.865809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ce5fd-90c8-41f7-b432-f72b7344589e None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:46.866791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-a3f633fb-b0e0-4eab-9d0a-651d5dbe327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:46.867753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 110/212] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:38:46.867956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ce5fd-90c8-41f7-b432-f72b7344589e None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:46.923335 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:46.950827 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 103/213] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:46.952506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2d5b4-021e-4bf6-a68d-a5f53803b969 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:46.952786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2d5b4-021e-4bf6-a68d-a5f53803b969 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:46.953060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2d5b4-021e-4bf6-a68d-a5f53803b969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:46.974349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-f5c2d5b4-021e-4bf6-a68d-a5f53803b969 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:46.975768 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-f5c2d5b4-021e-4bf6-a68d-a5f53803b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:46.995263 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ce5fd-90c8-41f7-b432-f72b7344589e None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.003421 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:47.050866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010da76-78e9-420a-90d4-498ecabe3fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.052007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010da76-78e9-420a-90d4-498ecabe3fba None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.052498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010da76-78e9-420a-90d4-498ecabe3fba None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.053131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010da76-78e9-420a-90d4-498ecabe3fba None admin] PATH_INFO: `/v3/roles/member` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.056597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 104/214] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:47.057294 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2d5b4-021e-4bf6-a68d-a5f53803b969 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:47.077051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010da76-78e9-420a-90d4-498ecabe3fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-b010da76-78e9-420a-90d4-498ecabe3fba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:47.082569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:47.083643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:47.084744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:47.087501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 111/215] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:47.094161 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010da76-78e9-420a-90d4-498ecabe3fba None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:47.172876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2d5b4-021e-4bf6-a68d-a5f53803b969 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.183837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:47.212022 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010da76-78e9-420a-90d4-498ecabe3fba None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.217278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0721b72-e9b0-4871-b222-2e0c7da6ef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.218260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721b72-e9b0-4871-b222-2e0c7da6ef56 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.218748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721b72-e9b0-4871-b222-2e0c7da6ef56 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.219202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721b72-e9b0-4871-b222-2e0c7da6ef56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.220716 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0721b72-e9b0-4871-b222-2e0c7da6ef56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:47.222566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0721b72-e9b0-4871-b222-2e0c7da6ef56 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:47.226167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 105/216] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:47.226167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0721b72-e9b0-4871-b222-2e0c7da6ef56 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:47.258860 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.260128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.260416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.260789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.261664 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:47.262016 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:47.263625 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:47.278533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 112/217] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:47.278533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:47.308836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:47.346703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0721b72-e9b0-4871-b222-2e0c7da6ef56 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.388837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9dc1-c9d3-491e-8cfc-e5bfaeb5f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.389697 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9dc1-c9d3-491e-8cfc-e5bfaeb5f306 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.389979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9dc1-c9d3-491e-8cfc-e5bfaeb5f306 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.390289 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9dc1-c9d3-491e-8cfc-e5bfaeb5f306 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.398250 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3617f5-52d5-466c-b04b-e2001302936c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.403846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d9dc1-c9d3-491e-8cfc-e5bfaeb5f306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-aa7d9dc1-c9d3-491e-8cfc-e5bfaeb5f306 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:47.410038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:47.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 106/218] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:47.413173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9dc1-c9d3-491e-8cfc-e5bfaeb5f306 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:47.442295 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf90669-c14d-4d17-89e3-ab573970530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.443342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf90669-c14d-4d17-89e3-ab573970530a None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.443869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf90669-c14d-4d17-89e3-ab573970530a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.444694 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf90669-c14d-4d17-89e3-ab573970530a None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.445420 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf90669-c14d-4d17-89e3-ab573970530a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:47.446981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf90669-c14d-4d17-89e3-ab573970530a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:47.457789 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:47.478021 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 113/219] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:38:47 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:47.479347 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf90669-c14d-4d17-89e3-ab573970530a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:47.555201 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9dc1-c9d3-491e-8cfc-e5bfaeb5f306 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.602713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4e4e7-5793-475d-b81c-48537502b810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.603403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4e4e7-5793-475d-b81c-48537502b810 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.603690 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4e4e7-5793-475d-b81c-48537502b810 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.603973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4e4e7-5793-475d-b81c-48537502b810 None admin] PATH_INFO: `/v3/users/a631906c67ac4c648bbd2c05e9f9d658` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.618680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:47.620893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4e4e7-5793-475d-b81c-48537502b810 None admin] RBAC: Authorizing `identity:get_user(user_id=a631906c67ac4c648bbd2c05e9f9d658)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:47.622561 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4e4e7-5793-475d-b81c-48537502b810 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:47.631408 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 107/220] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:47 2022] GET /identity/v3/users/a631906c67ac4c648bbd2c05e9f9d658 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:47.631408 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823d34-b20f-4eb0-8b3f-be42a81d7691 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.631408 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823d34-b20f-4eb0-8b3f-be42a81d7691 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.631408 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823d34-b20f-4eb0-8b3f-be42a81d7691 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.636150 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 108/221] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:47.636150 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81823d34-b20f-4eb0-8b3f-be42a81d7691 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:47.639987 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf90669-c14d-4d17-89e3-ab573970530a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.684516 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3b7e6-ee66-4644-aab3-db6e444d9411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.685613 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3b7e6-ee66-4644-aab3-db6e444d9411 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.686133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3b7e6-ee66-4644-aab3-db6e444d9411 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.686429 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3b7e6-ee66-4644-aab3-db6e444d9411 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.687606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3b7e6-ee66-4644-aab3-db6e444d9411 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:47.687606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3b7e6-ee66-4644-aab3-db6e444d9411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:47.688918 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3b7e6-ee66-4644-aab3-db6e444d9411 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:47.705062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 114/222] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:47.705062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3e263-9d0d-4e42-a8bc-3c9919a6b89a None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.705062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3e263-9d0d-4e42-a8bc-3c9919a6b89a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.705062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3e263-9d0d-4e42-a8bc-3c9919a6b89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.721533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-86b3e263-9d0d-4e42-a8bc-3c9919a6b89a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:47.722818 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-86b3e263-9d0d-4e42-a8bc-3c9919a6b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:47.788373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:47.798170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81823d34-b20f-4eb0-8b3f-be42a81d7691 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.816342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 115/223] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:47.817241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3e263-9d0d-4e42-a8bc-3c9919a6b89a None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:47.844158 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef9f4e-9684-4209-b880-6aaa93f0320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.844828 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef9f4e-9684-4209-b880-6aaa93f0320b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.845107 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef9f4e-9684-4209-b880-6aaa93f0320b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.845378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef9f4e-9684-4209-b880-6aaa93f0320b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.920638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef9f4e-9684-4209-b880-6aaa93f0320b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:47.922393 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef9f4e-9684-4209-b880-6aaa93f0320b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:47.924181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:47.957279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3e263-9d0d-4e42-a8bc-3c9919a6b89a None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:47.995585 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763a981-179b-4602-8b0c-9f79b7f14026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:47.996697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763a981-179b-4602-8b0c-9f79b7f14026 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:47.997262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763a981-179b-4602-8b0c-9f79b7f14026 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:47.997754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763a981-179b-4602-8b0c-9f79b7f14026 None admin] PATH_INFO: `/v3/users/e5b4629902714fb3b29af6f40b73e825` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:47.999320 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763a981-179b-4602-8b0c-9f79b7f14026 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b4629902714fb3b29af6f40b73e825)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:48.001076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763a981-179b-4602-8b0c-9f79b7f14026 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:48.003055 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 116/224] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:47 2022] GET /identity/v3/users/e5b4629902714fb3b29af6f40b73e825 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:48.008575 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763a981-179b-4602-8b0c-9f79b7f14026 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:48.028376 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 109/225] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:48.028708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:48.033924 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef9f4e-9684-4209-b880-6aaa93f0320b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:48.124173 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763a981-179b-4602-8b0c-9f79b7f14026 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:48.137774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef9f4e-9684-4209-b880-6aaa93f0320b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:48.166326 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439a4b0-7ab8-48b4-9c10-b9e3dc7edb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:48.167394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a4b0-7ab8-48b4-9c10-b9e3dc7edb3f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:48.167897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a4b0-7ab8-48b4-9c10-b9e3dc7edb3f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:48.168460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a4b0-7ab8-48b4-9c10-b9e3dc7edb3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:48.180016 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43dab-16f6-49e4-a07f-124a9f023827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:48.180845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43dab-16f6-49e4-a07f-124a9f023827 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:48.181244 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439a4b0-7ab8-48b4-9c10-b9e3dc7edb3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:48.182346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43dab-16f6-49e4-a07f-124a9f023827 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:48.182892 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43dab-16f6-49e4-a07f-124a9f023827 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-4439a4b0-7ab8-48b4-9c10-b9e3dc7edb3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:48.187087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:48.188156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 117/226] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:38:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:48.189788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439a4b0-7ab8-48b4-9c10-b9e3dc7edb3f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:48.191590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c43dab-16f6-49e4-a07f-124a9f023827 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:48.193600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c43dab-16f6-49e4-a07f-124a9f023827 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:48.196274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 110/227] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:48 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:48.197256 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43dab-16f6-49e4-a07f-124a9f023827 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:48.205639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:48.303197 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439a4b0-7ab8-48b4-9c10-b9e3dc7edb3f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:48.314556 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:48.338896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43dab-16f6-49e4-a07f-124a9f023827 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:48.339355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412997df-c946-4e30-8a67-f7131a6ffb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:48.340073 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412997df-c946-4e30-8a67-f7131a6ffb36 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:48.340353 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412997df-c946-4e30-8a67-f7131a6ffb36 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:48.340624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412997df-c946-4e30-8a67-f7131a6ffb36 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:48.341811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412997df-c946-4e30-8a67-f7131a6ffb36 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:48.343497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412997df-c946-4e30-8a67-f7131a6ffb36 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:48.350159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 118/228] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:48 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:48.350159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412997df-c946-4e30-8a67-f7131a6ffb36 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:48.395526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:48.397023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:48.397505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:48.398381 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:48.399613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:48.400135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:48.402520 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:48.420006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 111/229] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:38:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:48.420006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:48.433152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:48.469386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412997df-c946-4e30-8a67-f7131a6ffb36 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:48.507152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff8e05-5823-4d98-8265-e39828ffe78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:48.507976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff8e05-5823-4d98-8265-e39828ffe78f None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:48.508264 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff8e05-5823-4d98-8265-e39828ffe78f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:48.508591 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff8e05-5823-4d98-8265-e39828ffe78f None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233/users/e5b4629902714fb3b29af6f40b73e825/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:48.511817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff8e05-5823-4d98-8265-e39828ffe78f None admin] RBAC: Authorizing `identity:create_grant(project_id=713f2c71f07c4deeb316712af011e233, user_id=e5b4629902714fb3b29af6f40b73e825, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:48.513940 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff8e05-5823-4d98-8265-e39828ffe78f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:48.531149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 119/230] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:48 2022] PUT /identity/v3/projects/713f2c71f07c4deeb316712af011e233/users/e5b4629902714fb3b29af6f40b73e825/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:48.532209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff8e05-5823-4d98-8265-e39828ffe78f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:48.554215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b40604-cc21-452e-9fd6-76f9c78b305c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:48.580820 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:48.593403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:48.594468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:48.594990 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:48.595503 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:48.596552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] RBAC: Adding query filter params (role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=713f2c71f07c4deeb316712af011e233, user.id=a631906c67ac4c648bbd2c05e9f9d658) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:48.597079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:48.598860 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:48.606007 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 112/231] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:48 2022] GET /identity/v3/role_assignments?user.id=a631906c67ac4c648bbd2c05e9f9d658&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:48.672506 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff8e05-5823-4d98-8265-e39828ffe78f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:48.726593 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:48.734876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d42e7-229f-4d13-a4d1-78077b32ec68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:48.734876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d42e7-229f-4d13-a4d1-78077b32ec68 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:48.734876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d42e7-229f-4d13-a4d1-78077b32ec68 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:48.734876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d42e7-229f-4d13-a4d1-78077b32ec68 None admin] PATH_INFO: `/v3/users/e5b4629902714fb3b29af6f40b73e825` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:48.738650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d42e7-229f-4d13-a4d1-78077b32ec68 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b4629902714fb3b29af6f40b73e825)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:48.740379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d42e7-229f-4d13-a4d1-78077b32ec68 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:48.742141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 120/232] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:48 2022] GET /identity/v3/users/e5b4629902714fb3b29af6f40b73e825 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:48.750187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:48.889543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda72f4-bb53-4366-8e66-871a415a7333 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:48.938620 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab06300-5a0c-4294-b4a5-d93bc94cc170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:48.939729 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab06300-5a0c-4294-b4a5-d93bc94cc170 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:48.940290 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab06300-5a0c-4294-b4a5-d93bc94cc170 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:48.940857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab06300-5a0c-4294-b4a5-d93bc94cc170 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:48.947000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab06300-5a0c-4294-b4a5-d93bc94cc170 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:48.948843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab06300-5a0c-4294-b4a5-d93bc94cc170 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:48.951010 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 113/233] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:48 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:48.956902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d42e7-229f-4d13-a4d1-78077b32ec68 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:49.072988 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d42e7-229f-4d13-a4d1-78077b32ec68 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:49.110947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fab16-9ede-485a-b21d-749d37b93454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:49.111648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fab16-9ede-485a-b21d-749d37b93454 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:49.111928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fab16-9ede-485a-b21d-749d37b93454 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:49.112200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fab16-9ede-485a-b21d-749d37b93454 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233/users/e5b4629902714fb3b29af6f40b73e825/roles/8d072b58656b442f922c3287a5430603` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:49.114478 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fab16-9ede-485a-b21d-749d37b93454 None admin] RBAC: Authorizing `identity:create_grant(project_id=713f2c71f07c4deeb316712af011e233, user_id=e5b4629902714fb3b29af6f40b73e825, role_id=8d072b58656b442f922c3287a5430603)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:49.116066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fab16-9ede-485a-b21d-749d37b93454 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:49.131612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 121/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:48 2022] PUT /identity/v3/projects/713f2c71f07c4deeb316712af011e233/users/e5b4629902714fb3b29af6f40b73e825/roles/8d072b58656b442f922c3287a5430603 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:49.848060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d055a8-f38e-4bbb-bf4d-22c434b33329 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:49.848631 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d055a8-f38e-4bbb-bf4d-22c434b33329 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:49.849104 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d055a8-f38e-4bbb-bf4d-22c434b33329 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:49.851038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 114/235] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:49.863907 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13ab7-a70e-4f8c-acd6-816fadc2556a None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:49.864447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13ab7-a70e-4f8c-acd6-816fadc2556a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:49.864910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13ab7-a70e-4f8c-acd6-816fadc2556a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:49.887806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-60d13ab7-a70e-4f8c-acd6-816fadc2556a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:49.889297 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-60d13ab7-a70e-4f8c-acd6-816fadc2556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:49.964226 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:49.982449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 122/236] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:50.013306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d055a8-f38e-4bbb-bf4d-22c434b33329 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:50.048302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8008d3-7cb8-4a51-86fc-cd4bbec90e0b None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:50.048893 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8008d3-7cb8-4a51-86fc-cd4bbec90e0b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:50.049334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8008d3-7cb8-4a51-86fc-cd4bbec90e0b None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:50.051215 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 123/237] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:50.057553 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a35fd-b117-4ea3-aebe-28d8a7df8d3c None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:50.058093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a35fd-b117-4ea3-aebe-28d8a7df8d3c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:50.058578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a35fd-b117-4ea3-aebe-28d8a7df8d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:50.083059 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-851a35fd-b117-4ea3-aebe-28d8a7df8d3c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:50.083059 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-851a35fd-b117-4ea3-aebe-28d8a7df8d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:50.126310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:50.167077 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 124/238] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:50.198599 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a35fd-b117-4ea3-aebe-28d8a7df8d3c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:50.228309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d055a8-f38e-4bbb-bf4d-22c434b33329 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:50.305912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbbe1b-b654-4c80-9ed0-5cd8e8cb63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:50.306618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbbe1b-b654-4c80-9ed0-5cd8e8cb63f5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:50.306904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbbe1b-b654-4c80-9ed0-5cd8e8cb63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:50.307174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbbe1b-b654-4c80-9ed0-5cd8e8cb63f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:50.330078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbbe1b-b654-4c80-9ed0-5cd8e8cb63f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-4cbbbe1b-b654-4c80-9ed0-5cd8e8cb63f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:50.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:50.343778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:50.344808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:50.348261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 115/239] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:50.353234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbbe1b-b654-4c80-9ed0-5cd8e8cb63f5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:50.410823 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a35fd-b117-4ea3-aebe-28d8a7df8d3c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:50.449798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:50.462893 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c1e7e-c5bc-477c-b78f-e3efa022035a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:50.463582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c1e7e-c5bc-477c-b78f-e3efa022035a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:50.463862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c1e7e-c5bc-477c-b78f-e3efa022035a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:50.464132 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c1e7e-c5bc-477c-b78f-e3efa022035a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:50.471239 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c1e7e-c5bc-477c-b78f-e3efa022035a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:50.472915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c1e7e-c5bc-477c-b78f-e3efa022035a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:50.475166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 125/240] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:50.481168 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c1e7e-c5bc-477c-b78f-e3efa022035a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:50.516124 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbbe1b-b654-4c80-9ed0-5cd8e8cb63f5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:50.555522 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:50.570512 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:50.570512 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:50.570858 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:50.571117 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:50.572043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:50.572339 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:50.573928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:50.585117 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 116/241] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:50.590161 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:50.697195 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e4d0a-e9da-4b49-8217-cd6d090942a0 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:50.712344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c1e7e-c5bc-477c-b78f-e3efa022035a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:50.751205 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0261a3-39c3-4db5-8983-315479fefb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:50.752259 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0261a3-39c3-4db5-8983-315479fefb59 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:50.752750 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0261a3-39c3-4db5-8983-315479fefb59 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:50.753230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0261a3-39c3-4db5-8983-315479fefb59 None admin] PATH_INFO: `/v3/users/cff3683b33184675ae950718d2c6aea2` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:50.764016 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180ac6e-8377-49b0-9c94-a61be8a8851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:50.764675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ac6e-8377-49b0-9c94-a61be8a8851a None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:50.764957 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ac6e-8377-49b0-9c94-a61be8a8851a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:50.765234 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ac6e-8377-49b0-9c94-a61be8a8851a None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:50.765947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180ac6e-8377-49b0-9c94-a61be8a8851a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:50.767731 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180ac6e-8377-49b0-9c94-a61be8a8851a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:50.770772 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0261a3-39c3-4db5-8983-315479fefb59 None admin] RBAC: Authorizing `identity:get_user(user_id=cff3683b33184675ae950718d2c6aea2)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:50.772837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0261a3-39c3-4db5-8983-315479fefb59 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:50.775351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 117/242] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:50 2022] GET /identity/v3/users/cff3683b33184675ae950718d2c6aea2 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:50.781439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0261a3-39c3-4db5-8983-315479fefb59 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:50.817434 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 126/243] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:38:50 2022] POST /identity/v3/users => generated 293 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:38:50.822919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:50.898625 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0261a3-39c3-4db5-8983-315479fefb59 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:50.937493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4f771-7883-48d2-a717-cb7187fcec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:50.938579 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f771-7883-48d2-a717-cb7187fcec96 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:50.939082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f771-7883-48d2-a717-cb7187fcec96 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:50.939573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f771-7883-48d2-a717-cb7187fcec96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.016481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4f771-7883-48d2-a717-cb7187fcec96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:51.019054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4f771-7883-48d2-a717-cb7187fcec96 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:51.056010 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:51.129912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 118/244] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:38:51.136104 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180ac6e-8377-49b0-9c94-a61be8a8851a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:51.159564 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d594-d35e-471f-8686-dd575db5ff29 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:51.159814 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d594-d35e-471f-8686-dd575db5ff29 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:51.160069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d594-d35e-471f-8686-dd575db5ff29 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.161770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 119/245] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:51.175110 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21425f-acd4-43cb-9baa-05f8d04319bf None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:51.175746 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21425f-acd4-43cb-9baa-05f8d04319bf None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:51.176382 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21425f-acd4-43cb-9baa-05f8d04319bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.196837 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:51.202640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-fb21425f-acd4-43cb-9baa-05f8d04319bf None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:51.204155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-fb21425f-acd4-43cb-9baa-05f8d04319bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:51.262388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180ac6e-8377-49b0-9c94-a61be8a8851a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:51.300885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 120/246] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:51.312266 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e0207-d598-4a02-b791-aae26f9b991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:51.313308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e0207-d598-4a02-b791-aae26f9b991f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:51.313799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e0207-d598-4a02-b791-aae26f9b991f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:51.314306 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e0207-d598-4a02-b791-aae26f9b991f None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.315781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e0207-d598-4a02-b791-aae26f9b991f None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:51.317486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e0207-d598-4a02-b791-aae26f9b991f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:51.319941 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 127/247] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:51 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:51.326983 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21425f-acd4-43cb-9baa-05f8d04319bf None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:51.332394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e0207-d598-4a02-b791-aae26f9b991f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:51.403699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:51.448927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21425f-acd4-43cb-9baa-05f8d04319bf None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:51.492226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152e63d-bff9-454f-8b34-98085cd4cb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:51.492755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e0207-d598-4a02-b791-aae26f9b991f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:51.492890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152e63d-bff9-454f-8b34-98085cd4cb02 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:51.493142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152e63d-bff9-454f-8b34-98085cd4cb02 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:51.493417 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152e63d-bff9-454f-8b34-98085cd4cb02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.495151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152e63d-bff9-454f-8b34-98085cd4cb02 None admin] RBAC: Adding query filter params (role.id=9fc42581f1f84e28b6d65cdc76126fdc, scope.project.id=713f2c71f07c4deeb316712af011e233, user.id=cff3683b33184675ae950718d2c6aea2) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:51.495480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152e63d-bff9-454f-8b34-98085cd4cb02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:51.496977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152e63d-bff9-454f-8b34-98085cd4cb02 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:51.505125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 121/248] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:51 2022] GET /identity/v3/role_assignments?user.id=cff3683b33184675ae950718d2c6aea2&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=9fc42581f1f84e28b6d65cdc76126fdc => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:51.506855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f326-5f88-4efb-995f-b8014a22e619 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:51.507319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f326-5f88-4efb-995f-b8014a22e619 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:51.507773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f326-5f88-4efb-995f-b8014a22e619 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.509297 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 122/249] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:51.514537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:51.516542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7b9ab-48c3-4716-9959-c72015d372f1 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:51.516833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7b9ab-48c3-4716-9959-c72015d372f1 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:51.517143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7b9ab-48c3-4716-9959-c72015d372f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.540692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-5dd7b9ab-48c3-4716-9959-c72015d372f1 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:51.542740 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-5dd7b9ab-48c3-4716-9959-c72015d372f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:51.548741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f99c77-b486-470c-b36c-cb68b06069a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:51.549989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99c77-b486-470c-b36c-cb68b06069a5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:51.550545 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99c77-b486-470c-b36c-cb68b06069a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:51.551114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99c77-b486-470c-b36c-cb68b06069a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.566391 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f99c77-b486-470c-b36c-cb68b06069a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-39f99c77-b486-470c-b36c-cb68b06069a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:51.573061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:51.575653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:38:51.576827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:51.579769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 128/250] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:38:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:51.587521 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f99c77-b486-470c-b36c-cb68b06069a5 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:51.652542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 123/251] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:51.729678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f99c77-b486-470c-b36c-cb68b06069a5 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:51.752794 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7b9ab-48c3-4716-9959-c72015d372f1 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:51.788416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:51.789084 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:51.789366 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:51.789643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:51.790422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:51.790894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:51.792458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:51.807661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 129/252] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:51.814042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:51.867193 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:51.887532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:51.941079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7b9ab-48c3-4716-9959-c72015d372f1 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:51.962973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0272f-dc6a-402c-acc1-b4bc231cf352 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:52.010931 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cb226-e3da-4673-a245-6e02a7efcb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:52.011206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a152b8-3a80-4a34-8b4c-bfb92b5aff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:52.011622 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cb226-e3da-4673-a245-6e02a7efcb7d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.011748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a152b8-3a80-4a34-8b4c-bfb92b5aff46 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.011958 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cb226-e3da-4673-a245-6e02a7efcb7d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.012081 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a152b8-3a80-4a34-8b4c-bfb92b5aff46 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.012250 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cb226-e3da-4673-a245-6e02a7efcb7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.012358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a152b8-3a80-4a34-8b4c-bfb92b5aff46 None admin] PATH_INFO: `/v3/users/e5b4629902714fb3b29af6f40b73e825` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.013470 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a152b8-3a80-4a34-8b4c-bfb92b5aff46 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b4629902714fb3b29af6f40b73e825)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:52.015083 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a152b8-3a80-4a34-8b4c-bfb92b5aff46 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:52.021513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 130/253] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:51 2022] GET /identity/v3/users/e5b4629902714fb3b29af6f40b73e825 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:38:52.023020 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76637865-771d-466d-8da7-69ca83338a1e None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.023304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76637865-771d-466d-8da7-69ca83338a1e None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.023579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76637865-771d-466d-8da7-69ca83338a1e None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.024966 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 131/254] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:52.027237 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76637865-771d-466d-8da7-69ca83338a1e None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:52.028593 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645cb226-e3da-4673-a245-6e02a7efcb7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-645cb226-e3da-4673-a245-6e02a7efcb7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:52.034209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:52.035302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:52.036327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 124/255] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:38:52.042146 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97011f-88d4-4f11-b56a-fc8e993fa999 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.042146 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97011f-88d4-4f11-b56a-fc8e993fa999 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.042146 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97011f-88d4-4f11-b56a-fc8e993fa999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.057973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-bc97011f-88d4-4f11-b56a-fc8e993fa999 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:52.059181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-bc97011f-88d4-4f11-b56a-fc8e993fa999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:52.133984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 125/256] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:52.134928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97011f-88d4-4f11-b56a-fc8e993fa999 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:52.151382 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76637865-771d-466d-8da7-69ca83338a1e None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:52.153252 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:52.198657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:52.201071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2ba8b-94ac-4c72-aaff-9a72a8d98c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:52.201862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ba8b-94ac-4c72-aaff-9a72a8d98c38 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.202234 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ba8b-94ac-4c72-aaff-9a72a8d98c38 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.202601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ba8b-94ac-4c72-aaff-9a72a8d98c38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.245626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97011f-88d4-4f11-b56a-fc8e993fa999 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:52.275142 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2ba8b-94ac-4c72-aaff-9a72a8d98c38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:52.276824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2ba8b-94ac-4c72-aaff-9a72a8d98c38 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:52.321839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:52.322514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.322793 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.323118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.323917 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] RBAC: Adding query filter params (name=member) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:52.324221 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:52.325667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:52.345592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 126/257] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:52.348296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:52.474774 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 132/258] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:52.475704 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2ba8b-94ac-4c72-aaff-9a72a8d98c38 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:52.482428 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:52.555606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb3ddc-4262-4f8d-868f-43af230d3310 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:52.571184 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:52.597888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbd32b-8ee8-417f-a3c1-5342cb59b37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:52.598550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbd32b-8ee8-417f-a3c1-5342cb59b37b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.598836 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbd32b-8ee8-417f-a3c1-5342cb59b37b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.599147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbd32b-8ee8-417f-a3c1-5342cb59b37b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.611952 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbd32b-8ee8-417f-a3c1-5342cb59b37b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-1ddbd32b-8ee8-417f-a3c1-5342cb59b37b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:52.617970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:52.619099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:52.620120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 127/259] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:38:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:52.620870 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbd32b-8ee8-417f-a3c1-5342cb59b37b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:52.738470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbd32b-8ee8-417f-a3c1-5342cb59b37b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:52.763057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2ba8b-94ac-4c72-aaff-9a72a8d98c38 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:52.775968 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4355eb-12b4-4125-97db-fdbc5794ecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:52.776670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4355eb-12b4-4125-97db-fdbc5794ecb3 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.777022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4355eb-12b4-4125-97db-fdbc5794ecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.777306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4355eb-12b4-4125-97db-fdbc5794ecb3 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.779140 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4355eb-12b4-4125-97db-fdbc5794ecb3 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:52.780664 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4355eb-12b4-4125-97db-fdbc5794ecb3 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:52.782686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 128/260] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:52 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:52.783362 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4355eb-12b4-4125-97db-fdbc5794ecb3 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:52.830917 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb7baf-6102-48bc-84c9-92c765f5b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:52.831626 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb7baf-6102-48bc-84c9-92c765f5b088 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.831908 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb7baf-6102-48bc-84c9-92c765f5b088 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.832189 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb7baf-6102-48bc-84c9-92c765f5b088 None admin] PATH_INFO: `/v3/users/a631906c67ac4c648bbd2c05e9f9d658` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.832632 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:52.839342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb7baf-6102-48bc-84c9-92c765f5b088 None admin] RBAC: Authorizing `identity:get_user(user_id=a631906c67ac4c648bbd2c05e9f9d658)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:52.840873 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb7baf-6102-48bc-84c9-92c765f5b088 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:52.843092 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 133/261] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:52 2022] GET /identity/v3/users/a631906c67ac4c648bbd2c05e9f9d658 => generated 297 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:52.843970 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb7baf-6102-48bc-84c9-92c765f5b088 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:52.903165 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4355eb-12b4-4125-97db-fdbc5794ecb3 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:52.954513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:52.957629 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:52.958352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:52.958640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:52.959012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:52.959748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:52.960063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:52.961493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:52.982968 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 129/262] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:38:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:52.983672 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:53.017731 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb7baf-6102-48bc-84c9-92c765f5b088 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:53.077383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:53.078494 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:53.079798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:53.080309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:53.081513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=713f2c71f07c4deeb316712af011e233, user.id=e5b4629902714fb3b29af6f40b73e825) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:53.083340 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:53.086166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:53.098137 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 134/263] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:52 2022] GET /identity/v3/role_assignments?user.id=e5b4629902714fb3b29af6f40b73e825&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:53.100152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:53.204363 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a9319-3b23-4070-89a1-ff8640802251 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:53.207198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:53.277539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583945-2c2b-4ff9-bf1e-9c360ccd6be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:53.279131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583945-2c2b-4ff9-bf1e-9c360ccd6be3 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:53.279692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583945-2c2b-4ff9-bf1e-9c360ccd6be3 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:53.280202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583945-2c2b-4ff9-bf1e-9c360ccd6be3 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:53.281811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba583945-2c2b-4ff9-bf1e-9c360ccd6be3 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:53.284207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba583945-2c2b-4ff9-bf1e-9c360ccd6be3 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:53.287500 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 130/264] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:52 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:53.293811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583945-2c2b-4ff9-bf1e-9c360ccd6be3 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:53.294833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28ec24-db91-498b-927b-2968ded66407 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:53.368111 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649c4df-b901-45f0-b319-2c4a10946c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:53.373719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649c4df-b901-45f0-b319-2c4a10946c8b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:53.374426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649c4df-b901-45f0-b319-2c4a10946c8b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:53.374958 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649c4df-b901-45f0-b319-2c4a10946c8b None admin] PATH_INFO: `/v3/users/e5b4629902714fb3b29af6f40b73e825` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:53.376593 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649c4df-b901-45f0-b319-2c4a10946c8b None admin] RBAC: Authorizing `identity:get_user(user_id=e5b4629902714fb3b29af6f40b73e825)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:53.378348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649c4df-b901-45f0-b319-2c4a10946c8b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:53.383071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 135/265] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:53 2022] GET /identity/v3/users/e5b4629902714fb3b29af6f40b73e825 => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:53.394255 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649c4df-b901-45f0-b319-2c4a10946c8b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:53.409524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:53.517319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649c4df-b901-45f0-b319-2c4a10946c8b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:53.521125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583945-2c2b-4ff9-bf1e-9c360ccd6be3 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:53.561506 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110278c1-b685-41f2-9194-8d1e41c1e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:53.562692 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110278c1-b685-41f2-9194-8d1e41c1e1ce None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:53.563215 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110278c1-b685-41f2-9194-8d1e41c1e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:53.563715 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110278c1-b685-41f2-9194-8d1e41c1e1ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:53.616909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63451b-0d61-49c8-9579-275218f64979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:53.618310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63451b-0d61-49c8-9579-275218f64979 None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:53.618310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63451b-0d61-49c8-9579-275218f64979 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:53.618310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63451b-0d61-49c8-9579-275218f64979 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233/users/a631906c67ac4c648bbd2c05e9f9d658/roles/914dcb9114e54c75a514ce6af60ffbb3` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:53.623000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63451b-0d61-49c8-9579-275218f64979 None admin] RBAC: Authorizing `identity:create_grant(project_id=713f2c71f07c4deeb316712af011e233, user_id=a631906c67ac4c648bbd2c05e9f9d658, role_id=914dcb9114e54c75a514ce6af60ffbb3)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:53.624626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63451b-0d61-49c8-9579-275218f64979 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:53.658136 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 131/266] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:53 2022] PUT /identity/v3/projects/713f2c71f07c4deeb316712af011e233/users/a631906c67ac4c648bbd2c05e9f9d658/roles/914dcb9114e54c75a514ce6af60ffbb3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:38:53.663490 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110278c1-b685-41f2-9194-8d1e41c1e1ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:53.665384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110278c1-b685-41f2-9194-8d1e41c1e1ce None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:53.669779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:53.801068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 136/267] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:38:53.806951 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63451b-0d61-49c8-9579-275218f64979 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:53.885594 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:53.966985 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188761e1-e056-433b-9715-d84b72c82cf0 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:53.967273 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188761e1-e056-433b-9715-d84b72c82cf0 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:53.967855 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188761e1-e056-433b-9715-d84b72c82cf0 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:53.969816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 137/268] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:38:53.976389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd0c75-5dd9-48c2-92cf-ef48df71e1de None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:53.977082 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd0c75-5dd9-48c2-92cf-ef48df71e1de None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:53.977592 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd0c75-5dd9-48c2-92cf-ef48df71e1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:53.993978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63451b-0d61-49c8-9579-275218f64979 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:54.002586 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-66fd0c75-5dd9-48c2-92cf-ef48df71e1de None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:54.004548 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-66fd0c75-5dd9-48c2-92cf-ef48df71e1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:54.044947 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48067ef1-59ab-46d0-ab7b-8c144f0e0139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:54.046601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48067ef1-59ab-46d0-ab7b-8c144f0e0139 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.047181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48067ef1-59ab-46d0-ab7b-8c144f0e0139 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.047653 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48067ef1-59ab-46d0-ab7b-8c144f0e0139 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.049775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48067ef1-59ab-46d0-ab7b-8c144f0e0139 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:54.052452 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48067ef1-59ab-46d0-ab7b-8c144f0e0139 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:54.055663 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 132/269] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:53 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:54.067252 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48067ef1-59ab-46d0-ab7b-8c144f0e0139 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:54.109341 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 138/270] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:54.112073 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf6d7-1207-478d-a602-f76671fc9ce7 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.112679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf6d7-1207-478d-a602-f76671fc9ce7 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.113395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf6d7-1207-478d-a602-f76671fc9ce7 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.116353 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 139/271] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:54.124018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f357c-2584-4b15-b33f-6704b453d7e6 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.124693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f357c-2584-4b15-b33f-6704b453d7e6 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.125199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f357c-2584-4b15-b33f-6704b453d7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.151726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-1a2f357c-2584-4b15-b33f-6704b453d7e6 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:54.153255 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-1a2f357c-2584-4b15-b33f-6704b453d7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:54.185076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:54.217554 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:54.227812 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48067ef1-59ab-46d0-ab7b-8c144f0e0139 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:54.247416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 140/272] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:54.247416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f357c-2584-4b15-b33f-6704b453d7e6 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:54.291972 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:54.293643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.294458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.294972 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.296670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] RBAC: Adding query filter params (role.id=8d072b58656b442f922c3287a5430603, scope.project.id=713f2c71f07c4deeb316712af011e233, user.id=e5b4629902714fb3b29af6f40b73e825) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:54.297477 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:54.299299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:54.314818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 133/273] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:54 2022] GET /identity/v3/role_assignments?user.id=e5b4629902714fb3b29af6f40b73e825&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=8d072b58656b442f922c3287a5430603 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:54.314818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:54.404740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f357c-2584-4b15-b33f-6704b453d7e6 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:54.447452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410ffd3-f9ba-4810-9458-d10aacf025c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:54.448578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ffd3-f9ba-4810-9458-d10aacf025c7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.449039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ffd3-f9ba-4810-9458-d10aacf025c7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.449488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ffd3-f9ba-4810-9458-d10aacf025c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.462992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410ffd3-f9ba-4810-9458-d10aacf025c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-5410ffd3-f9ba-4810-9458-d10aacf025c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:54.469013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:54.470454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:54.474198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882805-0094-4407-8f9f-745f63e69332 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:54.474502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 141/274] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:54.476151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ae69-f1db-4440-af06-7b7653879373 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.476624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ae69-f1db-4440-af06-7b7653879373 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.477155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ae69-f1db-4440-af06-7b7653879373 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.483293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 142/275] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:54.484583 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970ae69-f1db-4440-af06-7b7653879373 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:54.514669 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:54.518759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc610dec-8ab7-497e-85ec-5c45abab17e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:54.520127 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610dec-8ab7-497e-85ec-5c45abab17e9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.520127 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610dec-8ab7-497e-85ec-5c45abab17e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.520127 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610dec-8ab7-497e-85ec-5c45abab17e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.521317 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc610dec-8ab7-497e-85ec-5c45abab17e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:54.523011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc610dec-8ab7-497e-85ec-5c45abab17e9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:54.525235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 134/276] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:54.526780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d9529-6493-48a4-88fb-a3142c58c054 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.527092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d9529-6493-48a4-88fb-a3142c58c054 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.527398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d9529-6493-48a4-88fb-a3142c58c054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.553904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-4c4d9529-6493-48a4-88fb-a3142c58c054 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:54.555998 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-4c4d9529-6493-48a4-88fb-a3142c58c054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:54.596300 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:54.632730 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970ae69-f1db-4440-af06-7b7653879373 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:54.657295 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 135/277] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:54.662150 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d9529-6493-48a4-88fb-a3142c58c054 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:54.681271 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:54.681922 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.682334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.682666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.683445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:54.683760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:54.685181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:54.695162 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 143/278] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:54.695890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:54.788132 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:54.791342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d9529-6493-48a4-88fb-a3142c58c054 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:54.811403 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2ae2d-0170-473a-a6d6-ba2bbee696a0 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:54.843143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2c62f-ccd2-4f82-b305-a8dd3a0e05c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:54.843998 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2c62f-ccd2-4f82-b305-a8dd3a0e05c0 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.844326 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2c62f-ccd2-4f82-b305-a8dd3a0e05c0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.844782 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2c62f-ccd2-4f82-b305-a8dd3a0e05c0 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.845947 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2c62f-ccd2-4f82-b305-a8dd3a0e05c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:54.848543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2c62f-ccd2-4f82-b305-a8dd3a0e05c0 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:54.867827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb84e40-6056-4fd9-9161-21489174825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:54.868900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb84e40-6056-4fd9-9161-21489174825b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:54.869389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb84e40-6056-4fd9-9161-21489174825b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:54.869879 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb84e40-6056-4fd9-9161-21489174825b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:54.883306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 136/279] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:38:54 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:54.884701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2c62f-ccd2-4f82-b305-a8dd3a0e05c0 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:54.888211 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb84e40-6056-4fd9-9161-21489174825b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-3fb84e40-6056-4fd9-9161-21489174825b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:54.902253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:54.903491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:38:54.904571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 144/280] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:54.907648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb84e40-6056-4fd9-9161-21489174825b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:54.920742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:55.044200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2c62f-ccd2-4f82-b305-a8dd3a0e05c0 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:55.063991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb84e40-6056-4fd9-9161-21489174825b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:55.100837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd2adc-9a52-4143-9977-a652c2b11640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:55.101600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2adc-9a52-4143-9977-a652c2b11640 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:55.101989 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2adc-9a52-4143-9977-a652c2b11640 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:55.102306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2adc-9a52-4143-9977-a652c2b11640 None admin] PATH_INFO: `/v3/users/3021c88c9cb3438bb4333e4066b51f19` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:55.110542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:55.111628 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:55.112148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:55.112669 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:55.113423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:55.113817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:55.115283 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:55.119269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd2adc-9a52-4143-9977-a652c2b11640 None admin] RBAC: Authorizing `identity:get_user(user_id=3021c88c9cb3438bb4333e4066b51f19)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:55.124881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd2adc-9a52-4143-9977-a652c2b11640 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:55.124881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 137/281] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:54 2022] GET /identity/v3/users/3021c88c9cb3438bb4333e4066b51f19 => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:55.131834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 145/282] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:55.133433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd2adc-9a52-4143-9977-a652c2b11640 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:55.140389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:55.160488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:55.241982 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:55.274708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd2adc-9a52-4143-9977-a652c2b11640 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:55.289150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346b8b8-885c-43fb-8976-c0976ccea5b5 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:55.323825 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:55.324776 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:55.325063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:55.325339 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:55.326358 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] RBAC: Adding query filter params (role.id=914dcb9114e54c75a514ce6af60ffbb3, user.id=3021c88c9cb3438bb4333e4066b51f19, scope.system=all) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:55.326996 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:55.328802 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:55.333309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624d4ef-5948-4f42-a612-791fd5f3adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:55.334073 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624d4ef-5948-4f42-a612-791fd5f3adcf None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:55.334442 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624d4ef-5948-4f42-a612-791fd5f3adcf None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:55.334705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624d4ef-5948-4f42-a612-791fd5f3adcf None admin] PATH_INFO: `/v3/users/cff3683b33184675ae950718d2c6aea2` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:55.334942 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 138/283] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:38:55 2022] GET /identity/v3/role_assignments?user.id=3021c88c9cb3438bb4333e4066b51f19&scope.system=all&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:55.336041 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624d4ef-5948-4f42-a612-791fd5f3adcf None admin] RBAC: Authorizing `identity:get_user(user_id=cff3683b33184675ae950718d2c6aea2)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:55.337551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624d4ef-5948-4f42-a612-791fd5f3adcf None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:55.340751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 146/284] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:55 2022] GET /identity/v3/users/cff3683b33184675ae950718d2c6aea2 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:55.345844 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:55.463778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e80dd-7bc1-4186-94a3-b2c2efb02e35 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:55.503535 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54d5b4-d25d-4768-8c43-1986e29dfde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:55.504227 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d5b4-d25d-4768-8c43-1986e29dfde4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:55.504513 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d5b4-d25d-4768-8c43-1986e29dfde4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:55.504795 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d5b4-d25d-4768-8c43-1986e29dfde4 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:55.506122 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54d5b4-d25d-4768-8c43-1986e29dfde4 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:55.507650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54d5b4-d25d-4768-8c43-1986e29dfde4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:55.509743 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 139/285] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:55 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:55.515743 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624d4ef-5948-4f42-a612-791fd5f3adcf None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:55.608792 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:55.635704 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624d4ef-5948-4f42-a612-791fd5f3adcf None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:55.678887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c833f-9fb8-4bc3-97a6-8ae2e061d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:55.679633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c833f-9fb8-4bc3-97a6-8ae2e061d513 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:55.679952 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c833f-9fb8-4bc3-97a6-8ae2e061d513 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:55.680242 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c833f-9fb8-4bc3-97a6-8ae2e061d513 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233/users/cff3683b33184675ae950718d2c6aea2/roles/9fc42581f1f84e28b6d65cdc76126fdc` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:55.682323 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c833f-9fb8-4bc3-97a6-8ae2e061d513 None admin] RBAC: Authorizing `identity:create_grant(project_id=713f2c71f07c4deeb316712af011e233, user_id=cff3683b33184675ae950718d2c6aea2, role_id=9fc42581f1f84e28b6d65cdc76126fdc)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:55.683800 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c833f-9fb8-4bc3-97a6-8ae2e061d513 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:55.705537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 147/286] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:38:55 2022] PUT /identity/v3/projects/713f2c71f07c4deeb316712af011e233/users/cff3683b33184675ae950718d2c6aea2/roles/9fc42581f1f84e28b6d65cdc76126fdc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:56.098364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbcd78-c20d-4845-86e7-cd00ea1331c6 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:56.098633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbcd78-c20d-4845-86e7-cd00ea1331c6 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:56.098945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbcd78-c20d-4845-86e7-cd00ea1331c6 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:56.100871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 140/287] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:56.106987 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ebb9c-4953-4bea-abc4-73eed7f7a42d None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:56.107356 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ebb9c-4953-4bea-abc4-73eed7f7a42d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:56.107561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ebb9c-4953-4bea-abc4-73eed7f7a42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:56.126985 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-8f6ebb9c-4953-4bea-abc4-73eed7f7a42d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:56.128221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-8f6ebb9c-4953-4bea-abc4-73eed7f7a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:56.216708 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 148/288] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:56.240586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbcd78-c20d-4845-86e7-cd00ea1331c6 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:56.249963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:56.363993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbcd78-c20d-4845-86e7-cd00ea1331c6 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:56.412686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67c2c-6e6d-4b12-9681-a68f6a31d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:56.413373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67c2c-6e6d-4b12-9681-a68f6a31d0c9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:56.413654 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67c2c-6e6d-4b12-9681-a68f6a31d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:56.413999 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67c2c-6e6d-4b12-9681-a68f6a31d0c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:56.425511 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c67c2c-6e6d-4b12-9681-a68f6a31d0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-c5c67c2c-6e6d-4b12-9681-a68f6a31d0c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:56.431045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:56.432120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:56.433792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 141/289] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:56.438665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ebb9c-4953-4bea-abc4-73eed7f7a42d None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:56.506911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:56.570316 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ebb9c-4953-4bea-abc4-73eed7f7a42d None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:56.621163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:56.621734 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:56.622021 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:56.622329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:56.623076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:56.623386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:56.624800 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:56.634360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 149/290] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:56.639819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67c2c-6e6d-4b12-9681-a68f6a31d0c9 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:56.751573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:56.804399 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67c2c-6e6d-4b12-9681-a68f6a31d0c9 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:56.863458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717542cd-fc47-401f-82d3-604c30faf7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:56.864586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717542cd-fc47-401f-82d3-604c30faf7f1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:56.865171 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717542cd-fc47-401f-82d3-604c30faf7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:56.865668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717542cd-fc47-401f-82d3-604c30faf7f1 None admin] PATH_INFO: `/v3/users/a631906c67ac4c648bbd2c05e9f9d658` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:56.867439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717542cd-fc47-401f-82d3-604c30faf7f1 None admin] RBAC: Authorizing `identity:get_user(user_id=a631906c67ac4c648bbd2c05e9f9d658)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:56.869244 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717542cd-fc47-401f-82d3-604c30faf7f1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:56.875978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 142/291] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:56 2022] GET /identity/v3/users/a631906c67ac4c648bbd2c05e9f9d658 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:56.879350 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:57.064047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c770e-f2a0-488d-89c4-dee737b516b7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:57.112392 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:57.126549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fd6de-eb18-4dc2-8596-41e66b432e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:57.127120 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fd6de-eb18-4dc2-8596-41e66b432e88 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:57.127412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fd6de-eb18-4dc2-8596-41e66b432e88 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:57.127693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fd6de-eb18-4dc2-8596-41e66b432e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:57.227432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fd6de-eb18-4dc2-8596-41e66b432e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:57.228933 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fd6de-eb18-4dc2-8596-41e66b432e88 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:57.362892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 150/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:57.369719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717542cd-fc47-401f-82d3-604c30faf7f1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:57.507444 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717542cd-fc47-401f-82d3-604c30faf7f1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:57.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034af0c-fb7c-42f2-8084-4b0d052c8b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:57.554913 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034af0c-fb7c-42f2-8084-4b0d052c8b0e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:57.555200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034af0c-fb7c-42f2-8084-4b0d052c8b0e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:57.555484 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034af0c-fb7c-42f2-8084-4b0d052c8b0e None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:57.556869 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034af0c-fb7c-42f2-8084-4b0d052c8b0e None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:57.558439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034af0c-fb7c-42f2-8084-4b0d052c8b0e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:57.560537 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 143/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:57 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:57.567753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fd6de-eb18-4dc2-8596-41e66b432e88 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:57.624297 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:57.705309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582940f-3a6e-4c7a-be0f-ad1e1397e5a7 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:57.705648 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582940f-3a6e-4c7a-be0f-ad1e1397e5a7 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:57.705875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582940f-3a6e-4c7a-be0f-ad1e1397e5a7 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:57.708050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 144/294] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:57.714065 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9252-c699-4a12-a260-667775b5feb3 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:57.714391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9252-c699-4a12-a260-667775b5feb3 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:57.714679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9252-c699-4a12-a260-667775b5feb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:57.722062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:57.739002 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fd6de-eb18-4dc2-8596-41e66b432e88 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:57.759420 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-130a9252-c699-4a12-a260-667775b5feb3 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:57.760599 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-130a9252-c699-4a12-a260-667775b5feb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:57.803213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30410c8c-13b2-4d7f-b14b-14af9ad592c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:57.804210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410c8c-13b2-4d7f-b14b-14af9ad592c5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:57.804502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410c8c-13b2-4d7f-b14b-14af9ad592c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:57.805064 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410c8c-13b2-4d7f-b14b-14af9ad592c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:57.805895 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30410c8c-13b2-4d7f-b14b-14af9ad592c5 None admin] RBAC: Adding query filter params (role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=713f2c71f07c4deeb316712af011e233, user.id=a631906c67ac4c648bbd2c05e9f9d658) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:57.806400 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30410c8c-13b2-4d7f-b14b-14af9ad592c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:57.808499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30410c8c-13b2-4d7f-b14b-14af9ad592c5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:57.816824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 151/295] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:38:57 2022] GET /identity/v3/role_assignments?user.id=a631906c67ac4c648bbd2c05e9f9d658&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:57.818307 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca287c-481b-46d0-af24-10f406883f7b None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:57.818597 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca287c-481b-46d0-af24-10f406883f7b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:57.818876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca287c-481b-46d0-af24-10f406883f7b None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:57.821438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 152/296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:57.829311 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37566a-d621-483e-b8ee-6b32c453398f None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:57.830004 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37566a-d621-483e-b8ee-6b32c453398f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:57.830316 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37566a-d621-483e-b8ee-6b32c453398f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:57.868170 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-9f37566a-d621-483e-b8ee-6b32c453398f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:57.869608 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-9f37566a-d621-483e-b8ee-6b32c453398f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:57.889736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 145/297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:57.967644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9252-c699-4a12-a260-667775b5feb3 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.019427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 153/298] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:58.030805 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9bdb6-b48d-4433-ae5e-940df27f1e0f None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.031173 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9bdb6-b48d-4433-ae5e-940df27f1e0f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.031368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9bdb6-b48d-4433-ae5e-940df27f1e0f None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.032808 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 154/299] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:38:58.038882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430652a-d846-4e66-98c4-93c3a0a6e5b1 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.039241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430652a-d846-4e66-98c4-93c3a0a6e5b1 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.039531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430652a-d846-4e66-98c4-93c3a0a6e5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.060911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-d430652a-d846-4e66-98c4-93c3a0a6e5b1 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:58.062204 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-d430652a-d846-4e66-98c4-93c3a0a6e5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:58.079321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:58.128918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9252-c699-4a12-a260-667775b5feb3 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:58.158825 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 155/300] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:38:58.160061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430652a-d846-4e66-98c4-93c3a0a6e5b1 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.178000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:58.188990 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595260a-252a-4f42-bf52-cb1d6d8f3a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:58.189669 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595260a-252a-4f42-bf52-cb1d6d8f3a15 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.190011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595260a-252a-4f42-bf52-cb1d6d8f3a15 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.190356 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595260a-252a-4f42-bf52-cb1d6d8f3a15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.202157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595260a-252a-4f42-bf52-cb1d6d8f3a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-e595260a-252a-4f42-bf52-cb1d6d8f3a15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:58.207261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:58.208361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:58.209490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 146/301] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:38:58.210314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595260a-252a-4f42-bf52-cb1d6d8f3a15 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.289399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430652a-d846-4e66-98c4-93c3a0a6e5b1 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:58.322651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595260a-252a-4f42-bf52-cb1d6d8f3a15 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:58.328713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eb31c-3df5-4ec5-954e-e05f46f48f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:58.329713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eb31c-3df5-4ec5-954e-e05f46f48f6d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.330435 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eb31c-3df5-4ec5-954e-e05f46f48f6d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.330779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eb31c-3df5-4ec5-954e-e05f46f48f6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.331984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2eb31c-3df5-4ec5-954e-e05f46f48f6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:58.333631 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2eb31c-3df5-4ec5-954e-e05f46f48f6d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:58.335807 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 156/302] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:38:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:58.341084 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eb31c-3df5-4ec5-954e-e05f46f48f6d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.359304 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7c9d-e711-4843-bd37-3232cb2953d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:58.359959 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7c9d-e711-4843-bd37-3232cb2953d1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.360249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7c9d-e711-4843-bd37-3232cb2953d1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.360525 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7c9d-e711-4843-bd37-3232cb2953d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.362412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:58.372411 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a7c9d-e711-4843-bd37-3232cb2953d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-102a7c9d-e711-4843-bd37-3232cb2953d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:58.377563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:58.378767 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 147/303] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:58.380235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7c9d-e711-4843-bd37-3232cb2953d1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.439822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:58.458521 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eb31c-3df5-4ec5-954e-e05f46f48f6d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:58.495730 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:58.496374 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.496660 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.496938 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.497314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7c9d-e711-4843-bd37-3232cb2953d1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:58.497683 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:58.497991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:58.499421 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:58.508782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 157/304] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:58.509651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.541246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbc37c-6e17-4d65-a5ca-a7884ae40287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:58.541903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbc37c-6e17-4d65-a5ca-a7884ae40287 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.542214 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbc37c-6e17-4d65-a5ca-a7884ae40287 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.542606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbc37c-6e17-4d65-a5ca-a7884ae40287 None admin] PATH_INFO: `/v3/groups` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.543289 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbc37c-6e17-4d65-a5ca-a7884ae40287 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:58.544777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbc37c-6e17-4d65-a5ca-a7884ae40287 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:58.563343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 148/305] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:38:58 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:38:58.564590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbc37c-6e17-4d65-a5ca-a7884ae40287 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.611145 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:58.634384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61593c84-d3a4-4920-b8d8-188dfb506bf4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:58.677758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ab4de-94a3-4ce5-88ed-bf58a666ef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:58.678473 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ab4de-94a3-4ce5-88ed-bf58a666ef9a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.678767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ab4de-94a3-4ce5-88ed-bf58a666ef9a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.679095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ab4de-94a3-4ce5-88ed-bf58a666ef9a None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.679897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ab4de-94a3-4ce5-88ed-bf58a666ef9a None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:58.680214 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ab4de-94a3-4ce5-88ed-bf58a666ef9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:58.681663 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ab4de-94a3-4ce5-88ed-bf58a666ef9a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:58.697687 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 158/306] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:58.697687 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86ee8d-fde8-46be-8c53-d0a9c93fcc89 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.697687 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86ee8d-fde8-46be-8c53-d0a9c93fcc89 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.697687 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86ee8d-fde8-46be-8c53-d0a9c93fcc89 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.697687 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 159/307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:38:58.701537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86ee8d-fde8-46be-8c53-d0a9c93fcc89 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbc37c-6e17-4d65-a5ca-a7884ae40287 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:58.723378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:58.751689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744bd4b-ab88-4e1b-819b-2045373fc459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:58.752407 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744bd4b-ab88-4e1b-819b-2045373fc459 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.752683 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744bd4b-ab88-4e1b-819b-2045373fc459 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.752967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744bd4b-ab88-4e1b-819b-2045373fc459 None admin] PATH_INFO: `/v3/users/559ddbcdf530467d9d0db7ab3e3645dd` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.769927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744bd4b-ab88-4e1b-819b-2045373fc459 None admin] RBAC: Authorizing `identity:get_user(user_id=559ddbcdf530467d9d0db7ab3e3645dd)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:58.771732 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744bd4b-ab88-4e1b-819b-2045373fc459 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:58.773540 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 149/308] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:58 2022] GET /identity/v3/users/559ddbcdf530467d9d0db7ab3e3645dd => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:58.775014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9080d-d020-450c-ae19-385a0fcc7e17 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.775305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9080d-d020-450c-ae19-385a0fcc7e17 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.775583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9080d-d020-450c-ae19-385a0fcc7e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.795679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-1bb9080d-d020-450c-ae19-385a0fcc7e17 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:38:58.796822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-1bb9080d-d020-450c-ae19-385a0fcc7e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:38:58.833570 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86ee8d-fde8-46be-8c53-d0a9c93fcc89 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:58.874068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519b32-edcf-4a4c-bfb1-79bfaaac0a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:58.874776 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519b32-edcf-4a4c-bfb1-79bfaaac0a31 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:58.875053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519b32-edcf-4a4c-bfb1-79bfaaac0a31 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:58.875586 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519b32-edcf-4a4c-bfb1-79bfaaac0a31 None admin] PATH_INFO: `/v3/users/3021c88c9cb3438bb4333e4066b51f19` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:58.875716 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 150/309] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:38:58.876616 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519b32-edcf-4a4c-bfb1-79bfaaac0a31 None admin] RBAC: Authorizing `identity:get_user(user_id=3021c88c9cb3438bb4333e4066b51f19)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:58.878087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519b32-edcf-4a4c-bfb1-79bfaaac0a31 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:58.884763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 160/310] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:58 2022] GET /identity/v3/users/3021c88c9cb3438bb4333e4066b51f19 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:58.884954 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9080d-d020-450c-ae19-385a0fcc7e17 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.886375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519b32-edcf-4a4c-bfb1-79bfaaac0a31 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:58.901520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:58.998010 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:58.999902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519b32-edcf-4a4c-bfb1-79bfaaac0a31 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:59.001981 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9080d-d020-450c-ae19-385a0fcc7e17 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:59.037438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67d37d-9917-491a-b97a-05ddb75d9dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:59.038284 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67d37d-9917-491a-b97a-05ddb75d9dab None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:59.038575 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67d37d-9917-491a-b97a-05ddb75d9dab None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:59.038859 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67d37d-9917-491a-b97a-05ddb75d9dab None admin] PATH_INFO: `/v3/system/users/3021c88c9cb3438bb4333e4066b51f19/roles/914dcb9114e54c75a514ce6af60ffbb3` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:59.039860 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:59.040370 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67d37d-9917-491a-b97a-05ddb75d9dab None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3021c88c9cb3438bb4333e4066b51f19, role_id=914dcb9114e54c75a514ce6af60ffbb3)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:59.040496 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:59.040773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:59.041046 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:59.041782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67d37d-9917-491a-b97a-05ddb75d9dab None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:59.041906 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] RBAC: Adding query filter params (role.id=9fc42581f1f84e28b6d65cdc76126fdc, user.id=559ddbcdf530467d9d0db7ab3e3645dd, scope.system=all) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:59.042177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:59.043644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:59.058309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 151/311] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:38:58 2022] GET /identity/v3/role_assignments?user.id=559ddbcdf530467d9d0db7ab3e3645dd&scope.system=all&role.id=9fc42581f1f84e28b6d65cdc76126fdc => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:59.058309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:59.069540 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 161/312] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:38:58 2022] PUT /identity/v3/system/users/3021c88c9cb3438bb4333e4066b51f19/roles/914dcb9114e54c75a514ce6af60ffbb3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:38:59.213552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910b420-1310-4393-8c2d-24020ce70f8d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:59.258865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2084f-f680-4c95-a5ea-bba8ef290f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:59.259600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2084f-f680-4c95-a5ea-bba8ef290f80 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:59.259884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2084f-f680-4c95-a5ea-bba8ef290f80 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:59.260163 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2084f-f680-4c95-a5ea-bba8ef290f80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:59.275427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2084f-f680-4c95-a5ea-bba8ef290f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-8ac2084f-f680-4c95-a5ea-bba8ef290f80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:38:59.280667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:38:59.281876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:38:59.282980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 152/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:38:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:38:59.290882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67d37d-9917-491a-b97a-05ddb75d9dab None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:59.389788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:59.419961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67d37d-9917-491a-b97a-05ddb75d9dab None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:59.457787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:59.458484 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:59.458870 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:59.459150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:59.459907 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:38:59.460222 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:59.461655 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:59.471817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 162/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:38:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:59.477700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2084f-f680-4c95-a5ea-bba8ef290f80 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:59.505086 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:59.596654 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2084f-f680-4c95-a5ea-bba8ef290f80 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:59.634350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0f2ab-6953-4166-b213-e81ca6ed07e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:59.635474 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0f2ab-6953-4166-b213-e81ca6ed07e5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:59.635807 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0f2ab-6953-4166-b213-e81ca6ed07e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:59.636087 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0f2ab-6953-4166-b213-e81ca6ed07e5 None admin] PATH_INFO: `/v3/users/cff3683b33184675ae950718d2c6aea2` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:59.637321 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0f2ab-6953-4166-b213-e81ca6ed07e5 None admin] RBAC: Authorizing `identity:get_user(user_id=cff3683b33184675ae950718d2c6aea2)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:59.638898 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0f2ab-6953-4166-b213-e81ca6ed07e5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:59.640938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 153/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:38:59 2022] GET /identity/v3/users/cff3683b33184675ae950718d2c6aea2 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:38:59.646809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:38:59.754474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0792e-912e-4635-8610-a9d1d3014788 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:38:59.791951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9824c-37bb-4a12-b752-dd78980a8dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:38:59.792624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9824c-37bb-4a12-b752-dd78980a8dd1 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:38:59.793008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9824c-37bb-4a12-b752-dd78980a8dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:38:59.793296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9824c-37bb-4a12-b752-dd78980a8dd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:38:59.797638 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:38:59.867315 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9824c-37bb-4a12-b752-dd78980a8dd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:38:59.868817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9824c-37bb-4a12-b752-dd78980a8dd1 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:38:59.973098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 163/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:38:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:38:59.979026 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0f2ab-6953-4166-b213-e81ca6ed07e5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:00.076329 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:00.085184 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0f2ab-6953-4166-b213-e81ca6ed07e5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:00.121366 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd68dd-b9d6-4acf-b61c-8ac516e1f49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:00.122079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd68dd-b9d6-4acf-b61c-8ac516e1f49b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:00.122394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd68dd-b9d6-4acf-b61c-8ac516e1f49b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:00.122675 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd68dd-b9d6-4acf-b61c-8ac516e1f49b None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:00.123886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd68dd-b9d6-4acf-b61c-8ac516e1f49b None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:00.125377 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd68dd-b9d6-4acf-b61c-8ac516e1f49b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:00.127543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 154/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:38:59 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:00.133365 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9824c-37bb-4a12-b752-dd78980a8dd1 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:00.169039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:00.237666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9824c-37bb-4a12-b752-dd78980a8dd1 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:00.275973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3df1f-50ee-4a10-837c-c8e142909f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:00.276616 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3df1f-50ee-4a10-837c-c8e142909f92 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:00.276906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3df1f-50ee-4a10-837c-c8e142909f92 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:00.277175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3df1f-50ee-4a10-837c-c8e142909f92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:00.277988 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3df1f-50ee-4a10-837c-c8e142909f92 None admin] RBAC: Adding query filter params (role.id=9fc42581f1f84e28b6d65cdc76126fdc, scope.project.id=713f2c71f07c4deeb316712af011e233, user.id=cff3683b33184675ae950718d2c6aea2) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:00.278327 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3df1f-50ee-4a10-837c-c8e142909f92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:00.280339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3df1f-50ee-4a10-837c-c8e142909f92 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:00.287752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 164/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:00 2022] GET /identity/v3/role_assignments?user.id=cff3683b33184675ae950718d2c6aea2&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=9fc42581f1f84e28b6d65cdc76126fdc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:00.716723 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d656a-f93c-4441-a803-d111f7cccab1 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:00.717333 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d656a-f93c-4441-a803-d111f7cccab1 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:00.717333 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d656a-f93c-4441-a803-d111f7cccab1 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:00.718897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 155/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:00.724179 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c395f-1775-4385-9008-f02483c2da48 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:00.724751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c395f-1775-4385-9008-f02483c2da48 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:00.724751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c395f-1775-4385-9008-f02483c2da48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:00.744232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-e35c395f-1775-4385-9008-f02483c2da48 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:00.745458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-e35c395f-1775-4385-9008-f02483c2da48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:00.821379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 165/320] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:00.843794 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d656a-f93c-4441-a803-d111f7cccab1 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:00.953870 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d656a-f93c-4441-a803-d111f7cccab1 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:00.989162 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ad9e1-6b39-4623-85ba-2dad9be03a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:00.989831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad9e1-6b39-4623-85ba-2dad9be03a53 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:00.990252 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad9e1-6b39-4623-85ba-2dad9be03a53 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:00.990520 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad9e1-6b39-4623-85ba-2dad9be03a53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:00.991738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ad9e1-6b39-4623-85ba-2dad9be03a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:00.993925 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ad9e1-6b39-4623-85ba-2dad9be03a53 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:01.001328 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 156/321] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:01.002723 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c395f-1775-4385-9008-f02483c2da48 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:01.035561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:01.115404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c395f-1775-4385-9008-f02483c2da48 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:01.155370 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b89f43-0985-4e98-8d25-347bf73964d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:01.156102 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b89f43-0985-4e98-8d25-347bf73964d3 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.156387 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b89f43-0985-4e98-8d25-347bf73964d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.156717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b89f43-0985-4e98-8d25-347bf73964d3 None admin] PATH_INFO: `/v3/groups` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.157426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b89f43-0985-4e98-8d25-347bf73964d3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:01.158962 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b89f43-0985-4e98-8d25-347bf73964d3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:01.162572 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f626-7e39-42c4-b85a-58a82a13c5a7 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.162886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f626-7e39-42c4-b85a-58a82a13c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.163149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f626-7e39-42c4-b85a-58a82a13c5a7 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.164759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 157/322] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:01.170828 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf38cde-1f72-40f8-9784-803775eac394 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.172475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf38cde-1f72-40f8-9784-803775eac394 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.172874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf38cde-1f72-40f8-9784-803775eac394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.190900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 166/323] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:39:01 2022] POST /identity/v3/groups => generated 226 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:39:01.197410 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-4bf38cde-1f72-40f8-9784-803775eac394 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:01.198726 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-4bf38cde-1f72-40f8-9784-803775eac394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:01.229686 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5cc10-d7f2-46da-b825-bd13f986e280 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.229903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5cc10-d7f2-46da-b825-bd13f986e280 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.230608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:01.230761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5cc10-d7f2-46da-b825-bd13f986e280 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.231777 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 167/324] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:39:01.238331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801435a2-cd53-4de7-a91d-f2fa817fd884 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.238651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801435a2-cd53-4de7-a91d-f2fa817fd884 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.238954 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801435a2-cd53-4de7-a91d-f2fa817fd884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.260262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-801435a2-cd53-4de7-a91d-f2fa817fd884 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:01.261544 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-801435a2-cd53-4de7-a91d-f2fa817fd884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:01.287991 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 158/325] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:01.310646 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf38cde-1f72-40f8-9784-803775eac394 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:01.343497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 168/326] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:01.366782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801435a2-cd53-4de7-a91d-f2fa817fd884 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:01.393141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:01.448834 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf38cde-1f72-40f8-9784-803775eac394 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:01.496756 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36088645-3fff-4a42-a4a5-9d3ae1bf550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:01.496756 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36088645-3fff-4a42-a4a5-9d3ae1bf550e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.496756 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36088645-3fff-4a42-a4a5-9d3ae1bf550e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.496756 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36088645-3fff-4a42-a4a5-9d3ae1bf550e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.507713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36088645-3fff-4a42-a4a5-9d3ae1bf550e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:01.512592 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801435a2-cd53-4de7-a91d-f2fa817fd884 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-36088645-3fff-4a42-a4a5-9d3ae1bf550e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:01.513588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:01.515053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:01.516090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 159/327] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:01.521020 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36088645-3fff-4a42-a4a5-9d3ae1bf550e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:01.544344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:01.559000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0d084-0a0b-4f05-b10c-67382349ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:01.559684 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0d084-0a0b-4f05-b10c-67382349ac56 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.559969 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0d084-0a0b-4f05-b10c-67382349ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.560245 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0d084-0a0b-4f05-b10c-67382349ac56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.572802 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0d084-0a0b-4f05-b10c-67382349ac56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-2cb0d084-0a0b-4f05-b10c-67382349ac56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:01.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:01.579383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:01.580459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 169/328] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:01.587694 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0d084-0a0b-4f05-b10c-67382349ac56 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:01.648845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36088645-3fff-4a42-a4a5-9d3ae1bf550e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:01.701760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:01.705835 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:01.707592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.708095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.708661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.709741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] RBAC: Adding query filter params (name=reader) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:01.710320 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:01.712040 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:01.722442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 160/329] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:01.730654 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:01.731598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0d084-0a0b-4f05-b10c-67382349ac56 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:01.771172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:01.771816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.772207 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.772413 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.773121 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:01.773435 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:01.774902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:01.784510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 170/330] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:01.790327 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:01.846775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:01.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7080db4-ed0e-40ad-81c5-c392ac15d04e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:01.917176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5b46d-5eb3-4b45-b7d9-f126dd1ef3dd None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:01.921346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98ca97-7ccb-44ff-8a81-80e94f65f7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:01.922376 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98ca97-7ccb-44ff-8a81-80e94f65f7d2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.922881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98ca97-7ccb-44ff-8a81-80e94f65f7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.923376 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98ca97-7ccb-44ff-8a81-80e94f65f7d2 None admin] PATH_INFO: `/v3/users/559ddbcdf530467d9d0db7ab3e3645dd` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.924915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98ca97-7ccb-44ff-8a81-80e94f65f7d2 None admin] RBAC: Authorizing `identity:get_user(user_id=559ddbcdf530467d9d0db7ab3e3645dd)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:01.927403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98ca97-7ccb-44ff-8a81-80e94f65f7d2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:01.931127 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 161/331] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:39:01 2022] GET /identity/v3/users/559ddbcdf530467d9d0db7ab3e3645dd => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:01.938215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98ca97-7ccb-44ff-8a81-80e94f65f7d2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:01.967554 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc391-0748-47c9-8297-4ec3d473c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:01.968237 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc391-0748-47c9-8297-4ec3d473c049 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:01.968868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc391-0748-47c9-8297-4ec3d473c049 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:01.968868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc391-0748-47c9-8297-4ec3d473c049 None admin] PATH_INFO: `/v3/users/3021c88c9cb3438bb4333e4066b51f19` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:01.970017 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcc391-0748-47c9-8297-4ec3d473c049 None admin] RBAC: Authorizing `identity:get_user(user_id=3021c88c9cb3438bb4333e4066b51f19)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:01.971654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcc391-0748-47c9-8297-4ec3d473c049 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:01.976031 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 171/332] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:39:01 2022] GET /identity/v3/users/3021c88c9cb3438bb4333e4066b51f19 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:01.981942 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc391-0748-47c9-8297-4ec3d473c049 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:02.004627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:02.064840 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98ca97-7ccb-44ff-8a81-80e94f65f7d2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:02.104490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f8787-4926-494d-85f1-7b6191de84a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:02.105839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f8787-4926-494d-85f1-7b6191de84a8 None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:02.106323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f8787-4926-494d-85f1-7b6191de84a8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:02.106766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f8787-4926-494d-85f1-7b6191de84a8 None admin] PATH_INFO: `/v3/system/users/559ddbcdf530467d9d0db7ab3e3645dd/roles/9fc42581f1f84e28b6d65cdc76126fdc` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:02.109700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f8787-4926-494d-85f1-7b6191de84a8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=559ddbcdf530467d9d0db7ab3e3645dd, role_id=9fc42581f1f84e28b6d65cdc76126fdc)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:02.111431 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f8787-4926-494d-85f1-7b6191de84a8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:02.126798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 162/333] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:39:01 2022] PUT /identity/v3/system/users/559ddbcdf530467d9d0db7ab3e3645dd/roles/9fc42581f1f84e28b6d65cdc76126fdc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:02.146480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc391-0748-47c9-8297-4ec3d473c049 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:02.187275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de88006-6334-4959-a181-a1c307112043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:02.187922 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88006-6334-4959-a181-a1c307112043 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:02.188201 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88006-6334-4959-a181-a1c307112043 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:02.188552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88006-6334-4959-a181-a1c307112043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:02.189460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de88006-6334-4959-a181-a1c307112043 None admin] RBAC: Adding query filter params (role.id=914dcb9114e54c75a514ce6af60ffbb3, user.id=3021c88c9cb3438bb4333e4066b51f19, scope.system=all) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:02.189803 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de88006-6334-4959-a181-a1c307112043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:02.191378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de88006-6334-4959-a181-a1c307112043 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:02.206454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 172/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:39:01 2022] GET /identity/v3/role_assignments?user.id=3021c88c9cb3438bb4333e4066b51f19&scope.system=all&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:03.816995 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1de68-b623-452d-b0af-ddf8d09d468d None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:03.817568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1de68-b623-452d-b0af-ddf8d09d468d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:03.817762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1de68-b623-452d-b0af-ddf8d09d468d None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:03.819691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 163/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:03.828160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe38cd4-58b1-4396-85f3-7961407a6b4f None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:03.828648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe38cd4-58b1-4396-85f3-7961407a6b4f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:03.828757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe38cd4-58b1-4396-85f3-7961407a6b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:03.848662 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-9fe38cd4-58b1-4396-85f3-7961407a6b4f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:03.849993 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-9fe38cd4-58b1-4396-85f3-7961407a6b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:03.926056 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:03.936310 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 173/336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:03.978525 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1de68-b623-452d-b0af-ddf8d09d468d None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:03.995197 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:04.028210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea185-f003-4f83-a989-ca07df042800 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:04.028588 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea185-f003-4f83-a989-ca07df042800 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:04.028775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea185-f003-4f83-a989-ca07df042800 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:04.030488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 174/337] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:39:04.035751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a461-a345-482f-ac81-c9021b0dacb8 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:04.036037 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a461-a345-482f-ac81-c9021b0dacb8 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:04.036317 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a461-a345-482f-ac81-c9021b0dacb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:04.071230 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-c2e3a461-a345-482f-ac81-c9021b0dacb8 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:04.072531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-c2e3a461-a345-482f-ac81-c9021b0dacb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:04.129144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1de68-b623-452d-b0af-ddf8d09d468d None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:04.192617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8467f8-109a-441f-b35e-9790d96a2ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:04.193358 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8467f8-109a-441f-b35e-9790d96a2ab2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:04.193638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8467f8-109a-441f-b35e-9790d96a2ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:04.193918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8467f8-109a-441f-b35e-9790d96a2ab2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:04.223204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8467f8-109a-441f-b35e-9790d96a2ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:04.225061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 175/338] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-4b8467f8-109a-441f-b35e-9790d96a2ab2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:04.229138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:04.230353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:04.231508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 164/339] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:04.239054 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3a461-a345-482f-ac81-c9021b0dacb8 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:04.261650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8467f8-109a-441f-b35e-9790d96a2ab2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:04.389856 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3a461-a345-482f-ac81-c9021b0dacb8 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:04.390264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:04.435160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:04.445309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8467f8-109a-441f-b35e-9790d96a2ab2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:04.449850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:04.450982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:04.451519 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:04.452026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:04.453021 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:04.453591 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:04.460172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:04.470872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 176/340] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:04.496465 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:04.511741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b56a7e-e294-4264-aa74-26627608886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:04.512442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b56a7e-e294-4264-aa74-26627608886f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:04.512724 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b56a7e-e294-4264-aa74-26627608886f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:04.513003 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b56a7e-e294-4264-aa74-26627608886f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:04.534168 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b56a7e-e294-4264-aa74-26627608886f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-f3b56a7e-e294-4264-aa74-26627608886f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:04.543718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:04.544896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:04.545910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 165/341] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:39:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:04.559114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b56a7e-e294-4264-aa74-26627608886f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:04.756662 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278aba9-3eba-4041-8bf2-95e527d63c06 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:04.794583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b56a7e-e294-4264-aa74-26627608886f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:04.849331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:04.851084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:04.851652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:04.852232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:04.853319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92942e-7984-44d7-a683-7d7c3c994bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:04.855917 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92942e-7984-44d7-a683-7d7c3c994bae None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:04.855917 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92942e-7984-44d7-a683-7d7c3c994bae None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:04.856181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:04.856267 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92942e-7984-44d7-a683-7d7c3c994bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:04.856749 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:04.857711 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:04.859850 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:04.872146 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 166/342] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:04.880519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:04.954478 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:04.963799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92942e-7984-44d7-a683-7d7c3c994bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:04.965880 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92942e-7984-44d7-a683-7d7c3c994bae None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:05.017828 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216d44b-26ee-4a79-83d5-34899e07417a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:05.077050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4ba83-5d33-46d3-900a-620e1a97e5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:05.078059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4ba83-5d33-46d3-900a-620e1a97e5f1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.078542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4ba83-5d33-46d3-900a-620e1a97e5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.078989 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4ba83-5d33-46d3-900a-620e1a97e5f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.095090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 177/343] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:05.103002 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92942e-7984-44d7-a683-7d7c3c994bae None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:05.161553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:05.171799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4ba83-5d33-46d3-900a-620e1a97e5f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:05.173426 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4ba83-5d33-46d3-900a-620e1a97e5f1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:05.216654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:05.228489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92942e-7984-44d7-a683-7d7c3c994bae None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:05.266605 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6e17b-791c-469d-b7ec-28fc7cfde151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:05.267373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e17b-791c-469d-b7ec-28fc7cfde151 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.267766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e17b-791c-469d-b7ec-28fc7cfde151 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.268110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e17b-791c-469d-b7ec-28fc7cfde151 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.269274 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6e17b-791c-469d-b7ec-28fc7cfde151 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:05.270908 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6e17b-791c-469d-b7ec-28fc7cfde151 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:05.273084 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 178/344] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:05 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:05.274481 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84f289-87f7-4d50-af28-4b01bb6d192d None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.274765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84f289-87f7-4d50-af28-4b01bb6d192d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.275040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84f289-87f7-4d50-af28-4b01bb6d192d None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.276473 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 179/345] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:05.287626 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84f289-87f7-4d50-af28-4b01bb6d192d None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:05.293671 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 167/346] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:05.299073 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108863-df03-4b35-8d04-cf53a709b214 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108863-df03-4b35-8d04-cf53a709b214 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.299700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108863-df03-4b35-8d04-cf53a709b214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.320322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-64108863-df03-4b35-8d04-cf53a709b214 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:05.321542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-64108863-df03-4b35-8d04-cf53a709b214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:05.398227 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 168/347] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:05.401296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108863-df03-4b35-8d04-cf53a709b214 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:05.437457 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84f289-87f7-4d50-af28-4b01bb6d192d None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:05.449025 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:05.484502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe12e0-d598-4f11-87e2-d2a34ac53440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:05.485169 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe12e0-d598-4f11-87e2-d2a34ac53440 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.485463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe12e0-d598-4f11-87e2-d2a34ac53440 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.485725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe12e0-d598-4f11-87e2-d2a34ac53440 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.492338 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe12e0-d598-4f11-87e2-d2a34ac53440 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:05.493807 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe12e0-d598-4f11-87e2-d2a34ac53440 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:05.502241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 180/348] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:05 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:05.503148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe12e0-d598-4f11-87e2-d2a34ac53440 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:05.531364 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:05.544140 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108863-df03-4b35-8d04-cf53a709b214 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:05.597288 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0d07c-3d80-4014-a054-d5a9b30c8567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:05.599270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0d07c-3d80-4014-a054-d5a9b30c8567 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.600067 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0d07c-3d80-4014-a054-d5a9b30c8567 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.600587 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0d07c-3d80-4014-a054-d5a9b30c8567 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.602369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0d07c-3d80-4014-a054-d5a9b30c8567 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:05.604672 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0d07c-3d80-4014-a054-d5a9b30c8567 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:05.607169 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 169/349] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:05 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:05.608201 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0d07c-3d80-4014-a054-d5a9b30c8567 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:05.632718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe12e0-d598-4f11-87e2-d2a34ac53440 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:05.672775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec29669-a7bf-400c-84be-d0cac8f87601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:05.673623 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec29669-a7bf-400c-84be-d0cac8f87601 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.674009 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec29669-a7bf-400c-84be-d0cac8f87601 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.674413 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec29669-a7bf-400c-84be-d0cac8f87601 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.697711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec29669-a7bf-400c-84be-d0cac8f87601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-0ec29669-a7bf-400c-84be-d0cac8f87601 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:05.705167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:05.706345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 181/350] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:05.707396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94742b-4333-4f0d-a63e-70c309be8281 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.708237 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94742b-4333-4f0d-a63e-70c309be8281 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.708237 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94742b-4333-4f0d-a63e-70c309be8281 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.711883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 182/351] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:05.711883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94742b-4333-4f0d-a63e-70c309be8281 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:05.737676 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:05.752901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0d07c-3d80-4014-a054-d5a9b30c8567 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:05.795938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce676d-e689-43b6-9a5a-5bb12c7056c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:05.796915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce676d-e689-43b6-9a5a-5bb12c7056c9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.797403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce676d-e689-43b6-9a5a-5bb12c7056c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.797894 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce676d-e689-43b6-9a5a-5bb12c7056c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.798948 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce676d-e689-43b6-9a5a-5bb12c7056c9 None admin] RBAC: Adding query filter params (group.id=72466bfd7e5b4776bb37f6e4eaecfbd0, role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=cfd603839dbc47efa27c6534b49c18c1) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:05.799991 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce676d-e689-43b6-9a5a-5bb12c7056c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:05.801734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce676d-e689-43b6-9a5a-5bb12c7056c9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:05.810310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 170/352] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:05 2022] GET /identity/v3/role_assignments?group.id=72466bfd7e5b4776bb37f6e4eaecfbd0&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:05.812059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac05b8d-e27c-4ceb-adf8-2689c6b9c819 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.812607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac05b8d-e27c-4ceb-adf8-2689c6b9c819 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.813099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac05b8d-e27c-4ceb-adf8-2689c6b9c819 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.814778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 171/353] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:05.815852 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac05b8d-e27c-4ceb-adf8-2689c6b9c819 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:05.830504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:05.864571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94742b-4333-4f0d-a63e-70c309be8281 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:05.913232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912011bb-5b0c-46b6-8b44-5ef08587bc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:05.913957 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912011bb-5b0c-46b6-8b44-5ef08587bc1b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.914571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912011bb-5b0c-46b6-8b44-5ef08587bc1b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.914865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912011bb-5b0c-46b6-8b44-5ef08587bc1b None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.916364 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912011bb-5b0c-46b6-8b44-5ef08587bc1b None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:05.919085 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912011bb-5b0c-46b6-8b44-5ef08587bc1b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:05.921122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 183/354] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:05 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:05.922264 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cfef7-7a35-44eb-ae78-f93fcae6cbf1 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:05.922552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cfef7-7a35-44eb-ae78-f93fcae6cbf1 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:05.922832 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cfef7-7a35-44eb-ae78-f93fcae6cbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:05.947441 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-3f0cfef7-7a35-44eb-ae78-f93fcae6cbf1 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:05.949311 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-3f0cfef7-7a35-44eb-ae78-f93fcae6cbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:05.957144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac05b8d-e27c-4ceb-adf8-2689c6b9c819 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:06.013937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec55b5f-97d6-4b6a-901e-2effe7d6c688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:06.014626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec55b5f-97d6-4b6a-901e-2effe7d6c688 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:06.014912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec55b5f-97d6-4b6a-901e-2effe7d6c688 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:06.015330 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec55b5f-97d6-4b6a-901e-2effe7d6c688 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:06.016091 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec55b5f-97d6-4b6a-901e-2effe7d6c688 None admin] RBAC: Adding query filter params (name=member) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:06.018253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec55b5f-97d6-4b6a-901e-2effe7d6c688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:06.020261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec55b5f-97d6-4b6a-901e-2effe7d6c688 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:06.038139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 172/355] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:06.038139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f895-e672-43a9-ac5e-709dfe32b775 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:06.038139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f895-e672-43a9-ac5e-709dfe32b775 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:06.038542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f895-e672-43a9-ac5e-709dfe32b775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:06.064265 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-c843f895-e672-43a9-ac5e-709dfe32b775 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:06.065612 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-c843f895-e672-43a9-ac5e-709dfe32b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:06.076078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 184/356] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:06.076985 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cfef7-7a35-44eb-ae78-f93fcae6cbf1 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:06.092928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:06.202129 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 173/357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:06.202497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:06.203406 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843f895-e672-43a9-ac5e-709dfe32b775 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:06.213294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cfef7-7a35-44eb-ae78-f93fcae6cbf1 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:06.265265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:06.266809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:06.266809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:06.266809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:06.267465 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] RBAC: Adding query filter params (group.id=72466bfd7e5b4776bb37f6e4eaecfbd0, role.id=8d072b58656b442f922c3287a5430603, scope.project.id=cfd603839dbc47efa27c6534b49c18c1) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:06.270178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:06.270178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:06.282149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 185/358] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:06 2022] GET /identity/v3/role_assignments?group.id=72466bfd7e5b4776bb37f6e4eaecfbd0&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=8d072b58656b442f922c3287a5430603 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:06.282149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:06.413752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bfbae-f764-463a-804a-38ec054f0947 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:06.504505 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb7a80-d08f-4b5e-b90f-38516a4ee837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:06.507499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb7a80-d08f-4b5e-b90f-38516a4ee837 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:06.508896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb7a80-d08f-4b5e-b90f-38516a4ee837 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:06.509312 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843f895-e672-43a9-ac5e-709dfe32b775 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:06.509652 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb7a80-d08f-4b5e-b90f-38516a4ee837 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:06.538305 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb7a80-d08f-4b5e-b90f-38516a4ee837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-6edb7a80-d08f-4b5e-b90f-38516a4ee837 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:06.550426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:06.551582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:39:06.552861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:06.554051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf11d9-4ea2-45e6-916d-283c2f273223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:06.555361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf11d9-4ea2-45e6-916d-283c2f273223 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:06.556685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf11d9-4ea2-45e6-916d-283c2f273223 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:06.556685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf11d9-4ea2-45e6-916d-283c2f273223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:06.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 186/359] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:06.558846 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb7a80-d08f-4b5e-b90f-38516a4ee837 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:06.564817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:06.584759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:06.673920 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf11d9-4ea2-45e6-916d-283c2f273223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:06.675561 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf11d9-4ea2-45e6-916d-283c2f273223 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:06.774480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb7a80-d08f-4b5e-b90f-38516a4ee837 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:06.813393 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bf1f2-d7a5-4fc7-91f2-f654ab816fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:06.814636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bf1f2-d7a5-4fc7-91f2-f654ab816fcc None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:06.814801 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bf1f2-d7a5-4fc7-91f2-f654ab816fcc None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:06.815223 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bf1f2-d7a5-4fc7-91f2-f654ab816fcc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:06.825566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 174/360] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:06.827155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf998-8374-4dc2-8695-70bb715d567a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:06.827678 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf998-8374-4dc2-8695-70bb715d567a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:06.828181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf998-8374-4dc2-8695-70bb715d567a None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:06.830013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 175/361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:06.831972 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bf1f2-d7a5-4fc7-91f2-f654ab816fcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:06.832453 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf998-8374-4dc2-8695-70bb715d567a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-1a5bf1f2-d7a5-4fc7-91f2-f654ab816fcc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:06.838621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:06.839785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:06.840806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 187/362] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:39:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:06.841552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bf1f2-d7a5-4fc7-91f2-f654ab816fcc None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:06.898505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:06.915431 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:06.960209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf998-8374-4dc2-8695-70bb715d567a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:06.960754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bf1f2-d7a5-4fc7-91f2-f654ab816fcc None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:07.002604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.003064 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db61c9-d4d5-4dce-9dfd-5eadb08e97a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.003771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db61c9-d4d5-4dce-9dfd-5eadb08e97a7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.004118 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db61c9-d4d5-4dce-9dfd-5eadb08e97a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.004399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db61c9-d4d5-4dce-9dfd-5eadb08e97a7 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.004632 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.005133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.005748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.006043 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db61c9-d4d5-4dce-9dfd-5eadb08e97a7 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:07.007203 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] RBAC: Adding query filter params (name=reader) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:07.007617 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db61c9-d4d5-4dce-9dfd-5eadb08e97a7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:07.007983 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:07.009899 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 188/363] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:06 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:07.010489 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:07.011327 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9939a-4fd6-41cb-a2bc-626a24272b90 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.011629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9939a-4fd6-41cb-a2bc-626a24272b90 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.011897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9939a-4fd6-41cb-a2bc-626a24272b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.029970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 176/364] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:07.032786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-e1c9939a-4fd6-41cb-a2bc-626a24272b90 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:07.034055 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:07.034330 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-e1c9939a-4fd6-41cb-a2bc-626a24272b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:07.113119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 189/365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:07.114324 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9939a-4fd6-41cb-a2bc-626a24272b90 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:07.162786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd800dd-ad17-457c-a682-001ea9930dba None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:07.175069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:07.186332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:07.205984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.207191 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.207706 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.208199 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.210037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:07.210786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:07.212441 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:07.224169 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 177/366] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:07.225389 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:07.234854 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9939a-4fd6-41cb-a2bc-626a24272b90 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:07.275997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec01ba4-d3d2-496d-a087-a39bd6de0f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.276728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec01ba4-d3d2-496d-a087-a39bd6de0f19 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.277023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec01ba4-d3d2-496d-a087-a39bd6de0f19 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.277304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec01ba4-d3d2-496d-a087-a39bd6de0f19 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.278846 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec01ba4-d3d2-496d-a087-a39bd6de0f19 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:07.280363 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec01ba4-d3d2-496d-a087-a39bd6de0f19 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:07.282660 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 190/367] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:07.283458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec01ba4-d3d2-496d-a087-a39bd6de0f19 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:07.345096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139512-67a1-4a9d-aaf4-d2c130f1d33e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:07.382882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddd022-efbf-47c2-b1cf-6794b4b5dd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.384057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddd022-efbf-47c2-b1cf-6794b4b5dd33 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.384613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddd022-efbf-47c2-b1cf-6794b4b5dd33 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.385121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddd022-efbf-47c2-b1cf-6794b4b5dd33 None admin] PATH_INFO: `/v3/users/559ddbcdf530467d9d0db7ab3e3645dd` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.386693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddd022-efbf-47c2-b1cf-6794b4b5dd33 None admin] RBAC: Authorizing `identity:get_user(user_id=559ddbcdf530467d9d0db7ab3e3645dd)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:07.388562 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddd022-efbf-47c2-b1cf-6794b4b5dd33 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:07.390558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 178/368] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/users/559ddbcdf530467d9d0db7ab3e3645dd => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:07.391851 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddd022-efbf-47c2-b1cf-6794b4b5dd33 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:07.401621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec01ba4-d3d2-496d-a087-a39bd6de0f19 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:07.435108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:07.439857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:07.442195 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed71cc3-d973-45aa-be56-ebda01309b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.443562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed71cc3-d973-45aa-be56-ebda01309b1c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.444570 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed71cc3-d973-45aa-be56-ebda01309b1c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.445165 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed71cc3-d973-45aa-be56-ebda01309b1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.458387 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed71cc3-d973-45aa-be56-ebda01309b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-2ed71cc3-d973-45aa-be56-ebda01309b1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:07.464401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:07.465476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:07.466582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:07.469058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 191/369] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:07.470189 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed71cc3-d973-45aa-be56-ebda01309b1c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:07.510085 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddd022-efbf-47c2-b1cf-6794b4b5dd33 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:07.547241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac44d4f-5627-4143-9850-1a658c332ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.548044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac44d4f-5627-4143-9850-1a658c332ff4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.548446 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac44d4f-5627-4143-9850-1a658c332ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.548917 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac44d4f-5627-4143-9850-1a658c332ff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.583493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed71cc3-d973-45aa-be56-ebda01309b1c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:07.622883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.623877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.624444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.624892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.625982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] RBAC: Adding query filter params (group.id=72466bfd7e5b4776bb37f6e4eaecfbd0, role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=713f2c71f07c4deeb316712af011e233) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:07.626562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:07.628241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:07.628835 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac44d4f-5627-4143-9850-1a658c332ff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:07.630520 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac44d4f-5627-4143-9850-1a658c332ff4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:07.635963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 192/370] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/role_assignments?group.id=72466bfd7e5b4776bb37f6e4eaecfbd0&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:07.637488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:07.719425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:07.734220 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:07.785888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c53d-b94a-406d-88ea-e8ee92296dbe None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:07.835577 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:07.836701 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:07.837230 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:07.837744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:07.838896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] RBAC: Adding query filter params (role.id=9fc42581f1f84e28b6d65cdc76126fdc, user.id=559ddbcdf530467d9d0db7ab3e3645dd, scope.system=all) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:07.840078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:07.841840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:07.858641 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 193/371] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/role_assignments?user.id=559ddbcdf530467d9d0db7ab3e3645dd&scope.system=all&role.id=9fc42581f1f84e28b6d65cdc76126fdc => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:07.860119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:07.880554 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 179/372] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:07.904411 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac44d4f-5627-4143-9850-1a658c332ff4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:08.007012 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea8692-11ca-425f-a020-dbc5bb3d5ede None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:08.026592 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:08.055216 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:08.056206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.056598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.056981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.057866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:08.058313 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:08.060930 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:08.072971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 194/373] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:08.079479 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:08.106535 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac44d4f-5627-4143-9850-1a658c332ff4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:08.147515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755627d-7e94-4bc3-9655-e69aaee3eef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:08.148574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755627d-7e94-4bc3-9655-e69aaee3eef8 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.149061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755627d-7e94-4bc3-9655-e69aaee3eef8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.149548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755627d-7e94-4bc3-9655-e69aaee3eef8 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.153781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755627d-7e94-4bc3-9655-e69aaee3eef8 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:08.155726 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755627d-7e94-4bc3-9655-e69aaee3eef8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:08.158009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 180/374] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:07 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:08.166738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755627d-7e94-4bc3-9655-e69aaee3eef8 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:08.182906 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:08.227654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712bfd-e75b-4e5e-9590-29ad6cf9a680 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:08.265600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea6fd-2517-4881-bd32-9406c569388d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:08.266692 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea6fd-2517-4881-bd32-9406c569388d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.267233 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea6fd-2517-4881-bd32-9406c569388d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.267738 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea6fd-2517-4881-bd32-9406c569388d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.295353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755627d-7e94-4bc3-9655-e69aaee3eef8 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:08.317421 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:08.336475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04841cf-0746-4ed3-ae06-6a55fc2f1a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:08.337156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04841cf-0746-4ed3-ae06-6a55fc2f1a73 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.337444 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04841cf-0746-4ed3-ae06-6a55fc2f1a73 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.337726 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04841cf-0746-4ed3-ae06-6a55fc2f1a73 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.339055 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04841cf-0746-4ed3-ae06-6a55fc2f1a73 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:08.340553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04841cf-0746-4ed3-ae06-6a55fc2f1a73 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:08.342779 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 181/375] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:08 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:08.349002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04841cf-0746-4ed3-ae06-6a55fc2f1a73 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:08.349836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecea6fd-2517-4881-bd32-9406c569388d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:08.351636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecea6fd-2517-4881-bd32-9406c569388d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:08.437793 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:08.470159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04841cf-0746-4ed3-ae06-6a55fc2f1a73 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:08.471328 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 195/376] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:39:08.476756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea6fd-2517-4881-bd32-9406c569388d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:08.512164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c07ce-8adb-4de8-b548-3c30f9911292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:08.512822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c07ce-8adb-4de8-b548-3c30f9911292 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.513108 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c07ce-8adb-4de8-b548-3c30f9911292 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.513386 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c07ce-8adb-4de8-b548-3c30f9911292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.514229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c07ce-8adb-4de8-b548-3c30f9911292 None admin] RBAC: Adding query filter params (group.id=72466bfd7e5b4776bb37f6e4eaecfbd0, role.id=8d072b58656b442f922c3287a5430603, scope.project.id=713f2c71f07c4deeb316712af011e233) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:08.514546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c07ce-8adb-4de8-b548-3c30f9911292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:08.517800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c07ce-8adb-4de8-b548-3c30f9911292 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:08.535445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 182/377] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:08 2022] GET /identity/v3/role_assignments?group.id=72466bfd7e5b4776bb37f6e4eaecfbd0&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=8d072b58656b442f922c3287a5430603 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:08.588624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:08.603814 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f5223-c8cd-4921-9b95-aa7f2fd4d092 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.604187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f5223-c8cd-4921-9b95-aa7f2fd4d092 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.604369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f5223-c8cd-4921-9b95-aa7f2fd4d092 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.605844 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 183/378] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:39:08.611626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b482-1321-4797-9f57-546930f2df54 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.611961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b482-1321-4797-9f57-546930f2df54 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.612193 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b482-1321-4797-9f57-546930f2df54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.647417 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-cc21b482-1321-4797-9f57-546930f2df54 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:08.648679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-cc21b482-1321-4797-9f57-546930f2df54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:08.653071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea6fd-2517-4881-bd32-9406c569388d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:08.722546 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f91c56-7b7f-44c9-ad0e-6d066694f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:08.725019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f91c56-7b7f-44c9-ad0e-6d066694f16d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.725646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f91c56-7b7f-44c9-ad0e-6d066694f16d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.726351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f91c56-7b7f-44c9-ad0e-6d066694f16d None admin] PATH_INFO: `/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.729053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f91c56-7b7f-44c9-ad0e-6d066694f16d None admin] RBAC: Authorizing `identity:get_project(project_id=0c33208ac31c49d19dce8ea7c7b40c7a)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:08.731818 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f91c56-7b7f-44c9-ad0e-6d066694f16d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:08.736901 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 196/379] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:08 2022] GET /identity/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a => generated 344 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:08.745149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f91c56-7b7f-44c9-ad0e-6d066694f16d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:08.855696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 184/380] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:08.867433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c2df-b11b-46a2-a383-976762d2c232 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.867433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c2df-b11b-46a2-a383-976762d2c232 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.867748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c2df-b11b-46a2-a383-976762d2c232 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.869234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 185/381] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:39:08.880070 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae678-aa80-4e99-b30f-45aa5a671bbd None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.880758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae678-aa80-4e99-b30f-45aa5a671bbd None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.881270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae678-aa80-4e99-b30f-45aa5a671bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.892124 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f91c56-7b7f-44c9-ad0e-6d066694f16d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:08.903985 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-b83ae678-aa80-4e99-b30f-45aa5a671bbd None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:08.905256 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-b83ae678-aa80-4e99-b30f-45aa5a671bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:08.931745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b512da0-5b15-4ec6-8390-d53f19e2937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:08.932424 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b512da0-5b15-4ec6-8390-d53f19e2937c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:08.932708 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b512da0-5b15-4ec6-8390-d53f19e2937c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:08.932982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b512da0-5b15-4ec6-8390-d53f19e2937c None admin] PATH_INFO: `/v3/groups/924a45b4669f49f78c81e2f8f3c400a8` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:08.938589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b512da0-5b15-4ec6-8390-d53f19e2937c None admin] RBAC: Authorizing `identity:get_group(group_id=924a45b4669f49f78c81e2f8f3c400a8)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:08.940080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b512da0-5b15-4ec6-8390-d53f19e2937c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:08.942480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 197/382] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:08 2022] GET /identity/v3/groups/924a45b4669f49f78c81e2f8f3c400a8 => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:08.943524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b512da0-5b15-4ec6-8390-d53f19e2937c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:08.953644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:08.990020 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:08.990254 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 186/383] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:08.994891 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae678-aa80-4e99-b30f-45aa5a671bbd None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:09.063097 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b512da0-5b15-4ec6-8390-d53f19e2937c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:09.111946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040eee04-a4ba-4350-b353-714bb3c2d6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:09.112535 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040eee04-a4ba-4350-b353-714bb3c2d6ac None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:09.112817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040eee04-a4ba-4350-b353-714bb3c2d6ac None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:09.113101 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040eee04-a4ba-4350-b353-714bb3c2d6ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:09.127154 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040eee04-a4ba-4350-b353-714bb3c2d6ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-040eee04-a4ba-4350-b353-714bb3c2d6ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:09.133048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:09.134158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:09.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 198/384] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:09.135947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040eee04-a4ba-4350-b353-714bb3c2d6ac None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:09.141627 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae678-aa80-4e99-b30f-45aa5a671bbd None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:09.189299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:09.189949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:09.191291 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:09.191638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:09.192394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] RBAC: Adding query filter params (group.id=924a45b4669f49f78c81e2f8f3c400a8, role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=0c33208ac31c49d19dce8ea7c7b40c7a) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:09.192707 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:09.194672 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:09.201049 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 187/385] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:08 2022] GET /identity/v3/role_assignments?group.id=924a45b4669f49f78c81e2f8f3c400a8&scope.project.id=0c33208ac31c49d19dce8ea7c7b40c7a&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:09.201954 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:09.231367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:09.290353 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040eee04-a4ba-4350-b353-714bb3c2d6ac None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:09.313562 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:09.346738 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51a494-4a74-47e1-9b04-69a292a3cc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:09.347446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51a494-4a74-47e1-9b04-69a292a3cc9e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:09.347736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51a494-4a74-47e1-9b04-69a292a3cc9e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:09.348216 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51a494-4a74-47e1-9b04-69a292a3cc9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:09.366079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51a494-4a74-47e1-9b04-69a292a3cc9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:09.371401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2fe9d-80e9-4311-8464-56c4437d8ac9 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-3c51a494-4a74-47e1-9b04-69a292a3cc9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:09.373809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:09.375051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:09.376114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:09.376804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 199/386] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:39:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:09.382993 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51a494-4a74-47e1-9b04-69a292a3cc9e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:09.420842 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:09.421902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:09.422529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:09.422817 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:09.423731 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] RBAC: Adding query filter params (name=member) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:09.424048 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:09.426031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:09.439242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 188/387] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:09.445465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:09.535981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51a494-4a74-47e1-9b04-69a292a3cc9e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:09.573486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:09.587217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:09.588048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:09.590512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:09.590512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:09.590512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:09.590512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:09.593110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:09.603871 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 200/388] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:09.611252 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:09.621731 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68498d7-c512-4dab-b80f-3fa9607856ec None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:09.683154 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:09.710228 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac302b2-fee5-43e2-bc42-c99a6a6018f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:09.710903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac302b2-fee5-43e2-bc42-c99a6a6018f6 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:09.711173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac302b2-fee5-43e2-bc42-c99a6a6018f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:09.711439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac302b2-fee5-43e2-bc42-c99a6a6018f6 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:09.712796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac302b2-fee5-43e2-bc42-c99a6a6018f6 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:09.715150 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac302b2-fee5-43e2-bc42-c99a6a6018f6 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:09.716800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 189/389] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:09 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:09.724120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac302b2-fee5-43e2-bc42-c99a6a6018f6 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:09.844800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac302b2-fee5-43e2-bc42-c99a6a6018f6 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:09.853958 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f32afb-719f-496b-ab3a-d3968e290116 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:09.889090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389f352-d35b-4e2b-a0d3-8a2a1954bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:09.889774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389f352-d35b-4e2b-a0d3-8a2a1954bb7d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:09.890063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389f352-d35b-4e2b-a0d3-8a2a1954bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:09.890370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389f352-d35b-4e2b-a0d3-8a2a1954bb7d None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:09.892471 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389f352-d35b-4e2b-a0d3-8a2a1954bb7d None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:09.893969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389f352-d35b-4e2b-a0d3-8a2a1954bb7d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:09.899105 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 190/390] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:09 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:09.907106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389f352-d35b-4e2b-a0d3-8a2a1954bb7d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:09.926327 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692aa27-a84d-4adc-af62-8d299a90bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:09.927500 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692aa27-a84d-4adc-af62-8d299a90bb3b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:09.928902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692aa27-a84d-4adc-af62-8d299a90bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:09.929410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692aa27-a84d-4adc-af62-8d299a90bb3b None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:09.931796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692aa27-a84d-4adc-af62-8d299a90bb3b None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:09.935553 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692aa27-a84d-4adc-af62-8d299a90bb3b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:09.937887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 201/391] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:09 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:09.944413 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692aa27-a84d-4adc-af62-8d299a90bb3b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:10.021894 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:10.052163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:10.099893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389f352-d35b-4e2b-a0d3-8a2a1954bb7d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:10.129708 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692aa27-a84d-4adc-af62-8d299a90bb3b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:10.159987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde30c5-9069-4af8-bd3c-9b427595068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:10.160998 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde30c5-9069-4af8-bd3c-9b427595068e None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:10.161290 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde30c5-9069-4af8-bd3c-9b427595068e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:10.161671 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde30c5-9069-4af8-bd3c-9b427595068e None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1/groups/72466bfd7e5b4776bb37f6e4eaecfbd0/roles/914dcb9114e54c75a514ce6af60ffbb3` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:10.166056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde30c5-9069-4af8-bd3c-9b427595068e None admin] RBAC: Authorizing `identity:create_grant(project_id=cfd603839dbc47efa27c6534b49c18c1, group_id=72466bfd7e5b4776bb37f6e4eaecfbd0, role_id=914dcb9114e54c75a514ce6af60ffbb3)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:10.167628 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde30c5-9069-4af8-bd3c-9b427595068e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:10.183039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f30d1-cacb-45d8-963d-f08d945047bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:10.184057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f30d1-cacb-45d8-963d-f08d945047bc None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:10.184474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f30d1-cacb-45d8-963d-f08d945047bc None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:10.184762 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f30d1-cacb-45d8-963d-f08d945047bc None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:10.188974 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f30d1-cacb-45d8-963d-f08d945047bc None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:10.191025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f30d1-cacb-45d8-963d-f08d945047bc None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:10.193276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 202/392] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:09 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:10.196781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 191/393] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:39:09 2022] PUT /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1/groups/72466bfd7e5b4776bb37f6e4eaecfbd0/roles/914dcb9114e54c75a514ce6af60ffbb3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:10.199260 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f30d1-cacb-45d8-963d-f08d945047bc None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:10.327792 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f30d1-cacb-45d8-963d-f08d945047bc None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:10.357906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:10.369040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefc2bd-83dd-4a72-b23a-d14b81184dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:10.369749 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefc2bd-83dd-4a72-b23a-d14b81184dbc None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:10.370031 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefc2bd-83dd-4a72-b23a-d14b81184dbc None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:10.370378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefc2bd-83dd-4a72-b23a-d14b81184dbc None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1/groups/72466bfd7e5b4776bb37f6e4eaecfbd0/roles/8d072b58656b442f922c3287a5430603` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:10.372784 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefc2bd-83dd-4a72-b23a-d14b81184dbc None admin] RBAC: Authorizing `identity:create_grant(project_id=cfd603839dbc47efa27c6534b49c18c1, group_id=72466bfd7e5b4776bb37f6e4eaecfbd0, role_id=8d072b58656b442f922c3287a5430603)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:10.374328 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefc2bd-83dd-4a72-b23a-d14b81184dbc None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:10.394075 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 203/394] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:39:10 2022] PUT /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1/groups/72466bfd7e5b4776bb37f6e4eaecfbd0/roles/8d072b58656b442f922c3287a5430603 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:10.699646 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad64345-d35b-4a0f-b86f-f8aa966a76b1 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:10.699939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad64345-d35b-4a0f-b86f-f8aa966a76b1 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:10.700222 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad64345-d35b-4a0f-b86f-f8aa966a76b1 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:10.701927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 192/395] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:10.707475 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054e2c0-3fff-463a-b980-1429e773dbc8 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:10.707765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054e2c0-3fff-463a-b980-1429e773dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:10.708129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054e2c0-3fff-463a-b980-1429e773dbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:10.729799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-6054e2c0-3fff-463a-b980-1429e773dbc8 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:10.731038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-6054e2c0-3fff-463a-b980-1429e773dbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:10.820254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 204/396] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:10.845666 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad64345-d35b-4a0f-b86f-f8aa966a76b1 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:10.992005 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad64345-d35b-4a0f-b86f-f8aa966a76b1 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:11.021163 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:11.041904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee0e90-a83a-4e13-8b15-3f54eff3828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:11.042644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee0e90-a83a-4e13-8b15-3f54eff3828a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.042935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee0e90-a83a-4e13-8b15-3f54eff3828a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.043221 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee0e90-a83a-4e13-8b15-3f54eff3828a None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:11.057292 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee0e90-a83a-4e13-8b15-3f54eff3828a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-5cee0e90-a83a-4e13-8b15-3f54eff3828a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:11.065439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:11.066699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:11.068734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 193/397] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:11.079393 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054e2c0-3fff-463a-b980-1429e773dbc8 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:11.246497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054e2c0-3fff-463a-b980-1429e773dbc8 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:11.268399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:11.294388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:11.295100 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.295380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.295651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:11.296395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] RBAC: Adding query filter params (name=member) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:11.296700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:11.298170 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:11.310310 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 205/398] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:11.315945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee0e90-a83a-4e13-8b15-3f54eff3828a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:11.433542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee0e90-a83a-4e13-8b15-3f54eff3828a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:11.473744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b72210-dfdc-4665-ab94-fa8f85b5a860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:11.474614 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72210-dfdc-4665-ab94-fa8f85b5a860 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.474901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72210-dfdc-4665-ab94-fa8f85b5a860 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.475232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72210-dfdc-4665-ab94-fa8f85b5a860 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:11.478804 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b72210-dfdc-4665-ab94-fa8f85b5a860 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:11.480415 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b72210-dfdc-4665-ab94-fa8f85b5a860 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:11.482331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 194/399] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:11 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:11.487865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:11.578218 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dd4f7-d66d-4e09-8c54-f959f15f5f38 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.578445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dd4f7-d66d-4e09-8c54-f959f15f5f38 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.578680 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dd4f7-d66d-4e09-8c54-f959f15f5f38 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:11.580064 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 195/400] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:39:11.591882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52a0d5-e57a-4a0f-98aa-aa6547c04fe4 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.592227 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52a0d5-e57a-4a0f-98aa-aa6547c04fe4 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.592503 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52a0d5-e57a-4a0f-98aa-aa6547c04fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:11.622956 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-9f52a0d5-e57a-4a0f-98aa-aa6547c04fe4 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:11.623696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-9f52a0d5-e57a-4a0f-98aa-aa6547c04fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:11.644325 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5697307-543e-4fbb-b3c4-6c31e596989a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:11.725049 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:11.726865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8977ef4-425f-4dd5-b99b-8e386f00a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:11.727570 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8977ef4-425f-4dd5-b99b-8e386f00a13c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.727857 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8977ef4-425f-4dd5-b99b-8e386f00a13c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.728141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8977ef4-425f-4dd5-b99b-8e386f00a13c None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:11.729445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8977ef4-425f-4dd5-b99b-8e386f00a13c None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:11.730945 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8977ef4-425f-4dd5-b99b-8e386f00a13c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:11.732986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 206/401] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:11 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:11.739954 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8977ef4-425f-4dd5-b99b-8e386f00a13c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:11.753757 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 196/402] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:11.779955 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52a0d5-e57a-4a0f-98aa-aa6547c04fe4 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:11.854574 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:11.935509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8977ef4-425f-4dd5-b99b-8e386f00a13c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:11.941063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52a0d5-e57a-4a0f-98aa-aa6547c04fe4 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:11.973703 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9efd91-dfb3-453c-85b9-122d477067f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:11.974522 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9efd91-dfb3-453c-85b9-122d477067f2 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.974816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9efd91-dfb3-453c-85b9-122d477067f2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.975103 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9efd91-dfb3-453c-85b9-122d477067f2 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233/groups/72466bfd7e5b4776bb37f6e4eaecfbd0/roles/914dcb9114e54c75a514ce6af60ffbb3` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:11.977109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9efd91-dfb3-453c-85b9-122d477067f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=713f2c71f07c4deeb316712af011e233, group_id=72466bfd7e5b4776bb37f6e4eaecfbd0, role_id=914dcb9114e54c75a514ce6af60ffbb3)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:11.977780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27af5da-d3ea-4bae-9baa-1702f51675e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:11.978549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27af5da-d3ea-4bae-9baa-1702f51675e1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.978667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9efd91-dfb3-453c-85b9-122d477067f2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:11.978906 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27af5da-d3ea-4bae-9baa-1702f51675e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.979274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27af5da-d3ea-4bae-9baa-1702f51675e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:11.993132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27af5da-d3ea-4bae-9baa-1702f51675e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:11.996003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 207/403] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:39:11 2022] PUT /identity/v3/projects/713f2c71f07c4deeb316712af011e233/groups/72466bfd7e5b4776bb37f6e4eaecfbd0/roles/914dcb9114e54c75a514ce6af60ffbb3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:39:11.997463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909d47e-4ed9-43bc-9dc1-fbe8694d2a0b None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:11.997954 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909d47e-4ed9-43bc-9dc1-fbe8694d2a0b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:11.998445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909d47e-4ed9-43bc-9dc1-fbe8694d2a0b None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.000244 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 208/404] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-d27af5da-d3ea-4bae-9baa-1702f51675e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:12.005174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:12.006487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:12.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:12.010025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f296-b6c6-406b-bb05-f311821a846a None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.010524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f296-b6c6-406b-bb05-f311821a846a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.010962 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f296-b6c6-406b-bb05-f311821a846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.020053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 197/405] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:39:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:12.022077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27af5da-d3ea-4bae-9baa-1702f51675e1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:12.039500 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-b8f9f296-b6c6-406b-bb05-f311821a846a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:12.041208 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-b8f9f296-b6c6-406b-bb05-f311821a846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:12.099966 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:12.124295 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 209/406] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:12.163646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9f296-b6c6-406b-bb05-f311821a846a None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:12.177640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27af5da-d3ea-4bae-9baa-1702f51675e1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:12.215421 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:12.216199 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.216552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.216901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.217740 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:12.218150 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:12.219817 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:12.230072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 198/407] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:12.236160 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:12.253840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:12.344508 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9f296-b6c6-406b-bb05-f311821a846a None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:12.381445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224ab16-7a9f-494b-a41d-8ecc514485df None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:12.383788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:12.386736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f74ed-d84e-4238-b7b0-1ab123169cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:12.388590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f74ed-d84e-4238-b7b0-1ab123169cf3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.388590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f74ed-d84e-4238-b7b0-1ab123169cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.388590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f74ed-d84e-4238-b7b0-1ab123169cf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.404840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f74ed-d84e-4238-b7b0-1ab123169cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-908f74ed-d84e-4238-b7b0-1ab123169cf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:12.409920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:12.411209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:12.412543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 210/408] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:12.421124 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f74ed-d84e-4238-b7b0-1ab123169cf3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:12.429843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62680f-cd5c-417c-ba7a-39c088a89773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:12.430902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62680f-cd5c-417c-ba7a-39c088a89773 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.432409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62680f-cd5c-417c-ba7a-39c088a89773 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.432921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62680f-cd5c-417c-ba7a-39c088a89773 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.434428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62680f-cd5c-417c-ba7a-39c088a89773 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:12.436197 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62680f-cd5c-417c-ba7a-39c088a89773 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:12.444475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 199/409] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:12 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:12.445820 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62680f-cd5c-417c-ba7a-39c088a89773 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:12.566201 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62680f-cd5c-417c-ba7a-39c088a89773 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:12.612342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098dcca-55a1-4a35-80c2-d3c4fa5b0878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:12.613462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098dcca-55a1-4a35-80c2-d3c4fa5b0878 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.614570 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098dcca-55a1-4a35-80c2-d3c4fa5b0878 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.614950 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098dcca-55a1-4a35-80c2-d3c4fa5b0878 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.618878 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098dcca-55a1-4a35-80c2-d3c4fa5b0878 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:12.621331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098dcca-55a1-4a35-80c2-d3c4fa5b0878 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:12.624265 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 200/410] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:12 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:12.630462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098dcca-55a1-4a35-80c2-d3c4fa5b0878 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:12.660089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:12.673547 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f74ed-d84e-4238-b7b0-1ab123169cf3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:12.683885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:12.758177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:12.758837 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.759120 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.759398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.760140 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:12.760448 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:12.761844 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:12.762080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098dcca-55a1-4a35-80c2-d3c4fa5b0878 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:12.783876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 211/411] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:12.790021 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:12.826929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118917b-86f8-4096-9a5c-890ea17ccda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:12.827656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118917b-86f8-4096-9a5c-890ea17ccda8 None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.827942 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118917b-86f8-4096-9a5c-890ea17ccda8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.828219 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118917b-86f8-4096-9a5c-890ea17ccda8 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233/groups/72466bfd7e5b4776bb37f6e4eaecfbd0/roles/8d072b58656b442f922c3287a5430603` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.835609 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118917b-86f8-4096-9a5c-890ea17ccda8 None admin] RBAC: Authorizing `identity:create_grant(project_id=713f2c71f07c4deeb316712af011e233, group_id=72466bfd7e5b4776bb37f6e4eaecfbd0, role_id=8d072b58656b442f922c3287a5430603)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:12.837115 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118917b-86f8-4096-9a5c-890ea17ccda8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:12.882223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 201/412] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:39:12 2022] PUT /identity/v3/projects/713f2c71f07c4deeb316712af011e233/groups/72466bfd7e5b4776bb37f6e4eaecfbd0/roles/8d072b58656b442f922c3287a5430603 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:12.884073 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c18f83-afcb-418d-a580-dd701ac1e84c None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.884404 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c18f83-afcb-418d-a580-dd701ac1e84c None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.884689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c18f83-afcb-418d-a580-dd701ac1e84c None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.886453 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 202/413] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:12.892318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeafea-8869-4375-85b1-b03142ff8d69 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.892698 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeafea-8869-4375-85b1-b03142ff8d69 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.892903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeafea-8869-4375-85b1-b03142ff8d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.924304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728908f-2806-42fe-a095-20cf98316f50 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:12.935368 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-a5aeafea-8869-4375-85b1-b03142ff8d69 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:12.936896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-a5aeafea-8869-4375-85b1-b03142ff8d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:12.970076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fcdfb-4aab-4d4a-b5ad-845f7a5300bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:12.971395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fcdfb-4aab-4d4a-b5ad-845f7a5300bf None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:12.971899 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fcdfb-4aab-4d4a-b5ad-845f7a5300bf None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:12.972607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fcdfb-4aab-4d4a-b5ad-845f7a5300bf None admin] PATH_INFO: `/v3/groups/924a45b4669f49f78c81e2f8f3c400a8` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:12.975620 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fcdfb-4aab-4d4a-b5ad-845f7a5300bf None admin] RBAC: Authorizing `identity:get_group(group_id=924a45b4669f49f78c81e2f8f3c400a8)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:12.979212 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fcdfb-4aab-4d4a-b5ad-845f7a5300bf None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:12.986044 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 212/414] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:12 2022] GET /identity/v3/groups/924a45b4669f49f78c81e2f8f3c400a8 => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:12.992014 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fcdfb-4aab-4d4a-b5ad-845f7a5300bf None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:13.047920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:13.067905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 203/415] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:13.105202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aeafea-8869-4375-85b1-b03142ff8d69 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:13.148255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:13.152196 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fcdfb-4aab-4d4a-b5ad-845f7a5300bf None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:13.193385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca1faf-fcec-4786-a91e-7fe6da2a92ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:13.193964 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca1faf-fcec-4786-a91e-7fe6da2a92ce None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.194659 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca1faf-fcec-4786-a91e-7fe6da2a92ce None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.195001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca1faf-fcec-4786-a91e-7fe6da2a92ce None admin] PATH_INFO: `/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.196644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca1faf-fcec-4786-a91e-7fe6da2a92ce None admin] RBAC: Authorizing `identity:get_project(project_id=0c33208ac31c49d19dce8ea7c7b40c7a)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:13.198272 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca1faf-fcec-4786-a91e-7fe6da2a92ce None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:13.200484 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 213/416] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:12 2022] GET /identity/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:13.201697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3c21-a01f-40c2-9dfb-ef37c0ff6bcf None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.201983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3c21-a01f-40c2-9dfb-ef37c0ff6bcf None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.202337 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3c21-a01f-40c2-9dfb-ef37c0ff6bcf None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.203725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 214/417] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:13.208863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf3c21-a01f-40c2-9dfb-ef37c0ff6bcf None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:13.247510 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aeafea-8869-4375-85b1-b03142ff8d69 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:13.283832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae8cb6-14b0-487f-a88b-418e699afe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:13.284670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8cb6-14b0-487f-a88b-418e699afe65 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.285044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8cb6-14b0-487f-a88b-418e699afe65 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.285574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8cb6-14b0-487f-a88b-418e699afe65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.298411 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae8cb6-14b0-487f-a88b-418e699afe65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-ceae8cb6-14b0-487f-a88b-418e699afe65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:13.305375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:13.306719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:13.308060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:13.313590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 204/418] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:39:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:13.313590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7c0dd-498c-4c61-98a0-838fb7e984e4 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.313590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7c0dd-498c-4c61-98a0-838fb7e984e4 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.313590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7c0dd-498c-4c61-98a0-838fb7e984e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.330586 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:13.339321 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-1bc7c0dd-498c-4c61-98a0-838fb7e984e4 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:13.341028 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-1bc7c0dd-498c-4c61-98a0-838fb7e984e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:13.345423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf3c21-a01f-40c2-9dfb-ef37c0ff6bcf None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:13.390254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3751c75-cf20-4fae-bed1-d8abd4175bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:13.391945 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3751c75-cf20-4fae-bed1-d8abd4175bf2 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.392467 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3751c75-cf20-4fae-bed1-d8abd4175bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.392973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3751c75-cf20-4fae-bed1-d8abd4175bf2 None admin] PATH_INFO: `/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a/groups/924a45b4669f49f78c81e2f8f3c400a8/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.396336 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3751c75-cf20-4fae-bed1-d8abd4175bf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c33208ac31c49d19dce8ea7c7b40c7a, group_id=924a45b4669f49f78c81e2f8f3c400a8, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:13.398492 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3751c75-cf20-4fae-bed1-d8abd4175bf2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:13.420749 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 215/419] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:39:13 2022] PUT /identity/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a/groups/924a45b4669f49f78c81e2f8f3c400a8/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:13.422134 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3751c75-cf20-4fae-bed1-d8abd4175bf2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:13.424399 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 205/420] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:13.448985 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7c0dd-498c-4c61-98a0-838fb7e984e4 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:13.470847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:13.574531 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7c0dd-498c-4c61-98a0-838fb7e984e4 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:13.590448 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3751c75-cf20-4fae-bed1-d8abd4175bf2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:13.613489 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e67bd2-fcf2-4a3a-b6a0-8486cb6b2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:13.614467 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67bd2-fcf2-4a3a-b6a0-8486cb6b2dae None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.614963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67bd2-fcf2-4a3a-b6a0-8486cb6b2dae None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.615336 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67bd2-fcf2-4a3a-b6a0-8486cb6b2dae None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.627456 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e67bd2-fcf2-4a3a-b6a0-8486cb6b2dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-86e67bd2-fcf2-4a3a-b6a0-8486cb6b2dae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:13.633530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:13.634961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:13.636239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:13.638063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 206/421] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:13.640644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:13.641511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.641969 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.642439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.643403 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:13.643876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:13.644650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e67bd2-fcf2-4a3a-b6a0-8486cb6b2dae None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:13.645842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:13.663709 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 216/422] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:13.669410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:13.703176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:13.787630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:13.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e67bd2-fcf2-4a3a-b6a0-8486cb6b2dae None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:13.828722 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba2756-9a2b-4e58-8e59-46b67d837dd1 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:13.884589 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:13.886290 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.886564 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.886843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.887591 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:13.887897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:13.891254 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:13.899833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f7cfa-4ca2-4dce-a734-74d71b3d11c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:13.900668 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7cfa-4ca2-4dce-a734-74d71b3d11c5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:13.900951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7cfa-4ca2-4dce-a734-74d71b3d11c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:13.901226 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7cfa-4ca2-4dce-a734-74d71b3d11c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:13.914248 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 207/423] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:13.920093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:14.031152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f7cfa-4ca2-4dce-a734-74d71b3d11c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:14.033765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f7cfa-4ca2-4dce-a734-74d71b3d11c5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:14.065952 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:14.107164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d738c3-1cf8-4b68-b209-4edc854048d3 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:14.154772 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6687a5-4e80-455f-97b4-64bb80ff8131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:14.156140 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6687a5-4e80-455f-97b4-64bb80ff8131 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:14.156400 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6687a5-4e80-455f-97b4-64bb80ff8131 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:14.156681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6687a5-4e80-455f-97b4-64bb80ff8131 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:14.181782 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:14.256074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 217/424] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:14.259875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f7cfa-4ca2-4dce-a734-74d71b3d11c5 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:14.308603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6687a5-4e80-455f-97b4-64bb80ff8131 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:14.311570 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6687a5-4e80-455f-97b4-64bb80ff8131 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:14.463381 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f7cfa-4ca2-4dce-a734-74d71b3d11c5 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:14.494845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 208/425] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:14.498536 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:14.500510 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6687a5-4e80-455f-97b4-64bb80ff8131 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:14.551099 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2385f-ae4b-4d0c-af5a-7dab4ffc11f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:14.552204 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2385f-ae4b-4d0c-af5a-7dab4ffc11f1 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:14.552697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2385f-ae4b-4d0c-af5a-7dab4ffc11f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:14.553128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2385f-ae4b-4d0c-af5a-7dab4ffc11f1 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:14.556599 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:14.556755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2385f-ae4b-4d0c-af5a-7dab4ffc11f1 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:14.557750 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2385f-ae4b-4d0c-af5a-7dab4ffc11f1 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:14.560047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 218/426] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:14 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:14.565074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2385f-ae4b-4d0c-af5a-7dab4ffc11f1 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:14.680639 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6687a5-4e80-455f-97b4-64bb80ff8131 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:14.741426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2385f-ae4b-4d0c-af5a-7dab4ffc11f1 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:14.758378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b3425-8712-4ec1-a595-a69e55c5d3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:14.759063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3425-8712-4ec1-a595-a69e55c5d3d2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:14.759342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3425-8712-4ec1-a595-a69e55c5d3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:14.759615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3425-8712-4ec1-a595-a69e55c5d3d2 None admin] PATH_INFO: `/v3/projects/cfd603839dbc47efa27c6534b49c18c1` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:14.762805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b3425-8712-4ec1-a595-a69e55c5d3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd603839dbc47efa27c6534b49c18c1)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:14.764473 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b3425-8712-4ec1-a595-a69e55c5d3d2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:14.766670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 209/427] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:14 2022] GET /identity/v3/projects/cfd603839dbc47efa27c6534b49c18c1 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:14.772941 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b3425-8712-4ec1-a595-a69e55c5d3d2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:14.797599 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58403ba8-1070-4606-898e-e510decdb73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:14.798311 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58403ba8-1070-4606-898e-e510decdb73b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:14.798598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58403ba8-1070-4606-898e-e510decdb73b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:14.798873 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58403ba8-1070-4606-898e-e510decdb73b None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:14.801651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58403ba8-1070-4606-898e-e510decdb73b None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:14.805346 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58403ba8-1070-4606-898e-e510decdb73b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:14.810810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 219/428] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:14 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:14.813540 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58403ba8-1070-4606-898e-e510decdb73b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:14.902441 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:14.948084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b3425-8712-4ec1-a595-a69e55c5d3d2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:14.959229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:14.971473 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58403ba8-1070-4606-898e-e510decdb73b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:14.987520 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a5aac-09c7-4dca-ba71-49531b1ee020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:14.988178 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a5aac-09c7-4dca-ba71-49531b1ee020 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:14.988457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a5aac-09c7-4dca-ba71-49531b1ee020 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:14.988732 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a5aac-09c7-4dca-ba71-49531b1ee020 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:14.990156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a5aac-09c7-4dca-ba71-49531b1ee020 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:14.991740 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a5aac-09c7-4dca-ba71-49531b1ee020 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:14.993718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 210/429] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:14 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:14.995272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca739d-0d8d-4cdd-879b-c2cd7696fe8a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:14.995557 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca739d-0d8d-4cdd-879b-c2cd7696fe8a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:14.995831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca739d-0d8d-4cdd-879b-c2cd7696fe8a None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:14.997218 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 211/430] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:15.002334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca739d-0d8d-4cdd-879b-c2cd7696fe8a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:15.016737 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c58744-951f-46f8-8c0e-4e7e9acfb7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:15.017762 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c58744-951f-46f8-8c0e-4e7e9acfb7d4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.018124 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c58744-951f-46f8-8c0e-4e7e9acfb7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.018486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c58744-951f-46f8-8c0e-4e7e9acfb7d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.020087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c58744-951f-46f8-8c0e-4e7e9acfb7d4 None admin] RBAC: Adding query filter params (group.id=72466bfd7e5b4776bb37f6e4eaecfbd0, role.id=8d072b58656b442f922c3287a5430603, scope.project.id=cfd603839dbc47efa27c6534b49c18c1) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:15.020474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c58744-951f-46f8-8c0e-4e7e9acfb7d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:15.022645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c58744-951f-46f8-8c0e-4e7e9acfb7d4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:15.030677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 220/431] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:14 2022] GET /identity/v3/role_assignments?group.id=72466bfd7e5b4776bb37f6e4eaecfbd0&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=8d072b58656b442f922c3287a5430603 => generated 611 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:15.032089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83b21f-6cfb-4bf5-bfbd-fe0028c2bb86 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.032381 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83b21f-6cfb-4bf5-bfbd-fe0028c2bb86 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.032678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83b21f-6cfb-4bf5-bfbd-fe0028c2bb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.052266 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-ad83b21f-6cfb-4bf5-bfbd-fe0028c2bb86 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:15.053741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-ad83b21f-6cfb-4bf5-bfbd-fe0028c2bb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:15.141177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 221/432] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:15.145347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca739d-0d8d-4cdd-879b-c2cd7696fe8a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:15.147224 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9844a7-89a6-4969-9764-7fc77c21e333 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.147498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9844a7-89a6-4969-9764-7fc77c21e333 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.147858 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9844a7-89a6-4969-9764-7fc77c21e333 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.154259 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 222/433] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:39:15.160706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411df12-9bb0-4c9b-ba9b-0b881e4d3f06 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.161100 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411df12-9bb0-4c9b-ba9b-0b881e4d3f06 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.161270 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411df12-9bb0-4c9b-ba9b-0b881e4d3f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.186787 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:15.187446 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.187725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.188016 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.188843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] RBAC: Adding query filter params (group.id=72466bfd7e5b4776bb37f6e4eaecfbd0, role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=cfd603839dbc47efa27c6534b49c18c1) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:15.189150 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:15.190687 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:15.193817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-9411df12-9bb0-4c9b-ba9b-0b881e4d3f06 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:15.195585 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-9411df12-9bb0-4c9b-ba9b-0b881e4d3f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:15.198360 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 212/434] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:15 2022] GET /identity/v3/role_assignments?group.id=72466bfd7e5b4776bb37f6e4eaecfbd0&scope.project.id=cfd603839dbc47efa27c6534b49c18c1&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:15.199235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:15.246607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:15.247842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:15.281057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 223/435] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:15.317419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411df12-9bb0-4c9b-ba9b-0b881e4d3f06 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:15.319359 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d3377-c29c-48d7-8c9f-64b4dc5f096f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:15.358495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89749cdf-1f03-4f70-9d73-f1d816764e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:15.359170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89749cdf-1f03-4f70-9d73-f1d816764e97 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.359448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89749cdf-1f03-4f70-9d73-f1d816764e97 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.359721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89749cdf-1f03-4f70-9d73-f1d816764e97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.375261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89749cdf-1f03-4f70-9d73-f1d816764e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-89749cdf-1f03-4f70-9d73-f1d816764e97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:15.380889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:15.383390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:15.385833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 213/436] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:15.390185 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89749cdf-1f03-4f70-9d73-f1d816764e97 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:15.439601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411df12-9bb0-4c9b-ba9b-0b881e4d3f06 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:15.484668 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472c61e-b3fb-48d8-8206-71e42f04bc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:15.485371 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472c61e-b3fb-48d8-8206-71e42f04bc73 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.485652 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472c61e-b3fb-48d8-8206-71e42f04bc73 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.485898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472c61e-b3fb-48d8-8206-71e42f04bc73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.500438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472c61e-b3fb-48d8-8206-71e42f04bc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-b472c61e-b3fb-48d8-8206-71e42f04bc73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:15.505853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:15.507015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:15.508051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 224/437] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:39:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:15.513241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472c61e-b3fb-48d8-8206-71e42f04bc73 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:15.522970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:15.527532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89749cdf-1f03-4f70-9d73-f1d816764e97 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:15.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:15.581118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:15.581770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.582050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.582397 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.583139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] RBAC: Adding query filter params (name=member) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:15.583586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:15.585006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:15.595643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 214/438] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:39:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:15.602264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:15.656573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472c61e-b3fb-48d8-8206-71e42f04bc73 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:15.705742 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:15.706422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.706708 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.706986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.707724 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:15.708040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:15.709497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:15.721300 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 225/439] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:15.726806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:15.753291 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ba9a4-e318-4423-8e4c-6935b0f25391 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:15.800633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a93c8d-4b2f-44e4-901c-40ec56fc1fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:15.801293 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93c8d-4b2f-44e4-901c-40ec56fc1fea None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.801576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93c8d-4b2f-44e4-901c-40ec56fc1fea None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.801851 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93c8d-4b2f-44e4-901c-40ec56fc1fea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.848127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:15.852791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:15.856816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397df65-c163-4931-9a1c-b2ca9aff5e93 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:15.901979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77104ca-5c12-4cb7-aab5-8181b6616b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:15.902793 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77104ca-5c12-4cb7-aab5-8181b6616b1c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:15.903147 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77104ca-5c12-4cb7-aab5-8181b6616b1c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:15.903438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77104ca-5c12-4cb7-aab5-8181b6616b1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:15.906229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a93c8d-4b2f-44e4-901c-40ec56fc1fea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:15.909121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a93c8d-4b2f-44e4-901c-40ec56fc1fea None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:15.983761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77104ca-5c12-4cb7-aab5-8181b6616b1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:15.985150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77104ca-5c12-4cb7-aab5-8181b6616b1c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:16.018166 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 215/440] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:39:16.020800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee37156-464a-462d-8579-7bcfbdb38718 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.021098 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee37156-464a-462d-8579-7bcfbdb38718 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.021381 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee37156-464a-462d-8579-7bcfbdb38718 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.022965 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 216/441] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:16.027037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee37156-464a-462d-8579-7bcfbdb38718 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.079166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:16.086471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 226/442] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:16.087823 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fa671-c367-48f5-bb75-5c76a10d343d None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.088117 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fa671-c367-48f5-bb75-5c76a10d343d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.088398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fa671-c367-48f5-bb75-5c76a10d343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.096374 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:16.107703 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-f26fa671-c367-48f5-bb75-5c76a10d343d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:16.108909 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-f26fa671-c367-48f5-bb75-5c76a10d343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:16.138390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee37156-464a-462d-8579-7bcfbdb38718 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.173044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff36f6d-623d-42c4-9cb2-b37e946f08b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.173695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff36f6d-623d-42c4-9cb2-b37e946f08b4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.173975 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff36f6d-623d-42c4-9cb2-b37e946f08b4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.174302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff36f6d-623d-42c4-9cb2-b37e946f08b4 None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.174543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 227/443] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:16.175507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff36f6d-623d-42c4-9cb2-b37e946f08b4 None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:16.175655 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fa671-c367-48f5-bb75-5c76a10d343d None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.177024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff36f6d-623d-42c4-9cb2-b37e946f08b4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:16.179413 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 217/444] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:16.184655 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff36f6d-623d-42c4-9cb2-b37e946f08b4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.277711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fa671-c367-48f5-bb75-5c76a10d343d None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.295018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff36f6d-623d-42c4-9cb2-b37e946f08b4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.303341 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:16.312289 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffaab4d-49b3-4bcf-91c3-798369f4199d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.312960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffaab4d-49b3-4bcf-91c3-798369f4199d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.313248 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffaab4d-49b3-4bcf-91c3-798369f4199d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.313529 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffaab4d-49b3-4bcf-91c3-798369f4199d None admin] PATH_INFO: `/v3/projects/713f2c71f07c4deeb316712af011e233` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.314746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffaab4d-49b3-4bcf-91c3-798369f4199d None admin] RBAC: Authorizing `identity:get_project(project_id=713f2c71f07c4deeb316712af011e233)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:16.316207 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffaab4d-49b3-4bcf-91c3-798369f4199d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:16.317942 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 228/445] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/projects/713f2c71f07c4deeb316712af011e233 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:16.318757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffaab4d-49b3-4bcf-91c3-798369f4199d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.327515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:16.334201 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef19de-b21b-4aef-ba5c-578cad119345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.334884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef19de-b21b-4aef-ba5c-578cad119345 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.335162 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef19de-b21b-4aef-ba5c-578cad119345 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.335435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef19de-b21b-4aef-ba5c-578cad119345 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.336631 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef19de-b21b-4aef-ba5c-578cad119345 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:16.338124 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef19de-b21b-4aef-ba5c-578cad119345 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:16.339761 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 218/446] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:16.340529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef19de-b21b-4aef-ba5c-578cad119345 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.424320 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffaab4d-49b3-4bcf-91c3-798369f4199d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.442596 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef19de-b21b-4aef-ba5c-578cad119345 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.459702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e77c-30cc-4016-a246-41b9ea6ca8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.460373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e77c-30cc-4016-a246-41b9ea6ca8e9 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.460659 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e77c-30cc-4016-a246-41b9ea6ca8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.460935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e77c-30cc-4016-a246-41b9ea6ca8e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.471414 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e77c-30cc-4016-a246-41b9ea6ca8e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:16.476733 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef303213-23b5-4678-96d2-0366d16f92c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-b279e77c-30cc-4016-a246-41b9ea6ca8e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:16.476951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:16.478279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:16.479748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:16.480588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef303213-23b5-4678-96d2-0366d16f92c9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.480588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef303213-23b5-4678-96d2-0366d16f92c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.480588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef303213-23b5-4678-96d2-0366d16f92c9 None admin] PATH_INFO: `/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.480848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 229/447] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:16.480848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e77c-30cc-4016-a246-41b9ea6ca8e9 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.483839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef303213-23b5-4678-96d2-0366d16f92c9 None admin] RBAC: Authorizing `identity:get_group(group_id=72466bfd7e5b4776bb37f6e4eaecfbd0)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:16.485526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef303213-23b5-4678-96d2-0366d16f92c9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:16.487408 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 219/448] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/groups/72466bfd7e5b4776bb37f6e4eaecfbd0 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:16.488190 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef303213-23b5-4678-96d2-0366d16f92c9 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.537970 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:16.556235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:16.585775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e77c-30cc-4016-a246-41b9ea6ca8e9 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.593521 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef303213-23b5-4678-96d2-0366d16f92c9 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.621166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.621813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.622115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.622410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.623221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] RBAC: Adding query filter params (group.id=72466bfd7e5b4776bb37f6e4eaecfbd0, role.id=914dcb9114e54c75a514ce6af60ffbb3, scope.project.id=713f2c71f07c4deeb316712af011e233) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:16.623541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:16.624998 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:16.630974 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.631606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.631884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.632157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.632875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:16.633177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:16.634644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:16.636441 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 230/449] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/role_assignments?group.id=72466bfd7e5b4776bb37f6e4eaecfbd0&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=914dcb9114e54c75a514ce6af60ffbb3 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:16.637889 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.651297 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 220/450] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:16.652651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:16.762469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19bf90-e5e2-440a-83c4-1ac00f5f93d0 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.779233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75030af7-cc16-48d6-b0ce-980893163a49 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:16.783986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:16.812796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.813463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:16.813713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.814014 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.814562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.815370 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] RBAC: Adding query filter params (group.id=72466bfd7e5b4776bb37f6e4eaecfbd0, role.id=8d072b58656b442f922c3287a5430603, scope.project.id=713f2c71f07c4deeb316712af011e233) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:16.816120 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:16.817740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:16.824017 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8782c-8d62-4e7b-a820-48ef986c161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:16.825139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8782c-8d62-4e7b-a820-48ef986c161a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:16.825139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8782c-8d62-4e7b-a820-48ef986c161a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:16.825536 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8782c-8d62-4e7b-a820-48ef986c161a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:16.826608 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 231/451] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/role_assignments?group.id=72466bfd7e5b4776bb37f6e4eaecfbd0&scope.project.id=713f2c71f07c4deeb316712af011e233&role.id=8d072b58656b442f922c3287a5430603 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:16.920344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8782c-8d62-4e7b-a820-48ef986c161a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:16.922075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8782c-8d62-4e7b-a820-48ef986c161a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:17.035325 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 221/452] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:17.041254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:17.149822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9bced-161a-4cc4-9025-a1ebce45feb9 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:17.185855 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf669c7-5ed5-4159-8a24-a79d03df7c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:17.186580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf669c7-5ed5-4159-8a24-a79d03df7c0d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:17.186868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf669c7-5ed5-4159-8a24-a79d03df7c0d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:17.187150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf669c7-5ed5-4159-8a24-a79d03df7c0d None admin] PATH_INFO: `/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:17.188417 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf669c7-5ed5-4159-8a24-a79d03df7c0d None admin] RBAC: Authorizing `identity:get_project(project_id=0c33208ac31c49d19dce8ea7c7b40c7a)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:17.189898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf669c7-5ed5-4159-8a24-a79d03df7c0d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:17.191724 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 232/453] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:39:17 2022] GET /identity/v3/projects/0c33208ac31c49d19dce8ea7c7b40c7a => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:17.198180 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8782c-8d62-4e7b-a820-48ef986c161a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:17.226943 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:17.303646 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8782c-8d62-4e7b-a820-48ef986c161a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:17.339802 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6abbf-f7ce-454c-bc58-288efedc12c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:17.340451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6abbf-f7ce-454c-bc58-288efedc12c0 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:17.340736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6abbf-f7ce-454c-bc58-288efedc12c0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:17.341013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6abbf-f7ce-454c-bc58-288efedc12c0 None admin] PATH_INFO: `/v3/groups/924a45b4669f49f78c81e2f8f3c400a8` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:17.342265 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6abbf-f7ce-454c-bc58-288efedc12c0 None admin] RBAC: Authorizing `identity:get_group(group_id=924a45b4669f49f78c81e2f8f3c400a8)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:17.343757 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6abbf-f7ce-454c-bc58-288efedc12c0 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:17.345388 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 222/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:39:17 2022] GET /identity/v3/groups/924a45b4669f49f78c81e2f8f3c400a8 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:17.352347 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf669c7-5ed5-4159-8a24-a79d03df7c0d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:17.410481 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:17.471293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf669c7-5ed5-4159-8a24-a79d03df7c0d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:17.509665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274c8c-d29a-4c20-a17c-12b8526cd11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:17.510359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274c8c-d29a-4c20-a17c-12b8526cd11e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:17.510650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274c8c-d29a-4c20-a17c-12b8526cd11e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:17.510928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274c8c-d29a-4c20-a17c-12b8526cd11e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:17.511752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70274c8c-d29a-4c20-a17c-12b8526cd11e None admin] RBAC: Adding query filter params (group.id=924a45b4669f49f78c81e2f8f3c400a8, role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=0c33208ac31c49d19dce8ea7c7b40c7a) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:17.512077 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70274c8c-d29a-4c20-a17c-12b8526cd11e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:17.513584 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70274c8c-d29a-4c20-a17c-12b8526cd11e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:17.520840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 233/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:39:17 2022] GET /identity/v3/role_assignments?group.id=924a45b4669f49f78c81e2f8f3c400a8&scope.project.id=0c33208ac31c49d19dce8ea7c7b40c7a&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:19.896589 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05a913-5908-4605-ab26-382f47c75d6b None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:19.897167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05a913-5908-4605-ab26-382f47c75d6b None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:19.897167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05a913-5908-4605-ab26-382f47c75d6b None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:19.898811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 223/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:19.905115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cf48d-562e-4389-bd92-bb65740a8925 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:19.906053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cf48d-562e-4389-bd92-bb65740a8925 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:19.906621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cf48d-562e-4389-bd92-bb65740a8925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:19.931530 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-459cf48d-562e-4389-bd92-bb65740a8925 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:19.933210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-459cf48d-562e-4389-bd92-bb65740a8925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:20.036889 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 234/457] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:20.073649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05a913-5908-4605-ab26-382f47c75d6b None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:20.145357 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0be2f-c250-43a3-8f44-6950e5c86f98 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.146177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0be2f-c250-43a3-8f44-6950e5c86f98 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.146669 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0be2f-c250-43a3-8f44-6950e5c86f98 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.148552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 235/458] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:20.153503 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908141-7722-4d29-8f78-daf427513fb8 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.154139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908141-7722-4d29-8f78-daf427513fb8 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.154639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908141-7722-4d29-8f78-daf427513fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.191090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-f1908141-7722-4d29-8f78-daf427513fb8 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:20.192996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-f1908141-7722-4d29-8f78-daf427513fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:20.225005 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:20.235812 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05a913-5908-4605-ab26-382f47c75d6b None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:20.245995 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:20.276176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 236/459] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:20.306996 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:20.307597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.307884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.308165 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.315459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908141-7722-4d29-8f78-daf427513fb8 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:20.326050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:20.327613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:20.354846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:20.356059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:20.357092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:20.357092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:20.357092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:20.357092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 224/460] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:20.367776 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:20.476669 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908141-7722-4d29-8f78-daf427513fb8 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:20.501058 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34452b89-031f-4526-9552-9bddbd4ab194 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:20.518981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38922485-5a3b-426c-b4a6-a4c65f4fd4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:20.519801 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38922485-5a3b-426c-b4a6-a4c65f4fd4be None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.520082 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38922485-5a3b-426c-b4a6-a4c65f4fd4be None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.520357 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38922485-5a3b-426c-b4a6-a4c65f4fd4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.534294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38922485-5a3b-426c-b4a6-a4c65f4fd4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:20.535992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38922485-5a3b-426c-b4a6-a4c65f4fd4be None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:20.544858 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbf94e-2cc4-4e82-bb7a-866774267dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:20.546453 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbf94e-2cc4-4e82-bb7a-866774267dfe None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.546975 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbf94e-2cc4-4e82-bb7a-866774267dfe None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.547980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbf94e-2cc4-4e82-bb7a-866774267dfe None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.549229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbf94e-2cc4-4e82-bb7a-866774267dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:20.549745 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbf94e-2cc4-4e82-bb7a-866774267dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:20.551585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbf94e-2cc4-4e82-bb7a-866774267dfe None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-38922485-5a3b-426c-b4a6-a4c65f4fd4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:20.554565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:20.555713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:20.556754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:20.556754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:20.556754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:20.556754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 237/461] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:20.562071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc3d58-4476-45c9-b3aa-9d1e069ecf0a None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.562071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc3d58-4476-45c9-b3aa-9d1e069ecf0a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.562071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc3d58-4476-45c9-b3aa-9d1e069ecf0a None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.562071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 238/462] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:20.563057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc3d58-4476-45c9-b3aa-9d1e069ecf0a None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:20.577584 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 225/463] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:20.579826 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093da92-15d5-4252-82d8-ad1c3a24878a None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.580114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093da92-15d5-4252-82d8-ad1c3a24878a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.580392 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093da92-15d5-4252-82d8-ad1c3a24878a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.604345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:20.608276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-5093da92-15d5-4252-82d8-ad1c3a24878a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:20.610854 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-5093da92-15d5-4252-82d8-ad1c3a24878a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:20.626468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:20.704622 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc3d58-4476-45c9-b3aa-9d1e069ecf0a None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:20.709483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 226/464] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:20.710490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093da92-15d5-4252-82d8-ad1c3a24878a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:20.749634 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:20.750802 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.751334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.751870 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.752925 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:20.753480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:20.755223 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:20.774986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 239/465] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:20.776431 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:20.827324 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093da92-15d5-4252-82d8-ad1c3a24878a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:20.873788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e367fa-239b-412e-bdb1-f96d159d0914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:20.874855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e367fa-239b-412e-bdb1-f96d159d0914 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.875554 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e367fa-239b-412e-bdb1-f96d159d0914 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.876018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e367fa-239b-412e-bdb1-f96d159d0914 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.877431 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e367fa-239b-412e-bdb1-f96d159d0914 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:20.879487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e367fa-239b-412e-bdb1-f96d159d0914 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:20.880409 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:20.894923 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:20.903815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03c80b-9cf1-4631-85a2-08d8a7306242 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:20.911921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 227/466] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:39:20 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:39:20.913394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e367fa-239b-412e-bdb1-f96d159d0914 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:20.970609 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03c7fb-16a2-4673-89ab-36bc3c8fe292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:20.971548 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c7fb-16a2-4673-89ab-36bc3c8fe292 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:20.971911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c7fb-16a2-4673-89ab-36bc3c8fe292 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:20.972246 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c7fb-16a2-4673-89ab-36bc3c8fe292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:20.991207 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03c7fb-16a2-4673-89ab-36bc3c8fe292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:20.993804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03c7fb-16a2-4673-89ab-36bc3c8fe292 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-4f03c7fb-16a2-4673-89ab-36bc3c8fe292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:21.018446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:21.019633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:21.020773 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:21.020773 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:21.020773 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:21.020773 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 240/467] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:21.021390 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3cf3-ca0d-4e76-8f30-7c2f22765447 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.022179 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3cf3-ca0d-4e76-8f30-7c2f22765447 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.022728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3cf3-ca0d-4e76-8f30-7c2f22765447 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.025108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 241/468] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:21.031925 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9dfdd-7fdb-490a-bf40-4e92e8554147 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.032526 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9dfdd-7fdb-490a-bf40-4e92e8554147 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.033295 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9dfdd-7fdb-490a-bf40-4e92e8554147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.057709 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-5ea9dfdd-7fdb-490a-bf40-4e92e8554147 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:21.058956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-5ea9dfdd-7fdb-490a-bf40-4e92e8554147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:21.066875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e367fa-239b-412e-bdb1-f96d159d0914 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:21.120527 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d67b23-94af-4a3f-8f22-0338e6c493b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:21.121115 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d67b23-94af-4a3f-8f22-0338e6c493b7 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.121403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d67b23-94af-4a3f-8f22-0338e6c493b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.121682 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d67b23-94af-4a3f-8f22-0338e6c493b7 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.122469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d67b23-94af-4a3f-8f22-0338e6c493b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:21.124075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d67b23-94af-4a3f-8f22-0338e6c493b7 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:21.148969 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 242/469] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:21.150072 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9dfdd-7fdb-490a-bf40-4e92e8554147 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:21.162331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 228/470] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:39:20 2022] POST /identity/v3/users => generated 248 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:39:21.179535 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d67b23-94af-4a3f-8f22-0338e6c493b7 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:21.248157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:21.258760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:21.308387 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9dfdd-7fdb-490a-bf40-4e92e8554147 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:21.320939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d67b23-94af-4a3f-8f22-0338e6c493b7 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:21.353958 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:21.354681 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.354965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.355251 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.355976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:21.356289 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:21.357832 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:21.374660 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 243/471] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:21.377642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:21.377642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.377642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.377642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.380285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:21.392113 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:21.396002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:21.417208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:21.418384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:21.419419 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:21.419419 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:21.419419 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:21.426316 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 229/472] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:21.431161 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:21.507010 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cf38b-c14b-4773-b5a6-bbd3214d2c64 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:21.541846 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:21.553115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779287-741f-4024-a59b-81d438003c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:21.554511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779287-741f-4024-a59b-81d438003c62 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.555294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779287-741f-4024-a59b-81d438003c62 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.556218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779287-741f-4024-a59b-81d438003c62 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.557195 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f779287-741f-4024-a59b-81d438003c62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:21.558801 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f779287-741f-4024-a59b-81d438003c62 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:21.560177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446fb82-8d25-490e-b191-99d4d9e98cd6 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:21.561858 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:21.594952 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 244/473] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:39:21 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:39:21.596618 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c18a7-db71-44a5-9cd6-c52e65f3b09c None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.596911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c18a7-db71-44a5-9cd6-c52e65f3b09c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.597369 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c18a7-db71-44a5-9cd6-c52e65f3b09c None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.599249 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 245/474] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:21.606077 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f8cc1-6c0c-4585-9d2d-cd4ad4b275ff None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.606435 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f8cc1-6c0c-4585-9d2d-cd4ad4b275ff None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.606664 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f8cc1-6c0c-4585-9d2d-cd4ad4b275ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.629610 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:21.630345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.630629 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.630904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.632033 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:21.632356 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:21.633819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:21.637518 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-905f8cc1-6c0c-4585-9d2d-cd4ad4b275ff None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:21.639167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-905f8cc1-6c0c-4585-9d2d-cd4ad4b275ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:21.654243 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 230/475] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:21.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:21.735220 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 246/476] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:21.774089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f8cc1-6c0c-4585-9d2d-cd4ad4b275ff None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:21.827200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e677e-057d-40a8-8048-4f29872d674f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:21.878551 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931e54c-9ec0-43f4-a852-2ca53965fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:21.879471 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931e54c-9ec0-43f4-a852-2ca53965fe0c None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.879811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931e54c-9ec0-43f4-a852-2ca53965fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.880154 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931e54c-9ec0-43f4-a852-2ca53965fe0c None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.880866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931e54c-9ec0-43f4-a852-2ca53965fe0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:21.883267 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931e54c-9ec0-43f4-a852-2ca53965fe0c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:21.915047 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 231/477] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:39:21 2022] POST /identity/v3/users => generated 249 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:39:21.935008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f8cc1-6c0c-4585-9d2d-cd4ad4b275ff None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:21.963218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:21.976136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:21.976824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:21.977178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:21.977457 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:21.978292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:21.979924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:21.990444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.sql.core [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:39:21.990444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:39:21.990444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [parameters: {'id': '11d77bd9bd544073925d074186704f14', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:39:21.990444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277712) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application result.read() Jul 28 08:39:22.009188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:39:22.010415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application yield resource Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise self.value Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application session.commit() Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:39:22.011596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:39:22.012661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:39:22.015739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application result.read() Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:39:22.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [parameters: {'id': '11d77bd9bd544073925d074186704f14', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:22.020405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:22.027351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 247/478] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:39:21 2022] POST /identity/v3/roles => generated 144 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:22.029565 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931e54c-9ec0-43f4-a852-2ca53965fe0c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:22.048444 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:22.152566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931e54c-9ec0-43f4-a852-2ca53965fe0c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:22.198844 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4d844-4f23-476c-ba71-4d8bae985553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:22.199526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d844-4f23-476c-ba71-4d8bae985553 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:22.199813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d844-4f23-476c-ba71-4d8bae985553 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:22.200090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d844-4f23-476c-ba71-4d8bae985553 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:22.216369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4d844-4f23-476c-ba71-4d8bae985553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-dfe4d844-4f23-476c-ba71-4d8bae985553 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:22.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:22.224273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:22.226650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 232/479] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:39:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:22.231243 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:22.442286 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b79b90-99ad-4832-aff8-75b482a1cd6a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:22.503025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d7d3a-f04b-4cda-9f25-2566825b5413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:22.503682 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d7d3a-f04b-4cda-9f25-2566825b5413 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:22.503960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d7d3a-f04b-4cda-9f25-2566825b5413 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:22.504231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d7d3a-f04b-4cda-9f25-2566825b5413 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:22.505014 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d7d3a-f04b-4cda-9f25-2566825b5413 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:22.505379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d7d3a-f04b-4cda-9f25-2566825b5413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:22.506790 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d7d3a-f04b-4cda-9f25-2566825b5413 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:22.515888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 248/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:23.510400 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c74e9-5050-414c-9893-dcf2e06d0d38 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:23.511403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c74e9-5050-414c-9893-dcf2e06d0d38 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:23.511911 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c74e9-5050-414c-9893-dcf2e06d0d38 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:23.513879 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 233/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:23.519810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb648cc-b97a-4ec2-aa4e-41d52727577d None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:23.520293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb648cc-b97a-4ec2-aa4e-41d52727577d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:23.520392 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb648cc-b97a-4ec2-aa4e-41d52727577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:23.543521 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-6cb648cc-b97a-4ec2-aa4e-41d52727577d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:23.544727 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-6cb648cc-b97a-4ec2-aa4e-41d52727577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:23.595294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:23.640462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 249/482] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:23.673482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c74e9-5050-414c-9893-dcf2e06d0d38 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:23.792506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:23.827858 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c74e9-5050-414c-9893-dcf2e06d0d38 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:23.873713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80626d3b-e38d-4378-a72b-ca6e2ebf90a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:23.874468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80626d3b-e38d-4378-a72b-ca6e2ebf90a9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:23.874758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80626d3b-e38d-4378-a72b-ca6e2ebf90a9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:23.875038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80626d3b-e38d-4378-a72b-ca6e2ebf90a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:23.895117 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80626d3b-e38d-4378-a72b-ca6e2ebf90a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-80626d3b-e38d-4378-a72b-ca6e2ebf90a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:23.900907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:23.902056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:23.903144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 234/483] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:23.910707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb648cc-b97a-4ec2-aa4e-41d52727577d None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.036578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb648cc-b97a-4ec2-aa4e-41d52727577d None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:24.072685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfc7ed-dee8-4666-a3bb-a4ff1643c4d9 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.072897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfc7ed-dee8-4666-a3bb-a4ff1643c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.073147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfc7ed-dee8-4666-a3bb-a4ff1643c4d9 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.074789 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 235/484] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:24.078153 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:24.078818 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.079095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.079367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.080113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:24.080417 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:24.081808 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:24.085013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5fd6-e18d-4a0f-a717-66fe86c03415 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.085470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5fd6-e18d-4a0f-a717-66fe86c03415 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.085748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5fd6-e18d-4a0f-a717-66fe86c03415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.101364 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 250/485] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:24.107341 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.124085 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-770c5fd6-e18d-4a0f-a717-66fe86c03415 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:24.125321 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-770c5fd6-e18d-4a0f-a717-66fe86c03415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:24.165283 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:24.250131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 236/486] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:24.250131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b447a2-6cc6-4d7d-8c8d-1b0556e04231 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.250131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b447a2-6cc6-4d7d-8c8d-1b0556e04231 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.250131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b447a2-6cc6-4d7d-8c8d-1b0556e04231 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.252194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 237/487] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:24.267696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc14f8-9a1d-465c-8f7b-d041c39ed70a None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.268051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc14f8-9a1d-465c-8f7b-d041c39ed70a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.268281 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc14f8-9a1d-465c-8f7b-d041c39ed70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.289198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-87bc14f8-9a1d-465c-8f7b-d041c39ed70a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:24.290544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-87bc14f8-9a1d-465c-8f7b-d041c39ed70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:24.291983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34409cb4-9faf-442b-8651-12a7babe8a57 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:24.329687 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:24.330789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.331308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.331810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.344406 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:24.346423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:24.357071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:24.370141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 238/488] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:24.370141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc14f8-9a1d-465c-8f7b-d041c39ed70a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:24.372129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:24.373175 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:24.374231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:24.374231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:24.374231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:24.376497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 251/489] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:24.383011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.481782 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc14f8-9a1d-465c-8f7b-d041c39ed70a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:24.493031 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:24.510725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a819c4a-53cf-44f0-9cb2-e27a16af5145 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:24.524020 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21783e77-cb77-4351-bf27-954f7f7c1c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:24.524733 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21783e77-cb77-4351-bf27-954f7f7c1c5d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.524980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21783e77-cb77-4351-bf27-954f7f7c1c5d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.525255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21783e77-cb77-4351-bf27-954f7f7c1c5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.537451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21783e77-cb77-4351-bf27-954f7f7c1c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-21783e77-cb77-4351-bf27-954f7f7c1c5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:24.542763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:24.543845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:24.544886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 239/490] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:24.545638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21783e77-cb77-4351-bf27-954f7f7c1c5d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.561131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:24.562606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.563070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.563501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.564381 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:24.564850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:24.566463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:24.584508 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 252/491] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:24.585722 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.612763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:24.660905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21783e77-cb77-4351-bf27-954f7f7c1c5d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:24.695911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f567d-256b-4b1f-aeb1-2df1886c15ee None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:24.702008 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99000ecf-144b-4cd7-9573-d90154a0f57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:24.702726 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000ecf-144b-4cd7-9573-d90154a0f57f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.703011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000ecf-144b-4cd7-9573-d90154a0f57f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.703287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000ecf-144b-4cd7-9573-d90154a0f57f None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.715777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99000ecf-144b-4cd7-9573-d90154a0f57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-99000ecf-144b-4cd7-9573-d90154a0f57f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:24.721313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:24.722469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 240/492] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:24.723529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99000ecf-144b-4cd7-9573-d90154a0f57f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.741408 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cf6e2-ac35-4956-9fa3-9611fc4c987f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:24.742068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cf6e2-ac35-4956-9fa3-9611fc4c987f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.742374 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cf6e2-ac35-4956-9fa3-9611fc4c987f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.742678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cf6e2-ac35-4956-9fa3-9611fc4c987f None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.743488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cf6e2-ac35-4956-9fa3-9611fc4c987f None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:24.743822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cf6e2-ac35-4956-9fa3-9611fc4c987f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:24.745155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cf6e2-ac35-4956-9fa3-9611fc4c987f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:24.762679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 253/493] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:24.762679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa65c5-ff18-4354-bea2-b00b8b31b486 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.762679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa65c5-ff18-4354-bea2-b00b8b31b486 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.762679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa65c5-ff18-4354-bea2-b00b8b31b486 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.762679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 254/494] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:24.762679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa65c5-ff18-4354-bea2-b00b8b31b486 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.776687 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:24.846008 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99000ecf-144b-4cd7-9573-d90154a0f57f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:24.864706 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:24.885653 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:24.886317 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.886608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.886832 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa65c5-ff18-4354-bea2-b00b8b31b486 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:24.886963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.895262 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:24.896863 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:24.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:24.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:24.913060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 241/495] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:39:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:24.913792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:24.929562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5ded9-ca25-4ef2-b90b-1c1fa8cd12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:24.930406 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ded9-ca25-4ef2-b90b-1c1fa8cd12e0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.931094 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ded9-ca25-4ef2-b90b-1c1fa8cd12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.931378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ded9-ca25-4ef2-b90b-1c1fa8cd12e0 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.932125 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5ded9-ca25-4ef2-b90b-1c1fa8cd12e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:24.932434 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5ded9-ca25-4ef2-b90b-1c1fa8cd12e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:24.933816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5ded9-ca25-4ef2-b90b-1c1fa8cd12e0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:24.943025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 255/496] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:24.948176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74728acd-c987-4a1f-8e49-e0ceae48c991 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:24.948520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74728acd-c987-4a1f-8e49-e0ceae48c991 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:24.948928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74728acd-c987-4a1f-8e49-e0ceae48c991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:24.967097 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-74728acd-c987-4a1f-8e49-e0ceae48c991 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:24.968329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-74728acd-c987-4a1f-8e49-e0ceae48c991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:25.025728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45acc71-c195-4308-82ce-28a7912740ad None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.036331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:25.042081 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 256/497] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:25.043056 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74728acd-c987-4a1f-8e49-e0ceae48c991 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.062448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.063183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.063412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.063800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.076517 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:25.078076 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:25.093910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:25.095372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:25.096558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:25.096558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:25.096558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:25.096558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 242/498] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:25.096558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.129786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:25.161979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74728acd-c987-4a1f-8e49-e0ceae48c991 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.200426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bcc16-702e-4ee5-8cbf-f56399984c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.201080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bcc16-702e-4ee5-8cbf-f56399984c8c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.201369 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bcc16-702e-4ee5-8cbf-f56399984c8c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.201658 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bcc16-702e-4ee5-8cbf-f56399984c8c None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.202510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bcc16-702e-4ee5-8cbf-f56399984c8c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:25.202828 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bcc16-702e-4ee5-8cbf-f56399984c8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:25.204354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bcc16-702e-4ee5-8cbf-f56399984c8c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:25.207540 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721ed2-f15b-4c71-adfe-f5a7cb1a9d71 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.245056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.245481 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 257/499] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:25.245749 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.246033 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.246346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.246921 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc93f45-ca5f-4d3b-9e02-7e5f69f4bb14 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.247231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc93f45-ca5f-4d3b-9e02-7e5f69f4bb14 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.247509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc93f45-ca5f-4d3b-9e02-7e5f69f4bb14 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.249941 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 258/500] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:25.250235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc93f45-ca5f-4d3b-9e02-7e5f69f4bb14 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.261094 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:25.262679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:25.278298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:25.279668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:25.280703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:25.280703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:25.280703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:25.280703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 243/501] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:25.280703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.319877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:25.375022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:25.377284 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc93f45-ca5f-4d3b-9e02-7e5f69f4bb14 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.393880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275b3ef-20cd-447f-a35f-ba01055d0a8e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.429385 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.430017 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.430334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.430607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.431314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:25.431625 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:25.433035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:25.435255 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c671c-d07f-421e-a854-bea58c5bf950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.436360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c671c-d07f-421e-a854-bea58c5bf950 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.436865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c671c-d07f-421e-a854-bea58c5bf950 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.437365 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c671c-d07f-421e-a854-bea58c5bf950 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.445944 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 244/502] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:25.446862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.451456 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c671c-d07f-421e-a854-bea58c5bf950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-fb2c671c-d07f-421e-a854-bea58c5bf950 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:25.456612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:25.457887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 259/503] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:25.459141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab60b-becc-43fa-aa1d-a2b7671ac856 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.460155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab60b-becc-43fa-aa1d-a2b7671ac856 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.460155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab60b-becc-43fa-aa1d-a2b7671ac856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.478036 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-d68ab60b-becc-43fa-aa1d-a2b7671ac856 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:25.479311 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-d68ab60b-becc-43fa-aa1d-a2b7671ac856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:25.547713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 260/504] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:25.548589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab60b-becc-43fa-aa1d-a2b7671ac856 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.566503 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18abf-08ba-4a39-962f-b22d60eec86c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.602311 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:25.606448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75e29-679e-482d-b077-09203ec4986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.607130 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75e29-679e-482d-b077-09203ec4986b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.607408 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75e29-679e-482d-b077-09203ec4986b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.607684 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75e29-679e-482d-b077-09203ec4986b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.621146 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:25.659619 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab60b-becc-43fa-aa1d-a2b7671ac856 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.682828 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e75e29-679e-482d-b077-09203ec4986b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:25.684334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e75e29-679e-482d-b077-09203ec4986b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:25.700453 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.701109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.701394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.701674 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.702470 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:25.702780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:25.704205 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:25.716600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 261/505] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:25.717471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.788402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 245/506] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:25.789268 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75e29-679e-482d-b077-09203ec4986b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.827732 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9851ae5-17b3-4eca-878e-4c69f33edc28 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.834710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:25.837907 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:25.863334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.863983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.864385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.864664 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.871960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:25.873438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:25.886511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:25.887700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 262/507] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:25.889199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:25.899293 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75e29-679e-482d-b077-09203ec4986b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:25.937595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:25.938189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:25.938485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:25.938770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:25.939515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] RBAC: Adding query filter params (name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:25.939830 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:25.941330 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:25.951002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 246/508] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:25.954296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.000517 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06852d-f59f-4007-9b29-ccc800976cc2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.035589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.036333 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.036614 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.036889 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.048566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.050063 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.061296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f152b-598e-43d7-be7a-eea0a3a2754e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.064556 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:26.064747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:26.065842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:26.067581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:26.067581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:26.067581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:26.067581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 263/509] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:26.067581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.087895 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:26.101580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.102319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.102630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.102899 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.110942 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.112449 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:26.124494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:26.125696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 247/510] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:26.126881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.184692 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f753001-f8aa-409c-9a7e-778c1173fd33 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.219177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.219748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.220034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.220313 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.230678 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a11db-83d9-43aa-ae81-63794afcb0d0 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.231171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.232645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:26.247185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:26.248332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:26.249373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:26.249373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:26.249373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:26.249373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 264/511] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:26.249373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.270823 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.271510 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.271773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.272067 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.272890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:26.273209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.274745 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.314646 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 248/512] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:26.315579 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.336782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:26.342596 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:26.371582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe31a76-222f-4918-8047-2ea67891c337 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.407363 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.408011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.408318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.408563 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.420039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.421559 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.424311 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548c0da-64be-4566-b65b-70017e7758a4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:26.437629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:26.438953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:26.440339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:26.440339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:26.440339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:26.440339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 265/513] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:26.440339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.464999 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.465635 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.465914 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.466215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.466927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:26.467231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.468645 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.480157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 249/514] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:26.481228 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.542234 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55f807-f219-4708-b8b1-0bcab0fdf5f6 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.566234 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:26.570786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:26.582878 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.582878 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.582878 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.582878 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.583541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:26.583541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.584560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.593074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19179054-51a7-4692-80d4-479fec9cfa41 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.603409 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 266/515] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:26.604224 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.631529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.632275 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.632560 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.632833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.633596 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:26.633913 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.635370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.649778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 250/516] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:26.650964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.717656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e4796-fb01-47bd-9de8-47cb4e172aa3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.754806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9ae09-49f6-4cde-b503-7ce83479c530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.755401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9ae09-49f6-4cde-b503-7ce83479c530 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.755686 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9ae09-49f6-4cde-b503-7ce83479c530 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.755963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9ae09-49f6-4cde-b503-7ce83479c530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.763314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac555499-9e08-4b48-98e5-0792f0695923 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.803636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.804287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.804569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.804844 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.805558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:26.805868 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.807346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.809957 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:26.820358 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 251/517] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:26.821205 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.824478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:26.834915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9ae09-49f6-4cde-b503-7ce83479c530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.837145 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9ae09-49f6-4cde-b503-7ce83479c530 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.930819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa4543-e6e1-4ec3-8e2a-867fbe1e4dd1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:26.946060 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 267/518] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:26.946960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9ae09-49f6-4cde-b503-7ce83479c530 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:26.973600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846347d-2ff6-4772-872c-99d3d9174e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:26.974322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846347d-2ff6-4772-872c-99d3d9174e58 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:26.974609 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846347d-2ff6-4772-872c-99d3d9174e58 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:26.974908 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846347d-2ff6-4772-872c-99d3d9174e58 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:26.975609 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846347d-2ff6-4772-872c-99d3d9174e58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:26.977083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846347d-2ff6-4772-872c-99d3d9174e58 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:26.999435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 252/519] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:39:26 2022] POST /identity/v3/users => generated 247 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:39:27.007501 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846347d-2ff6-4772-872c-99d3d9174e58 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.034973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:27.058851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9ae09-49f6-4cde-b503-7ce83479c530 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.090734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:27.104108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f2ba5-eae7-4d6d-9eed-30e9a7646cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.104763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f2ba5-eae7-4d6d-9eed-30e9a7646cd2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.105062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f2ba5-eae7-4d6d-9eed-30e9a7646cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.105346 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f2ba5-eae7-4d6d-9eed-30e9a7646cd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.149809 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846347d-2ff6-4772-872c-99d3d9174e58 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.194566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f2ba5-eae7-4d6d-9eed-30e9a7646cd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.194788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.195773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.195773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.196068 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.196184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f2ba5-eae7-4d6d-9eed-30e9a7646cd2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.204722 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.207096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:27.221949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:27.223272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:27.224665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:27.224665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:27.224665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:27.224665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:27.224665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:27.224665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:27.224665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:27.224665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:27.226625 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 253/520] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:27.228183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.299467 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:27.327763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 268/521] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:27.327763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f2ba5-eae7-4d6d-9eed-30e9a7646cd2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.357501 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f9f7c-9c5e-4c14-97ec-9112dbdc3c15 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.379709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:27.399020 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.399687 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.399972 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.400250 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.408444 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.409959 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:27.421742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:27.423054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:39:27.424338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:27.428336 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 254/522] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:27.429340 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.441078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f2ba5-eae7-4d6d-9eed-30e9a7646cd2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.481388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.482274 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.482641 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.482982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.494364 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.495931 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:27.511892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:27.513240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:27.514775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:27.514775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:27.514775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:27.514775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 269/523] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:27.515226 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.545026 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc3a5-ac7d-47fc-b063-3e3e2d929b03 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.550052 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:27.589273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.589934 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.590404 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.590752 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.591541 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:27.591849 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.593381 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.619305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 255/524] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:27.619305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.632880 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f66d8-7072-4906-8d7f-76e7f3ad676a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.662581 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:27.670527 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.671203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.671493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.671769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.687789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.687789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:27.702763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:27.703932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:27.704999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:27.704999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:27.704999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:27.704999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 270/525] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:27.704999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.739826 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2eea5-e4d5-44c8-8140-fcb17de481ba None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.776945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.777614 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.777874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.778179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.778993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:27.779301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.780792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.796065 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:27.805459 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 256/526] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:27.805459 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.816381 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204ed31-7618-48e0-9591-f888d8f7954e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.855094 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.855738 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.856022 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.856307 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.857006 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:27.857309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.858740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.871878 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 271/527] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:27.872806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.909337 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:27.917761 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3764d8-4bd2-4767-936f-8891532e7ac4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:27.956598 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:27.957327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:27.957608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:27.957890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:27.958781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=90103f3776ce4eeb90bde66ed546aa87) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:27.959090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:27.960618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:27.966812 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 257/528] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/role_assignments?user.id=90103f3776ce4eeb90bde66ed546aa87&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:27.967748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:27.998727 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc1502-d1e1-46cc-a523-fb6f7acf143b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.051249 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.052663 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.053180 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.053658 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.054684 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:28.055564 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.058289 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.077233 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 272/529] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:28.078828 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.083963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:28.100977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf5206-ec04-4288-b23a-a9c0b8b77a80 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.144933 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfea88a-7e10-45aa-8dd5-99986ffa5e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.146045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfea88a-7e10-45aa-8dd5-99986ffa5e13 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.146553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfea88a-7e10-45aa-8dd5-99986ffa5e13 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.147042 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfea88a-7e10-45aa-8dd5-99986ffa5e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.186607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:28.217718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0f08b-9c22-47eb-8a3c-b0c5ee0eec6d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.236023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfea88a-7e10-45aa-8dd5-99986ffa5e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.237922 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfea88a-7e10-45aa-8dd5-99986ffa5e13 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.253797 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.254587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.254847 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.255135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.262817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.264309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:28.278144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:28.279254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:28.280285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 273/530] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:28.280867 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.346777 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:28.354483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 258/531] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:28.355679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfea88a-7e10-45aa-8dd5-99986ffa5e13 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.392066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb32b2-bc87-4533-9c5c-c238b04311c4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.428095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.428988 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.429348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.429747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.437180 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.438500 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:28.438976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:28.452096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:28.453468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:28.454796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:28.454796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:28.454796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:28.454796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:28.454796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:28.454796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:28.454796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:28.454796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:28.456449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 274/532] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:28.457492 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.478724 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfea88a-7e10-45aa-8dd5-99986ffa5e13 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.516890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.517903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.518409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.518834 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.519873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:28.520344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.522208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.540790 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 259/533] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:28.542023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.565336 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4e5aa-c7b3-4b5f-9643-3ba6165b8144 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.592373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:28.606259 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.607039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.607423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.607799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.620950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.622739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:28.639384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:28.641321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:28.642542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:28.642542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:28.642542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:28.646353 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 275/534] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:28.647301 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.667737 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d37e22-d7f0-474a-8fe7-ec1de9e7db42 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.695180 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:28.712737 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.713890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.714512 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.715038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.716043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:28.716677 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.718607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.733261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 260/535] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:28.734568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.762281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddaddf-de19-479b-ae74-e0b2fbca86b8 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.798776 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.799418 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.799700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.799977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.800782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=0a4ddbb2e9ae43eb8c4efaa7e82a05b4) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:28.801098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.802731 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.814287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 276/536] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/role_assignments?user.id=0a4ddbb2e9ae43eb8c4efaa7e82a05b4&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:28.815320 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.855376 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28b8a4-f283-492e-b841-13544dda2d6c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:28.896281 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:28.897351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:28.897881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:28.898416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:28.899358 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:28.899896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:28.901617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:28.902753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:28.917170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 261/537] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:28.937994 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:28.979477 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd23c-40cf-4492-872f-604dcbdbfd24 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:29.018473 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef6254-6d0c-4db1-8f74-bfd71bcbcc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:29.019280 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef6254-6d0c-4db1-8f74-bfd71bcbcc8f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:29.019663 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef6254-6d0c-4db1-8f74-bfd71bcbcc8f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:29.020038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef6254-6d0c-4db1-8f74-bfd71bcbcc8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:29.021090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef6254-6d0c-4db1-8f74-bfd71bcbcc8f None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=2a4683819f3f40daafc42098b161c364) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:29.021509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef6254-6d0c-4db1-8f74-bfd71bcbcc8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:29.023235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef6254-6d0c-4db1-8f74-bfd71bcbcc8f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:29.030302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 277/538] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/role_assignments?user.id=2a4683819f3f40daafc42098b161c364&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:29.045315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:29.088927 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c268-0405-4973-a107-5fd3c3cef58a None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:29.089315 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c268-0405-4973-a107-5fd3c3cef58a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:29.089502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c268-0405-4973-a107-5fd3c3cef58a None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:29.091359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 278/539] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:39:29.097305 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f253e1-491a-4c41-ae25-505ee6a6e7c0 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:29.097644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f253e1-491a-4c41-ae25-505ee6a6e7c0 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:29.097866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f253e1-491a-4c41-ae25-505ee6a6e7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:29.127712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-57f253e1-491a-4c41-ae25-505ee6a6e7c0 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:29.128943 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-57f253e1-491a-4c41-ae25-505ee6a6e7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:29.140568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201eba3-b07a-41bb-a608-f1dad352fd0c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:29.220011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:29.220683 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:29.220975 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:29.221313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:29.235783 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:29.237317 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:29.252618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:29.253727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:29.254880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:29.254880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:29.254880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:29.254880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:29.254880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:29.254880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:29.254880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:29.254880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:29.255400 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 262/540] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:29.261499 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:29.277708 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 279/541] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:29.301455 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f253e1-491a-4c41-ae25-505ee6a6e7c0 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:29.364160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:29.401978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:29.405134 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5952c-6665-446d-bc78-292062f9166b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:29.450816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f253e1-491a-4c41-ae25-505ee6a6e7c0 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:29.488438 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:29.489000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:29.490177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:29.491174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:29.491977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:29.492274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:29.494029 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:29.508226 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d01bc7-2b92-4eaf-82db-f903a7b9afe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:29.508963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d01bc7-2b92-4eaf-82db-f903a7b9afe9 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:29.509247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d01bc7-2b92-4eaf-82db-f903a7b9afe9 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:29.509525 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d01bc7-2b92-4eaf-82db-f903a7b9afe9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:29.522500 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 263/542] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:29.527488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d01bc7-2b92-4eaf-82db-f903a7b9afe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:29.527667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-85d01bc7-2b92-4eaf-82db-f903a7b9afe9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:29.533000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:29.534398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:29.535642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:29.538584 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 280/543] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:29.551430 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d01bc7-2b92-4eaf-82db-f903a7b9afe9 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:29.647668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95747d0-e898-4350-b923-e90fb5f92e60 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:29.700287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:29.700964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:29.701190 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:29.701464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:29.702304 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=a1abd448d02d41cf9d969fe9b403259e) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:29.702613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:29.704095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:29.717345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 264/544] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:29 2022] GET /identity/v3/role_assignments?user.id=a1abd448d02d41cf9d969fe9b403259e&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:29.778587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:29.786953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d01bc7-2b92-4eaf-82db-f903a7b9afe9 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:29.839450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:29.840099 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:29.840379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:29.840656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:29.841395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:29.841703 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:29.843137 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:29.858475 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 281/545] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:29.864662 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:29.893330 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:30.014732 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de61a84-0be9-4245-bfa8-136e3bc8a684 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:30.053069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437415d7-35e4-4cf2-a0e2-47961abfb31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:30.053644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437415d7-35e4-4cf2-a0e2-47961abfb31d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:30.053922 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437415d7-35e4-4cf2-a0e2-47961abfb31d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:30.054241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437415d7-35e4-4cf2-a0e2-47961abfb31d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:30.065851 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437415d7-35e4-4cf2-a0e2-47961abfb31d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:30.067506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437415d7-35e4-4cf2-a0e2-47961abfb31d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-437415d7-35e4-4cf2-a0e2-47961abfb31d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:30.083255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:30.084364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:30.085747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:30.085747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:30.085747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:30.085747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 265/546] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:30.090210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:30.220761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65f2fd-9dfc-4a98-8fcf-01222007c20f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:30.270900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:30.271566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:30.271850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:30.272133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:30.272869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:30.273185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:30.274676 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:30.288074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c9ce4-a445-4fe1-a729-201907f58f60 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:30.288440 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c9ce4-a445-4fe1-a729-201907f58f60 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:30.288705 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c9ce4-a445-4fe1-a729-201907f58f60 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:30.289010 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 282/547] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:30.290215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 266/548] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:30.295299 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:30.300287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8cd93-51a1-42bf-9ba0-153be129cadc None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:30.300621 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8cd93-51a1-42bf-9ba0-153be129cadc None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:30.300839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8cd93-51a1-42bf-9ba0-153be129cadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:30.324923 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:30.350661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-95b8cd93-51a1-42bf-9ba0-153be129cadc None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:30.352098 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-95b8cd93-51a1-42bf-9ba0-153be129cadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:30.411471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7f22a-f57d-4859-8294-1c87d9902f8e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:30.467614 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 267/549] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:30.488689 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:30.489457 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:30.489748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:30.490034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:30.503751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:30.505357 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:30.523941 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8cd93-51a1-42bf-9ba0-153be129cadc None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:30.532606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:30.533789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:30.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 283/550] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:30.536331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:30.541996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:30.658786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8cd93-51a1-42bf-9ba0-153be129cadc None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:30.724090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:30.726980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b28cd-5473-4d6e-9c7b-fb53572f63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:30.728325 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b28cd-5473-4d6e-9c7b-fb53572f63f5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:30.728838 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b28cd-5473-4d6e-9c7b-fb53572f63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:30.729364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b28cd-5473-4d6e-9c7b-fb53572f63f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:30.753563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b28cd-5473-4d6e-9c7b-fb53572f63f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-329b28cd-5473-4d6e-9c7b-fb53572f63f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:30.759183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:30.760278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:30.761303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:30.772367 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 268/551] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:30.775725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7b65e-606a-4d26-b7f2-3098968a2602 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:30.776910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b28cd-5473-4d6e-9c7b-fb53572f63f5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:30.837603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:30.839021 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:30.839517 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:30.840003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:30.841063 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:30.841584 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:30.843512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:30.883322 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 284/552] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:39:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:30.889621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:30.924499 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:30.971645 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b28cd-5473-4d6e-9c7b-fb53572f63f5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:31.045230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:31.045886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.046192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.046465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.047212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] RBAC: Adding query filter params (name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:31.047519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:31.054893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:31.080727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 269/553] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:31.082797 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26752776-3364-4189-a3b3-1e688a3855e4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:31.086885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:31.121330 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bbe59-8ed4-4075-9f74-d35a0d374d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:31.121982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bbe59-8ed4-4075-9f74-d35a0d374d44 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.122340 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bbe59-8ed4-4075-9f74-d35a0d374d44 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.122617 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bbe59-8ed4-4075-9f74-d35a0d374d44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.125486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:31.199627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bbe59-8ed4-4075-9f74-d35a0d374d44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:31.200998 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bbe59-8ed4-4075-9f74-d35a0d374d44 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:31.311511 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf32a93-8fee-4bdd-a1a4-0dd8df9089cb None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:31.312522 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 285/554] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:31.314082 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade3901-12c0-45b0-a6a9-09c83b7c6438 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.314409 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade3901-12c0-45b0-a6a9-09c83b7c6438 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.314692 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade3901-12c0-45b0-a6a9-09c83b7c6438 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.316312 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 286/555] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:31.321967 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade3901-12c0-45b0-a6a9-09c83b7c6438 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:31.346212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:31.359712 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e9c5b-97b4-4c93-81e9-0c1e6054c767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:31.360775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e9c5b-97b4-4c93-81e9-0c1e6054c767 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.360894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:31.361379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e9c5b-97b4-4c93-81e9-0c1e6054c767 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.361865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e9c5b-97b4-4c93-81e9-0c1e6054c767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.374202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e9c5b-97b4-4c93-81e9-0c1e6054c767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:31.376084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e9c5b-97b4-4c93-81e9-0c1e6054c767 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-de0e9c5b-97b4-4c93-81e9-0c1e6054c767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:31.391718 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:31.392987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:31.394023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:31.394023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:31.394023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:31.396528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 270/556] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:31.403017 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd927d4-6bd5-421b-8c7a-ff02ceb3bb94 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.403770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd927d4-6bd5-421b-8c7a-ff02ceb3bb94 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.404282 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd927d4-6bd5-421b-8c7a-ff02ceb3bb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.431260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-8dd927d4-6bd5-421b-8c7a-ff02ceb3bb94 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:31.432504 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-8dd927d4-6bd5-421b-8c7a-ff02ceb3bb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:31.451263 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade3901-12c0-45b0-a6a9-09c83b7c6438 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:31.498874 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:31.500042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.500575 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.501136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.513427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:31.515537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:31.526725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 271/557] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:31.527729 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd927d4-6bd5-421b-8c7a-ff02ceb3bb94 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:31.545018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:31.546334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:31.547399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:31.547399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:31.547399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:31.552596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 287/558] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:31.554146 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:31.657218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:31.664530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd927d4-6bd5-421b-8c7a-ff02ceb3bb94 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:31.676692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:31.678055 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09acbfc-f871-46dc-b35e-35744d9ca125 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:31.712478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:31.713167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.713361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.713638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.714413 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:31.714734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:31.716172 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:31.731367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0984e-4e6b-4b14-9d93-104a52e1d657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:31.732040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0984e-4e6b-4b14-9d93-104a52e1d657 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.732322 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0984e-4e6b-4b14-9d93-104a52e1d657 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.732601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0984e-4e6b-4b14-9d93-104a52e1d657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.736308 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 272/559] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:31.737276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:31.746058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0984e-4e6b-4b14-9d93-104a52e1d657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-5cc0984e-4e6b-4b14-9d93-104a52e1d657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:31.751680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:31.752782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:31.753815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 288/560] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:31.755172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241655-78be-4e9e-bc62-ecd4fa963d28 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.755172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241655-78be-4e9e-bc62-ecd4fa963d28 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.755172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241655-78be-4e9e-bc62-ecd4fa963d28 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.756167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 289/561] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:31.758983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87241655-78be-4e9e-bc62-ecd4fa963d28 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:31.865363 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2329a-20d8-4d36-9b4c-0774de369fc4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:31.892902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87241655-78be-4e9e-bc62-ecd4fa963d28 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:31.913092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:31.914268 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.914803 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.915278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.916179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:31.916807 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:31.918247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:31.936651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad890ba-5815-4ac9-86ae-4f69bbbe982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:31.936895 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 273/562] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:31.937034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad890ba-5815-4ac9-86ae-4f69bbbe982e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.937034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad890ba-5815-4ac9-86ae-4f69bbbe982e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.937034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad890ba-5815-4ac9-86ae-4f69bbbe982e None admin] PATH_INFO: `/v3/users/nova` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.937307 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:31.946769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad890ba-5815-4ac9-86ae-4f69bbbe982e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:31.948277 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad890ba-5815-4ac9-86ae-4f69bbbe982e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-9ad890ba-5815-4ac9-86ae-4f69bbbe982e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:31.963463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:31.964600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:31.965740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:31.969523 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 290/563] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:39:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:31.970923 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4270ca-bee2-47ea-b0c9-9024f7ad0efd None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:31.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4270ca-bee2-47ea-b0c9-9024f7ad0efd None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:31.971494 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4270ca-bee2-47ea-b0c9-9024f7ad0efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:31.978798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:31.985848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:31.991890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-da4270ca-bee2-47ea-b0c9-9024f7ad0efd None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:31.993152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-da4270ca-bee2-47ea-b0c9-9024f7ad0efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:32.075379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3dbf-7a63-4d96-90ea-ede5ee8ff1ec None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.090130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 291/564] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:32.090130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4270ca-bee2-47ea-b0c9-9024f7ad0efd None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.128051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.128717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.129013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.129326 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.130052 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] RBAC: Adding query filter params (name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:32.130392 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.131825 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:32.143794 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 274/565] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:32.145176 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.221647 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4270ca-bee2-47ea-b0c9-9024f7ad0efd None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.257543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:32.265423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.266122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.267248 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.267248 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.267452 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16301ec7-99cf-422c-a941-3c6a169e7906 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.269866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:32.274868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.277756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:32.293755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:32.295008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 292/566] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:32.296184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.309479 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.310252 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.310557 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.310837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.311658 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:32.311972 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.313494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:32.332649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 275/567] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:32.337798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.407670 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a5f1f-0b82-4e43-8d07-e960acdf5736 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.447747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802be5a-c5db-4daf-bb9a-068a89630254 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.449051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79eae8-4a51-4f63-8593-6c62484ce2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.450643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79eae8-4a51-4f63-8593-6c62484ce2eb None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.450937 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79eae8-4a51-4f63-8593-6c62484ce2eb None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.451218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79eae8-4a51-4f63-8593-6c62484ce2eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.463752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79eae8-4a51-4f63-8593-6c62484ce2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-be79eae8-4a51-4f63-8593-6c62484ce2eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:32.469302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:32.470437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:32.471510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:32.474502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 293/568] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:32.475844 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4f767-82f7-43c0-8e4b-90bc3ed23ff9 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.476099 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4f767-82f7-43c0-8e4b-90bc3ed23ff9 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.476400 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4f767-82f7-43c0-8e4b-90bc3ed23ff9 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.477726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 294/569] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:32.478482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4f767-82f7-43c0-8e4b-90bc3ed23ff9 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.488311 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.488988 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.489330 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.489606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.502952 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.504441 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:32.520548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:32.521748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:32.523216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:32.523216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:32.523216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:32.523216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 276/570] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:32.523216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.526204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:32.546390 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:32.601720 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4f767-82f7-43c0-8e4b-90bc3ed23ff9 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.634032 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044b244-8953-4dcd-b1f1-c9ea079edcb9 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.647238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.647883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.648166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.648465 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.649276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:32.649588 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.651094 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:32.670207 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 295/571] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:32.671087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.675679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf261c5-6bd9-4889-814f-dc915ff65d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.675679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf261c5-6bd9-4889-814f-dc915ff65d86 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.675679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf261c5-6bd9-4889-814f-dc915ff65d86 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.675679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf261c5-6bd9-4889-814f-dc915ff65d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.687060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf261c5-6bd9-4889-814f-dc915ff65d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.688838 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf261c5-6bd9-4889-814f-dc915ff65d86 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-ecf261c5-6bd9-4889-814f-dc915ff65d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:32.705427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:32.706657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:32.707724 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:32.707724 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:32.707724 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:32.707724 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 277/572] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:32.708285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44390efe-7340-4a01-a4f1-5c35a988384b None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.708285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44390efe-7340-4a01-a4f1-5c35a988384b None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.708488 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44390efe-7340-4a01-a4f1-5c35a988384b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.729864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-44390efe-7340-4a01-a4f1-5c35a988384b None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:32.731670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-44390efe-7340-4a01-a4f1-5c35a988384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:32.791707 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:32.792379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5f23-8e03-4bee-bada-6e7c3056ec41 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.808937 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:32.813460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 278/573] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:32.814384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44390efe-7340-4a01-a4f1-5c35a988384b None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.829194 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.829840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.830141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.830424 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.831159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:32.831472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.832941 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:32.841896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 296/574] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:32.842879 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.932473 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44390efe-7340-4a01-a4f1-5c35a988384b None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.952755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79760fa8-f6e1-4ec6-9939-6924c80cb683 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:32.971910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.972553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.972838 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.973112 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.973896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:32.974223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.975647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:32.992371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 279/575] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:32.992371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:32.992973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:32.994521 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:32.994799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:32.995070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:32.995880 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=9b3e9589dc214e90a6ace523306dffc0) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:32.996190 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:32.997657 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.003807 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 297/576] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/role_assignments?user.id=9b3e9589dc214e90a6ace523306dffc0&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:33.004705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.054950 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:33.072482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:33.129437 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216523e-9fac-4b86-944b-1e62490a65ec None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.152936 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3ff83-b6c2-428d-824e-38a34a07972e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.178722 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.179514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.179800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.180081 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.180809 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:33.181121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.182667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.196870 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758df8d-a257-4056-aca7-89caae756530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.197546 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758df8d-a257-4056-aca7-89caae756530 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.197826 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758df8d-a257-4056-aca7-89caae756530 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.198099 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758df8d-a257-4056-aca7-89caae756530 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.200883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 280/577] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:33.203245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.211867 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758df8d-a257-4056-aca7-89caae756530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-9758df8d-a257-4056-aca7-89caae756530 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:33.218007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:33.219645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 298/578] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:33.228551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758df8d-a257-4056-aca7-89caae756530 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.331307 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2053b-832e-4a88-ae29-ddf6471c085e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.337846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:33.354540 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758df8d-a257-4056-aca7-89caae756530 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.355812 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:33.380630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.381363 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.381581 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.381851 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.393513 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.393833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.394504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.394783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.395057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.395194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.403427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.404920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:33.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:33.415801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:33.418619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:33.418619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:33.418619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:33.421549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:33.422661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:39:33.426482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:33.428314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 281/579] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:33.428314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.433965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 299/580] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:33.435980 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.544024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8f35a-67b2-40b8-8e1c-68526abf5043 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.552009 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5b6bc-92bd-4084-aefc-faa121d0e300 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.580918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.581648 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.581878 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.582186 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.589883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.591482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.594383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.595031 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.595318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.595606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.596343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:33.596657 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.598059 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:33.604319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:33.605350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 282/581] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:33.606763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.616421 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 300/582] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:33.618224 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.627824 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:33.646770 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:33.724702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd039e3-0781-49b2-a4d1-d2647b66e266 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.727653 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2892452-74b9-4d50-b303-6d72c727a491 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.760464 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.761117 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.761407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.761691 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.762546 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:33.762865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.764348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.765236 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.765891 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.766319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.766600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.767862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:33.767862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.769137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.787442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 283/583] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:33.787698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 301/584] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:33.787698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.788471 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.871733 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:33.874960 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:33.893361 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb4713-dcba-4600-8c84-14cc4fd9cd33 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.902303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f315e8-6751-4877-aec2-43f75408f241 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:33.931136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.931802 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.932082 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.932363 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.938292 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:33.938945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:33.939686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:33.939978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:33.941123 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:33.941461 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.942912 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:33.943032 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.944730 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:33.961348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 284/585] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:33.961348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:33.961650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:33.963042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:33.964122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:33.964122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:33.964122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:33.964122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 302/586] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:33.964122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.073544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2179e-60ae-4f17-b977-11ffeedbb3e1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.076429 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bc201-b6eb-4d82-8c90-6522dd665007 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.110890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.111571 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.111859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.112136 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.114261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.114973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.115262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.115577 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.116021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:34.118218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:34.126804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.127553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.128556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.129047 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:34.141355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:34.142550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 303/587] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:34.143725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:34.144702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:34.145915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:34.147247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:34.147247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:34.147247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:34.150262 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 285/588] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:34.151143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.254552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc88bb-2643-4ef2-a2e7-8a47bd33ee80 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.259690 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054a10-e205-4743-8e71-a6abddb25896 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.290778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418c14c-05b3-422d-a1a7-692b2c9c691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.291656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418c14c-05b3-422d-a1a7-692b2c9c691a None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.292046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418c14c-05b3-422d-a1a7-692b2c9c691a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.292448 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418c14c-05b3-422d-a1a7-692b2c9c691a None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/90103f3776ce4eeb90bde66ed546aa87/roles/cc9e44cb256e4496ba5fd23548c12437` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.297437 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.298178 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.298539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.298889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.299683 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:34.300097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.301610 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.309241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418c14c-05b3-422d-a1a7-692b2c9c691a None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=90103f3776ce4eeb90bde66ed546aa87, role_id=cc9e44cb256e4496ba5fd23548c12437)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.310712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418c14c-05b3-422d-a1a7-692b2c9c691a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.318830 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 286/589] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:34.318830 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.338141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 304/590] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:39:34 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/90103f3776ce4eeb90bde66ed546aa87/roles/cc9e44cb256e4496ba5fd23548c12437 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:34.338141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418c14c-05b3-422d-a1a7-692b2c9c691a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.394708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:34.398901 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:34.448622 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8978-ebbb-45bf-80ea-88e1cdf84152 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.483908 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418c14c-05b3-422d-a1a7-692b2c9c691a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.497711 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.498938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.499687 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.499974 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.501127 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:34.501447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.503005 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.529192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 287/591] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:34.531520 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.538713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.539788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.540074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.540400 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.541135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:34.541450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.542898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.556567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 305/592] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:34.557640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.665320 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2e6a-f69f-4707-8260-f7d87f39286b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.675016 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:34.682313 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fabfa-f4c8-4624-9c6b-1d8961dec8c3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.685454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:34.711671 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.712340 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.712734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.712935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.720995 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.722673 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.726580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.727535 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.728033 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.728512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:34.736730 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:34.737857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:39:34.739159 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:34.743037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 288/593] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:34.743557 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.745515 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.747491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:34.764293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:34.765480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:34.766541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:34.766541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:34.766541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:34.768869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 306/594] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:34.774766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.862770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42f204-0728-4ebb-8702-ebf8ad54d168 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.899258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.899962 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.900276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.900479 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.906236 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3d90c-eeef-4032-80a1-fb05175dbe6d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:34.908053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.910139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:34.922442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:34.923595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:34.924736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:34.924736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:34.924736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:34.924736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:34.924736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:34.924736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:34.924736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:34.924736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:34.928496 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 289/595] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:34.929420 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:34.949929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:34.951106 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:34.951635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:34.952127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:34.953223 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:34.953758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:34.955498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:34.960192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:34.977796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 307/596] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:34.979491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.002541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:35.057775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22320526-e45c-46f4-ba91-16850e0deb57 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.094381 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.095056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.095339 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.095616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.096321 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:35.096629 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:35.098026 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:35.104202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ae57c-4e32-4768-b454-53506ce2a17a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.115314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 290/597] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:35.115314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.148622 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54149ac-5ebc-4e0a-90f9-eddd84cbcbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.149281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54149ac-5ebc-4e0a-90f9-eddd84cbcbc3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.149575 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54149ac-5ebc-4e0a-90f9-eddd84cbcbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.149822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54149ac-5ebc-4e0a-90f9-eddd84cbcbc3 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.150622 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54149ac-5ebc-4e0a-90f9-eddd84cbcbc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:35.150944 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54149ac-5ebc-4e0a-90f9-eddd84cbcbc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:35.152385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54149ac-5ebc-4e0a-90f9-eddd84cbcbc3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:35.164786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 308/598] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:35.170251 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20414ab-7d7b-48ec-8d09-076bf3742d00 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.170251 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20414ab-7d7b-48ec-8d09-076bf3742d00 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.170251 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20414ab-7d7b-48ec-8d09-076bf3742d00 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.170251 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 309/599] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:35.170251 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20414ab-7d7b-48ec-8d09-076bf3742d00 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.214999 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:35.233215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a394-d656-43fa-84e2-59e6ff2982a2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.254437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:35.268685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.269021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.269276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.269551 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.281198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:35.282779 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:35.284262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20414ab-7d7b-48ec-8d09-076bf3742d00 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:35.297873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:35.299209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:35.300880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:35.300880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:35.300880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:35.300880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 291/600] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:35.300880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.329171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96378f-4e14-4fb0-9f33-187ba5603a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.329837 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96378f-4e14-4fb0-9f33-187ba5603a92 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.330137 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96378f-4e14-4fb0-9f33-187ba5603a92 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.330422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96378f-4e14-4fb0-9f33-187ba5603a92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.337438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96378f-4e14-4fb0-9f33-187ba5603a92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:35.339009 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96378f-4e14-4fb0-9f33-187ba5603a92 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-3e96378f-4e14-4fb0-9f33-187ba5603a92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:35.350902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:35.352127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:35.353172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 310/601] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:35.353757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1b23-2908-4ccb-bee2-ca8adfbbe40f None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.353757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1b23-2908-4ccb-bee2-ca8adfbbe40f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.353919 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1b23-2908-4ccb-bee2-ca8adfbbe40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.372332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-ad0d1b23-2908-4ccb-bee2-ca8adfbbe40f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:35.373547 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-ad0d1b23-2908-4ccb-bee2-ca8adfbbe40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:35.417326 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b2c7-9239-49c6-b435-c9cba7868980 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.447301 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 311/602] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:35.447977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d1b23-2908-4ccb-bee2-ca8adfbbe40f None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.470443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42084436-77da-4393-b7f3-3f9fc867054a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.471083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42084436-77da-4393-b7f3-3f9fc867054a None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.471447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42084436-77da-4393-b7f3-3f9fc867054a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.471727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42084436-77da-4393-b7f3-3f9fc867054a None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/0a4ddbb2e9ae43eb8c4efaa7e82a05b4/roles/cc9e44cb256e4496ba5fd23548c12437` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.483346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:35.492811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42084436-77da-4393-b7f3-3f9fc867054a None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=0a4ddbb2e9ae43eb8c4efaa7e82a05b4, role_id=cc9e44cb256e4496ba5fd23548c12437)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:35.494695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42084436-77da-4393-b7f3-3f9fc867054a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:35.522143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 292/603] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:39:35 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/0a4ddbb2e9ae43eb8c4efaa7e82a05b4/roles/cc9e44cb256e4496ba5fd23548c12437 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:35.522143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42084436-77da-4393-b7f3-3f9fc867054a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.534043 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:35.581511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d1b23-2908-4ccb-bee2-ca8adfbbe40f None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.638677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.639503 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.639890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.640274 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.640401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42084436-77da-4393-b7f3-3f9fc867054a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.641193 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:35.641624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:35.644665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:35.662063 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 312/604] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:35.663163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.681345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.681999 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.682347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.682628 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.683411 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:35.683723 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:35.685388 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:35.701306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 293/605] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:35.702336 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.765444 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:35.811216 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d805-b7f3-41f8-b148-b0d8b0ab411e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.828423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c37088-5e09-4fad-ae14-08addc0e98cc None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:35.839573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:35.855377 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a636ea4-6b6b-4d6b-9e25-299698b0e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.856503 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a636ea4-6b6b-4d6b-9e25-299698b0e41a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.857096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a636ea4-6b6b-4d6b-9e25-299698b0e41a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.857582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a636ea4-6b6b-4d6b-9e25-299698b0e41a None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.868237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:35.869001 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:35.869357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:35.869712 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:35.871440 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a636ea4-6b6b-4d6b-9e25-299698b0e41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:35.877211 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-5a636ea4-6b6b-4d6b-9e25-299698b0e41a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:35.877849 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:35.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:35.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:35.880837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:35.882938 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 313/606] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:35.883957 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a636ea4-6b6b-4d6b-9e25-299698b0e41a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:35.899189 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:35.900939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:35.902234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:35.902234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:35.902234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:35.902234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:35.902234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:35.902234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:35.902234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:35.902234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:35.903432 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 294/607] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:35.904413 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:36.006601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a636ea4-6b6b-4d6b-9e25-299698b0e41a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:36.040880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bdb79-a332-46b1-b759-54f607c95ecc None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:36.046301 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aaa25-fbe5-4b1e-b9fb-5a02157e8c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:36.047007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aaa25-fbe5-4b1e-b9fb-5a02157e8c0f None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.047292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aaa25-fbe5-4b1e-b9fb-5a02157e8c0f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.047579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aaa25-fbe5-4b1e-b9fb-5a02157e8c0f None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/2a4683819f3f40daafc42098b161c364/roles/cc9e44cb256e4496ba5fd23548c12437` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.053582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:36.065695 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47aaa25-fbe5-4b1e-b9fb-5a02157e8c0f None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=2a4683819f3f40daafc42098b161c364, role_id=cc9e44cb256e4496ba5fd23548c12437)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:36.067298 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47aaa25-fbe5-4b1e-b9fb-5a02157e8c0f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:36.084657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:36.085606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.086077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.086553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.087533 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] RBAC: Adding query filter params (name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:36.088009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:36.089672 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:36.092647 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 314/608] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:39:35 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/2a4683819f3f40daafc42098b161c364/roles/cc9e44cb256e4496ba5fd23548c12437 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:36.102133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aaa25-fbe5-4b1e-b9fb-5a02157e8c0f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:36.107785 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 295/609] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:36.114380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:36.150607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:36.243000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aaa25-fbe5-4b1e-b9fb-5a02157e8c0f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:36.286613 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:36.287709 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.288218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.288712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.289727 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:36.290308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:36.292089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:36.307177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 315/610] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:36.313069 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:36.325300 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c32cb-1b75-4dd1-8582-9b196e0056fe None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:36.368435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:36.369766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.370554 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.371075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.387459 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:36.389364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:36.410379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:36.411619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:36.412668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:36.412668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:36.412668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:36.415660 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 296/611] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:36.423766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:36.439217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:36.445505 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1212bf-9047-4f3b-b8a0-bc3f2018a51f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:36.479085 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:36.519667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b98f87-9ace-4cc8-9c63-f1cf0a487db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:36.520997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b98f87-9ace-4cc8-9c63-f1cf0a487db5 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.521524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b98f87-9ace-4cc8-9c63-f1cf0a487db5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.522025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b98f87-9ace-4cc8-9c63-f1cf0a487db5 None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/a1abd448d02d41cf9d969fe9b403259e/roles/cc9e44cb256e4496ba5fd23548c12437` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.541018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b98f87-9ace-4cc8-9c63-f1cf0a487db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=a1abd448d02d41cf9d969fe9b403259e, role_id=cc9e44cb256e4496ba5fd23548c12437)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:36.543029 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b98f87-9ace-4cc8-9c63-f1cf0a487db5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:36.561543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 316/612] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:39:36 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/a1abd448d02d41cf9d969fe9b403259e/roles/cc9e44cb256e4496ba5fd23548c12437 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:36.563541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3af17-0cde-4c97-856e-f90ba2003126 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.564123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3af17-0cde-4c97-856e-f90ba2003126 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.564629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3af17-0cde-4c97-856e-f90ba2003126 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.570286 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 317/613] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:36.574327 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e905f-89c6-49a0-a4eb-03c4b3906ab0 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.574974 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e905f-89c6-49a0-a4eb-03c4b3906ab0 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.575479 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e905f-89c6-49a0-a4eb-03c4b3906ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.599338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078465-6c18-43a6-b043-ce884dfc298b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:36.603325 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-451e905f-89c6-49a0-a4eb-03c4b3906ab0 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:36.605070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-451e905f-89c6-49a0-a4eb-03c4b3906ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:36.694715 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 318/614] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:36.724669 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:36.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.725607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e905f-89c6-49a0-a4eb-03c4b3906ab0 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:36.725735 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.726018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.726818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:36.727134 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:36.728653 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:36.742828 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 297/615] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:36.749120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:36.864708 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:36.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:36.906531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e905f-89c6-49a0-a4eb-03c4b3906ab0 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:36.909860 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c77ad-403d-46f1-9008-60bdd4f82333 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:36.960839 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1531d1-313d-4825-8ce6-cfa8d16e93ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:36.961936 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1531d1-313d-4825-8ce6-cfa8d16e93ba None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:36.962496 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1531d1-313d-4825-8ce6-cfa8d16e93ba None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:36.963561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1531d1-313d-4825-8ce6-cfa8d16e93ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:36.979980 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1531d1-313d-4825-8ce6-cfa8d16e93ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-4e1531d1-313d-4825-8ce6-cfa8d16e93ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:36.993736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:36.994992 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:36.996034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:36.999149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 319/616] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:37.002887 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:37.003788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:37.004248 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:37.004766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:37.006531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1531d1-313d-4825-8ce6-cfa8d16e93ba None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:37.023094 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:37.024628 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:37.044549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:37.045926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:37.047318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 298/617] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:37.059981 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:37.190792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a722027-0070-4dcc-b016-9cac72cd89ef None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:37.206306 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1531d1-313d-4825-8ce6-cfa8d16e93ba None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:37.231503 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:37.232700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:37.233435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:37.234030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:37.235233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:37.235895 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:37.237588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:37.257151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:37.257787 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 299/618] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:39:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:37.270164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee7200-120c-4555-963a-c776113e330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:37.270164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee7200-120c-4555-963a-c776113e330c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:37.270164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee7200-120c-4555-963a-c776113e330c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:37.270164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee7200-120c-4555-963a-c776113e330c None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:37.270873 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee7200-120c-4555-963a-c776113e330c None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:37.271041 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee7200-120c-4555-963a-c776113e330c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:37.271516 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:37.273814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee7200-120c-4555-963a-c776113e330c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:37.274329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:37.292252 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 320/619] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:37.297960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee7200-120c-4555-963a-c776113e330c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:37.446144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee7200-120c-4555-963a-c776113e330c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:37.490471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:37.491159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:37.491443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:37.491728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:37.503898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:37.505558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:37.524265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:37.525493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:37.526562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:37.526562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:37.526562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:37.526562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 321/620] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:37.535193 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:37.556199 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f39a5-79c6-4396-9ac6-3ac0116246da None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:37.595594 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:37.644626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:37.645346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:37.645632 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:37.645911 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:37.669826 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd72e7-a916-4ea4-b21d-43f8631006b1 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:37.679287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:37.681177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:37.710427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:37.711346 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:37.711815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:37.712246 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:37.713120 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:37.713593 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:37.715178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:37.720661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:37.721964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:37.723400 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:37.723400 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:37.723400 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:37.724666 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 300/621] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:37.730195 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:37.732710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 322/622] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:37.739086 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:37.849632 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:37.930927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b93bb-42d9-425d-8198-ece3d0e294a8 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:37.934708 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2543ce-d8d6-4d2b-99b7-954f6dad4e74 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:37.954802 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:38.003527 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.004109 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.004978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.005265 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.005995 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:38.006334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:38.007762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:38.011676 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.012356 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.012639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.012914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] PATH_INFO: `/v3/users/nova` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.029193 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:38.030725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:38.037796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 301/623] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:38.044076 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:38.048486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:38.049576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:38.051348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 323/624] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:39:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:38.056718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:38.172679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6029a-06fb-4326-aea9-b6a30eabdb5e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:38.196099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ad33d-440f-4f5a-973f-c913cc9037bd None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:38.210691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:38.221604 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dafcd-f47b-4c87-9f87-cfc6cbe77e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.223607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dafcd-f47b-4c87-9f87-cfc6cbe77e09 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.224064 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dafcd-f47b-4c87-9f87-cfc6cbe77e09 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.224769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dafcd-f47b-4c87-9f87-cfc6cbe77e09 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.226036 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dafcd-f47b-4c87-9f87-cfc6cbe77e09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:38.226971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dafcd-f47b-4c87-9f87-cfc6cbe77e09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:38.229174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dafcd-f47b-4c87-9f87-cfc6cbe77e09 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:38.234747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.235515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.235866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.236207 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.243853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:38.245415 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:38.257268 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 324/625] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:39:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:38.257796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:38.259035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:38.260315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:38.260315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:38.260315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:38.260315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:38.260315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:38.260315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:38.260315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:38.260315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:38.263056 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f303ec-4d55-4670-814d-e297a9a0457f None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.263726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f303ec-4d55-4670-814d-e297a9a0457f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.264385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f303ec-4d55-4670-814d-e297a9a0457f None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.265342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 302/626] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:38.266364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:38.267293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 325/627] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:38.268194 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f303ec-4d55-4670-814d-e297a9a0457f None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:38.307550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:38.388097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b228b-66d4-4049-9ed3-0ceb4f04e6da None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:38.403155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f303ec-4d55-4670-814d-e297a9a0457f None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:38.425187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1c048-ff0b-4910-a0f8-5474b55950ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.426149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1c048-ff0b-4910-a0f8-5474b55950ac None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.426149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1c048-ff0b-4910-a0f8-5474b55950ac None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.426452 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1c048-ff0b-4910-a0f8-5474b55950ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.441392 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454e34-bfcd-4041-aeac-10ea98db5c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.442603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454e34-bfcd-4041-aeac-10ea98db5c91 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.443122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454e34-bfcd-4041-aeac-10ea98db5c91 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.443573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454e34-bfcd-4041-aeac-10ea98db5c91 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.444931 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454e34-bfcd-4041-aeac-10ea98db5c91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:38.445402 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454e34-bfcd-4041-aeac-10ea98db5c91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:38.447195 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454e34-bfcd-4041-aeac-10ea98db5c91 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:38.465902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 326/628] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:38.467673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94344c2b-650c-4ca1-ad68-39082974e1ab None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.468143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94344c2b-650c-4ca1-ad68-39082974e1ab None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.468599 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94344c2b-650c-4ca1-ad68-39082974e1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.473620 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:38.489814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-94344c2b-650c-4ca1-ad68-39082974e1ab None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:38.491393 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-94344c2b-650c-4ca1-ad68-39082974e1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:38.509953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1c048-ff0b-4910-a0f8-5474b55950ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:38.511403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1c048-ff0b-4910-a0f8-5474b55950ac None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:38.570139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 327/629] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:38.570139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94344c2b-650c-4ca1-ad68-39082974e1ab None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:38.586497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:38.636889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 303/630] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:38.640143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff3692-51fb-475a-8e04-4c62874e689a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.640596 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff3692-51fb-475a-8e04-4c62874e689a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.641060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff3692-51fb-475a-8e04-4c62874e689a None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.643714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 304/631] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:38.644622 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff3692-51fb-475a-8e04-4c62874e689a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:38.678568 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94344c2b-650c-4ca1-ad68-39082974e1ab None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:38.719053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7932b-4147-4f31-acba-1a5077575484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.720102 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7932b-4147-4f31-acba-1a5077575484 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.720388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7932b-4147-4f31-acba-1a5077575484 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.720672 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7932b-4147-4f31-acba-1a5077575484 None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/9b3e9589dc214e90a6ace523306dffc0/roles/cc9e44cb256e4496ba5fd23548c12437` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.728961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:38.736919 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7932b-4147-4f31-acba-1a5077575484 None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=9b3e9589dc214e90a6ace523306dffc0, role_id=cc9e44cb256e4496ba5fd23548c12437)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:38.738439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7932b-4147-4f31-acba-1a5077575484 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:38.754233 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 328/632] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:39:38 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/9b3e9589dc214e90a6ace523306dffc0/roles/cc9e44cb256e4496ba5fd23548c12437 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:38.755277 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7932b-4147-4f31-acba-1a5077575484 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:38.779484 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff3692-51fb-475a-8e04-4c62874e689a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:38.837803 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471c7f0-82b6-4394-af03-b0e7aea08ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.839264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471c7f0-82b6-4394-af03-b0e7aea08ce5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.840253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471c7f0-82b6-4394-af03-b0e7aea08ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.840780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471c7f0-82b6-4394-af03-b0e7aea08ce5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.858178 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471c7f0-82b6-4394-af03-b0e7aea08ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:38.863102 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a471c7f0-82b6-4394-af03-b0e7aea08ce5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:38.867009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:38.868313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:38.869447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:38.874935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 305/633] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:38.876597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b69d7-8b54-4c53-9cf1-9bbb6fa6e519 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.876986 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b69d7-8b54-4c53-9cf1-9bbb6fa6e519 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.877424 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b69d7-8b54-4c53-9cf1-9bbb6fa6e519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.897916 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-aa2b69d7-8b54-4c53-9cf1-9bbb6fa6e519 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:38.899414 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-aa2b69d7-8b54-4c53-9cf1-9bbb6fa6e519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:38.920185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7932b-4147-4f31-acba-1a5077575484 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:38.970154 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59699ba-3e06-4379-854c-ef41dcd81882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:38.970832 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59699ba-3e06-4379-854c-ef41dcd81882 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:38.971119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59699ba-3e06-4379-854c-ef41dcd81882 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:38.971424 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59699ba-3e06-4379-854c-ef41dcd81882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:38.982498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59699ba-3e06-4379-854c-ef41dcd81882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:38.984000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59699ba-3e06-4379-854c-ef41dcd81882 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:38.990608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 306/634] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:38.995314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b69d7-8b54-4c53-9cf1-9bbb6fa6e519 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-c59699ba-3e06-4379-854c-ef41dcd81882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:39.000419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:39.001817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:39.003108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:39.003108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:39.003108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:39.005128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 329/635] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:39.006571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81aac2-f0ad-457f-ab9b-f8916334b005 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.006965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81aac2-f0ad-457f-ab9b-f8916334b005 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.007319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81aac2-f0ad-457f-ab9b-f8916334b005 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.013071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 330/636] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:39.014629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81aac2-f0ad-457f-ab9b-f8916334b005 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.070379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:39.121909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b69d7-8b54-4c53-9cf1-9bbb6fa6e519 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.132284 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81aac2-f0ad-457f-ab9b-f8916334b005 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.150894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:39.161390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2831c-f4c1-48f5-8236-794c4998595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.162047 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2831c-f4c1-48f5-8236-794c4998595a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.162369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2831c-f4c1-48f5-8236-794c4998595a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.162651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2831c-f4c1-48f5-8236-794c4998595a None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.163385 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2831c-f4c1-48f5-8236-794c4998595a None admin] RBAC: Adding query filter params (name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:39.163693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2831c-f4c1-48f5-8236-794c4998595a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:39.165110 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2831c-f4c1-48f5-8236-794c4998595a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:39.169605 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.170385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.170740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.171083 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.171800 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:39.172109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:39.173478 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:39.175208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 307/637] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:39.176720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f90f3-5b95-4b1d-8d0c-3527ee4e0d15 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.177226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f90f3-5b95-4b1d-8d0c-3527ee4e0d15 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.177704 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f90f3-5b95-4b1d-8d0c-3527ee4e0d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.191810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 331/638] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:39.192833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.201202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-4e9f90f3-5b95-4b1d-8d0c-3527ee4e0d15 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:39.202748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-4e9f90f3-5b95-4b1d-8d0c-3527ee4e0d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:39.285077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 308/639] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:39.285077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f90f3-5b95-4b1d-8d0c-3527ee4e0d15 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.308671 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d5a3d-f1dd-4f61-9964-5ad891a1925b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.347620 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a801b-d07c-46b9-9ff1-c2c986ecbcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.348441 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:39.349283 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a801b-d07c-46b9-9ff1-c2c986ecbcd7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.349636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a801b-d07c-46b9-9ff1-c2c986ecbcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.350009 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a801b-d07c-46b9-9ff1-c2c986ecbcd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.361424 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a801b-d07c-46b9-9ff1-c2c986ecbcd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-750a801b-d07c-46b9-9ff1-c2c986ecbcd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:39.366996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:39.368110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 332/640] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:39.374354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a801b-d07c-46b9-9ff1-c2c986ecbcd7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.407579 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f90f3-5b95-4b1d-8d0c-3527ee4e0d15 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.416472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:39.445264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.445939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.446257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.446539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.457193 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:39.459127 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:39.474938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:39.476071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:39.477118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:39.477118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:39.477118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:39.477118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 309/641] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:39.477118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.490130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a801b-d07c-46b9-9ff1-c2c986ecbcd7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.526611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.527277 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.527562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.527869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.535950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:39.537488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:39.550035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:39.551141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:39.552360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:39.552360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:39.552360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:39.552360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:39.552360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:39.552360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:39.552360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:39.552360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:39.556148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 333/642] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:39.557020 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.592976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a339b-06dc-469c-986d-cbd897c4cebc None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.601903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:39.635128 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da84f2c-326a-48a8-8454-6cba3c637e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.635814 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da84f2c-326a-48a8-8454-6cba3c637e58 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.636110 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da84f2c-326a-48a8-8454-6cba3c637e58 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.636379 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da84f2c-326a-48a8-8454-6cba3c637e58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.649771 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da84f2c-326a-48a8-8454-6cba3c637e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-8da84f2c-326a-48a8-8454-6cba3c637e58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:39.656380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:39.657921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:39.659246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:39.662738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 310/643] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:39.664042 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da84f2c-326a-48a8-8454-6cba3c637e58 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.679763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:39.684504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c1b33-a7b3-4c77-832f-de66fd6876f9 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.729650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.730342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.730632 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.731020 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.731798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:39.732190 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:39.733590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:39.748110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 334/644] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:39.748110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.771696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da84f2c-326a-48a8-8454-6cba3c637e58 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.806597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.807231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.807507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.807777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.808482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:39.808788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:39.810238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:39.823659 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 311/645] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:39.824340 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.853703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:39.864711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960484dc-0fe0-4591-b3aa-5f98bd84a344 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.906963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.908076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.908076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.908302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.908935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:39.909250 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:39.910765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:39.924117 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 335/646] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:39.924841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:39.947520 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef41d1d-6188-4bab-bd65-e64d4d5ca111 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:39.951407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:39.985336 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:39.985980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:39.986414 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:39.986699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:39.987433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] RBAC: Adding query filter params (name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:39.987744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:39.989188 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:39.997964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 312/647] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:39.998887 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.040636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e713a-7b13-4e39-a504-037544b24de2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.077733 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.078432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.078719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.078998 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.090334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:40.092056 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:40.094685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:40.107418 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee58fe9-d6b5-441a-9829-e1ef96997ddf None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:40.110981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:40.117452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:40.118804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:40.118804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:40.118804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:40.118804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 336/648] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:40.118804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.147621 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.148396 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.148744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.149095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.156955 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:40.158549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:40.170402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:40.171786 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:40.173200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 313/649] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:40.178396 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.193322 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:40.229768 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e04f73-50de-49ca-bbed-0ac4dc1a65de None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.269405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.270047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.270490 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.270699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=90103f3776ce4eeb90bde66ed546aa87) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:40.271966 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:40.273427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:40.290151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 337/650] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:40 2022] GET /identity/v3/role_assignments?user.id=90103f3776ce4eeb90bde66ed546aa87&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:40.290151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.295427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f89c5d-97d1-4679-b042-e285fd770992 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.340483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.341314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.341917 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.342234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.359313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:40.361001 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:40.373691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:40.384423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:40.385709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:40.386961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:40.386961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:40.386961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:40.386961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 314/651] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:40.387680 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.427916 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877384c-f627-42f9-926d-9b3d63fea499 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.490433 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.491126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.491468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.491803 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.492619 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:40.492927 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:40.494375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:40.497852 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:40.517927 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 338/652] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:40.517927 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.526083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c0741-304a-4576-ba87-20f3af41894e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.575115 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.576689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.576689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.576689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.577365 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:40.577740 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:40.579453 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:40.611547 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 315/653] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:39:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:40.612467 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.681833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccf913-8f5a-4d8a-bafc-e6d99cf1eb06 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.709372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:40.733347 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.734265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.734663 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.735054 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.735890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:40.736296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:40.737850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:40.753847 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 339/654] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:40.755199 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.774563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfc9f7-b977-4077-a726-3c1c89eeb919 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.824111 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.824800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.825084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.825359 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.830908 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:40.834076 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:40.835799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:40.851739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:40.853013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:40.854679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 316/655] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:40.855644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:40.885917 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5269-6055-4bd4-b95d-521154295b0c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:40.924737 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbd797-91e0-442e-8f6f-4a2e6bc4823a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:40.925401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbd797-91e0-442e-8f6f-4a2e6bc4823a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:40.925693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbd797-91e0-442e-8f6f-4a2e6bc4823a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:40.926151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbd797-91e0-442e-8f6f-4a2e6bc4823a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:40.982750 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f41f4d-0c5d-4b3d-bbad-566644cec97c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:41.002553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:41.012519 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbd797-91e0-442e-8f6f-4a2e6bc4823a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:41.014028 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbd797-91e0-442e-8f6f-4a2e6bc4823a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:41.021287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcc043-363b-4cb5-a116-417d6152ebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:41.021961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcc043-363b-4cb5-a116-417d6152ebc1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.022271 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcc043-363b-4cb5-a116-417d6152ebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.022553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcc043-363b-4cb5-a116-417d6152ebc1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.031602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcc043-363b-4cb5-a116-417d6152ebc1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:41.033125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcc043-363b-4cb5-a116-417d6152ebc1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-67bcc043-363b-4cb5-a116-417d6152ebc1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:41.047147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:41.048592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:41.049758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 317/656] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:41.050567 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854901c-a686-4688-9199-72e110b51318 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.050567 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854901c-a686-4688-9199-72e110b51318 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.050567 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854901c-a686-4688-9199-72e110b51318 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.052034 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 318/657] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:41.055422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854901c-a686-4688-9199-72e110b51318 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:41.197099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854901c-a686-4688-9199-72e110b51318 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:41.215738 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:41.238821 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:41.239517 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.239791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.240062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.240906 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:41.241206 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:41.242725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:41.265074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 319/658] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:39:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:41.265074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:41.273474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 340/659] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:41.274569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d91e-1a93-41c1-afd3-ce75b1542763 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.274910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d91e-1a93-41c1-afd3-ce75b1542763 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.275239 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d91e-1a93-41c1-afd3-ce75b1542763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.293382 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-4e10d91e-1a93-41c1-afd3-ce75b1542763 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:41.294612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-4e10d91e-1a93-41c1-afd3-ce75b1542763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:41.302753 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:41.366167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 341/660] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:41.366167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10d91e-1a93-41c1-afd3-ce75b1542763 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:41.387713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e01d3-9c32-45cb-9f7c-bb5e6926014f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:41.424625 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:41.425261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.425536 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.425808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.426708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:41.427011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:41.428486 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:41.445841 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 320/661] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:39:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:41.446713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:41.469367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:41.487052 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10d91e-1a93-41c1-afd3-ce75b1542763 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:41.523715 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f520c7-6f88-4673-8fee-bc88aa1b9e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:41.524373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f520c7-6f88-4673-8fee-bc88aa1b9e5c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.524659 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f520c7-6f88-4673-8fee-bc88aa1b9e5c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.524927 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f520c7-6f88-4673-8fee-bc88aa1b9e5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.548239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:41.555896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7da8-80d8-4ab5-bb81-fe0c9a42c104 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:41.593928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:41.594657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.594938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.595216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.606533 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:41.608022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:41.610272 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f520c7-6f88-4673-8fee-bc88aa1b9e5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:41.611984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f520c7-6f88-4673-8fee-bc88aa1b9e5c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:41.626009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:41.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:41.628582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:41.628582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:41.628582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:41.628582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 321/662] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:41.628582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:41.710892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:41.733061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 342/663] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:41.733061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f520c7-6f88-4673-8fee-bc88aa1b9e5c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:41.747293 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6de47-3c74-4dc8-ac7f-1aff5fbc6493 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:41.782318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08c08f-aeab-44b7-adec-cc178f1d7fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:41.782984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08c08f-aeab-44b7-adec-cc178f1d7fd0 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.783269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08c08f-aeab-44b7-adec-cc178f1d7fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.783552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08c08f-aeab-44b7-adec-cc178f1d7fd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.794921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08c08f-aeab-44b7-adec-cc178f1d7fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-aa08c08f-aeab-44b7-adec-cc178f1d7fd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:41.800458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:41.801681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 322/664] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:41.802832 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08c08f-aeab-44b7-adec-cc178f1d7fd0 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:41.816166 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:41.843789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f520c7-6f88-4673-8fee-bc88aa1b9e5c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:41.879308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49017d-e14e-412a-af50-8079b77961fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:41.879977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49017d-e14e-412a-af50-8079b77961fc None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.880259 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49017d-e14e-412a-af50-8079b77961fc None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.880532 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49017d-e14e-412a-af50-8079b77961fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.917141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08c08f-aeab-44b7-adec-cc178f1d7fd0 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:41.939990 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:41.958532 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49017d-e14e-412a-af50-8079b77961fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:41.960011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49017d-e14e-412a-af50-8079b77961fc None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:41.960915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:41.961556 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:41.961836 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:41.962249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:41.963627 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:41.963627 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:41.965104 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:41.983298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 323/665] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:41.983298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.069415 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:42.070917 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 343/666] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:42.071963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49017d-e14e-412a-af50-8079b77961fc None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.111407 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06189ae0-ff0a-4e1f-b97e-7b2c4ed26363 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.145391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.146060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.146375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.146652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.156865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.158382 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.165928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:42.173853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:42.175230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:42.176384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:42.176384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:42.176384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:42.176384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 324/667] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:42.176384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.188210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49017d-e14e-412a-af50-8079b77961fc None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.226697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.227235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.227669 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.228573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:42.229054 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.230787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.239811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 344/668] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:42.240798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.286973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd61c-a020-4f20-b783-9084370247ed None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.305063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:42.328125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.328801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.329075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.329356 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.336288 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.337769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.348757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e19f-d60c-4779-aef9-8f2f9ab8e437 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:42.349391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:42.350502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 325/669] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:42.351619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.392924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.393959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.394496 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.394962 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.406677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.408389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.417972 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:42.426504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:42.427675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:42.428864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:42.428864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:42.428864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:42.428864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 345/670] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:42.428864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.469078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd31306-e2a9-4658-9789-ce21de316a0c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.505373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.505932 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.506262 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.506541 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.507245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:42.507555 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.508968 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.520906 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 326/671] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:42.521738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.541375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368124a4-2106-484a-8ac8-038c90841c5b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.564274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:42.576835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.577500 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.577786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.578073 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.589758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.591305 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:42.606148 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:42.607319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:42.608830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:42.608830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:42.608830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:42.608830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 346/672] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:42.608830 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.629975 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83d859-c3e6-4025-a998-7cdc11ac2aab None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.661843 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:42.667190 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c511e2-ea07-4cda-85c9-e95c7887b7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.668132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c511e2-ea07-4cda-85c9-e95c7887b7c9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.668604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c511e2-ea07-4cda-85c9-e95c7887b7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.669071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c511e2-ea07-4cda-85c9-e95c7887b7c9 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.670443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c511e2-ea07-4cda-85c9-e95c7887b7c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:42.670985 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c511e2-ea07-4cda-85c9-e95c7887b7c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.673651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c511e2-ea07-4cda-85c9-e95c7887b7c9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.692583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 327/673] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:42.695175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f446127-323b-4283-bf6d-39bd4fca0116 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.695702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f446127-323b-4283-bf6d-39bd4fca0116 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.696166 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f446127-323b-4283-bf6d-39bd4fca0116 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.703607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 328/674] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:42.704485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f446127-323b-4283-bf6d-39bd4fca0116 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.720108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb306b-7ca9-47a1-a921-dc5b8b1d9777 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.753542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.754197 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.754485 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.754836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.755583 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:42.755857 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.757211 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.768971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 347/675] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:42.769877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.806174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:42.811147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f446127-323b-4283-bf6d-39bd4fca0116 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.844999 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.845652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.845937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.846248 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.852962 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.854563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:42.866900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:42.868125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 329/676] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:42.869233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:42.875987 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a3e3-63db-48ca-81e3-d079c1db6a3b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:42.889482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:42.910404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ee72f-f714-4db7-8232-05706fc4006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:42.911157 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ee72f-f714-4db7-8232-05706fc4006f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.911436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ee72f-f714-4db7-8232-05706fc4006f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.911703 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ee72f-f714-4db7-8232-05706fc4006f None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.912398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ee72f-f714-4db7-8232-05706fc4006f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:42.912698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ee72f-f714-4db7-8232-05706fc4006f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:42.914080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ee72f-f714-4db7-8232-05706fc4006f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:42.925303 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 348/677] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:42.926537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acad6f-93db-496e-8919-a27d9daffe7d None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:42.926825 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acad6f-93db-496e-8919-a27d9daffe7d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:42.927101 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acad6f-93db-496e-8919-a27d9daffe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:42.949916 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-83acad6f-93db-496e-8919-a27d9daffe7d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:42.951961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-83acad6f-93db-496e-8919-a27d9daffe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:42.972188 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111009e0-3316-41e1-ad2a-c74a77da6c31 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.006080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.006649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.006929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.007202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.008014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=0a4ddbb2e9ae43eb8c4efaa7e82a05b4) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:43.008323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:43.009791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:43.015605 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 330/678] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:42 2022] GET /identity/v3/role_assignments?user.id=0a4ddbb2e9ae43eb8c4efaa7e82a05b4&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:43.016404 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.023569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 349/679] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:43.025705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acad6f-93db-496e-8919-a27d9daffe7d None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.058026 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:43.140671 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acad6f-93db-496e-8919-a27d9daffe7d None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.142783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:43.144402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea33948-2ffd-42f1-b620-21d434dce524 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.181401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.182791 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.182791 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.183024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.183221 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.183399 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.183508 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:43.184785 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.184908 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:43.185171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:43.193571 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:43.195443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:43.199826 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 350/680] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:43.200697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:43.207495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:43.208865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 331/681] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:43.210728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.310626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:43.319798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ad93-045e-4d2d-bf70-1d915bac0c91 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.341219 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c21c9c-ea2d-4495-976e-007e72d9f2cc None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.371757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.372537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.372922 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.373268 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] PATH_INFO: `/v3/users/swift` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.381415 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff31277-8b21-4078-a5bc-a263983ce4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.382038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:43.382256 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31277-8b21-4078-a5bc-a263983ce4a1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.382373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31277-8b21-4078-a5bc-a263983ce4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.382640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31277-8b21-4078-a5bc-a263983ce4a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.383741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:43.394851 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff31277-8b21-4078-a5bc-a263983ce4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:43.397910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:43.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 351/682] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:43.400782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-fff31277-8b21-4078-a5bc-a263983ce4a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:43.403882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:43.405075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:43.406179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:43.408285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 332/683] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:43.413846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff31277-8b21-4078-a5bc-a263983ce4a1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.436635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:43.537964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff31277-8b21-4078-a5bc-a263983ce4a1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.539177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18024a-5c6a-422d-87ef-015d0d47d8da None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.582991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.582991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.582991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.582991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.583896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=2a4683819f3f40daafc42098b161c364) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:43.583896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:43.583896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:43.586013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.587389 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.587723 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.588164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.589016 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:43.589397 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:43.602504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 352/684] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/role_assignments?user.id=2a4683819f3f40daafc42098b161c364&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:43.602791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:43.602911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.623765 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 333/685] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:43.624929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.628427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:43.725062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:43.750791 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7301c-8e5b-45db-baed-6b64db42de79 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.756801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107c6d3-f07a-4f02-bdc9-392cc13dbaad None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:43.805193 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.805859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.806187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.806464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.807262 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:43.807606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:43.809062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:43.816939 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:43.817754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:43.818189 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:43.818564 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:43.820243 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:43.820619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 334/686] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:43.820767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:43.821514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.823316 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:43.846685 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 353/687] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:43.847667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:43.924608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:43.991741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ad767-cfc7-4ebc-bc08-a259ee71c62e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:44.003108 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a51304-73b9-4a8c-97b3-b39b2bcc3c55 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:44.057652 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:44.058968 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:44.059586 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:44.060541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:44.066281 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:44.067650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:44.068367 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:44.069456 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:44.071195 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=a1abd448d02d41cf9d969fe9b403259e) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:44.071969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:44.074815 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:44.085493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:44.087038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 335/688] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/role_assignments?user.id=a1abd448d02d41cf9d969fe9b403259e&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:44.088693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:44.094785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:44.100714 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:44.154203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:44.155472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:44.156572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:44.156572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:44.156572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:44.156572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 354/689] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:44.157547 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:44.336355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f32239-9e4f-4120-a699-afc8156645ac None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:44.364557 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b7751-5c86-4557-b2ca-da1f992518b8 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:44.411276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:44.417961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:44.418390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf242d7-96bb-49dc-86e9-b740b133da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:44.419041 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf242d7-96bb-49dc-86e9-b740b133da05 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:44.420017 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf242d7-96bb-49dc-86e9-b740b133da05 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:44.420331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf242d7-96bb-49dc-86e9-b740b133da05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:44.421744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:44.422664 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:44.423538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:44.425252 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:44.426581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:44.429787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:44.459380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 355/690] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:44.471107 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:44.542622 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf242d7-96bb-49dc-86e9-b740b133da05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:44.544638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf242d7-96bb-49dc-86e9-b740b133da05 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:44.552401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:44.640867 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3b7d6-adc0-4406-ac04-e899cd7cb551 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:44.697398 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:44.699042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:44.701915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:44.702776 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:44.718458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:44.721960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:44.750096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:44.752345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:44.754648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:44.758680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 356/691] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:39:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:44.766089 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 336/692] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:44.768351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:44.769010 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf242d7-96bb-49dc-86e9-b740b133da05 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:44.876243 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:45.029643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342925cb-b24d-48dc-9448-c903571819c7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:45.049206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:45.076598 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf242d7-96bb-49dc-86e9-b740b133da05 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:45.084824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:45.085976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:45.086530 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:45.087129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:45.088173 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:45.088696 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:45.090652 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:45.124377 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 357/693] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:39:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:45.135235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:45.174472 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:45.175304 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:45.175604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:45.175887 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:45.195985 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:45.197565 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:45.234347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:45.236674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:45.240478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:45.240478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:45.240478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:45.240478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 337/694] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:45.244101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:45.338252 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d91f54-67f6-4f24-9631-585551fbce9c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:45.377449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd72a6-3edd-4f3a-9d5f-8ee2cdbbb908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:45.378243 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd72a6-3edd-4f3a-9d5f-8ee2cdbbb908 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:45.378424 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd72a6-3edd-4f3a-9d5f-8ee2cdbbb908 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:45.378782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd72a6-3edd-4f3a-9d5f-8ee2cdbbb908 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:45.454636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:45.465372 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:45.496531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd72a6-3edd-4f3a-9d5f-8ee2cdbbb908 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:45.498071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd72a6-3edd-4f3a-9d5f-8ee2cdbbb908 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:45.498752 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea267bfb-a8d1-4764-a1f9-a5a77bb573b1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:45.577492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:45.579151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:45.579993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:45.580762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:45.582421 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:45.583555 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:45.588487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:45.615398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 338/695] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:45.619813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 358/696] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:39:45.631133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:45.645697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd72a6-3edd-4f3a-9d5f-8ee2cdbbb908 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:45.831919 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:45.912470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945ef30-685a-48f3-bf2d-baf1ba763c5e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:45.920144 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd72a6-3edd-4f3a-9d5f-8ee2cdbbb908 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:45.995928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008b9cb-a97f-46bf-b0e9-d0d467cda58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:45.996958 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:45.997110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b9cb-a97f-46bf-b0e9-d0d467cda58c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:45.997590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b9cb-a97f-46bf-b0e9-d0d467cda58c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:45.998026 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:45.998178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b9cb-a97f-46bf-b0e9-d0d467cda58c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:45.998529 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:45.998991 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.012361 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008b9cb-a97f-46bf-b0e9-d0d467cda58c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:46.015263 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008b9cb-a97f-46bf-b0e9-d0d467cda58c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:46.016480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:46.022617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:46.025879 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-0008b9cb-a97f-46bf-b0e9-d0d467cda58c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:46.039635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:46.040914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:46.042032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:46.042032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:46.042032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:46.042032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:46.042032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:46.042032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:46.042032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:46.042032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:46.043049 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 359/697] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:46.045422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb80d6-9d97-441a-9537-30b89465c71d None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:46.045920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb80d6-9d97-441a-9537-30b89465c71d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:46.046452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb80d6-9d97-441a-9537-30b89465c71d None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.055520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 360/698] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:46.058222 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f8727-29b9-4185-a6b3-25dda94c0c4c None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:46.058711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f8727-29b9-4185-a6b3-25dda94c0c4c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:46.059153 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f8727-29b9-4185-a6b3-25dda94c0c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:46.061480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:46.064470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:46.067133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:46.067133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:46.067133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:46.067133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 339/699] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:46.067133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:46.091919 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-a40f8727-29b9-4185-a6b3-25dda94c0c4c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:46.093882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-a40f8727-29b9-4185-a6b3-25dda94c0c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:46.207273 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 361/700] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:46.207273 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f8727-29b9-4185-a6b3-25dda94c0c4c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:46.254804 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a478d-1d0f-45ea-b522-aefc7540e08f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:46.307725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde2036-4733-4344-b823-0deae1fd96cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:46.308398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2036-4733-4344-b823-0deae1fd96cb None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:46.308692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2036-4733-4344-b823-0deae1fd96cb None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:46.308979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2036-4733-4344-b823-0deae1fd96cb None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.309380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:46.310555 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde2036-4733-4344-b823-0deae1fd96cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:46.310899 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde2036-4733-4344-b823-0deae1fd96cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:46.312583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde2036-4733-4344-b823-0deae1fd96cb None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:46.338155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 340/701] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:46.338155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584272-48a7-434b-b8ac-961d3114d05d None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:46.338155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584272-48a7-434b-b8ac-961d3114d05d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:46.338155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584272-48a7-434b-b8ac-961d3114d05d None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.342439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 341/702] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:46.344436 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35584272-48a7-434b-b8ac-961d3114d05d None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:46.385648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f8727-29b9-4185-a6b3-25dda94c0c4c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:46.423809 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:46.438152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:46.439222 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:46.439666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:46.440103 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.441314 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:46.441865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:46.444489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:46.460777 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 362/703] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:46.460777 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:46.526002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35584272-48a7-434b-b8ac-961d3114d05d None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:46.576250 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a950b-b6ef-4740-83b2-497e9f3b8187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:46.576936 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a950b-b6ef-4740-83b2-497e9f3b8187 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:46.577221 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a950b-b6ef-4740-83b2-497e9f3b8187 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:46.577487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a950b-b6ef-4740-83b2-497e9f3b8187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.599084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a950b-b6ef-4740-83b2-497e9f3b8187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:46.601929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a950b-b6ef-4740-83b2-497e9f3b8187 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:46.618226 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51475175-1955-4c15-87ae-87ab9014a9bf None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-d06a950b-b6ef-4740-83b2-497e9f3b8187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:46.629342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:46.632212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:46.643505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:46.643505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:46.643505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:46.643505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 342/704] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:46.643505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd5d92-c00b-492e-8f84-9266ac90596a None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:46.643505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd5d92-c00b-492e-8f84-9266ac90596a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:46.643505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd5d92-c00b-492e-8f84-9266ac90596a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.660477 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-95dd5d92-c00b-492e-8f84-9266ac90596a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:46.661844 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-95dd5d92-c00b-492e-8f84-9266ac90596a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:46.665902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:46.690889 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:46.692191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:46.693766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:46.693766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:46.695459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=9b3e9589dc214e90a6ace523306dffc0) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:46.696033 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:46.698607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:46.710050 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 363/705] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:46 2022] GET /identity/v3/role_assignments?user.id=9b3e9589dc214e90a6ace523306dffc0&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:46.711867 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:46.806261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 343/706] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:46.806261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd5d92-c00b-492e-8f84-9266ac90596a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:46.897717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:46.965258 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1c001-bffc-484e-b9c8-c0d1c9f216e3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:47.014449 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd5d92-c00b-492e-8f84-9266ac90596a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:47.018494 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:47.019671 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.020303 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.020765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.036632 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:47.039654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:47.065555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:47.067810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 364/707] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:47.070278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:47.081615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5791087-6722-4d21-8569-b9587827ce86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:47.082785 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5791087-6722-4d21-8569-b9587827ce86 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.083321 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5791087-6722-4d21-8569-b9587827ce86 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.083845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5791087-6722-4d21-8569-b9587827ce86 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.084859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5791087-6722-4d21-8569-b9587827ce86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:47.085475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5791087-6722-4d21-8569-b9587827ce86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:47.087179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5791087-6722-4d21-8569-b9587827ce86 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:47.101683 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 344/708] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:47.103396 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08a4e-4dc8-4670-82c6-121d91f8bf85 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.103920 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08a4e-4dc8-4670-82c6-121d91f8bf85 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.104372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08a4e-4dc8-4670-82c6-121d91f8bf85 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.106174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 345/709] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:47.107166 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe08a4e-4dc8-4670-82c6-121d91f8bf85 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:47.174946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:47.257699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:47.262787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7505e-79d5-4109-8d6c-583febc670f3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:47.271057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe08a4e-4dc8-4670-82c6-121d91f8bf85 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:47.314373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:47.315569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.316049 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.316545 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.317785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:47.320162 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:47.325228 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565102e-48b9-4de2-9d79-956a1152b5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:47.326722 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565102e-48b9-4de2-9d79-956a1152b5ab None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.327359 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565102e-48b9-4de2-9d79-956a1152b5ab None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.327930 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565102e-48b9-4de2-9d79-956a1152b5ab None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.328893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565102e-48b9-4de2-9d79-956a1152b5ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:47.329576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565102e-48b9-4de2-9d79-956a1152b5ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:47.331331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565102e-48b9-4de2-9d79-956a1152b5ab None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:47.338489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:47.339929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:39:47.340977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:47.342549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 365/710] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:39:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:47.342549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:47.362992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 346/711] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:47.362992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c49-36bd-440f-99cf-7f16f127f89c None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.362992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c49-36bd-440f-99cf-7f16f127f89c None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.362992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c49-36bd-440f-99cf-7f16f127f89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.387041 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-fa771c49-36bd-440f-99cf-7f16f127f89c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:47.388758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-fa771c49-36bd-440f-99cf-7f16f127f89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:47.484505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 347/712] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:47.485563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771c49-36bd-440f-99cf-7f16f127f89c None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:47.519203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc280-9223-43bb-b20a-3b3bd1e9f923 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:47.538680 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:47.568869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7398b-e300-44ee-910d-9c399735cfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:47.570290 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7398b-e300-44ee-910d-9c399735cfe8 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.570748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7398b-e300-44ee-910d-9c399735cfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.571141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7398b-e300-44ee-910d-9c399735cfe8 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.572445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7398b-e300-44ee-910d-9c399735cfe8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:47.574997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7398b-e300-44ee-910d-9c399735cfe8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:47.605812 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 366/713] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:39:47 2022] POST /identity/v3/users => generated 254 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:39:47.607426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7398b-e300-44ee-910d-9c399735cfe8 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:47.632179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:47.665197 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771c49-36bd-440f-99cf-7f16f127f89c None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:47.766360 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:47.767396 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.767869 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.769173 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.770745 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:47.771188 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:47.774931 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:47.790155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 348/714] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:39:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:47.790155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:47.848898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7398b-e300-44ee-910d-9c399735cfe8 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:47.913811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:47.914612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:47.914888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:47.915173 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:47.916028 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=813e8db425644f688e926aafe3249721, user.id=90103f3776ce4eeb90bde66ed546aa87) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:47.916346 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:47.917994 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:47.926426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 367/715] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:47 2022] GET /identity/v3/role_assignments?user.id=90103f3776ce4eeb90bde66ed546aa87&scope.project.id=813e8db425644f688e926aafe3249721&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:47.928160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:47.974395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:47.984211 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33773cd4-1947-46d5-8c73-1d12a440ce00 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:48.072524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33cdd-9f42-4343-90bf-d6cead61bd5f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:48.135201 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:48.136088 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:48.136383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:48.136757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:48.137526 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:48.137882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:48.139533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:48.150788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 368/716] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:39:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:48.196536 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:48.197378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:48.197692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:48.198029 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] PATH_INFO: `/v3/services/network` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:48.198779 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:48.200356 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:48.214056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:48.215387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:48.216428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 349/717] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:47 2022] GET /identity/v3/services/network => generated 88 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:48.222846 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:48.415158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42139159-d8b6-467e-b74a-a320576b0292 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:48.418485 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:48.457999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed72c-a9f2-4181-a650-4920b34ca5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:48.458695 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed72c-a9f2-4181-a650-4920b34ca5c9 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:48.458978 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed72c-a9f2-4181-a650-4920b34ca5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:48.459252 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed72c-a9f2-4181-a650-4920b34ca5c9 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:48.459964 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed72c-a9f2-4181-a650-4920b34ca5c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:48.460270 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed72c-a9f2-4181-a650-4920b34ca5c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:48.461676 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed72c-a9f2-4181-a650-4920b34ca5c9 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:48.467566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 369/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:39:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:48.473303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:48.509459 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:48.673584 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfce708-8d8f-4a4c-a00e-dbb0c25a7381 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:48.732474 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a48074-620d-4bb0-b0fb-2c730c1633e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:48.733155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a48074-620d-4bb0-b0fb-2c730c1633e5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:48.733434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a48074-620d-4bb0-b0fb-2c730c1633e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:48.733709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a48074-620d-4bb0-b0fb-2c730c1633e5 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:48.734476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a48074-620d-4bb0-b0fb-2c730c1633e5 None admin] RBAC: Adding query filter params (type=network) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:48.734788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a48074-620d-4bb0-b0fb-2c730c1633e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:48.736260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a48074-620d-4bb0-b0fb-2c730c1633e5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:48.748163 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 350/719] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:39:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:49.273654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5b2d-01a9-4077-81e5-3f8a934146b1 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:49.274295 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5b2d-01a9-4077-81e5-3f8a934146b1 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:49.274295 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5b2d-01a9-4077-81e5-3f8a934146b1 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:49.276276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 370/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:49.300971 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004be673-a173-4b15-93db-83a6a3db39dc None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:49.301309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004be673-a173-4b15-93db-83a6a3db39dc None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:49.301605 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004be673-a173-4b15-93db-83a6a3db39dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:49.331474 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-004be673-a173-4b15-93db-83a6a3db39dc None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:49.332822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-004be673-a173-4b15-93db-83a6a3db39dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:49.436947 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 351/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:49.495369 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef5b2d-01a9-4077-81e5-3f8a934146b1 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:49.646882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef5b2d-01a9-4077-81e5-3f8a934146b1 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:49.704276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ad628-dda0-4c88-879c-883c5c17595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:49.704956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ad628-dda0-4c88-879c-883c5c17595a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:49.705239 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ad628-dda0-4c88-879c-883c5c17595a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:49.705515 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ad628-dda0-4c88-879c-883c5c17595a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:49.722779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ad628-dda0-4c88-879c-883c5c17595a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-328ad628-dda0-4c88-879c-883c5c17595a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:49.728003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:49.730386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:49.732798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:49.734384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 371/722] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:49.738317 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004be673-a173-4b15-93db-83a6a3db39dc None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:49.758984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:49.958411 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004be673-a173-4b15-93db-83a6a3db39dc None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:50.041434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:50.042528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:50.043012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:50.043494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:50.044797 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:50.045327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:50.047905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:50.067794 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 352/723] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:50.076361 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ad628-dda0-4c88-879c-883c5c17595a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:50.112982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:50.200024 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ad628-dda0-4c88-879c-883c5c17595a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:50.242153 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5baf71-7fdb-4d78-8c47-b1924c2ddfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:50.243270 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5baf71-7fdb-4d78-8c47-b1924c2ddfd4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:50.243780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5baf71-7fdb-4d78-8c47-b1924c2ddfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:50.244395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5baf71-7fdb-4d78-8c47-b1924c2ddfd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:50.262145 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5baf71-7fdb-4d78-8c47-b1924c2ddfd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:50.264051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5baf71-7fdb-4d78-8c47-b1924c2ddfd4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-0d5baf71-7fdb-4d78-8c47-b1924c2ddfd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:50.287358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:50.289042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:50.290176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:50.290176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:50.290176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:50.299938 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 372/724] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:50.308874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:50.512523 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:50.544167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc1a56-9a4b-41b7-b59e-8ea660e7a4c3 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:50.544460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc1a56-9a4b-41b7-b59e-8ea660e7a4c3 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:50.544851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc1a56-9a4b-41b7-b59e-8ea660e7a4c3 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:50.546089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 373/725] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:50.564921 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14111eff-763b-4cd7-b214-92332e78d033 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:50.565404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14111eff-763b-4cd7-b214-92332e78d033 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:50.565891 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14111eff-763b-4cd7-b214-92332e78d033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:50.583953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d8d49-2375-4dd7-b932-ddc2eafd7106 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:50.607805 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-14111eff-763b-4cd7-b214-92332e78d033 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:50.609734 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-14111eff-763b-4cd7-b214-92332e78d033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:50.651459 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:50.652467 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:50.652943 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:50.653411 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:50.654699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:50.655220 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:50.657550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:50.678762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 353/726] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:50.699682 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:50.729896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 374/727] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:39:50.779897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14111eff-763b-4cd7-b214-92332e78d033 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:50.783888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:50.885938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d48a4b-ce1d-409d-bc9f-714e8a162cbd None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:50.933713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:50.949257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:50.949257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:50.949257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:50.949257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:50.960493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:50.962443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:50.977969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:50.979447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:50.980864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:50.981915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 354/728] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:50.988645 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:51.099977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14111eff-763b-4cd7-b214-92332e78d033 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:51.148440 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deec07-8fd2-45dd-9a6a-a63a520e2cb1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:51.193180 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:51.194670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.195169 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.195653 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.197027 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:51.197525 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:51.200060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:51.227482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301907-0b56-4828-a844-83cf44cec4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:51.228640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301907-0b56-4828-a844-83cf44cec4be None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.229146 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301907-0b56-4828-a844-83cf44cec4be None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.229651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301907-0b56-4828-a844-83cf44cec4be None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.265527 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81301907-0b56-4828-a844-83cf44cec4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:51.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 355/729] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:39:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-81301907-0b56-4828-a844-83cf44cec4be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:51.275809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:51.276959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:51.278053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:51.278821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 375/730] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 500 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:51.280062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:51.287639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301907-0b56-4828-a844-83cf44cec4be None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:51.381506 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:51.417940 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:51.457292 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4276088-2f23-49fe-a8cd-ff806d1f67ee None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:51.486588 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301907-0b56-4828-a844-83cf44cec4be None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:51.507822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015673a2-e6c1-42c7-8706-6af488bb6d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:51.508492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015673a2-e6c1-42c7-8706-6af488bb6d2f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.508775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015673a2-e6c1-42c7-8706-6af488bb6d2f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.509053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015673a2-e6c1-42c7-8706-6af488bb6d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.569876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76e9b4-68b3-4119-9a72-dfa01bc9448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:51.572842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76e9b4-68b3-4119-9a72-dfa01bc9448f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.573288 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76e9b4-68b3-4119-9a72-dfa01bc9448f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.573590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76e9b4-68b3-4119-9a72-dfa01bc9448f None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.574402 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76e9b4-68b3-4119-9a72-dfa01bc9448f None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:51.574710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76e9b4-68b3-4119-9a72-dfa01bc9448f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:51.576159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76e9b4-68b3-4119-9a72-dfa01bc9448f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:51.590035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 376/731] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:51.590035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ddca3-a547-4e46-a602-858f0ac78fc0 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.590035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ddca3-a547-4e46-a602-858f0ac78fc0 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.590035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ddca3-a547-4e46-a602-858f0ac78fc0 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.590035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 377/732] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:51.590035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ff658-2e3f-446e-aeac-e4e93b5566ca None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.591085 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ff658-2e3f-446e-aeac-e4e93b5566ca None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.591085 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ff658-2e3f-446e-aeac-e4e93b5566ca None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.596018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 378/733] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:51.597707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646d7cd-827b-4c3e-b520-33c0a3387e1e None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.598253 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646d7cd-827b-4c3e-b520-33c0a3387e1e None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.598701 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646d7cd-827b-4c3e-b520-33c0a3387e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.611220 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015673a2-e6c1-42c7-8706-6af488bb6d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:51.612898 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015673a2-e6c1-42c7-8706-6af488bb6d2f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:51.618787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-9646d7cd-827b-4c3e-b520-33c0a3387e1e None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:51.620031 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-9646d7cd-827b-4c3e-b520-33c0a3387e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:51.677747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:51.721692 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 379/734] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:51.724089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa741-e729-4063-9f2a-ba0d25317a34 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.724677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa741-e729-4063-9f2a-ba0d25317a34 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.725247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa741-e729-4063-9f2a-ba0d25317a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.751466 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 356/735] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:39:51.752874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015673a2-e6c1-42c7-8706-6af488bb6d2f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:51.758673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-541aa741-e729-4063-9f2a-ba0d25317a34 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:51.760882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-541aa741-e729-4063-9f2a-ba0d25317a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:51.834038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:51.857498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 380/736] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:51.859205 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aa741-e729-4063-9f2a-ba0d25317a34 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:51.890062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015673a2-e6c1-42c7-8706-6af488bb6d2f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:51.935926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:51.936908 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:51.937191 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:51.937629 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:51.952727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:51.955281 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:51.971649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:51.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:51.984876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:51.991633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:51.991633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:51.991633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:51.991633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 357/737] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:51.991633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:52.028601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aa741-e729-4063-9f2a-ba0d25317a34 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:52.090230 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:52.091438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.092087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.092765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.108046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:52.109766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:52.127571 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:52.128822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:52.131640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:52.131640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:52.131640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:52.131640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 381/738] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:52.131640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:52.164822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19d23-2b02-4933-804f-dbe28f93203e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:52.175112 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:52.206834 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72634539-c192-4299-a75e-91acf2708df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:52.207791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72634539-c192-4299-a75e-91acf2708df4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.208664 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72634539-c192-4299-a75e-91acf2708df4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.208664 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72634539-c192-4299-a75e-91acf2708df4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.230174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72634539-c192-4299-a75e-91acf2708df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-72634539-c192-4299-a75e-91acf2708df4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:52.235875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:52.237137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 358/739] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:52.238508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72634539-c192-4299-a75e-91acf2708df4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:52.272880 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571422-3f0e-407f-9f61-beb078b66154 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:52.311058 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:52.313603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301ac99-45d6-42f1-9d5d-443bf2e7dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:52.314390 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301ac99-45d6-42f1-9d5d-443bf2e7dd85 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.314696 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301ac99-45d6-42f1-9d5d-443bf2e7dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.314967 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301ac99-45d6-42f1-9d5d-443bf2e7dd85 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.315707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301ac99-45d6-42f1-9d5d-443bf2e7dd85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:52.317241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301ac99-45d6-42f1-9d5d-443bf2e7dd85 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:52.336274 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 382/740] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:39:52 2022] POST /identity/v3/services => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:39:52.337264 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301ac99-45d6-42f1-9d5d-443bf2e7dd85 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:52.367527 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72634539-c192-4299-a75e-91acf2708df4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:52.436252 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d2d40-c694-4512-a27c-859b47710ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:52.437205 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d2d40-c694-4512-a27c-859b47710ece None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.437528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d2d40-c694-4512-a27c-859b47710ece None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.437878 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d2d40-c694-4512-a27c-859b47710ece None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.438666 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d2d40-c694-4512-a27c-859b47710ece None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:52.438981 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d2d40-c694-4512-a27c-859b47710ece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:52.440460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d2d40-c694-4512-a27c-859b47710ece None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:52.457069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 359/741] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:52.463959 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb86b5-848c-46fd-8071-577643f4e23a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.464259 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb86b5-848c-46fd-8071-577643f4e23a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.464526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb86b5-848c-46fd-8071-577643f4e23a None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.466863 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 360/742] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:52.467885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb86b5-848c-46fd-8071-577643f4e23a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:52.511246 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:52.523246 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301ac99-45d6-42f1-9d5d-443bf2e7dd85 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:52.571574 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:52.572579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.573111 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.573407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.574572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:52.574572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:52.576073 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:52.593936 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 383/743] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:52.594961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:52.627708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb86b5-848c-46fd-8071-577643f4e23a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:52.656241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:52.688710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8c0f0-6a88-4766-ae2c-d9db498e9171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:52.689377 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c0f0-6a88-4766-ae2c-d9db498e9171 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.689667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c0f0-6a88-4766-ae2c-d9db498e9171 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.689946 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c0f0-6a88-4766-ae2c-d9db498e9171 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.691240 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8c0f0-6a88-4766-ae2c-d9db498e9171 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:52.691240 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8c0f0-6a88-4766-ae2c-d9db498e9171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:52.692679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8c0f0-6a88-4766-ae2c-d9db498e9171 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:52.703309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 361/744] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:52.704671 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b387e-06da-4621-9e88-8ef7f6269dbb None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.704960 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b387e-06da-4621-9e88-8ef7f6269dbb None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.705238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b387e-06da-4621-9e88-8ef7f6269dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.726421 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-720b387e-06da-4621-9e88-8ef7f6269dbb None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:52.727569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-720b387e-06da-4621-9e88-8ef7f6269dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:52.755225 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfeabb-dee8-40b3-b5db-85bded3b9d7b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:52.805038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:52.806387 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:52.806903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:52.807388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:52.818482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 362/745] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:39:52.818482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b387e-06da-4621-9e88-8ef7f6269dbb None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:52.828635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:52.831035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:52.850810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:52.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:52.853257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 384/746] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:39:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:52.854087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:52.882868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:52.981951 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b387e-06da-4621-9e88-8ef7f6269dbb None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:52.999725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:53.032177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76bede-3bcf-432a-b486-4ceb56213816 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:53.052661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:53.053461 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:53.053790 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:53.054012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:53.064389 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:53.066924 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:53.087706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:53.088416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:53.088853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:53.089285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:53.090656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:53.092024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:53.092024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:53.092024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:53.092024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:53.092024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:53.092024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:53.092024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:53.092024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:53.092823 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:53.092919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 363/747] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:53.092919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:53.112101 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:53.114225 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:53.133412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:53.134935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:53.144410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:53.144410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:53.144410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:53.144410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 385/748] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:53.144410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:53.250406 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:53.270900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f1d25-291d-48c4-93da-354bb1292ee5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:53.311407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e909ad9-80a9-4b31-801c-b8235124b76c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:53.317200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2da00-34cf-4b94-b414-0311053224e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:53.317935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2da00-34cf-4b94-b414-0311053224e5 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:53.320988 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2da00-34cf-4b94-b414-0311053224e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:53.320988 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2da00-34cf-4b94-b414-0311053224e5 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:53.320988 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2da00-34cf-4b94-b414-0311053224e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:53.321756 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2da00-34cf-4b94-b414-0311053224e5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:53.343899 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 364/749] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:39:53 2022] POST /identity/v3/services => generated 240 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:39:53.345161 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2da00-34cf-4b94-b414-0311053224e5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:53.405180 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:53.405991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:53.406177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:53.406447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:53.407563 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:53.407866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:53.410289 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:53.423433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:53.450435 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 386/750] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:39:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:53.455986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:53.549055 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2da00-34cf-4b94-b414-0311053224e5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:53.609014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:53.609983 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:53.610392 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:53.610843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:53.611715 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:53.612141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:53.614181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:53.632873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 365/751] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:53.633875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:53.677007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f508920-5ab1-421d-9eee-227009744a99 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:53.715417 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:53.730972 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:53.731926 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:53.732460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:53.732783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:53.733686 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:53.734125 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:53.735834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:53.752715 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 387/752] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:53.754307 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:53.819660 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:53.842906 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bb53d-0d0c-4da2-8fe9-fee2d4efc807 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:53.902745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974ddeb-8f20-47e4-8234-6c1c03653176 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:53.912130 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0dd6a-8876-46e9-9969-150f686c3f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:53.913339 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0dd6a-8876-46e9-9969-150f686c3f6f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:53.913818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0dd6a-8876-46e9-9969-150f686c3f6f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:53.914490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0dd6a-8876-46e9-9969-150f686c3f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:53.957223 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:53.958391 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:53.958921 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:53.959469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:53.961487 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=813e8db425644f688e926aafe3249721, user.id=9b3e9589dc214e90a6ace523306dffc0) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:53.962531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:53.965128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:53.975460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 388/753] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:53 2022] GET /identity/v3/role_assignments?user.id=9b3e9589dc214e90a6ace523306dffc0&scope.project.id=813e8db425644f688e926aafe3249721&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:53.978838 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:54.046578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:54.064370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0dd6a-8876-46e9-9969-150f686c3f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:54.065935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0dd6a-8876-46e9-9969-150f686c3f6f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:54.125950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015d769-ba25-49f5-80e1-be305ad037c2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:54.169436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:54.170697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:54.171239 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:54.171764 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:54.182241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:54.184717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:54.200425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:54.201861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:39:54.203131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:54.206548 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 389/754] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:39:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:54.212367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:54.273872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:54.311079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 366/755] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:39:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:39:54.317037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0dd6a-8876-46e9-9969-150f686c3f6f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:54.413417 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa909e-5f7a-4a70-8b2d-8eb417e89949 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:54.421505 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:54.491895 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:54.492626 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:54.492954 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:54.493242 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:54.494078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:54.494449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:54.496001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:54.517910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 390/756] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:39:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:54.523560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:54.565935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0dd6a-8876-46e9-9969-150f686c3f6f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:54.645135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:54.646839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:54.647673 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:54.648370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:54.679480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:54.682701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:54.704097 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf9d13-0333-4db9-902e-25dcf5c6b4a8 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:54.721935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:54.723416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:54.724970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:54.724970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:54.724970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:54.724970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 367/757] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:54.729610 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:54.770483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:54.771218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:54.771504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:54.771785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:54.784108 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:54.785804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:54.804587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:54.805989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:54.807525 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:54.807525 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:54.807525 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:54.807525 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 391/758] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:54.815241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:54.827448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:54.838872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:54.941885 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d394-aef3-457e-9877-35b1f457041e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:54.990826 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175d1c7-8ac2-454d-a5cb-d250739c1408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:54.991818 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d1c7-8ac2-454d-a5cb-d250739c1408 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:54.992268 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d1c7-8ac2-454d-a5cb-d250739c1408 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:54.992723 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d1c7-8ac2-454d-a5cb-d250739c1408 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:54.994587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175d1c7-8ac2-454d-a5cb-d250739c1408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:54.995943 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175d1c7-8ac2-454d-a5cb-d250739c1408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:54.997582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175d1c7-8ac2-454d-a5cb-d250739c1408 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:55.018580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848caa5-d4a7-4b42-b029-8451cb3cebc6 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:55.024257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 392/759] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:55.024257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f08612-67ba-4756-a23a-8df942746a6c None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.024257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f08612-67ba-4756-a23a-8df942746a6c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.024257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f08612-67ba-4756-a23a-8df942746a6c None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.025894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 393/760] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:55.031815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f08612-67ba-4756-a23a-8df942746a6c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:55.085891 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52422cfb-637b-4ad3-a3b6-acfba8fb88df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:55.086680 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422cfb-637b-4ad3-a3b6-acfba8fb88df None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.087096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422cfb-637b-4ad3-a3b6-acfba8fb88df None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.087448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422cfb-637b-4ad3-a3b6-acfba8fb88df None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.088448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52422cfb-637b-4ad3-a3b6-acfba8fb88df None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:55.088773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52422cfb-637b-4ad3-a3b6-acfba8fb88df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:55.091691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52422cfb-637b-4ad3-a3b6-acfba8fb88df None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:55.107341 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 368/761] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:55.109409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72e48-0115-43d6-8541-004813c62412 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.110142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72e48-0115-43d6-8541-004813c62412 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.110662 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72e48-0115-43d6-8541-004813c62412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.133770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-24a72e48-0115-43d6-8541-004813c62412 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:55.135679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-24a72e48-0115-43d6-8541-004813c62412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:55.157671 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:55.196335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f08612-67ba-4756-a23a-8df942746a6c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:55.233778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 369/762] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:39:55.235581 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a72e48-0115-43d6-8541-004813c62412 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:55.239213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:55.240093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.240530 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.241025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.254047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:55.257429 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:55.270952 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:55.280355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:55.281667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:55.283039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:55.283039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:55.283039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:55.283039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:55.283039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:55.283039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:55.283039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:55.283039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:55.290302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 394/763] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:55.292334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:55.396921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a72e48-0115-43d6-8541-004813c62412 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:55.472900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f4e04-1cbc-4900-95cd-f971802f46ac None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:55.474357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:55.475581 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.476585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.477069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.486514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:55.488364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:55.505647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:55.507014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:55.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:55.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:55.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:55.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:55.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:55.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:55.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:55.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:55.511051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 370/764] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:55.512627 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:55.541245 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:55.542321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.542811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.543422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.544432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:55.546473 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:55.550345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:55.561561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:55.563184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:39:55.564452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:55.567833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 395/765] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:39:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:55.569674 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:55.710217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:55.713425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8a2fd-08a2-4dba-a3d9-04ccfad2884b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:55.733456 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75aef05-532b-4c1c-a0ca-5c3ac064eebd None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:55.767459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:55.768957 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.769457 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.769943 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.770959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:55.771513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:55.773266 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:55.791523 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 396/766] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:55.793058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:55.802865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:55.803997 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.804543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.805071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.806535 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:55.807204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:55.810219 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:55.838293 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 371/767] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:55.840393 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:55.876062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:55.934991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27083e7-4832-49e1-a05e-430f413e8f0c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:55.982019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:55.982767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:55.983136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:55.983257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:55.984227 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:55.984480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:55.986125 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:55.991855 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 397/768] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:39:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:55.992574 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:56.025761 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e8de5-da89-4eaa-959d-4b621fd70d5c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:56.077938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3840-864f-487b-9112-bc1218652c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:56.078671 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3840-864f-487b-9112-bc1218652c6f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.078951 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3840-864f-487b-9112-bc1218652c6f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.079225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3840-864f-487b-9112-bc1218652c6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.080049 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3840-864f-487b-9112-bc1218652c6f None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=945e08e70ce943708b6eeeb0fd8acc5d) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:56.080354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3840-864f-487b-9112-bc1218652c6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:56.081859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3840-864f-487b-9112-bc1218652c6f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:56.095976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 372/769] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:39:55 2022] GET /identity/v3/role_assignments?user.id=945e08e70ce943708b6eeeb0fd8acc5d&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:56.095976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894e41f-1c57-4369-a0d2-45c26d087c9d None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.095976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894e41f-1c57-4369-a0d2-45c26d087c9d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.095976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894e41f-1c57-4369-a0d2-45c26d087c9d None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.095976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 373/770] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:56.095976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894e41f-1c57-4369-a0d2-45c26d087c9d None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:56.116174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:56.168759 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801be138-dfe4-4488-892d-e5a9cb9c0fea None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:56.208151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:56.233897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f76266-9764-4e51-b0b1-14e49e89f7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:56.235638 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f76266-9764-4e51-b0b1-14e49e89f7f5 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.236063 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f76266-9764-4e51-b0b1-14e49e89f7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.236543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f76266-9764-4e51-b0b1-14e49e89f7f5 None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/90103f3776ce4eeb90bde66ed546aa87/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.243036 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f76266-9764-4e51-b0b1-14e49e89f7f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=90103f3776ce4eeb90bde66ed546aa87, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:56.246739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f76266-9764-4e51-b0b1-14e49e89f7f5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:56.252276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894e41f-1c57-4369-a0d2-45c26d087c9d None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:56.279580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 398/771] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:39:55 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/90103f3776ce4eeb90bde66ed546aa87/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:39:56.283329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c28b-ecf7-49b9-b8e5-d1dff7cc57fa None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.284399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c28b-ecf7-49b9-b8e5-d1dff7cc57fa None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.285390 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c28b-ecf7-49b9-b8e5-d1dff7cc57fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.315694 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-4be2c28b-ecf7-49b9-b8e5-d1dff7cc57fa None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:56.317060 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-4be2c28b-ecf7-49b9-b8e5-d1dff7cc57fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:56.329634 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:56.330618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.331072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.331461 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.332929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:56.333422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:56.335180 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:56.343483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 374/772] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:39:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:56.353876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:56.454755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 399/773] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:39:56.480218 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2c28b-ecf7-49b9-b8e5-d1dff7cc57fa None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:56.577667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:56.637733 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb036-c458-4b00-b9fb-60bfa056923a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:56.653787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2c28b-ecf7-49b9-b8e5-d1dff7cc57fa None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:56.674580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:56.705325 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b11750-8597-4a66-b099-e343f519b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:56.705719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:56.706301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b11750-8597-4a66-b099-e343f519b269 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.706588 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.706758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b11750-8597-4a66-b099-e343f519b269 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.706977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.707236 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b11750-8597-4a66-b099-e343f519b269 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.707461 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] PATH_INFO: `/v3/services/network` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.708269 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:56.708410 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b11750-8597-4a66-b099-e343f519b269 None admin] RBAC: Adding query filter params (interface=public, service_id=12b6c9fc835543ea98982fedbc080b57, region_id=RegionOne) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:56.708857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b11750-8597-4a66-b099-e343f519b269 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:56.709869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:56.710541 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b11750-8597-4a66-b099-e343f519b269 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:56.720773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 375/774] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:39:56 2022] GET /identity/v3/endpoints?service_id=12b6c9fc835543ea98982fedbc080b57&interface=public®ion_id=RegionOne => generated 194 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:56.721534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:56.722806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:39:56.724011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:56.724952 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab808dc-484b-4d8e-96a9-28151a75ccab None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.724952 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab808dc-484b-4d8e-96a9-28151a75ccab None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.724952 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab808dc-484b-4d8e-96a9-28151a75ccab None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.725184 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 376/775] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:56.727789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 400/776] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:56 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:56.731925 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef12e2a-e0f6-4abf-85b2-a6d388cb850d None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.732302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef12e2a-e0f6-4abf-85b2-a6d388cb850d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.732586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef12e2a-e0f6-4abf-85b2-a6d388cb850d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.734517 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:56.754987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-cef12e2a-e0f6-4abf-85b2-a6d388cb850d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:56.756662 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-cef12e2a-e0f6-4abf-85b2-a6d388cb850d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:56.854427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 377/777] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:39:56.856403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef12e2a-e0f6-4abf-85b2-a6d388cb850d None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:56.882971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11387b4-3c3f-48d5-8cb6-209f83054047 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:56.924330 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a55a79-798d-42db-bcd2-4d905262b11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:56.925036 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55a79-798d-42db-bcd2-4d905262b11b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:56.925321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55a79-798d-42db-bcd2-4d905262b11b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:56.925593 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55a79-798d-42db-bcd2-4d905262b11b None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:56.926348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a55a79-798d-42db-bcd2-4d905262b11b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:56.927981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a55a79-798d-42db-bcd2-4d905262b11b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:56.942141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 401/778] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:39:56 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:56.942141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a55a79-798d-42db-bcd2-4d905262b11b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:56.993082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:57.013816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:57.059427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef12e2a-e0f6-4abf-85b2-a6d388cb850d None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:57.145612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a55a79-798d-42db-bcd2-4d905262b11b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:57.151031 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:57.151848 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:57.152134 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:57.152416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:57.153299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] RBAC: Adding query filter params (name=network) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:57.153710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:57.155331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:57.162540 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 378/779] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:39:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:57.171721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:57.223324 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c906a50-808a-467f-a404-f34f8765f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:57.224583 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906a50-808a-467f-a404-f34f8765f006 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:57.225280 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906a50-808a-467f-a404-f34f8765f006 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:57.225706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906a50-808a-467f-a404-f34f8765f006 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:57.241081 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c906a50-808a-467f-a404-f34f8765f006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-6c906a50-808a-467f-a404-f34f8765f006 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:57.247716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:57.249128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:57.250510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 402/780] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:57.258049 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c906a50-808a-467f-a404-f34f8765f006 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:57.326283 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d28dcd-3f8f-4da6-8508-8c3d9bb04eab None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:57.407720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:57.409988 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:57.410842 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:57.412850 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:57.414294 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] RBAC: Adding query filter params (type=network) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:57.415775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:57.417670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:57.422818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:57.429569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 379/781] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:39:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:57.437372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:57.451177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c906a50-808a-467f-a404-f34f8765f006 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:57.454357 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:57.534960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:57.535873 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:57.536133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:57.536333 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:57.537038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:57.537405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:57.539051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:57.561826 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 403/782] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:39:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:57.567676 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:57.651849 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e62da-a77b-45cc-9b21-651d0457b0ce None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:57.741048 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:57.741871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:57.742359 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:57.742684 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:57.744328 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] RBAC: Adding query filter params (interface=public, service_id=20d760ee0d35427c91b2b962b6fdafc5, region_id=RegionOne) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:57.744812 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:57.747281 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:57.764114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 380/783] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:39:57 2022] GET /identity/v3/endpoints?service_id=20d760ee0d35427c91b2b962b6fdafc5&interface=public®ion_id=RegionOne => generated 194 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:57.775596 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:57.778054 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0423e5-1fc2-46a3-9dd2-1b2c591b14a6 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:57.826564 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:57.827319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:57.827608 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:57.827933 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:57.847504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:57.849120 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:57.883877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:57.885128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:57.886898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:57.886898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:57.886898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:57.886898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 404/784] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:57.891474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:57.910691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:57.942597 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:57.945068 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6d145-e6f3-47b6-a4ad-3758437c1bbd None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:57.988018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a4006-eb1a-42ad-8566-42f6ceefa920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:57.989805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a4006-eb1a-42ad-8566-42f6ceefa920 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:57.990441 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a4006-eb1a-42ad-8566-42f6ceefa920 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:57.990946 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a4006-eb1a-42ad-8566-42f6ceefa920 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:57.991903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a4006-eb1a-42ad-8566-42f6ceefa920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:57.993545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a4006-eb1a-42ad-8566-42f6ceefa920 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:58.000206 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 381/785] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:39:57 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:58.112981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eb883-6128-4551-89f2-a8ed203be43f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:58.170722 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:58.171407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:58.171710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:58.171973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:58.172710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:58.173022 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:58.174617 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:58.195051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 405/786] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:39:58.200824 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a4006-eb1a-42ad-8566-42f6ceefa920 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:58.335596 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a4006-eb1a-42ad-8566-42f6ceefa920 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:58.370770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:58.385192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:58.385898 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:58.386217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:58.386595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] PATH_INFO: `/v3/users/swift` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:58.395588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:58.397105 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:39:58.415929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:58.417241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:58.418613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 382/787] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:58.423585 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:58.550637 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:58.586215 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa635311-9275-4c7c-9537-48002ae55590 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:58.641558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:58.642285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:58.642576 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:58.642857 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:58.643767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:58.644109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:58.645694 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:58.666997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 406/788] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:39:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:58.672840 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:58.873594 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf3edb-d6be-44e8-b0c6-b23529dcc99f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:58.921500 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:58.922264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:58.922577 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:58.922876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:58.941260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:58.942866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:39:58.966904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:58.968151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:39:58.969284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:39:58.969284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:39:58.969284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:58.969284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 383/789] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:58.974650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:59.141117 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8965bf-fb7b-4c3e-bc3d-34c0461d9020 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:59.190029 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab7a58-18b7-4022-bb5c-15585a359c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:59.190727 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab7a58-18b7-4022-bb5c-15585a359c6d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.191017 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab7a58-18b7-4022-bb5c-15585a359c6d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.191290 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab7a58-18b7-4022-bb5c-15585a359c6d None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.192143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab7a58-18b7-4022-bb5c-15585a359c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:59.192447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab7a58-18b7-4022-bb5c-15585a359c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:59.193885 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab7a58-18b7-4022-bb5c-15585a359c6d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:59.196231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:59.215343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 407/790] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:59.221897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:59.248372 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31f8f-1f16-4e29-9efe-680e20ba5784 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.249272 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31f8f-1f16-4e29-9efe-680e20ba5784 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.249866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31f8f-1f16-4e29-9efe-680e20ba5784 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.252181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 408/791] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:59.260395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e6ec2-f3b8-4aa2-a93b-600527e5df06 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.261213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e6ec2-f3b8-4aa2-a93b-600527e5df06 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.261907 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e6ec2-f3b8-4aa2-a93b-600527e5df06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.293257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-7b7e6ec2-f3b8-4aa2-a93b-600527e5df06 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:59.294239 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-7b7e6ec2-f3b8-4aa2-a93b-600527e5df06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:59.318260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:59.375793 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 409/792] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:39:59.377693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e500f16-5795-423d-808a-dfb9cd665c27 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.378292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e500f16-5795-423d-808a-dfb9cd665c27 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.378774 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e500f16-5795-423d-808a-dfb9cd665c27 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.380895 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 410/793] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:59.387573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2f058-3ad5-47d8-be6e-d444660cb056 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.388209 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2f058-3ad5-47d8-be6e-d444660cb056 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.388675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2f058-3ad5-47d8-be6e-d444660cb056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.407883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6475f-fbf1-45e4-808a-4be9958dcf00 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:59.418748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-4dc2f058-3ad5-47d8-be6e-d444660cb056 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:39:59.420186 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-4dc2f058-3ad5-47d8-be6e-d444660cb056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:39:59.456399 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:59.457746 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.458247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.459182 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.471471 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:59.473425 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:39:59.490566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:59.491866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:59.493082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:39:59.493082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:39:59.493082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:39:59.493082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:39:59.493082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:39:59.493082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:39:59.493082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:39:59.493082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:59.495746 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 384/794] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:39:59.501284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:59.529156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 411/795] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:39:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 28 08:39:59.530844 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2f058-3ad5-47d8-be6e-d444660cb056 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:59.614507 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:59.657882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:59.665138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d0832-91e5-457b-9158-c7665ac6ae38 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:59.671822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2f058-3ad5-47d8-be6e-d444660cb056 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:59.714047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:59.714934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.715324 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.715736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.716777 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:39:59.717104 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:39:59.719187 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:39:59.733129 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34ec61-0c77-409d-9497-de4b0bd25f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:59.734778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34ec61-0c77-409d-9497-de4b0bd25f15 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.735561 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34ec61-0c77-409d-9497-de4b0bd25f15 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.736371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34ec61-0c77-409d-9497-de4b0bd25f15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.738281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 412/796] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:39:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:39:59.739714 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:59.758259 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34ec61-0c77-409d-9497-de4b0bd25f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-9d34ec61-0c77-409d-9497-de4b0bd25f15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:59.769801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:59.772223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:39:59.774643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:39:59.778855 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 385/797] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:39:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:59.780568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34ec61-0c77-409d-9497-de4b0bd25f15 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:59.863697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94d5d3-9e63-4993-9ed4-d22909cc2d51 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:39:59.879258 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:39:59.902824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094a90b-cb5e-46ba-a156-8608359967d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:39:59.903511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094a90b-cb5e-46ba-a156-8608359967d1 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.903807 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094a90b-cb5e-46ba-a156-8608359967d1 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.904074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094a90b-cb5e-46ba-a156-8608359967d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.916818 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094a90b-cb5e-46ba-a156-8608359967d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-9094a90b-cb5e-46ba-a156-8608359967d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:39:59.924126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:39:59.926639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 413/798] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:39:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:39:59.928735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3499-d3b7-497d-ad0e-922aeba3319c None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:39:59.930490 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3499-d3b7-497d-ad0e-922aeba3319c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:39:59.930490 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3499-d3b7-497d-ad0e-922aeba3319c None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:39:59.930490 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 414/799] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:39:59.930906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e3499-d3b7-497d-ad0e-922aeba3319c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:39:59.968551 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34ec61-0c77-409d-9497-de4b0bd25f15 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:00.018633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5e00f-a080-42d1-a5a1-bf9a6ae09fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:00.019478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e00f-a080-42d1-a5a1-bf9a6ae09fe5 None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.019831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e00f-a080-42d1-a5a1-bf9a6ae09fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.020126 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e00f-a080-42d1-a5a1-bf9a6ae09fe5 None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/9b3e9589dc214e90a6ace523306dffc0/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.022298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5e00f-a080-42d1-a5a1-bf9a6ae09fe5 None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=9b3e9589dc214e90a6ace523306dffc0, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:00.023865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5e00f-a080-42d1-a5a1-bf9a6ae09fe5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:00.044089 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 386/800] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:39:59 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/9b3e9589dc214e90a6ace523306dffc0/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:40:00.045603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5e00f-a080-42d1-a5a1-bf9a6ae09fe5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:00.075181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:00.110431 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e3499-d3b7-497d-ad0e-922aeba3319c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:00.165705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df541ee-13dc-4e84-8d52-54cf27b85bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:00.166611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df541ee-13dc-4e84-8d52-54cf27b85bdf None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.167203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df541ee-13dc-4e84-8d52-54cf27b85bdf None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.167452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df541ee-13dc-4e84-8d52-54cf27b85bdf None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.168499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df541ee-13dc-4e84-8d52-54cf27b85bdf None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:00.168875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df541ee-13dc-4e84-8d52-54cf27b85bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:00.171772 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df541ee-13dc-4e84-8d52-54cf27b85bdf None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:00.184726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 415/801] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:39:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:00.186950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2cd9-4112-48a7-b6d8-d2eb2e5a7e25 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.187307 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2cd9-4112-48a7-b6d8-d2eb2e5a7e25 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.187785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2cd9-4112-48a7-b6d8-d2eb2e5a7e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.218523 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-3aaa2cd9-4112-48a7-b6d8-d2eb2e5a7e25 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:00.220867 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-3aaa2cd9-4112-48a7-b6d8-d2eb2e5a7e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:00.223213 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5e00f-a080-42d1-a5a1-bf9a6ae09fe5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:00.269050 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:00.272818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:00.273487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.273819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.274099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.275111 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:00.275565 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:00.277822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:00.298488 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 387/802] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:40:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:00.299657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:00.330399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 416/803] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:00.331495 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa2cd9-4112-48a7-b6d8-d2eb2e5a7e25 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:00.446057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:00.468764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e852e-1d5b-4a42-adc2-1dec9baa9dda None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:00.501215 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa2cd9-4112-48a7-b6d8-d2eb2e5a7e25 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:00.508675 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:00.509355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.509637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.509922 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.523042 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:00.524684 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:00.541601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:00.542867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:00.543979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:00.543979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:00.543979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:00.543979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 388/804] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:00.543979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:00.552986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:00.553677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.553964 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.554290 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.570710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:00.572833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:00.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:00.596281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:00.598354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:00.598354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:00.598354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:00.602937 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 417/805] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:00.604032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:00.636257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:00.698253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa133f-788a-41b4-a821-b0d4e215df79 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:00.733430 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:00.739436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f342234-c643-4d5c-abb1-f715378c5fb8 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:00.741096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c7c6-e351-492a-93a6-03d5c7083f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:00.742094 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c7c6-e351-492a-93a6-03d5c7083f80 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.742457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c7c6-e351-492a-93a6-03d5c7083f80 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.742725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c7c6-e351-492a-93a6-03d5c7083f80 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.743387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29c7c6-e351-492a-93a6-03d5c7083f80 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:00.744811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29c7c6-e351-492a-93a6-03d5c7083f80 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-ba29c7c6-e351-492a-93a6-03d5c7083f80 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:00.755641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:00.757101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:00.758257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 389/806] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:40:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:00.759245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28e4e6-5437-4af1-b734-4f5611fb144b None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.759245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28e4e6-5437-4af1-b734-4f5611fb144b None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.759245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28e4e6-5437-4af1-b734-4f5611fb144b None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.764508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 390/807] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:00.764508 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28e4e6-5437-4af1-b734-4f5611fb144b None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:00.798891 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d00856-5ca8-4a91-8437-25db307a48f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:00.799543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d00856-5ca8-4a91-8437-25db307a48f7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.799832 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d00856-5ca8-4a91-8437-25db307a48f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.800103 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d00856-5ca8-4a91-8437-25db307a48f7 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.800821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d00856-5ca8-4a91-8437-25db307a48f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:00.801164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d00856-5ca8-4a91-8437-25db307a48f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:00.802616 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d00856-5ca8-4a91-8437-25db307a48f7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:00.815558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 418/808] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:00.817004 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f039-054c-4295-b88d-796a76d295cf None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.817295 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f039-054c-4295-b88d-796a76d295cf None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.817573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f039-054c-4295-b88d-796a76d295cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.841459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-5804f039-054c-4295-b88d-796a76d295cf None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:00.842786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-5804f039-054c-4295-b88d-796a76d295cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:00.903947 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28e4e6-5437-4af1-b734-4f5611fb144b None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:00.930379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 419/809] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:00.930379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804f039-054c-4295-b88d-796a76d295cf None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:00.950665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:00.965658 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:00.966702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:00.967230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:00.967445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:00.968156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:00.968467 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:00.969910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:00.989012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 391/810] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:00.990086 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.061854 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:01.083273 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804f039-054c-4295-b88d-796a76d295cf None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.123771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.124565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.124907 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.125293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.126047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:01.126503 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.128050 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.131201 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f47116-c46f-4ebb-8007-4ee0da3a1495 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.133984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 420/811] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:40:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:01.134944 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.180209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.181188 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.181524 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.181893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.193306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.193306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:01.207744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:01.209009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:01.210299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 392/812] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:40:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:01.216241 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.229321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:01.275109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b09ca8-6499-4cde-8b9e-9bd0d8df71da None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.317752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.318491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.318803 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.319463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] PATH_INFO: `/v3/services/network` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.320163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.321621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:01.333436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:01.334763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:01.335836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 421/813] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:01 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:01.336828 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.371076 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a31e5-33d9-41f2-95cd-b89506e16655 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.392276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:01.422400 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11130416-6256-4236-a962-636184518e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.423639 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11130416-6256-4236-a962-636184518e59 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.424333 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11130416-6256-4236-a962-636184518e59 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.424897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11130416-6256-4236-a962-636184518e59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.437266 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11130416-6256-4236-a962-636184518e59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.439011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11130416-6256-4236-a962-636184518e59 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-11130416-6256-4236-a962-636184518e59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:01.455149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:01.456498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:40:01.458272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:01.462895 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 393/814] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:40:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:01.463969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11130416-6256-4236-a962-636184518e59 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.482569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ed6d3-fa14-4d3d-a1d5-1379ce99ca72 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.530010 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.530659 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.530965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.531372 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.532074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:01.532389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.533810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.537395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:01.548661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 422/815] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:40:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:01.548661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.607063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11130416-6256-4236-a962-636184518e59 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.654827 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.655726 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.656439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.656697 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.657525 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:01.657816 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.659829 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.672423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e333a-5ca4-43f5-b693-a4053c3bf32e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.687273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 394/816] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:40:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:01.687273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.721653 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.722995 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.723524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.723825 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.724722 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] RBAC: Adding query filter params (name=network) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:01.725273 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.726914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.733426 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:01.738806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 423/817] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:01.739637 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.813858 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:01.833837 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907411b-9859-4627-96ba-ab0142b9be1a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.866761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78506f-2e95-413c-90b9-2c5c82b419c4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:01.891514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.892607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.893675 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.894317 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.896083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:01.896476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.899247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.913474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760b28d-a169-4667-97c1-1f8c7f1547ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:01.914134 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760b28d-a169-4667-97c1-1f8c7f1547ed None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:01.914439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760b28d-a169-4667-97c1-1f8c7f1547ed None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:01.914719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760b28d-a169-4667-97c1-1f8c7f1547ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:01.915460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760b28d-a169-4667-97c1-1f8c7f1547ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:01.916864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760b28d-a169-4667-97c1-1f8c7f1547ed None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:01.936890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 395/818] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:40:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:01.936890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:01.958146 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 424/819] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:40:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:01.958146 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760b28d-a169-4667-97c1-1f8c7f1547ed None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:02.088374 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:02.100426 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c91c9-6181-4e59-972f-e74f2f32a64e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:02.119162 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760b28d-a169-4667-97c1-1f8c7f1547ed None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:02.152525 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:02.153956 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:02.159301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.159301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.159863 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:02.182469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:02.186232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:02.189051 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:02.190025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.190358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.190695 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:02.191698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] RBAC: Adding query filter params (type=network) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:02.192141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:02.194672 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:02.202409 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 425/820] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:02.204360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:02.218578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:02.220096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:02.222274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:02.222274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:02.222274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:02.228058 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 396/821] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:02.229593 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:02.359301 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1db21-7f96-4731-b863-4688ce5d9c31 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:02.366247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693d5db-bc19-4dd7-a9aa-699d61f3924e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:02.404190 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58561aa-2a41-4026-826e-6fda339f7af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:02.405174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58561aa-2a41-4026-826e-6fda339f7af3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.405620 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58561aa-2a41-4026-826e-6fda339f7af3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.405934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58561aa-2a41-4026-826e-6fda339f7af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:02.413073 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16f613-1e44-4e26-a246-907c6953da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:02.414359 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16f613-1e44-4e26-a246-907c6953da31 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.414835 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16f613-1e44-4e26-a246-907c6953da31 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.415287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16f613-1e44-4e26-a246-907c6953da31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:02.416202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16f613-1e44-4e26-a246-907c6953da31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:02.417799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16f613-1e44-4e26-a246-907c6953da31 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:02.441102 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:02.448320 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 397/822] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:40:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:02.449652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16f613-1e44-4e26-a246-907c6953da31 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:02.484587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:02.517518 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58561aa-2a41-4026-826e-6fda339f7af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:02.519898 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58561aa-2a41-4026-826e-6fda339f7af3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:02.595565 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16f613-1e44-4e26-a246-907c6953da31 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:02.638343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29bf1-5bf2-4096-b0a6-1117d7bf661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:02.639445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29bf1-5bf2-4096-b0a6-1117d7bf661a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.639993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29bf1-5bf2-4096-b0a6-1117d7bf661a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.641146 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29bf1-5bf2-4096-b0a6-1117d7bf661a None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:02.641892 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29bf1-5bf2-4096-b0a6-1117d7bf661a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:02.642237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29bf1-5bf2-4096-b0a6-1117d7bf661a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:02.644680 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29bf1-5bf2-4096-b0a6-1117d7bf661a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:02.663969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 398/823] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:02.665717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49b51-7fb9-4906-801a-636e12933d11 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.666013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49b51-7fb9-4906-801a-636e12933d11 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.666343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49b51-7fb9-4906-801a-636e12933d11 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:02.669679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 399/824] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:02.670994 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e49b51-7fb9-4906-801a-636e12933d11 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:02.679339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 426/825] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:40:02.680750 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455ca4-e644-4cc8-a609-2d221fdc2790 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.681040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455ca4-e644-4cc8-a609-2d221fdc2790 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.681318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455ca4-e644-4cc8-a609-2d221fdc2790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:02.711316 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-1c455ca4-e644-4cc8-a609-2d221fdc2790 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:02.712956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-1c455ca4-e644-4cc8-a609-2d221fdc2790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:02.768494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:02.806624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 427/826] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:02.807457 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c455ca4-e644-4cc8-a609-2d221fdc2790 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:02.825201 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:02.825697 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e49b51-7fb9-4906-801a-636e12933d11 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:02.872502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:02.873696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.874213 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.874716 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:02.885843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:02.887928 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:02.903481 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:02.904644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 400/827] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:02.911492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:02.946989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c455ca4-e644-4cc8-a609-2d221fdc2790 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:02.992950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:02.993817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:02.994122 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:02.994502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.007231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:03.008798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:03.028924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:03.030509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:03.031861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:03.031861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:03.031861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:03.031861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 428/828] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:03.031861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:03.052691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32535a0-db12-4b75-a898-7c8f5104bd38 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:03.074648 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:03.103546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e72c-4d71-497e-9d1a-1ecd041a632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:03.104401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e72c-4d71-497e-9d1a-1ecd041a632a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:03.104689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e72c-4d71-497e-9d1a-1ecd041a632a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:03.104970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e72c-4d71-497e-9d1a-1ecd041a632a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.115019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:03.119113 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078e72c-4d71-497e-9d1a-1ecd041a632a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-3078e72c-4d71-497e-9d1a-1ecd041a632a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:03.127539 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:03.128764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:03.130012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 401/829] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:40:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:03.130838 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e72c-4d71-497e-9d1a-1ecd041a632a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:03.174893 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38b0d1-3ba7-4136-9e0e-cdefcdcc6d1c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:03.232140 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:03.233548 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:03.234233 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:03.234735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.236097 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:03.236644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:03.239275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:03.256524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 429/830] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:03.257437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:03.295395 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e72c-4d71-497e-9d1a-1ecd041a632a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:03.339719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:03.340541 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:03.340925 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:03.341261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.342332 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:03.342728 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:03.344546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:03.365141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 402/831] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:03.366763 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:03.399349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa7191-74fb-4a00-a155-4143c4f3dae7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:03.433251 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:03.439485 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:03.457292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:03.458370 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:03.458813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:03.459341 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.460558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:03.461067 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:03.463032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:03.477350 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 430/832] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:40:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:03.483693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:03.533780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161990f1-d8d9-458e-8f43-b6fbc460b69d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:03.584880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e3864-28ca-4a7d-9808-3607f1aa2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:03.586466 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e3864-28ca-4a7d-9808-3607f1aa2db0 None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:03.586778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e3864-28ca-4a7d-9808-3607f1aa2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:03.587008 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e3864-28ca-4a7d-9808-3607f1aa2db0 None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/945e08e70ce943708b6eeeb0fd8acc5d/roles/cc9e44cb256e4496ba5fd23548c12437` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.612417 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e3864-28ca-4a7d-9808-3607f1aa2db0 None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=945e08e70ce943708b6eeeb0fd8acc5d, role_id=cc9e44cb256e4496ba5fd23548c12437)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:03.613949 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e3864-28ca-4a7d-9808-3607f1aa2db0 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:03.624417 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74759da2-a53f-43aa-afd3-1b6d46eb4a60 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:03.634563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 403/833] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:40:03 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/945e08e70ce943708b6eeeb0fd8acc5d/roles/cc9e44cb256e4496ba5fd23548c12437 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:40:03.635888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e3864-28ca-4a7d-9808-3607f1aa2db0 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:03.682564 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:03.683386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:03.683621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:03.683902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.698204 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:03.699619 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:03.714711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:03.716080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:03.717342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 431/834] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:03.726332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:03.806015 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:03.817752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:03.833280 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e3864-28ca-4a7d-9808-3607f1aa2db0 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:03.883309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:03.884889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:03.885642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:03.886141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.901234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:03.903860 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:03.917781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d0d7b-1775-47a4-a63a-856ec3fb9fd7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:03.927978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:03.929226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:03.930769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:03.930769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:03.930769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:03.930769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 404/835] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:03.935845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:03.970683 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:03.972105 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:03.973287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:03.974133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:03.975447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:03.975955 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:03.978507 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:03.998801 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 432/836] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:04.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:04.074083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03c78-f83c-42ec-a789-4d13bb7ce68d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:04.115418 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:04.116525 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.117062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.117586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.118565 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:04.118693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d42da-41d7-4a70-b4a6-ebfbc26bea2d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:04.119279 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:04.121329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:04.122657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:04.132631 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:04.140261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 405/837] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:04.147588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:04.170447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9595b-a95b-47ce-afc3-28c65ffd765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:04.171961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9595b-a95b-47ce-afc3-28c65ffd765c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.172610 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9595b-a95b-47ce-afc3-28c65ffd765c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.173188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9595b-a95b-47ce-afc3-28c65ffd765c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.174952 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9595b-a95b-47ce-afc3-28c65ffd765c None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=813e8db425644f688e926aafe3249721, user.id=90103f3776ce4eeb90bde66ed546aa87) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:04.175592 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9595b-a95b-47ce-afc3-28c65ffd765c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:04.177981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9595b-a95b-47ce-afc3-28c65ffd765c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:04.190317 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 433/838] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:04 2022] GET /identity/v3/role_assignments?user.id=90103f3776ce4eeb90bde66ed546aa87&scope.project.id=813e8db425644f688e926aafe3249721&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:04.306659 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30f8c4-d89a-44d1-b1b1-85e309823ec9 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:04.360195 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:04.360982 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.361413 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.361798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.371552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:04.372204 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121988b-6ad9-4202-86fd-b396c973513c None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.372499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121988b-6ad9-4202-86fd-b396c973513c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.372778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121988b-6ad9-4202-86fd-b396c973513c None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.373287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:04.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 434/839] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:04.380618 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377179f3-7f92-4083-a77e-8775381188d8 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.381231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377179f3-7f92-4083-a77e-8775381188d8 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.381718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377179f3-7f92-4083-a77e-8775381188d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:04.390992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:04.392384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:04.393774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 406/840] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:40:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:04.402479 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:04.410848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-377179f3-7f92-4083-a77e-8775381188d8 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:04.412637 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-377179f3-7f92-4083-a77e-8775381188d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:04.484442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:04.538521 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 435/841] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:04.562667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486a343-7994-4815-94f2-d15129441f32 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:04.568872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377179f3-7f92-4083-a77e-8775381188d8 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:04.616986 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:04.617732 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.618013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.618717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.621473 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:04.621790 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:04.623329 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:04.657715 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 407/842] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:40:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:04.666616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:04.726865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:04.735706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377179f3-7f92-4083-a77e-8775381188d8 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:04.791018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:04.792164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.792561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.792864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.793746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:04.795847 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:04.807442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d559c1-2259-4181-8c5c-edf833c84d2a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:04.810378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:04.812040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:04.813469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:04.815416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 436/843] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:04.827782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:04.843587 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:04.874026 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273afc56-ab83-4710-af69-cc080cd8c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:04.875693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273afc56-ab83-4710-af69-cc080cd8c794 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.876462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273afc56-ab83-4710-af69-cc080cd8c794 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.877246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273afc56-ab83-4710-af69-cc080cd8c794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.953901 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ddb13-0aba-4eaf-a812-b73eae84eb34 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:04.960299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273afc56-ab83-4710-af69-cc080cd8c794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:04.962787 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273afc56-ab83-4710-af69-cc080cd8c794 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:04.997197 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:04.997765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:04.998082 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:04.998401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:04.999111 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:04.999430 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:05.001465 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:05.013725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 437/844] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:05.016650 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:05.055753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:05.131953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 408/845] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:40:05.138297 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273afc56-ab83-4710-af69-cc080cd8c794 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:05.142513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79a749-5eff-40b1-af5e-79f01b23af69 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:05.162303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:05.194352 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:05.195780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:05.196224 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:05.196671 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:05.197515 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:05.198362 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:05.200260 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:05.214139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 438/846] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:05.287602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273afc56-ab83-4710-af69-cc080cd8c794 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:05.334646 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:05.336009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:05.336873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:05.337677 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:05.354774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:05.356739 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:05.374344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:05.375575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:05.376686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:05.376686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:05.376686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:05.379343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 409/847] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:05.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:05.512864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:05.528350 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bd2d8-5937-401a-88c8-4c2800eac08f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:05.588869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:05.590534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:05.591315 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:05.592105 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:05.593751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:05.594631 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:05.597558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:05.621099 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 439/848] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:05.628794 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:05.748979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:05.811619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc75c3b-71ae-4dce-89f6-0031038e68ad None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:05.869574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0947692-eb40-4d3c-9385-394a7ffa1343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:05.870396 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947692-eb40-4d3c-9385-394a7ffa1343 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:05.870684 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947692-eb40-4d3c-9385-394a7ffa1343 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:05.870963 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947692-eb40-4d3c-9385-394a7ffa1343 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:05.879187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0947692-eb40-4d3c-9385-394a7ffa1343 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:05.880893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0947692-eb40-4d3c-9385-394a7ffa1343 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a0947692-eb40-4d3c-9385-394a7ffa1343 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:05.896225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:05.897487 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:05.898717 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 410/849] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:05.907805 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:06.030145 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390440da-6381-489f-aed9-fec4f7c7414e None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:06.030634 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390440da-6381-489f-aed9-fec4f7c7414e None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:06.030834 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390440da-6381-489f-aed9-fec4f7c7414e None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:06.032406 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 411/850] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:06.038096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157891b-7f24-45ab-9425-4bf177889d99 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:06.038716 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157891b-7f24-45ab-9425-4bf177889d99 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:06.039344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157891b-7f24-45ab-9425-4bf177889d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:06.042725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382721fd-8f13-4d98-bdb6-e6435948279e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:06.063099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-c157891b-7f24-45ab-9425-4bf177889d99 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:06.064616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-c157891b-7f24-45ab-9425-4bf177889d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:06.083011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:06.084076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:06.084589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:06.085093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:06.086091 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:06.086679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:06.088351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:06.101884 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 440/851] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:06.111761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:06.129808 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:06.153475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 412/852] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:06.183948 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157891b-7f24-45ab-9425-4bf177889d99 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:06.250345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:06.252662 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e033a-7316-4f55-9fe5-415f55f2e4c4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:06.296691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157891b-7f24-45ab-9425-4bf177889d99 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:06.298852 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:06.299981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:06.300648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:06.301327 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:06.302494 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=813e8db425644f688e926aafe3249721, user.id=9b3e9589dc214e90a6ace523306dffc0) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:06.303124 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:06.305023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:06.313116 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 441/853] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:06 2022] GET /identity/v3/role_assignments?user.id=9b3e9589dc214e90a6ace523306dffc0&scope.project.id=813e8db425644f688e926aafe3249721&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:06.351239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b844fe1-c473-44cb-b6de-4db8ecfa345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:06.352177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b844fe1-c473-44cb-b6de-4db8ecfa345e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:06.352613 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b844fe1-c473-44cb-b6de-4db8ecfa345e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:06.352921 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b844fe1-c473-44cb-b6de-4db8ecfa345e None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:06.368668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b844fe1-c473-44cb-b6de-4db8ecfa345e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-6b844fe1-c473-44cb-b6de-4db8ecfa345e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:06.376131 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:06.377630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:40:06.379097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:06.379961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 413/854] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:40:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:06.388304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:06.493090 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:06.520485 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241697-ebdc-4c5b-bc4b-cf3858074a9d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:06.560176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:06.561248 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:06.561790 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:06.562325 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:06.563296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:06.563838 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:06.565443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:06.575429 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 442/855] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:40:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:06.590793 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b844fe1-c473-44cb-b6de-4db8ecfa345e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:06.731507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b844fe1-c473-44cb-b6de-4db8ecfa345e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:06.753063 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:06.773101 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57691d3-eb75-49aa-aaa6-d2c139c19dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:06.773776 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57691d3-eb75-49aa-aaa6-d2c139c19dec None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:06.774059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57691d3-eb75-49aa-aaa6-d2c139c19dec None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:06.775941 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57691d3-eb75-49aa-aaa6-d2c139c19dec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:06.787425 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57691d3-eb75-49aa-aaa6-d2c139c19dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:06.788966 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57691d3-eb75-49aa-aaa6-d2c139c19dec None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-b57691d3-eb75-49aa-aaa6-d2c139c19dec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:06.806310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:06.808114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:06.810626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:06.810626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:06.810626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:06.810626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 414/856] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:06.820715 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:06.964975 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ae7a0-a853-4e29-a957-2c0ae399644f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:07.025453 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:07.025453 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.025453 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.025453 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.026039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:07.026039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:07.026039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:07.028190 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:07.036405 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e50c-f316-4065-b51f-47c6330eaaff None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.036696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e50c-f316-4065-b51f-47c6330eaaff None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.036978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e50c-f316-4065-b51f-47c6330eaaff None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.038738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 415/857] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:07.044550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce346f-5624-44a3-8fb3-91559d66fef7 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.044785 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce346f-5624-44a3-8fb3-91559d66fef7 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.045037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce346f-5624-44a3-8fb3-91559d66fef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.047815 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 443/858] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:07.055484 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:07.071709 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-27ce346f-5624-44a3-8fb3-91559d66fef7 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:07.073323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-27ce346f-5624-44a3-8fb3-91559d66fef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:07.199436 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 416/859] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:07.211958 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508e3bf-e471-4926-b807-02363a731732 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:07.227295 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce346f-5624-44a3-8fb3-91559d66fef7 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:07.258660 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:07.260047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.260674 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.261190 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.271069 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:07.273191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:07.288131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:07.289436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:07.290779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:07.293951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 444/860] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:40:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:07.299948 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:07.370505 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:07.374651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:07.380255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce346f-5624-44a3-8fb3-91559d66fef7 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:07.418995 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027b8f-bb56-428f-ae71-0a229083e0fc None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:07.461956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:07.463083 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.463618 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.464173 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.465359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:07.465959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:07.468043 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:07.497998 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 445/861] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:40:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:07.499469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:07.501652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:07.502391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.502674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.502946 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.503601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:07.506011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:07.529232 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:07.530657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:07.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:07.537482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 417/862] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:40:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:07.539237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:07.626738 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a9b9-6b4c-4ea1-905b-99b5d107bf54 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:07.644783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:07.659178 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2fb91-2072-49d0-9156-9ce83efaf29d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:07.670410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf42902-8538-4fc4-abf1-7c8d68a3475f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:07.671594 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf42902-8538-4fc4-abf1-7c8d68a3475f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.672127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf42902-8538-4fc4-abf1-7c8d68a3475f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.672725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf42902-8538-4fc4-abf1-7c8d68a3475f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.702441 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:07.703772 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.703772 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.703772 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.704317 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:07.704630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:07.706456 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:07.717231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 418/863] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:07.718260 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:07.751215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:07.761419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf42902-8538-4fc4-abf1-7c8d68a3475f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:07.762959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf42902-8538-4fc4-abf1-7c8d68a3475f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:07.835450 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72733339-312a-4dd9-ae52-dcf21c608802 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:07.888223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366210-a5e0-456f-9bb8-d7716053afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:07.888865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366210-a5e0-456f-9bb8-d7716053afcd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.889182 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366210-a5e0-456f-9bb8-d7716053afcd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.889479 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366210-a5e0-456f-9bb8-d7716053afcd None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.890135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366210-a5e0-456f-9bb8-d7716053afcd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:07.890448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366210-a5e0-456f-9bb8-d7716053afcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:07.890934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 446/864] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:40:07.892351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490f5b3-d37a-4699-8217-5236057236f3 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.892675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490f5b3-d37a-4699-8217-5236057236f3 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.892952 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490f5b3-d37a-4699-8217-5236057236f3 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.894495 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 447/865] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:07.896700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366210-a5e0-456f-9bb8-d7716053afcd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:07.899718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490f5b3-d37a-4699-8217-5236057236f3 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:07.912295 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 419/866] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:07.912295 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515511-e2c5-485a-a845-e18c039c05ae None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:07.912295 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515511-e2c5-485a-a845-e18c039c05ae None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:07.912295 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515511-e2c5-485a-a845-e18c039c05ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:07.929106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-59515511-e2c5-485a-a845-e18c039c05ae None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:07.930123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:07.930573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-59515511-e2c5-485a-a845-e18c039c05ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:08.054740 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 420/867] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:08.077958 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59515511-e2c5-485a-a845-e18c039c05ae None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:08.094043 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490f5b3-d37a-4699-8217-5236057236f3 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:08.122988 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:08.147000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:08.148137 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:08.148649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:08.149145 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:08.166247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:08.167975 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:08.196426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:08.197677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:08.198876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:08.198876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:08.198876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:08.198876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 448/868] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:08.206331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:08.260589 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59515511-e2c5-485a-a845-e18c039c05ae None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:08.304580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:08.305338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4689ed-a631-4811-82cb-c7bcf14a651e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:08.306337 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4689ed-a631-4811-82cb-c7bcf14a651e None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:08.306882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4689ed-a631-4811-82cb-c7bcf14a651e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:08.307387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4689ed-a631-4811-82cb-c7bcf14a651e None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:08.308323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4689ed-a631-4811-82cb-c7bcf14a651e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:08.309982 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4689ed-a631-4811-82cb-c7bcf14a651e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:08.332618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 421/869] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:40:08 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:08.359942 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55fd0-18c5-4914-aa29-96de05bee9a0 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:08.429684 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:08.431550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:08.432437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:08.433203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:08.434546 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:08.435298 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:08.437447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:08.459739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 449/870] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:08.467224 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4689ed-a631-4811-82cb-c7bcf14a651e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:08.570980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:08.604079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4689ed-a631-4811-82cb-c7bcf14a651e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:08.643997 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:08.645016 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:08.645455 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:08.645927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:08.656139 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:08.657936 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:08.671044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:08.672264 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:08.673451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 422/871] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:08.679217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:08.804437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06929f7b-ed5a-4912-a375-1b7592159b3c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:08.842510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:08.857844 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaebc3c-e079-4e5e-b29c-1a8b1d3e8276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:08.859046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaebc3c-e079-4e5e-b29c-1a8b1d3e8276 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:08.859556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaebc3c-e079-4e5e-b29c-1a8b1d3e8276 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:08.860141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaebc3c-e079-4e5e-b29c-1a8b1d3e8276 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:08.861327 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaebc3c-e079-4e5e-b29c-1a8b1d3e8276 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:08.861851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaebc3c-e079-4e5e-b29c-1a8b1d3e8276 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:08.863743 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaebc3c-e079-4e5e-b29c-1a8b1d3e8276 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:08.879853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 450/872] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:08.889511 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:08.920453 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6d93a-b1ae-4e68-95cb-14863b55fa0a None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:08.920784 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6d93a-b1ae-4e68-95cb-14863b55fa0a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:08.921153 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6d93a-b1ae-4e68-95cb-14863b55fa0a None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:08.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 451/873] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:08.930354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7acb7a-feb9-40a1-890b-b526a2198a19 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:08.930655 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7acb7a-feb9-40a1-890b-b526a2198a19 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:08.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7acb7a-feb9-40a1-890b-b526a2198a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:08.951601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-7f7acb7a-feb9-40a1-890b-b526a2198a19 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:08.952817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-7f7acb7a-feb9-40a1-890b-b526a2198a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:09.039181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 452/874] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:08 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:09.061862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7acb7a-feb9-40a1-890b-b526a2198a19 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:09.080111 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92402d-8da9-4940-ae12-f980355e6498 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:09.118960 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:09.119897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:09.120505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:09.120973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:09.122003 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=945e08e70ce943708b6eeeb0fd8acc5d) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:09.122569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:09.124366 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:09.132134 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 423/875] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:08 2022] GET /identity/v3/role_assignments?user.id=945e08e70ce943708b6eeeb0fd8acc5d&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:09.218177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7acb7a-feb9-40a1-890b-b526a2198a19 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:09.234911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:09.261050 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:09.261780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:09.262079 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:09.262386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:09.263267 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:09.264797 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:09.275728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:09.277066 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:09.278130 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 453/876] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:09.284099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:09.295425 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:09.434926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add457b-289e-49dd-b0fb-805bfe227b8a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:09.477965 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebfa87-ace4-407e-9c07-aa5918744549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:09.478655 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebfa87-ace4-407e-9c07-aa5918744549 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:09.478942 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebfa87-ace4-407e-9c07-aa5918744549 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:09.479219 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebfa87-ace4-407e-9c07-aa5918744549 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:09.479919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebfa87-ace4-407e-9c07-aa5918744549 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:09.480230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebfa87-ace4-407e-9c07-aa5918744549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:09.481650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebfa87-ace4-407e-9c07-aa5918744549 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:09.487685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 424/877] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:09.494152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:09.627897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad22426-f46c-4df2-8af2-9ab891a468bf None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:09.696113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b16f3-ec06-40f1-a5f3-241cf6f6c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:09.696866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b16f3-ec06-40f1-a5f3-241cf6f6c464 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:09.697163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b16f3-ec06-40f1-a5f3-241cf6f6c464 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:09.697489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b16f3-ec06-40f1-a5f3-241cf6f6c464 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:09.698244 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b16f3-ec06-40f1-a5f3-241cf6f6c464 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:09.698572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b16f3-ec06-40f1-a5f3-241cf6f6c464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:09.700087 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b16f3-ec06-40f1-a5f3-241cf6f6c464 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:09.706129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 454/878] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:09 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:10.280334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedd21c-e2bf-4093-b14f-46a184192ac0 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:10.280939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedd21c-e2bf-4093-b14f-46a184192ac0 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:10.280939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedd21c-e2bf-4093-b14f-46a184192ac0 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:10.282694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 425/879] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:10.288888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a76a5-1788-4555-b740-489e8e6d7588 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:10.289462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a76a5-1788-4555-b740-489e8e6d7588 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:10.289462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a76a5-1788-4555-b740-489e8e6d7588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:10.312305 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-561a76a5-1788-4555-b740-489e8e6d7588 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:10.314397 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-561a76a5-1788-4555-b740-489e8e6d7588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:10.371550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:10.431100 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 455/880] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:10.454155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedd21c-e2bf-4093-b14f-46a184192ac0 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:10.542871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:10.585817 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedd21c-e2bf-4093-b14f-46a184192ac0 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:10.628436 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1896af-6d48-4e1f-9f32-6dc21b6ef9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:10.629153 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1896af-6d48-4e1f-9f32-6dc21b6ef9cf None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:10.629436 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1896af-6d48-4e1f-9f32-6dc21b6ef9cf None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:10.629715 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1896af-6d48-4e1f-9f32-6dc21b6ef9cf None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:10.630475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1896af-6d48-4e1f-9f32-6dc21b6ef9cf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:10.632111 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1896af-6d48-4e1f-9f32-6dc21b6ef9cf None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:10.657317 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 426/881] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:40:10 2022] POST /identity/v3/services => generated 269 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:10.712798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168be721-9b96-490a-9b59-48cc4f627f0e None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:10.713678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168be721-9b96-490a-9b59-48cc4f627f0e None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:10.714351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168be721-9b96-490a-9b59-48cc4f627f0e None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:10.718334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 456/882] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:10.725561 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853b89-edb2-4f9d-a8bd-e5f853ff144d None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:10.726319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853b89-edb2-4f9d-a8bd-e5f853ff144d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:10.726997 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853b89-edb2-4f9d-a8bd-e5f853ff144d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:10.751982 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-d9853b89-edb2-4f9d-a8bd-e5f853ff144d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:10.754398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-d9853b89-edb2-4f9d-a8bd-e5f853ff144d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:10.862170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 427/883] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:10.897403 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168be721-9b96-490a-9b59-48cc4f627f0e None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:11.036486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168be721-9b96-490a-9b59-48cc4f627f0e None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:11.082832 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:11.084050 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.084631 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.085216 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.086289 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:11.088150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:11.099596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:11.101023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:11.102257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:11.110208 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 457/884] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:11.115382 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9853b89-edb2-4f9d-a8bd-e5f853ff144d None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:11.175850 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:11.226233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9853b89-edb2-4f9d-a8bd-e5f853ff144d None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:11.261758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:11.262436 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.262720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.263000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.263696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:11.264002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:11.265427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:11.271312 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 428/885] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:11.277257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:11.320684 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:11.386392 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd7a4-5c03-404a-b216-d59ec16f3028 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:11.423929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791f93-e5b4-4470-88d1-238a764d92f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:11.424584 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791f93-e5b4-4470-88d1-238a764d92f0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.424884 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791f93-e5b4-4470-88d1-238a764d92f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.425180 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791f93-e5b4-4470-88d1-238a764d92f0 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.426030 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15791f93-e5b4-4470-88d1-238a764d92f0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:11.426608 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15791f93-e5b4-4470-88d1-238a764d92f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:11.429139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15791f93-e5b4-4470-88d1-238a764d92f0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:11.436409 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 458/886] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:11.443727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:11.572367 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d960ca-9783-40ea-8e9c-07b97d37e556 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:11.605017 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194b43-613a-4f85-a599-77512c1f00be None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.605464 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194b43-613a-4f85-a599-77512c1f00be None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.605604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:11.605784 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194b43-613a-4f85-a599-77512c1f00be None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.608030 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 459/887] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:11.613744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2135e8-fd35-4d46-865d-4a83a0a42f88 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.613986 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2135e8-fd35-4d46-865d-4a83a0a42f88 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.614396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2135e8-fd35-4d46-865d-4a83a0a42f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.615853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:11.616507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.616793 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.617075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.617857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] RBAC: Adding query filter params (interface=public, service_id=4c3a43552dd741c1b677c6bb0f5601f3, region_id=RegionOne) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:11.618202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:11.619608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:11.625961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 429/888] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:40:11 2022] GET /identity/v3/endpoints?service_id=4c3a43552dd741c1b677c6bb0f5601f3&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:11.632278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:11.633966 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-5a2135e8-fd35-4d46-865d-4a83a0a42f88 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:11.635285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-5a2135e8-fd35-4d46-865d-4a83a0a42f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:11.734249 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 460/889] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:11.756071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2135e8-fd35-4d46-865d-4a83a0a42f88 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:11.780542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c1e96-bb1d-4fab-8ec3-532367d5a77a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:11.797939 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:11.823803 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cab36-022c-4aa2-b2e6-52d1deb2a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:11.824451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cab36-022c-4aa2-b2e6-52d1deb2a96f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.824735 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cab36-022c-4aa2-b2e6-52d1deb2a96f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.825015 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cab36-022c-4aa2-b2e6-52d1deb2a96f None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.825691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cab36-022c-4aa2-b2e6-52d1deb2a96f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:11.827124 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cab36-022c-4aa2-b2e6-52d1deb2a96f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:11.833857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 430/890] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:40:11 2022] GET /identity/v3/services => generated 1268 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:40:11.898034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2135e8-fd35-4d46-865d-4a83a0a42f88 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:11.919406 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e40bc-d59f-4f31-a0aa-f0079ff54769 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.919840 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e40bc-d59f-4f31-a0aa-f0079ff54769 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.920079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e40bc-d59f-4f31-a0aa-f0079ff54769 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.921856 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 431/891] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:11.927976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce601b5-1b87-4b8b-9114-5a3a386c6e3a None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.928365 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce601b5-1b87-4b8b-9114-5a3a386c6e3a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.928630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce601b5-1b87-4b8b-9114-5a3a386c6e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.949310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-0ce601b5-1b87-4b8b-9114-5a3a386c6e3a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:11.949749 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31610b6c-f4a4-43e0-971e-c755d1c4f481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:11.950595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-0ce601b5-1b87-4b8b-9114-5a3a386c6e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:11.951058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31610b6c-f4a4-43e0-971e-c755d1c4f481 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:11.951568 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31610b6c-f4a4-43e0-971e-c755d1c4f481 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:11.952070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31610b6c-f4a4-43e0-971e-c755d1c4f481 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:11.966025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31610b6c-f4a4-43e0-971e-c755d1c4f481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-31610b6c-f4a4-43e0-971e-c755d1c4f481 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:11.972285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:11.973552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:11.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:11.977917 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 461/892] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:40:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:11.985306 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31610b6c-f4a4-43e0-971e-c755d1c4f481 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:12.018079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:12.050135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 432/893] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:12.068315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce601b5-1b87-4b8b-9114-5a3a386c6e3a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:12.122238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:12.138654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31610b6c-f4a4-43e0-971e-c755d1c4f481 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:12.181500 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:12.182226 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:12.182517 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:12.182906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:12.183835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:12.184149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:12.185625 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:12.196732 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 462/894] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:40:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:12.202221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:12.205647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce601b5-1b87-4b8b-9114-5a3a386c6e3a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:12.245526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f8d8d-60bc-4a5e-997e-ea5c11c1d81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:12.246203 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f8d8d-60bc-4a5e-997e-ea5c11c1d81d None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:12.246493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f8d8d-60bc-4a5e-997e-ea5c11c1d81d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:12.246774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f8d8d-60bc-4a5e-997e-ea5c11c1d81d None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:12.247498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f8d8d-60bc-4a5e-997e-ea5c11c1d81d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:12.248933 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f8d8d-60bc-4a5e-997e-ea5c11c1d81d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:12.266469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 433/895] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:40:12 2022] POST /identity/v3/services => generated 241 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:40:12.339128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f74902-9c00-42dd-95cf-7736a969651d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:12.393490 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:12.394278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:12.394709 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:12.395022 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:12.410465 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:12.412060 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:12.421093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:12.433368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:12.435232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:12.436333 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:12.436333 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:12.436333 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:12.436333 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 463/896] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:12.443604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f8d8d-60bc-4a5e-997e-ea5c11c1d81d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:12.491403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:12.566568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f8d8d-60bc-4a5e-997e-ea5c11c1d81d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:12.609689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:12.610397 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:12.610697 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:12.610972 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:12.611788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:12.612099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:12.613814 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:12.629781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 434/897] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:12.635892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:12.778035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2dd29-d08c-4b86-ad0b-05a390bc560f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:12.830814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76690f8-b83f-4546-8ebc-0527b452b8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:12.831506 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76690f8-b83f-4546-8ebc-0527b452b8a1 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:12.831811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76690f8-b83f-4546-8ebc-0527b452b8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:12.832240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76690f8-b83f-4546-8ebc-0527b452b8a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:12.841387 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76690f8-b83f-4546-8ebc-0527b452b8a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:12.842950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76690f8-b83f-4546-8ebc-0527b452b8a1 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-a76690f8-b83f-4546-8ebc-0527b452b8a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:12.856534 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:12.857903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:12.859247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 464/898] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:40:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:12.864389 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:12.979037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512999b-0082-4542-92da-097e1dca24dc None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:12.982322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:13.022375 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:13.023079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.023482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.023907 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.024905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:13.025223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:13.027282 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:13.039502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5129a2-eec3-4150-9ccd-7818667ce116 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.039843 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5129a2-eec3-4150-9ccd-7818667ce116 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.040015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5129a2-eec3-4150-9ccd-7818667ce116 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.042934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 465/899] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:13.047626 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d68bfb-a8e9-4197-98d4-e5202baeb4bc None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.047920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d68bfb-a8e9-4197-98d4-e5202baeb4bc None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.048188 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d68bfb-a8e9-4197-98d4-e5202baeb4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.057048 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 435/900] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:40:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:13.063395 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:13.076412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-22d68bfb-a8e9-4197-98d4-e5202baeb4bc None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:13.078684 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-22d68bfb-a8e9-4197-98d4-e5202baeb4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:13.149262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:13.175192 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 466/901] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:13.195946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d68bfb-a8e9-4197-98d4-e5202baeb4bc None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:13.243629 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ac810-a52b-4702-8c2b-44fcfeab5fb9 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:13.307968 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6c129-c463-4cd3-a591-db5d200c16d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:13.309833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c129-c463-4cd3-a591-db5d200c16d3 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.310827 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c129-c463-4cd3-a591-db5d200c16d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.311842 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c129-c463-4cd3-a591-db5d200c16d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.314981 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d68bfb-a8e9-4197-98d4-e5202baeb4bc None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:13.353313 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:13.354496 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.354496 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.354886 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.355584 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:13.356715 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:13.366875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:13.369789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:13.375560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 467/902] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:40:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:13.380043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:13.380378 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:13.436936 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:13.476851 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6c129-c463-4cd3-a591-db5d200c16d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:13.480815 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6c129-c463-4cd3-a591-db5d200c16d3 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:13.491788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0d141-c453-4702-b86a-87a372271557 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:13.532841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:13.533734 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.533947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.534142 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.534852 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:13.535284 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:13.536806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:13.542797 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 468/903] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:13.550530 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:13.641093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 436/904] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:40:13.642762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6c129-c463-4cd3-a591-db5d200c16d3 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:13.681277 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e69c5-0617-4835-b291-cac115af67a5 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:13.703699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:13.710563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:13.730717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:13.731899 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.732411 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.732927 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.733916 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:13.734454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:13.736343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:13.743225 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 469/905] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:13.752560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:13.768357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6c129-c463-4cd3-a591-db5d200c16d3 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:13.806623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:13.807500 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.807656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.807853 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.819809 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:13.821324 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:13.838120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:13.839641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:13.841039 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:13.841039 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:13.841039 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:13.844449 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 437/906] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:13.845925 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:13.889248 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd993dad-5810-4fa9-968b-9073d5f7cf30 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:13.932640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:13.933601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:13.934096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:13.934752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:13.935722 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] RBAC: Adding query filter params (interface=public, service_id=4267a4ce9db7417e838c6e32617af129, region_id=RegionOne) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:13.936355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:13.938171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:13.949032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 470/907] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:40:13 2022] GET /identity/v3/endpoints?service_id=4267a4ce9db7417e838c6e32617af129&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:13.958712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:13.965545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387264c-b055-4d95-99e2-dcabf46ed134 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:13.976163 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:14.005119 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:14.005772 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.006071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.006440 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.007271 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:14.007572 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:14.009024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:14.028620 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 438/908] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:14.029971 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:14.034228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:14.103385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9181b4-43e8-40a1-912a-7aa9c02e3f3f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:14.141733 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03949d4-2d36-42d5-afa6-50439e07217f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:14.160536 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a3fae-b384-462e-b6f7-ba128354b724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:14.161876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a3fae-b384-462e-b6f7-ba128354b724 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.162434 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a3fae-b384-462e-b6f7-ba128354b724 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.162954 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a3fae-b384-462e-b6f7-ba128354b724 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.163810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a3fae-b384-462e-b6f7-ba128354b724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:14.165459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a3fae-b384-462e-b6f7-ba128354b724 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:14.173966 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 471/909] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:40:13 2022] GET /identity/v3/services => generated 1497 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:40:14.190022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:14.191056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.191777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.192157 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921be40-91bf-4e75-88cf-4a0be63abc44 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.192563 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.192891 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921be40-91bf-4e75-88cf-4a0be63abc44 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.193502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921be40-91bf-4e75-88cf-4a0be63abc44 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.197863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 472/910] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:14.203826 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcd29f-ee5c-4a5e-be9d-af4a7d9eaf9c None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.205347 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcd29f-ee5c-4a5e-be9d-af4a7d9eaf9c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.205347 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcd29f-ee5c-4a5e-be9d-af4a7d9eaf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.210542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:14.214043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:14.235560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-75dcd29f-ee5c-4a5e-be9d-af4a7d9eaf9c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:14.236270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:14.240192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:14.240192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:14.240192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:14.240192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:14.240192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:14.240192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:14.240192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:14.240192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:14.240935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 439/911] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:14.243147 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-75dcd29f-ee5c-4a5e-be9d-af4a7d9eaf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:14.245877 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:14.279552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:14.358318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 473/912] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:14.379802 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcd29f-ee5c-4a5e-be9d-af4a7d9eaf9c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:14.399213 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b9ea9-ba41-49f3-be7b-80d60c3e164c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:14.439217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:14.460886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:14.461679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.461970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.462402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.463577 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:14.464094 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:14.466178 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:14.480758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 440/913] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:14.488339 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:14.515134 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcd29f-ee5c-4a5e-be9d-af4a7d9eaf9c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:14.556375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c76a8-80f6-4451-b5ce-6a7c4ce82f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:14.557278 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c76a8-80f6-4451-b5ce-6a7c4ce82f03 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.557685 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c76a8-80f6-4451-b5ce-6a7c4ce82f03 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.557965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c76a8-80f6-4451-b5ce-6a7c4ce82f03 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.558682 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c76a8-80f6-4451-b5ce-6a7c4ce82f03 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:14.560358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c76a8-80f6-4451-b5ce-6a7c4ce82f03 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-eb0c76a8-80f6-4451-b5ce-6a7c4ce82f03 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:14.571566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:14.572951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 474/914] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:14.575706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4384-e4a9-4bcb-9b66-dc00a31cc86b None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.575706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4384-e4a9-4bcb-9b66-dc00a31cc86b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.575706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4384-e4a9-4bcb-9b66-dc00a31cc86b None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.576864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 475/915] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:14.582169 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca4384-e4a9-4bcb-9b66-dc00a31cc86b None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:14.624852 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:14.646183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd6564-aa31-4268-a40b-286fd4999a7a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:14.697443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4da9f-625e-4e98-a18e-efe88499330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:14.698202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4da9f-625e-4e98-a18e-efe88499330d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.698499 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4da9f-625e-4e98-a18e-efe88499330d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.698781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4da9f-625e-4e98-a18e-efe88499330d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.699729 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4da9f-625e-4e98-a18e-efe88499330d None admin] RBAC: Adding query filter params (role.id=4b13ca2dcb004c4891a205a6f8d64042, scope.project.id=813e8db425644f688e926aafe3249721, user.id=945e08e70ce943708b6eeeb0fd8acc5d) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:14.700044 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4da9f-625e-4e98-a18e-efe88499330d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:14.701573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4da9f-625e-4e98-a18e-efe88499330d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:14.708898 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 441/916] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:14 2022] GET /identity/v3/role_assignments?user.id=945e08e70ce943708b6eeeb0fd8acc5d&scope.project.id=813e8db425644f688e926aafe3249721&role.id=4b13ca2dcb004c4891a205a6f8d64042 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:14.711118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ab360-a5de-46d9-ad3a-7eab14e202fa None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.711354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ab360-a5de-46d9-ad3a-7eab14e202fa None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.711633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ab360-a5de-46d9-ad3a-7eab14e202fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.739945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-f93ab360-a5de-46d9-ad3a-7eab14e202fa None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:14.741744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-f93ab360-a5de-46d9-ad3a-7eab14e202fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:14.752208 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca4384-e4a9-4bcb-9b66-dc00a31cc86b None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:14.755854 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:14.825151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:14.826753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:14.827506 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:14.828656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:14.830473 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:14.831411 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:14.834746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:14.846144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 442/917] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:14.854164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 476/918] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:14.861862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ab360-a5de-46d9-ad3a-7eab14e202fa None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:14.880143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:14.970875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:14.994013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ab360-a5de-46d9-ad3a-7eab14e202fa None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:15.049868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4c94-61da-4a3d-8d41-8a69e9495880 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:15.050197 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:15.051432 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:15.052069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:15.052687 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:15.053848 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:15.054531 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:15.057186 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:15.067935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 443/919] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:40:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:15.073950 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:15.120111 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:15.121184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:15.121639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:15.122146 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:15.123016 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:15.125315 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:15.139119 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:15.140673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:15.142042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:15.145912 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 477/920] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:15.152388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:15.208110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:15.225109 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ba7a1-d6a7-4fad-b9d9-c50bc030eca9 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:15.266623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c463ec6-50d2-4d6e-b045-889eb40151bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:15.267444 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c463ec6-50d2-4d6e-b045-889eb40151bb None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:15.267838 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c463ec6-50d2-4d6e-b045-889eb40151bb None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:15.268226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c463ec6-50d2-4d6e-b045-889eb40151bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:15.269201 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c463ec6-50d2-4d6e-b045-889eb40151bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:15.270971 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c463ec6-50d2-4d6e-b045-889eb40151bb None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:15.302419 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 444/921] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:40:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:15.315887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cfad0-a7b3-4a44-ae26-eef031049559 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:15.367281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:15.368761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:15.369235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:15.369704 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:15.370710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:15.371212 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:15.372906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:15.379858 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 478/922] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:15.386213 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c463ec6-50d2-4d6e-b045-889eb40151bb None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:15.389695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:15.528185 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c463ec6-50d2-4d6e-b045-889eb40151bb None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:15.572604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:15.573256 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:15.573543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:15.573822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:15.574555 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:15.574864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:15.576279 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:15.582617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 445/923] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:15.591972 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:15.718618 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:15.730481 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8274b3-b75f-454b-ad17-eeba902339be None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:15.775277 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdca3d-eb5e-411c-99bb-f00770954ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:15.776358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdca3d-eb5e-411c-99bb-f00770954ca8 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:15.776839 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdca3d-eb5e-411c-99bb-f00770954ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:15.777340 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdca3d-eb5e-411c-99bb-f00770954ca8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:15.778841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdca3d-eb5e-411c-99bb-f00770954ca8 None admin] RBAC: Adding query filter params (interface=public, service_id=771522e6f9214b0e8ea22c2368744b3b, region_id=RegionOne) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:15.779299 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdca3d-eb5e-411c-99bb-f00770954ca8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:15.780963 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdca3d-eb5e-411c-99bb-f00770954ca8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:15.789852 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 479/924] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:40:15 2022] GET /identity/v3/endpoints?service_id=771522e6f9214b0e8ea22c2368744b3b&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:15.796079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:15.896552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:15.937436 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdbba5-9732-45ca-866d-a980b9082aff None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:15.979758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0e5c0-3d94-4cca-9aa1-637f6da9a5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:15.980461 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e5c0-3d94-4cca-9aa1-637f6da9a5a4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:15.980769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e5c0-3d94-4cca-9aa1-637f6da9a5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:15.981058 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e5c0-3d94-4cca-9aa1-637f6da9a5a4 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:15.981775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0e5c0-3d94-4cca-9aa1-637f6da9a5a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:15.983348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0e5c0-3d94-4cca-9aa1-637f6da9a5a4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:15.990991 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 446/925] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:40:15 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:40:16.475865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f56ec9-5456-41ba-a9e2-8b0d8778e9aa None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:16.476447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f56ec9-5456-41ba-a9e2-8b0d8778e9aa None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:16.476723 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f56ec9-5456-41ba-a9e2-8b0d8778e9aa None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:16.478596 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 480/926] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:16.484575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ccd5b-c6b1-44f6-9eb6-a1c79e3292c1 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:16.485030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ccd5b-c6b1-44f6-9eb6-a1c79e3292c1 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:16.485423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ccd5b-c6b1-44f6-9eb6-a1c79e3292c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:16.506857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-d35ccd5b-c6b1-44f6-9eb6-a1c79e3292c1 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:16.508519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-d35ccd5b-c6b1-44f6-9eb6-a1c79e3292c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:16.590329 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 447/927] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:16.613290 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f56ec9-5456-41ba-a9e2-8b0d8778e9aa None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:16.744335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f56ec9-5456-41ba-a9e2-8b0d8778e9aa None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:16.786309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:16.787279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:16.787766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:16.788200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:16.789018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:16.790622 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:16.800914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:16.802678 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:40:16.804019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:16.806983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 481/928] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:40:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:16.815284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ccd5b-c6b1-44f6-9eb6-a1c79e3292c1 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:16.896319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:16.942736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ccd5b-c6b1-44f6-9eb6-a1c79e3292c1 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:16.986755 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa6edc-5dbd-459d-8c1c-c81437a055c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:16.987899 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa6edc-5dbd-459d-8c1c-c81437a055c7 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:16.988477 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa6edc-5dbd-459d-8c1c-c81437a055c7 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:16.988993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa6edc-5dbd-459d-8c1c-c81437a055c7 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:16.989945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa6edc-5dbd-459d-8c1c-c81437a055c7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:16.990502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa6edc-5dbd-459d-8c1c-c81437a055c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:16.992151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa6edc-5dbd-459d-8c1c-c81437a055c7 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:16.998080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 448/929] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:17.007531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:17.008889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230ca17-2648-4d02-93aa-ba26a9e5e86f None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.009184 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230ca17-2648-4d02-93aa-ba26a9e5e86f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.009478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230ca17-2648-4d02-93aa-ba26a9e5e86f None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.011233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 449/930] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:17.016719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed021e8-c178-4d0a-810f-7bac61efce12 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.017032 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed021e8-c178-4d0a-810f-7bac61efce12 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.017335 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed021e8-c178-4d0a-810f-7bac61efce12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.035156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:17.039215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-5ed021e8-c178-4d0a-810f-7bac61efce12 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:17.040707 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-5ed021e8-c178-4d0a-810f-7bac61efce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:17.130437 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 450/931] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:17.147799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed021e8-c178-4d0a-810f-7bac61efce12 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:17.169332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052c158-1300-4a9c-8dec-8bd5e5c1f15e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:17.215372 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:17.216200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.216489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.216770 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.217525 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:17.217831 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:17.219555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:17.226934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 482/932] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:17.236299 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:17.251814 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:17.306088 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed021e8-c178-4d0a-810f-7bac61efce12 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:17.344920 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467cffb-4008-4ef8-80e1-621078939f39 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:17.348124 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:17.348512 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5e5e-ebf6-4ac2-9f10-26da7f561df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:17.349684 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5e5e-ebf6-4ac2-9f10-26da7f561df9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.350644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5e5e-ebf6-4ac2-9f10-26da7f561df9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5e5e-ebf6-4ac2-9f10-26da7f561df9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.368609 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de5e5e-ebf6-4ac2-9f10-26da7f561df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-41de5e5e-ebf6-4ac2-9f10-26da7f561df9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:17.374468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:17.376143 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:17.377665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:17.380974 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 451/933] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:40:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:17.382638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5e5e-ebf6-4ac2-9f10-26da7f561df9 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:17.401422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd51aff-1743-4dbd-bb21-5848ef094694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:17.402727 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd51aff-1743-4dbd-bb21-5848ef094694 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.403294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd51aff-1743-4dbd-bb21-5848ef094694 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.403897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd51aff-1743-4dbd-bb21-5848ef094694 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.404892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd51aff-1743-4dbd-bb21-5848ef094694 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:17.406748 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd51aff-1743-4dbd-bb21-5848ef094694 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:17.440093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 483/934] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:40:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:17.520444 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5e5e-ebf6-4ac2-9f10-26da7f561df9 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:17.541079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:17.590170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d766654-5adb-4ccd-a3c6-f7f0a68c15b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:17.590170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d766654-5adb-4ccd-a3c6-f7f0a68c15b4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.590170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d766654-5adb-4ccd-a3c6-f7f0a68c15b4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.590170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d766654-5adb-4ccd-a3c6-f7f0a68c15b4 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.590736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d766654-5adb-4ccd-a3c6-f7f0a68c15b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:17.590736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d766654-5adb-4ccd-a3c6-f7f0a68c15b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:17.590736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d766654-5adb-4ccd-a3c6-f7f0a68c15b4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:17.600247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 452/935] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:40:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:17.607200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd51aff-1743-4dbd-bb21-5848ef094694 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:17.719769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1fb70-2005-4bea-839d-a8c5f8d63c74 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.720150 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1fb70-2005-4bea-839d-a8c5f8d63c74 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.720387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1fb70-2005-4bea-839d-a8c5f8d63c74 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.722152 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 453/936] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:17.727779 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbd955-e093-4128-a7b7-c32a369ca3ea None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.728135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbd955-e093-4128-a7b7-c32a369ca3ea None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.728340 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbd955-e093-4128-a7b7-c32a369ca3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.743449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd51aff-1743-4dbd-bb21-5848ef094694 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:17.749138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-83fbd955-e093-4128-a7b7-c32a369ca3ea None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:17.750399 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-83fbd955-e093-4128-a7b7-c32a369ca3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:17.784282 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:17.785685 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:17.786275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:17.786835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:17.795788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:17.801436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:17.804039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:17.821862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:17.823150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:17.824279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:17.824279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:17.824279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:17.827573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 484/937] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:17.833927 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:17.841860 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 454/938] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:17.857927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbd955-e093-4128-a7b7-c32a369ca3ea None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:17.972412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acb730-db67-410e-aab7-d5b9ab5c35fb None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.006030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbd955-e093-4128-a7b7-c32a369ca3ea None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.008344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:18.022752 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.023413 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.023696 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.023972 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.024693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:18.025001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.026440 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.040390 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 485/939] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:18.045854 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:18.055378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.056040 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.056330 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.056626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.057475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.059124 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.062310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: WARNING keystone.server.flask.common [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 08:40:18.083523 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.sql.core [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:40:18.083523 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:40:18.083523 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [parameters: {'id': '28d666cf8fbc46adbe304920553646f7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:40:18.083523 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277713) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 28d666cf8fbc46adbe304920553646f7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 28d666cf8fbc46adbe304920553646f7. Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:40:18.103457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result.read() Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 08:40:18.104583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application yield resource Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise self.value Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application session.commit() Jul 28 08:40:18.105811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:40:18.107030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:40:18.108175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result.read() Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:40:18.109247 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [parameters: {'id': '28d666cf8fbc46adbe304920553646f7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 08:40:18.116778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:18.117889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 28d666cf8fbc46adbe304920553646f7. Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.119084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 455/940] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:40:17 2022] POST /identity/v3/projects => generated 262 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:18.119928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:18.120032 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:18.180353 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414104c7-f4e4-49d1-94cb-c3020faa4644 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.226168 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a94b1-6b75-498d-86ed-35af85e9cafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.226998 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a94b1-6b75-498d-86ed-35af85e9cafd None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.227268 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a94b1-6b75-498d-86ed-35af85e9cafd None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.227547 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a94b1-6b75-498d-86ed-35af85e9cafd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.236785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a94b1-6b75-498d-86ed-35af85e9cafd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.238386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a94b1-6b75-498d-86ed-35af85e9cafd None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-185a94b1-6b75-498d-86ed-35af85e9cafd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:18.252640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:18.253971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 486/941] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82f570-5456-4d47-8768-9c6ddde86b3d None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.260466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82f570-5456-4d47-8768-9c6ddde86b3d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.261395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82f570-5456-4d47-8768-9c6ddde86b3d None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.261395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 487/942] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:18.262932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82f570-5456-4d47-8768-9c6ddde86b3d None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:18.266350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe8e0d-911b-4cdb-8a50-585eb1fa6ce1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.308284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d30830-3a75-4a8e-8ab3-5e19449d58a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.308888 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d30830-3a75-4a8e-8ab3-5e19449d58a0 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.309176 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d30830-3a75-4a8e-8ab3-5e19449d58a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.309455 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d30830-3a75-4a8e-8ab3-5e19449d58a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.317643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d30830-3a75-4a8e-8ab3-5e19449d58a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.319834 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d30830-3a75-4a8e-8ab3-5e19449d58a0 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-94d30830-3a75-4a8e-8ab3-5e19449d58a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:18.332882 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:18.334401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.335545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 456/943] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:18.336204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be7b1-fdb2-405d-a7bb-a4a408a84e18 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.336204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be7b1-fdb2-405d-a7bb-a4a408a84e18 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.336204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be7b1-fdb2-405d-a7bb-a4a408a84e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.358706 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-be3be7b1-fdb2-405d-a7bb-a4a408a84e18 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:18.360386 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-be3be7b1-fdb2-405d-a7bb-a4a408a84e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:18.377663 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:18.410251 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82f570-5456-4d47-8768-9c6ddde86b3d None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.426344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:18.470141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 457/944] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:18.470141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3be7b1-fdb2-405d-a7bb-a4a408a84e18 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:18.475808 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.477232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.477624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.478293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.479190 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:18.479511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.481019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.502177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 488/945] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:18.503466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:18.594534 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3be7b1-fdb2-405d-a7bb-a4a408a84e18 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.633345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3fc4c-32ad-42e0-9189-da1c78b65cc5 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.646057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.646738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.647022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.647301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.648053 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] RBAC: Adding query filter params (name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:18.648362 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.649860 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.674637 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 458/946] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:18.677639 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:18.706494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:18.706852 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.708575 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.709054 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.709581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.710458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.712723 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:18.730228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:18.731589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:18.732656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 489/947] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:18.746178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:18.798559 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:18.859127 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbf4e9-75a1-48ca-b8be-062007cbb532 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.906387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.907057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.907345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.907623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.918935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.920475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.922166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d7cd2-7baf-472e-944e-1dd4df3aaee9 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:18.937796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:18.939100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:18.940204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:18.940204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:18.940204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:18.940204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 459/948] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:18.945583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:18.982464 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:18.983599 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:18.984149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:18.984589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:18.985638 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:18.986160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:18.988004 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:18.993603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 490/949] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:19.001866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:19.015630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:19.063311 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdfc32-299e-43cb-ad0c-a9d25f28248d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:19.112421 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:19.113304 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:19.114084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:19.114252 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:19.115343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:19.115775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:19.117437 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:19.132642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 460/950] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:19.139079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:19.143824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:19.152964 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ebb1-c0d9-400e-bda0-dd2a62a06c41 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:19.193639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:19.194741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:19.195022 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:19.195296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:19.196057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:19.196363 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:19.197859 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:19.203860 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 491/951] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:19.209764 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:19.269993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365e28b-8b9e-489a-912f-4f245fa7ac6f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:19.296873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:19.322298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:19.322986 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:19.323278 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:19.323913 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:19.332129 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:19.333384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:19.346545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:19.347770 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:19.348885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:19.348885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:19.348885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:19.348885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:19.348885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:19.348885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:19.348885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:19.348885 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:19.350200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a6805-c988-4325-bb59-4aa901f4ab05 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:19.352878 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 461/952] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:19.354526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:19.391997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7ce72-6c48-4fae-950e-79355fc570d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:19.392654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7ce72-6c48-4fae-950e-79355fc570d3 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:19.393013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7ce72-6c48-4fae-950e-79355fc570d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:19.393297 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7ce72-6c48-4fae-950e-79355fc570d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:19.394132 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7ce72-6c48-4fae-950e-79355fc570d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:19.395778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7ce72-6c48-4fae-950e-79355fc570d3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:19.421303 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 492/953] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:40:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:40:19.512372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c334061-e50f-425d-9b5b-f599f5b04115 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:19.562703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d9efd-da2e-4a56-a097-dfd2c611603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:19.563798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d9efd-da2e-4a56-a097-dfd2c611603e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:19.564432 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d9efd-da2e-4a56-a097-dfd2c611603e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:19.564983 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d9efd-da2e-4a56-a097-dfd2c611603e None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:19.566198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d9efd-da2e-4a56-a097-dfd2c611603e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:19.566954 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d9efd-da2e-4a56-a097-dfd2c611603e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:19.569469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d9efd-da2e-4a56-a097-dfd2c611603e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:19.586692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 462/954] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:19.595416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7ce72-6c48-4fae-950e-79355fc570d3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:19.616796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:19.679788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fdfc8-a472-4244-bf72-7f2807e4fba3 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:19.680016 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fdfc8-a472-4244-bf72-7f2807e4fba3 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:19.680318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fdfc8-a472-4244-bf72-7f2807e4fba3 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:19.682028 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 463/955] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:19.688082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477abe5-dd4a-4179-a447-52cf2fea40b5 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:19.690335 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477abe5-dd4a-4179-a447-52cf2fea40b5 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:19.690790 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477abe5-dd4a-4179-a447-52cf2fea40b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:19.716821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7ce72-6c48-4fae-950e-79355fc570d3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:19.717891 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-a477abe5-dd4a-4179-a447-52cf2fea40b5 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:19.719524 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-a477abe5-dd4a-4179-a447-52cf2fea40b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:19.758994 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e126107-0bef-4e87-a632-4eca4f2dc88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:19.760109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126107-0bef-4e87-a632-4eca4f2dc88f None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:19.760109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126107-0bef-4e87-a632-4eca4f2dc88f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:19.760432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126107-0bef-4e87-a632-4eca4f2dc88f None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/945e08e70ce943708b6eeeb0fd8acc5d/roles/4b13ca2dcb004c4891a205a6f8d64042` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:19.760732 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:19.763693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e126107-0bef-4e87-a632-4eca4f2dc88f None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=945e08e70ce943708b6eeeb0fd8acc5d, role_id=4b13ca2dcb004c4891a205a6f8d64042)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:19.765921 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e126107-0bef-4e87-a632-4eca4f2dc88f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:19.783665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 493/956] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:40:19 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/945e08e70ce943708b6eeeb0fd8acc5d/roles/4b13ca2dcb004c4891a205a6f8d64042 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:40:19.808103 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 464/957] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:19.833388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e126107-0bef-4e87-a632-4eca4f2dc88f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:19.940872 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:19.973460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e126107-0bef-4e87-a632-4eca4f2dc88f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:20.015526 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:20.016207 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:20.016501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:20.016771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] PATH_INFO: `/v3/services/compute` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:20.017421 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:20.018853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:20.028997 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:20.030343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:20.031423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 494/958] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:20.040254 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477abe5-dd4a-4179-a447-52cf2fea40b5 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:20.160105 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477abe5-dd4a-4179-a447-52cf2fea40b5 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:20.198156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c44ee-3c2a-44a9-91c6-a8e5629440d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:20.199133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c44ee-3c2a-44a9-91c6-a8e5629440d7 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:20.199612 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c44ee-3c2a-44a9-91c6-a8e5629440d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:20.200079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c44ee-3c2a-44a9-91c6-a8e5629440d7 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:20.201003 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c44ee-3c2a-44a9-91c6-a8e5629440d7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:20.201512 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c44ee-3c2a-44a9-91c6-a8e5629440d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:20.203226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c44ee-3c2a-44a9-91c6-a8e5629440d7 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:20.209740 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 465/959] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:20.219274 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:20.331240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d188d-c7dd-4269-96b9-87dfa8510c2d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:20.368577 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824d67b-a4cf-4d04-84dd-1637bac65505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:20.369573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d67b-a4cf-4d04-84dd-1637bac65505 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:20.370086 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d67b-a4cf-4d04-84dd-1637bac65505 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:20.370587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d67b-a4cf-4d04-84dd-1637bac65505 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:20.371509 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824d67b-a4cf-4d04-84dd-1637bac65505 None admin] RBAC: Adding query filter params (type=compute) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:20.372014 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824d67b-a4cf-4d04-84dd-1637bac65505 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:20.373614 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824d67b-a4cf-4d04-84dd-1637bac65505 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:20.379967 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 495/960] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:20.994218 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1f9d-a606-44b8-99e0-5c095acc083e None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:20.994442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1f9d-a606-44b8-99e0-5c095acc083e None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:20.994711 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1f9d-a606-44b8-99e0-5c095acc083e None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:20.996595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 466/961] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:21.002635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017dc93-66a6-491d-9279-04e1a64c1d26 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:21.003228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017dc93-66a6-491d-9279-04e1a64c1d26 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:21.003428 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017dc93-66a6-491d-9279-04e1a64c1d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:21.012545 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:21.024840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-8017dc93-66a6-491d-9279-04e1a64c1d26 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:21.026099 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-8017dc93-66a6-491d-9279-04e1a64c1d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:21.105762 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 496/962] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:21.130962 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a1f9d-a606-44b8-99e0-5c095acc083e None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:21.165863 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:21.245345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a1f9d-a606-44b8-99e0-5c095acc083e None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:21.281451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:21.282099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:21.282415 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:21.282701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:21.283404 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:21.284945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:21.301708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.sql.core [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:40:21.301708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:40:21.301708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [parameters: {'user_id': '3c0516c874ab4e40b372cc39a303c834', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:40:21.301708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277713) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result.read() Jul 28 08:40:21.309083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:40:21.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application yield resource Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise self.value Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application session.commit() Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:40:21.311884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:40:21.313323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:40:21.315132 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application result.read() Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:40:21.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [parameters: {'user_id': '3c0516c874ab4e40b372cc39a303c834', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:21.324412 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 08:40:21.332585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 08:40:21.335000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:40:21.335000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:40:21.335000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:40:21.335000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:21.335000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 467/963] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:40:21 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:21.335705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017dc93-66a6-491d-9279-04e1a64c1d26 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:21.440011 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017dc93-66a6-491d-9279-04e1a64c1d26 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:21.477891 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b4696-89eb-44ee-bace-bf832dc92d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:21.478991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b4696-89eb-44ee-bace-bf832dc92d24 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:21.479529 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b4696-89eb-44ee-bace-bf832dc92d24 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:21.480021 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b4696-89eb-44ee-bace-bf832dc92d24 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:21.482438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:21.489379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b4696-89eb-44ee-bace-bf832dc92d24 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:21.491459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b4696-89eb-44ee-bace-bf832dc92d24 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-6e1b4696-89eb-44ee-bace-bf832dc92d24 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:21.508601 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:21.509841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:21.511093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 497/964] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:40:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:21.516274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:21.563292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79564-a5a0-492e-9004-2b18f6b31516 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:21.563660 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79564-a5a0-492e-9004-2b18f6b31516 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:21.563850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79564-a5a0-492e-9004-2b18f6b31516 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:21.565497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 498/965] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:21.571273 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cb10e-5061-4047-9efa-512c844f74f2 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:21.571636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cb10e-5061-4047-9efa-512c844f74f2 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:21.572033 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cb10e-5061-4047-9efa-512c844f74f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:21.595152 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-ee0cb10e-5061-4047-9efa-512c844f74f2 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:21.596681 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-ee0cb10e-5061-4047-9efa-512c844f74f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:21.627052 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:21.636833 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b764-2d03-4714-8405-4aadba54eb8f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:21.675181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:21.676034 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:21.676458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:21.676857 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 499/966] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:21.677250 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:21.678065 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:21.678432 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:21.683932 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:21.704514 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cb10e-5061-4047-9efa-512c844f74f2 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:21.732937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 468/967] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:40:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:21.823177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cb10e-5061-4047-9efa-512c844f74f2 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:21.854795 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:21.863702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:21.864766 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:21.865262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:21.865722 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:21.866645 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:21.868236 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:21.878835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:21.880385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:21.881633 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:21.884714 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 500/968] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:21.891262 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.007829 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48adcc-5b13-4484-aaed-5ee3c1b7369b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.043434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193da54-57c5-4eba-855b-1e691e540ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.044086 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193da54-57c5-4eba-855b-1e691e540ac7 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.044368 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193da54-57c5-4eba-855b-1e691e540ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.044641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193da54-57c5-4eba-855b-1e691e540ac7 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.045334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193da54-57c5-4eba-855b-1e691e540ac7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:22.045643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193da54-57c5-4eba-855b-1e691e540ac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:22.047078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193da54-57c5-4eba-855b-1e691e540ac7 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:22.052599 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 469/969] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:22.058361 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.096045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17084a-e06b-4ccc-8568-8a2d994d6592 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.096261 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17084a-e06b-4ccc-8568-8a2d994d6592 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.096519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17084a-e06b-4ccc-8568-8a2d994d6592 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.098097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 470/970] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:22.103292 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355479c-3e71-4fb4-bafb-d81f8cba2b16 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.103576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355479c-3e71-4fb4-bafb-d81f8cba2b16 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.103875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355479c-3e71-4fb4-bafb-d81f8cba2b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.123689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-6355479c-3e71-4fb4-bafb-d81f8cba2b16 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:22.125276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-6355479c-3e71-4fb4-bafb-d81f8cba2b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:22.136870 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:22.178937 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912060-2a7b-43d9-8efe-a4e80abb2387 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.207396 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 471/971] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:22.215652 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.216301 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.216590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.216874 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.217562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:22.217871 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:22.219323 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:22.224721 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 501/972] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:22.230550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355479c-3e71-4fb4-bafb-d81f8cba2b16 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.231780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.290818 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:22.339496 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed009f3e-62e3-49b3-9a18-fada75f1ed1e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.343842 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355479c-3e71-4fb4-bafb-d81f8cba2b16 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.374908 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd324f74-fd8f-47ba-b212-c4648129b430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.375735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd324f74-fd8f-47ba-b212-c4648129b430 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.375892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd324f74-fd8f-47ba-b212-c4648129b430 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.376136 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd324f74-fd8f-47ba-b212-c4648129b430 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.381982 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.382664 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.382969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.383240 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.383979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] RBAC: Adding query filter params (interface=admin, service_id=771522e6f9214b0e8ea22c2368744b3b, region_id=RegionOne) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:22.384285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:22.385905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:22.389554 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd324f74-fd8f-47ba-b212-c4648129b430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:22.392068 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 472/973] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jul 28 08:40:22 2022] GET /identity/v3/endpoints?service_id=771522e6f9214b0e8ea22c2368744b3b&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-dd324f74-fd8f-47ba-b212-c4648129b430 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:22.395971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:22.397386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:22.398621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:22.400762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.402980 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 502/974] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:40:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:22.404290 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd324f74-fd8f-47ba-b212-c4648129b430 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.414532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:22.517008 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd324f74-fd8f-47ba-b212-c4648129b430 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.519893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54402ee-5ac2-41c2-bd65-99c8f1edb6c8 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.530557 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:22.552881 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1246c0-6375-4b6a-92bb-4253815cb55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.553530 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1246c0-6375-4b6a-92bb-4253815cb55f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.553817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1246c0-6375-4b6a-92bb-4253815cb55f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.554131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1246c0-6375-4b6a-92bb-4253815cb55f None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.554864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1246c0-6375-4b6a-92bb-4253815cb55f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:22.555176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1246c0-6375-4b6a-92bb-4253815cb55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:22.555683 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8b281-2565-4090-a21a-1cba2cc9608d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.556333 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8b281-2565-4090-a21a-1cba2cc9608d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.556564 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1246c0-6375-4b6a-92bb-4253815cb55f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:22.556682 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8b281-2565-4090-a21a-1cba2cc9608d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.556871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8b281-2565-4090-a21a-1cba2cc9608d None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.557537 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8b281-2565-4090-a21a-1cba2cc9608d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:22.558977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8b281-2565-4090-a21a-1cba2cc9608d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:22.565091 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 503/975] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:40:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:22.566403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 473/976] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:40:22 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:40:22.566531 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8cc7-2dd8-4c44-a939-976a4b55574d None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.566794 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8cc7-2dd8-4c44-a939-976a4b55574d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.567080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8cc7-2dd8-4c44-a939-976a4b55574d None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.568551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 504/977] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:22.571591 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8b281-2565-4090-a21a-1cba2cc9608d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.574889 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6b40a-94f0-4256-95f9-0d5ce9b520e9 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.575555 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6b40a-94f0-4256-95f9-0d5ce9b520e9 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.575842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6b40a-94f0-4256-95f9-0d5ce9b520e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.599092 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-3fb6b40a-94f0-4256-95f9-0d5ce9b520e9 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:22.600351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-3fb6b40a-94f0-4256-95f9-0d5ce9b520e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:22.664378 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:22.680608 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 505/978] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:22.701992 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8b281-2565-4090-a21a-1cba2cc9608d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.704913 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6b40a-94f0-4256-95f9-0d5ce9b520e9 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.747504 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312704a-8569-4240-92be-293a0329463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.747504 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312704a-8569-4240-92be-293a0329463c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.747846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312704a-8569-4240-92be-293a0329463c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.748136 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312704a-8569-4240-92be-293a0329463c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.764105 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312704a-8569-4240-92be-293a0329463c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:22.765865 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312704a-8569-4240-92be-293a0329463c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-e312704a-8569-4240-92be-293a0329463c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:22.784585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:22.786349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:22.787623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:22.787623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:22.787623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:22.787623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 474/979] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:22.791792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312704a-8569-4240-92be-293a0329463c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:22.822925 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6b40a-94f0-4256-95f9-0d5ce9b520e9 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.825421 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:22.867638 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d989a5-f803-4bb3-bc8d-a301188d9789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.868269 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d989a5-f803-4bb3-bc8d-a301188d9789 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.868556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d989a5-f803-4bb3-bc8d-a301188d9789 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.868836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d989a5-f803-4bb3-bc8d-a301188d9789 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.869572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d989a5-f803-4bb3-bc8d-a301188d9789 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:22.870987 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d989a5-f803-4bb3-bc8d-a301188d9789 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:22.887161 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 506/980] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:40:22 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:22.910577 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312704a-8569-4240-92be-293a0329463c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:22.940814 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:22.964157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:22.964974 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:22.965319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:22.965665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:22.966514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:22.966890 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:22.968394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:22.983497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 475/981] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:22.989520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d989a5-f803-4bb3-bc8d-a301188d9789 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:23.105949 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d989a5-f803-4bb3-bc8d-a301188d9789 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:23.142795 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:23.143478 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:23.143776 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:23.144089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:23.152810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:23.154328 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:23.166688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:23.168089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:23.169483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 507/982] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:40:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:23.174889 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:23.291778 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49c65-e825-4c02-9479-6a63df2ca017 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:23.330140 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:23.330982 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:23.331357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:23.331780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:23.332687 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:23.333090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:23.334912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:23.354860 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 476/983] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:40:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:23.360542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:23.395290 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:23.505702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6009c5-5051-46b6-9922-6d9dd313a7f7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:23.542638 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81700cb4-f25f-4c34-9533-d3d05f7d4e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:23.543275 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700cb4-f25f-4c34-9533-d3d05f7d4e87 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:23.543636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700cb4-f25f-4c34-9533-d3d05f7d4e87 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:23.543979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700cb4-f25f-4c34-9533-d3d05f7d4e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:23.623624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81700cb4-f25f-4c34-9533-d3d05f7d4e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:23.625205 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81700cb4-f25f-4c34-9533-d3d05f7d4e87 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:23.652299 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:23.744652 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 508/984] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:23 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:40:23.751001 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:23.795223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:23.866568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9eb9c-a79f-4026-8152-b8baecb1d594 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:23.902532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:23.903187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:23.903473 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:23.903727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:23.915120 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:23.916666 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:23.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:23.933667 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:23.934829 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:23.934829 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:23.934829 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:23.934829 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 477/985] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:23.941458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81700cb4-f25f-4c34-9533-d3d05f7d4e87 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:24.084991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81700cb4-f25f-4c34-9533-d3d05f7d4e87 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:24.128062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:24.145843 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:24.146533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:24.146809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:24.147089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:24.147787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:24.148093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:24.149526 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:24.163632 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 509/986] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:24.169233 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:24.271167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:24.283305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911cc58-76e0-463a-b542-f0615ce9ac1e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:24.321482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:24.322154 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:24.322447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:24.322721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:24.329978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:24.331483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:24.344351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:24.352409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:24.353813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 478/987] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:24.354579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:24.476776 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000157a-0d96-4b17-873d-08586e0e2ca0 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:24.514078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57089016-5f9c-45aa-8f9e-13bc3abec53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:24.514853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57089016-5f9c-45aa-8f9e-13bc3abec53c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:24.515054 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57089016-5f9c-45aa-8f9e-13bc3abec53c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:24.515373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57089016-5f9c-45aa-8f9e-13bc3abec53c None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:24.516140 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57089016-5f9c-45aa-8f9e-13bc3abec53c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:24.516445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57089016-5f9c-45aa-8f9e-13bc3abec53c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:24.518001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57089016-5f9c-45aa-8f9e-13bc3abec53c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:24.532034 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 510/988] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:24.539158 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:24.610179 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb39a2-009f-40a7-8cb4-6ab4e7d7fd55 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:24.610501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb39a2-009f-40a7-8cb4-6ab4e7d7fd55 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:24.610876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb39a2-009f-40a7-8cb4-6ab4e7d7fd55 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:24.612379 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 511/989] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:24.618848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143428-aad8-4a47-8b14-f69dcc8546a3 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:24.619358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143428-aad8-4a47-8b14-f69dcc8546a3 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:24.619859 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143428-aad8-4a47-8b14-f69dcc8546a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:24.642639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-40143428-aad8-4a47-8b14-f69dcc8546a3 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:24.643958 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-40143428-aad8-4a47-8b14-f69dcc8546a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:24.674571 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ec735-e571-48ea-aa75-2953ea2b2562 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:24.689129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:24.720558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:24.721559 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:24.721822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:24.722098 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:24.723097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] RBAC: Adding query filter params (role.id=4b13ca2dcb004c4891a205a6f8d64042, scope.project.id=813e8db425644f688e926aafe3249721, user.id=945e08e70ce943708b6eeeb0fd8acc5d) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:24.723426 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:24.724916 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:24.731583 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 479/990] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:24 2022] GET /identity/v3/role_assignments?user.id=945e08e70ce943708b6eeeb0fd8acc5d&scope.project.id=813e8db425644f688e926aafe3249721&role.id=4b13ca2dcb004c4891a205a6f8d64042 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:24.744040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 512/991] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:24.768288 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:24.789135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:24.900161 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10bb40-3337-4a66-936c-9d69b65ff73d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:24.942265 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58fe57-258c-4ada-b5de-5d507312e483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:24.943064 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58fe57-258c-4ada-b5de-5d507312e483 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:24.943353 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58fe57-258c-4ada-b5de-5d507312e483 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:24.943627 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58fe57-258c-4ada-b5de-5d507312e483 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:24.944651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58fe57-258c-4ada-b5de-5d507312e483 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:24.946065 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58fe57-258c-4ada-b5de-5d507312e483 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-ae58fe57-258c-4ada-b5de-5d507312e483 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:24.955980 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:24.958066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:24.960030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 480/992] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:40:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:24.963660 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40143428-aad8-4a47-8b14-f69dcc8546a3 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:24.996660 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e54e43-ea9d-4f81-88de-416301a16965 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:24.996943 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e54e43-ea9d-4f81-88de-416301a16965 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:24.997194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e54e43-ea9d-4f81-88de-416301a16965 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:24.999076 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 481/993] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:25.004134 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538ea91-b124-4f9a-b33d-45dc875a3b1b None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:25.004415 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538ea91-b124-4f9a-b33d-45dc875a3b1b None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:25.004687 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538ea91-b124-4f9a-b33d-45dc875a3b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:25.025298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-0538ea91-b124-4f9a-b33d-45dc875a3b1b None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:25.026534 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-0538ea91-b124-4f9a-b33d-45dc875a3b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:25.080977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40143428-aad8-4a47-8b14-f69dcc8546a3 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:25.109266 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 482/994] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:25.118463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:25.119109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:25.119385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:25.119712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:25.120338 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:25.120846 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:25.122021 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:25.127459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 513/995] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:25.133035 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:25.133702 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538ea91-b124-4f9a-b33d-45dc875a3b1b None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:25.145187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:25.180635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:25.245371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538ea91-b124-4f9a-b33d-45dc875a3b1b None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:25.251407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d1291-45db-40ac-8fd7-983f123dca00 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:25.289543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:25.290483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:25.290790 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:25.291056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:25.291927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:25.292223 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:25.292476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:25.292906 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:25.293193 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:25.293475 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:25.294145 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:25.294383 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:25.295543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:25.305573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 483/996] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:40:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:25.306351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:25.308335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:25.309702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:25.310712 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:25.310809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 514/997] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:25.313497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:25.390825 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:25.427697 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb33e50-897a-4772-98f1-bbddd9a99056 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:25.438711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166968e-866d-4a62-9f7c-47bcb6183cf5 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:25.440896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:25.467344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa30ad-11b9-4a65-b7e7-a4b7939f060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:25.468273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa30ad-11b9-4a65-b7e7-a4b7939f060e None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:25.468838 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa30ad-11b9-4a65-b7e7-a4b7939f060e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:25.469268 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa30ad-11b9-4a65-b7e7-a4b7939f060e None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:25.470235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa30ad-11b9-4a65-b7e7-a4b7939f060e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:25.471615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa30ad-11b9-4a65-b7e7-a4b7939f060e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:25.480262 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:25.481007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:25.481303 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:25.481702 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:25.482541 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:25.482855 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:25.484310 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:25.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 515/998] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:25.495252 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 484/999] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:40:25 2022] POST /identity/v3/endpoints => generated 325 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:25.499405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:25.634551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d899e-1f74-4bc0-a516-a1f29a7f96f8 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:25.683502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:25.684173 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:25.684445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:25.684727 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:25.685445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] RBAC: Adding query filter params (type=compute) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:25.685757 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:25.687252 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:25.696546 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 516/1000] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:25.703535 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa30ad-11b9-4a65-b7e7-a4b7939f060e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:25.823649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa30ad-11b9-4a65-b7e7-a4b7939f060e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:25.853710 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:25.860391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f17ac-7e38-46be-8aaf-93825f727a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:25.861059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f17ac-7e38-46be-8aaf-93825f727a13 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:25.861361 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f17ac-7e38-46be-8aaf-93825f727a13 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:25.861686 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f17ac-7e38-46be-8aaf-93825f727a13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:25.862483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f17ac-7e38-46be-8aaf-93825f727a13 None admin] RBAC: Adding query filter params (interface=public, service_id=f629467d181049f8a696a3cbb1e8db8c, region_id=RegionOne) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:25.862850 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f17ac-7e38-46be-8aaf-93825f727a13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:25.864835 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f17ac-7e38-46be-8aaf-93825f727a13 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:25.874152 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 485/1001] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:40:25 2022] GET /identity/v3/endpoints?service_id=f629467d181049f8a696a3cbb1e8db8c&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:25.876839 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:25.906566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:25.986915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f26153-cfdc-4714-8e92-ab3834764702 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:26.023000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6a010-782a-4aac-9e8a-aada6028c834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:26.023643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6a010-782a-4aac-9e8a-aada6028c834 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:26.023922 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6a010-782a-4aac-9e8a-aada6028c834 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:26.024192 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6a010-782a-4aac-9e8a-aada6028c834 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:26.024853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6a010-782a-4aac-9e8a-aada6028c834 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:26.026259 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6a010-782a-4aac-9e8a-aada6028c834 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:26.033057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 517/1002] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:40:25 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:40:26.867846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9934-fe45-4476-a447-42d173fefa10 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:26.868449 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9934-fe45-4476-a447-42d173fefa10 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:26.868449 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9934-fe45-4476-a447-42d173fefa10 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:26.869984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 486/1003] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:26.875319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8eb32-3c45-4f9a-a797-5bd869206a35 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:26.875823 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8eb32-3c45-4f9a-a797-5bd869206a35 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:26.875914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8eb32-3c45-4f9a-a797-5bd869206a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:26.894798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-4cd8eb32-3c45-4f9a-a797-5bd869206a35 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:26.896015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-4cd8eb32-3c45-4f9a-a797-5bd869206a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:26.969890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 518/1004] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:26.992068 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9934-fe45-4476-a447-42d173fefa10 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:27.100743 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9934-fe45-4476-a447-42d173fefa10 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:27.137097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:27.137775 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:27.138061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:27.138374 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] PATH_INFO: `/v3/services/image` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:27.139024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:27.140438 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:27.150347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:27.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:27.153458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 487/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:40:26 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:27.157999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8eb32-3c45-4f9a-a797-5bd869206a35 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:27.182510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:27.268238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8eb32-3c45-4f9a-a797-5bd869206a35 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:27.304292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980aec50-c669-4391-a06c-25868c1cf2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:27.304950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aec50-c669-4391-a06c-25868c1cf2e5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:27.305233 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aec50-c669-4391-a06c-25868c1cf2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:27.305520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aec50-c669-4391-a06c-25868c1cf2e5 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:27.306244 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980aec50-c669-4391-a06c-25868c1cf2e5 None admin] RBAC: Adding query filter params (name=image) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:27.306556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980aec50-c669-4391-a06c-25868c1cf2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:27.307948 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980aec50-c669-4391-a06c-25868c1cf2e5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:27.324789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 519/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:27.327663 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:27.396415 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:27.440496 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a8457-6b22-4c35-a2e5-a750c461c21a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:27.476666 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b33bf-9f94-45fa-bf8f-d85f709627d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:27.477314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b33bf-9f94-45fa-bf8f-d85f709627d0 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:27.477599 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b33bf-9f94-45fa-bf8f-d85f709627d0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:27.477879 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b33bf-9f94-45fa-bf8f-d85f709627d0 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:27.478605 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b33bf-9f94-45fa-bf8f-d85f709627d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:27.478917 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b33bf-9f94-45fa-bf8f-d85f709627d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:27.480331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b33bf-9f94-45fa-bf8f-d85f709627d0 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:27.485759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 488/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:27.613865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265a10-9b7c-49c8-adc8-85e926484ff0 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:27.614176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265a10-9b7c-49c8-adc8-85e926484ff0 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:27.614547 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265a10-9b7c-49c8-adc8-85e926484ff0 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:27.615983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 520/1008] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:27.622346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fccef7-2e8d-4f9c-bac2-dbf8f064b88e None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:27.622729 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fccef7-2e8d-4f9c-bac2-dbf8f064b88e None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:27.622900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fccef7-2e8d-4f9c-bac2-dbf8f064b88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:27.644912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-23fccef7-2e8d-4f9c-bac2-dbf8f064b88e None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:27.646229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-23fccef7-2e8d-4f9c-bac2-dbf8f064b88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:27.725257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 489/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:27.750492 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f265a10-9b7c-49c8-adc8-85e926484ff0 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:27.854029 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:27.875975 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f265a10-9b7c-49c8-adc8-85e926484ff0 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:27.913487 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b37beb-e98a-45a9-92ee-afc4b83f0a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:27.914198 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b37beb-e98a-45a9-92ee-afc4b83f0a24 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:27.914491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b37beb-e98a-45a9-92ee-afc4b83f0a24 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:27.914759 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b37beb-e98a-45a9-92ee-afc4b83f0a24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:27.915953 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b37beb-e98a-45a9-92ee-afc4b83f0a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:27.917422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b37beb-e98a-45a9-92ee-afc4b83f0a24 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:27.919345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 521/1010] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:27.925371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fccef7-2e8d-4f9c-bac2-dbf8f064b88e None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:27.987839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:28.038270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fccef7-2e8d-4f9c-bac2-dbf8f064b88e None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:28.077157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69565e-069d-40ab-b179-f80bab88959c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:28.077810 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69565e-069d-40ab-b179-f80bab88959c None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:28.078094 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69565e-069d-40ab-b179-f80bab88959c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:28.078402 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69565e-069d-40ab-b179-f80bab88959c None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:28.079138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69565e-069d-40ab-b179-f80bab88959c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:28.080624 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69565e-069d-40ab-b179-f80bab88959c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:28.091764 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1c50-2b39-41ce-bc76-8068ae9434e5 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:28.092060 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1c50-2b39-41ce-bc76-8068ae9434e5 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:28.092325 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1c50-2b39-41ce-bc76-8068ae9434e5 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:28.093707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 522/1011] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:28.098924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcca3fb-9f95-4cc3-a518-bbb9e871d918 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:28.099217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcca3fb-9f95-4cc3-a518-bbb9e871d918 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:28.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcca3fb-9f95-4cc3-a518-bbb9e871d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:28.115573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 490/1012] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:40:27 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:28.128944 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-3bcca3fb-9f95-4cc3-a518-bbb9e871d918 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:28.130255 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-3bcca3fb-9f95-4cc3-a518-bbb9e871d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:28.214202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 523/1013] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:28 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:28.237168 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69565e-069d-40ab-b179-f80bab88959c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:28.370902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69565e-069d-40ab-b179-f80bab88959c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:28.381192 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:28.430498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:28.431839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:28.432394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:28.432914 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:28.434332 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:28.437217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:28.457896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:28.459322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:40:28.460398 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:28.461532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 491/1014] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:28.469889 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcca3fb-9f95-4cc3-a518-bbb9e871d918 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:28.573780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:28.602153 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcca3fb-9f95-4cc3-a518-bbb9e871d918 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:28.641211 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:28.641778 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:28.642060 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:28.642386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:28.643095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:28.643403 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:28.644934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:28.655780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 524/1015] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:28.657744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:28.773951 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21b82b-4f9a-4f6e-9573-6f2322ad9f0d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:28.815864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274e498-079a-4a1e-86f3-f532f71d0e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:28.816516 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274e498-079a-4a1e-86f3-f532f71d0e3d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:28.816794 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274e498-079a-4a1e-86f3-f532f71d0e3d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:28.817071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274e498-079a-4a1e-86f3-f532f71d0e3d None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:28.817758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274e498-079a-4a1e-86f3-f532f71d0e3d None admin] RBAC: Adding query filter params (type=compute) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:28.818061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274e498-079a-4a1e-86f3-f532f71d0e3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:28.819873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274e498-079a-4a1e-86f3-f532f71d0e3d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:28.829987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 492/1016] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:40:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:28.831169 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:28.939451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d59d0-8750-4f2e-9850-f0ddd58b08c2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:28.976818 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c262eef-257c-4940-a8eb-22e03545d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:28.977780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c262eef-257c-4940-a8eb-22e03545d50e None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:28.978259 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c262eef-257c-4940-a8eb-22e03545d50e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:28.978718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c262eef-257c-4940-a8eb-22e03545d50e None admin] PATH_INFO: `/v3/endpoints` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:28.979603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c262eef-257c-4940-a8eb-22e03545d50e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:28.981182 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c262eef-257c-4940-a8eb-22e03545d50e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:29.010206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 525/1017] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:40:28 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:29.554762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359418-e7a8-4c5b-b710-780fed4d94d4 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:29.555348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359418-e7a8-4c5b-b710-780fed4d94d4 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:29.555348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359418-e7a8-4c5b-b710-780fed4d94d4 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:29.556670 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 493/1018] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:29.562775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8ed1-72b2-4054-8906-e4745be3e968 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:29.563072 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8ed1-72b2-4054-8906-e4745be3e968 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:29.563502 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8ed1-72b2-4054-8906-e4745be3e968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:29.570944 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:29.588328 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-ac6a8ed1-72b2-4054-8906-e4745be3e968 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:29.589842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-ac6a8ed1-72b2-4054-8906-e4745be3e968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:29.666324 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 526/1019] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:29 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:29.690323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3359418-e7a8-4c5b-b710-780fed4d94d4 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:29.727762 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:29.802170 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3359418-e7a8-4c5b-b710-780fed4d94d4 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:29.849499 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e167c36-1dee-4e27-a94b-83c008ff182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:29.850052 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e167c36-1dee-4e27-a94b-83c008ff182f None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:29.850368 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e167c36-1dee-4e27-a94b-83c008ff182f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:29.850649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e167c36-1dee-4e27-a94b-83c008ff182f None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:29.851371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e167c36-1dee-4e27-a94b-83c008ff182f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:29.852819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e167c36-1dee-4e27-a94b-83c008ff182f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:29.870156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 494/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:40:29 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:30.215125 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463224a-b76f-4aa9-a3bb-07ddd28b31aa None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:30.215754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463224a-b76f-4aa9-a3bb-07ddd28b31aa None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:30.215754 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463224a-b76f-4aa9-a3bb-07ddd28b31aa None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:30.217404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 527/1021] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:30.223624 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4d2b5-e526-4787-a639-99bc5a7c9c52 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:30.223864 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4d2b5-e526-4787-a639-99bc5a7c9c52 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:30.224162 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4d2b5-e526-4787-a639-99bc5a7c9c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:30.243519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-30e4d2b5-e526-4787-a639-99bc5a7c9c52 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:30.244745 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-30e4d2b5-e526-4787-a639-99bc5a7c9c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:30.324796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 495/1022] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:30.349337 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463224a-b76f-4aa9-a3bb-07ddd28b31aa None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:30.460853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463224a-b76f-4aa9-a3bb-07ddd28b31aa None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:30.489468 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:30.498991 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b9e1a-94a9-46b4-b9b5-01ab82d91182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:30.499664 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b9e1a-94a9-46b4-b9b5-01ab82d91182 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:30.499952 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b9e1a-94a9-46b4-b9b5-01ab82d91182 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:30.500267 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b9e1a-94a9-46b4-b9b5-01ab82d91182 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:30.501438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b9e1a-94a9-46b4-b9b5-01ab82d91182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:30.502945 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b9e1a-94a9-46b4-b9b5-01ab82d91182 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:30.504799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 528/1023] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:30.510663 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4d2b5-e526-4787-a639-99bc5a7c9c52 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:30.533792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:30.625945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4d2b5-e526-4787-a639-99bc5a7c9c52 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:30.664652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09caa86-6297-4615-a256-5398719fe3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:30.665608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09caa86-6297-4615-a256-5398719fe3e2 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:30.665608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09caa86-6297-4615-a256-5398719fe3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:30.665810 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09caa86-6297-4615-a256-5398719fe3e2 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:30.666604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09caa86-6297-4615-a256-5398719fe3e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:30.668071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09caa86-6297-4615-a256-5398719fe3e2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:30.706223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 496/1024] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:40:30 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:31.135350 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591329f9-9217-4fd6-83fb-4688d18b6c92 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:31.135598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591329f9-9217-4fd6-83fb-4688d18b6c92 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:31.135888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591329f9-9217-4fd6-83fb-4688d18b6c92 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:31.137265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 529/1025] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:31.143071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3d78f-bf94-4204-9441-000a0aa74c97 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:31.143342 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3d78f-bf94-4204-9441-000a0aa74c97 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:31.143642 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3d78f-bf94-4204-9441-000a0aa74c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:31.163128 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-63f3d78f-bf94-4204-9441-000a0aa74c97 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:31.164348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-63f3d78f-bf94-4204-9441-000a0aa74c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:31.237061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 497/1026] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:31.260462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591329f9-9217-4fd6-83fb-4688d18b6c92 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:31.372723 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591329f9-9217-4fd6-83fb-4688d18b6c92 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:31.418730 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be290c-cd19-4f84-8aa6-ead9748a046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:31.419461 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be290c-cd19-4f84-8aa6-ead9748a046d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:31.419779 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be290c-cd19-4f84-8aa6-ead9748a046d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:31.420171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be290c-cd19-4f84-8aa6-ead9748a046d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:31.435525 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be290c-cd19-4f84-8aa6-ead9748a046d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-05be290c-cd19-4f84-8aa6-ead9748a046d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:31.441835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:31.443359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:31.452046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 530/1027] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:40:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:31.456586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3d78f-bf94-4204-9441-000a0aa74c97 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:31.468181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:31.571108 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3d78f-bf94-4204-9441-000a0aa74c97 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:31.611498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:31.612141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:31.612419 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:31.612746 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:31.613483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:31.613789 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:31.615237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:31.624140 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 498/1028] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:40:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:31.629934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be290c-cd19-4f84-8aa6-ead9748a046d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:31.693417 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:31.741516 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be290c-cd19-4f84-8aa6-ead9748a046d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:31.777821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef554df5-d422-47c8-8ed0-f0285ca7fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:31.778512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554df5-d422-47c8-8ed0-f0285ca7fee0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:31.778792 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554df5-d422-47c8-8ed0-f0285ca7fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:31.779064 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554df5-d422-47c8-8ed0-f0285ca7fee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:31.790510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef554df5-d422-47c8-8ed0-f0285ca7fee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:31.791974 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef554df5-d422-47c8-8ed0-f0285ca7fee0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-ef554df5-d422-47c8-8ed0-f0285ca7fee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:31.807741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:31.813960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:31.815131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:31.815131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:31.815131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:31.815131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 531/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:31.815429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:31.896569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:31.924095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867150aa-1e28-45ab-b4d0-26d1d70b49c8 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:31.924367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867150aa-1e28-45ab-b4d0-26d1d70b49c8 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:31.924646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867150aa-1e28-45ab-b4d0-26d1d70b49c8 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:31.926336 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 532/1030] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:31.931655 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596f9c7-f64f-41f2-b206-069a78521e52 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:31.931973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596f9c7-f64f-41f2-b206-069a78521e52 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:31.932222 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596f9c7-f64f-41f2-b206-069a78521e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:31.941015 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963b0be-0067-41af-ba8a-fd686828f0fe None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:31.953219 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-1596f9c7-f64f-41f2-b206-069a78521e52 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:31.954563 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-1596f9c7-f64f-41f2-b206-069a78521e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:31.978152 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:31.979133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:31.979414 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:31.979694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:31.980397 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:31.980705 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:31.982125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:31.994422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 499/1031] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:32.000069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.034776 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 533/1032] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:32.057210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596f9c7-f64f-41f2-b206-069a78521e52 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.100775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:32.108688 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa742b-0114-4e59-ac8a-0d85b9832552 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.137141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:32.146049 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.146748 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.147035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.147310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.154582 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.156057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.167878 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596f9c7-f64f-41f2-b206-069a78521e52 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:32.168036 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:32.169296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:32.170736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 500/1033] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:40:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:32.174936 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.212304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.212892 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.213181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.213460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] PATH_INFO: `/v3/services/image` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.214134 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.215533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:32.225951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:32.227404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:32.228443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 534/1034] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:32.233726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.286201 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1cf09-b2c7-4ed5-bada-61bd0c51be89 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.324364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.325142 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.325411 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.325720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.326571 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:32.326886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.328376 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.344227 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d984c-4a2c-4aa3-a89c-e031c07048f0 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.345913 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:32.347297 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 501/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:32.353625 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.385489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.386331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.386747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.387007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.387880 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] RBAC: Adding query filter params (name=image) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:32.388343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.390019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.396207 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 535/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:32.401756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.414719 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:32.474008 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e69a40-7a48-4b6c-bd68-3c86bee6608d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.512093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dbbc4-59e7-4c6d-a8b2-53f6bd1dc422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.512737 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dbbc4-59e7-4c6d-a8b2-53f6bd1dc422 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.513012 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dbbc4-59e7-4c6d-a8b2-53f6bd1dc422 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.513306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dbbc4-59e7-4c6d-a8b2-53f6bd1dc422 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.517500 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3d1e-95db-4d30-ad7a-5b6c39a73965 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.556563 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.557226 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.557511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.557788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.558522 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] RBAC: Adding query filter params (type=image) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:32.558833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.560237 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.566224 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 536/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:32.572474 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.595208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0dbbc4-59e7-4c6d-a8b2-53f6bd1dc422 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.596703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0dbbc4-59e7-4c6d-a8b2-53f6bd1dc422 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.605927 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:32.651856 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:32.684586 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51c174-0436-4a4d-852a-3640837d9b93 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.716912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 502/1038] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:40:32.722968 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dbbc4-59e7-4c6d-a8b2-53f6bd1dc422 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.724544 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.725140 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.725426 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.725706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.726524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] RBAC: Adding query filter params (interface=public, service_id=736da4382a4a4636a19036bca72d060e, region_id=RegionOne) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:32.726865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.728260 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.733803 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 537/1039] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/endpoints?service_id=736da4382a4a4636a19036bca72d060e&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:32.739369 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.837649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dbbc4-59e7-4c6d-a8b2-53f6bd1dc422 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.847705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:32.855877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36280e3-7b84-4992-8ac1-aabd5c974c38 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:32.874787 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.875458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.875746 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.876023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.886572 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.888069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.891433 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2727e1-22fa-4ee9-a690-6d646b9a4562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:32.892101 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2727e1-22fa-4ee9-a690-6d646b9a4562 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.892343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2727e1-22fa-4ee9-a690-6d646b9a4562 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.892640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2727e1-22fa-4ee9-a690-6d646b9a4562 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.893308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2727e1-22fa-4ee9-a690-6d646b9a4562 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:32.894783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2727e1-22fa-4ee9-a690-6d646b9a4562 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:32.895618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:32.901996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 538/1040] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:40:32.906427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8bc9-8974-46b4-9b9e-1a1318d6f7d8 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.906427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8bc9-8974-46b4-9b9e-1a1318d6f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.906427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8bc9-8974-46b4-9b9e-1a1318d6f7d8 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:32.906713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:32.908269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:32.909953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:32.909953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:32.909953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:32.910231 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 539/1041] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:32.910320 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 503/1042] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:32.912002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:32.912667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610414c-19c8-4bee-9cff-7d6155536383 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:32.912917 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610414c-19c8-4bee-9cff-7d6155536383 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:32.913212 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610414c-19c8-4bee-9cff-7d6155536383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:32.934397 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-0610414c-19c8-4bee-9cff-7d6155536383 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:32.935774 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-0610414c-19c8-4bee-9cff-7d6155536383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:33.016751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 540/1043] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:33.040559 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610414c-19c8-4bee-9cff-7d6155536383 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:33.064014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52870d-fe03-481e-8dad-5be3cef1c9ac None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:33.113178 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:33.113883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:33.114217 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:33.114502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:33.115347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:33.115925 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:33.117483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:33.136501 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 504/1044] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:33.137607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:33.151472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:33.168235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610414c-19c8-4bee-9cff-7d6155536383 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:33.210622 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c63c3-d6ba-4444-97e1-4a19ebcc787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:33.211583 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c63c3-d6ba-4444-97e1-4a19ebcc787d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:33.211876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c63c3-d6ba-4444-97e1-4a19ebcc787d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:33.212259 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c63c3-d6ba-4444-97e1-4a19ebcc787d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:33.216769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:33.225765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c63c3-d6ba-4444-97e1-4a19ebcc787d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-255c63c3-d6ba-4444-97e1-4a19ebcc787d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:33.231042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:33.232560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:40:33.234451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:33.238340 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 541/1045] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:40:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:33.239515 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c63c3-d6ba-4444-97e1-4a19ebcc787d None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:33.272435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ae313-dfd0-425d-9f8d-e5f28c343381 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:33.312307 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:33.312977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:33.313257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:33.313546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:33.321638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:33.323172 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:33.336198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:33.337338 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:33.338528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 505/1046] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:33.344081 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:33.356067 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c63c3-d6ba-4444-97e1-4a19ebcc787d None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:33.402606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:33.403401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:33.403696 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:33.403999 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:33.404782 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:33.405171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:33.406737 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:33.418159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 542/1047] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:33.423990 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:33.438386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:33.478077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8484d64-753d-4432-9b1c-07c2b971d50c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:33.498663 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:33.517245 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:33.517893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:33.518221 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:33.518495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:33.519322 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:33.519638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:33.521110 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:33.533820 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 506/1048] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:33.540364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:33.545425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bce41-55a2-48f7-a785-991d36e04004 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:33.588801 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2876c-d0fe-4d3a-ade7-c70a303d8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:33.589483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2876c-d0fe-4d3a-ade7-c70a303d8bb8 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:33.589763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2876c-d0fe-4d3a-ade7-c70a303d8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:33.590415 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2876c-d0fe-4d3a-ade7-c70a303d8bb8 None admin] PATH_INFO: `/v3/users/5986d7da093b41daa0ec9045dc3d8cd3` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:33.605756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2876c-d0fe-4d3a-ade7-c70a303d8bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=5986d7da093b41daa0ec9045dc3d8cd3)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:33.607301 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2876c-d0fe-4d3a-ade7-c70a303d8bb8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:33.609200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 543/1049] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:33 2022] GET /identity/v3/users/5986d7da093b41daa0ec9045dc3d8cd3 => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:33.615724 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2876c-d0fe-4d3a-ade7-c70a303d8bb8 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:33.667960 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0c281-47e3-4b68-bb60-bcbbbb162c89 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:33.692091 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:33.722568 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:33.723373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:33.723652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:33.723930 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:33.724753 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] RBAC: Adding query filter params (role.id=4b13ca2dcb004c4891a205a6f8d64042, scope.project.id=813e8db425644f688e926aafe3249721, user.id=90103f3776ce4eeb90bde66ed546aa87) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:33.725067 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:33.726663 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:33.733272 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 507/1050] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:33 2022] GET /identity/v3/role_assignments?user.id=90103f3776ce4eeb90bde66ed546aa87&scope.project.id=813e8db425644f688e926aafe3249721&role.id=4b13ca2dcb004c4891a205a6f8d64042 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:33.737186 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2876c-d0fe-4d3a-ade7-c70a303d8bb8 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:33.781695 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be79c4-e4f4-4df6-b1a8-62b512613f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:33.782396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be79c4-e4f4-4df6-b1a8-62b512613f18 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:33.782682 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be79c4-e4f4-4df6-b1a8-62b512613f18 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:33.783004 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be79c4-e4f4-4df6-b1a8-62b512613f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:33.870067 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be79c4-e4f4-4df6-b1a8-62b512613f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:33.871521 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be79c4-e4f4-4df6-b1a8-62b512613f18 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:33.945184 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:33.985374 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 544/1051] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:33 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:40:33.991096 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:34.046460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:34.123591 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415eef26-212a-4565-95ac-8f4b4f4e5c8a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:34.159817 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b502465-8fc7-483e-92ac-3ebde3e9ac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:34.160495 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b502465-8fc7-483e-92ac-3ebde3e9ac97 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:34.160783 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b502465-8fc7-483e-92ac-3ebde3e9ac97 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:34.161074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b502465-8fc7-483e-92ac-3ebde3e9ac97 None admin] PATH_INFO: `/v3/projects/cac2af9e3d3d4a8688161ec1886a3895` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:34.162316 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b502465-8fc7-483e-92ac-3ebde3e9ac97 None admin] RBAC: Authorizing `identity:get_project(project_id=cac2af9e3d3d4a8688161ec1886a3895)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:34.163796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b502465-8fc7-483e-92ac-3ebde3e9ac97 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:34.165716 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 508/1052] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:40:33 2022] GET /identity/v3/projects/cac2af9e3d3d4a8688161ec1886a3895 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:34.171834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be79c4-e4f4-4df6-b1a8-62b512613f18 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:34.279214 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be79c4-e4f4-4df6-b1a8-62b512613f18 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:34.314661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994380e5-78e2-42ab-83d5-2887f0e35af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:34.315298 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994380e5-78e2-42ab-83d5-2887f0e35af3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:34.315580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994380e5-78e2-42ab-83d5-2887f0e35af3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:34.315903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994380e5-78e2-42ab-83d5-2887f0e35af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:34.316704 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994380e5-78e2-42ab-83d5-2887f0e35af3 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=cac2af9e3d3d4a8688161ec1886a3895, user.id=5986d7da093b41daa0ec9045dc3d8cd3) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:34.317030 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994380e5-78e2-42ab-83d5-2887f0e35af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:34.318495 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994380e5-78e2-42ab-83d5-2887f0e35af3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:34.324805 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 545/1053] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:34 2022] GET /identity/v3/role_assignments?user.id=5986d7da093b41daa0ec9045dc3d8cd3&scope.project.id=cac2af9e3d3d4a8688161ec1886a3895&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:34.975270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284628ea-8737-4ef8-acee-bcf7e1377de6 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:34.975484 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284628ea-8737-4ef8-acee-bcf7e1377de6 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:34.975772 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284628ea-8737-4ef8-acee-bcf7e1377de6 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:34.977084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 509/1054] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:34.982659 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f390fe-a915-40e6-addd-e94b190340ff None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:34.983133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f390fe-a915-40e6-addd-e94b190340ff None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:34.983223 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f390fe-a915-40e6-addd-e94b190340ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:35.002569 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-29f390fe-a915-40e6-addd-e94b190340ff None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:35.003792 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-29f390fe-a915-40e6-addd-e94b190340ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:35.033488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:35.081542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 546/1055] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:35.103877 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284628ea-8737-4ef8-acee-bcf7e1377de6 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:35.215288 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284628ea-8737-4ef8-acee-bcf7e1377de6 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:35.226355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:35.254387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:35.255061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:35.255344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:35.255621 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] PATH_INFO: `/v3/services/image` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:35.256265 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:35.257673 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:35.267238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:35.268507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:40:35.270056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:35.273706 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 510/1056] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:40:35 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:35.274613 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f390fe-a915-40e6-addd-e94b190340ff None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:35.385309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f390fe-a915-40e6-addd-e94b190340ff None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:35.425452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:35.426139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:35.426440 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:35.426719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:35.427432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] RBAC: Adding query filter params (name=image) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:35.427742 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:35.429181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:35.435332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 547/1057] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:35.441240 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:35.545741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a52511-4fd5-420f-a858-f8fa0e42df47 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:35.586065 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa411c2-0083-42ee-a595-82c6f7c01829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:35.586753 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa411c2-0083-42ee-a595-82c6f7c01829 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:35.587037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa411c2-0083-42ee-a595-82c6f7c01829 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:35.587377 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa411c2-0083-42ee-a595-82c6f7c01829 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:35.588138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa411c2-0083-42ee-a595-82c6f7c01829 None admin] RBAC: Adding query filter params (type=image) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:35.588394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa411c2-0083-42ee-a595-82c6f7c01829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:35.589805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa411c2-0083-42ee-a595-82c6f7c01829 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:35.595493 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 511/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:35.601225 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:35.630040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:35.712475 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0c219-3973-444f-a94c-8ffa15b4bd26 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:35.749867 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47db1a-2baa-4c46-8165-b5ee74cf61c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:35.750566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47db1a-2baa-4c46-8165-b5ee74cf61c3 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:35.750850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47db1a-2baa-4c46-8165-b5ee74cf61c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:35.751129 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47db1a-2baa-4c46-8165-b5ee74cf61c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:35.751861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47db1a-2baa-4c46-8165-b5ee74cf61c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:35.753444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47db1a-2baa-4c46-8165-b5ee74cf61c3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:35.755043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5773df-d704-41cb-8dfd-1c4610e95413 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:35.755334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5773df-d704-41cb-8dfd-1c4610e95413 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:35.755619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5773df-d704-41cb-8dfd-1c4610e95413 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:35.757134 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 512/1059] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:35.762546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ede9-7634-40aa-a814-153dc9eef52f None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:35.762781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ede9-7634-40aa-a814-153dc9eef52f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:35.763022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ede9-7634-40aa-a814-153dc9eef52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:35.776362 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 548/1060] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:40:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:35.783932 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-f051ede9-7634-40aa-a814-153dc9eef52f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:35.785284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-f051ede9-7634-40aa-a814-153dc9eef52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:35.832283 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:35.890600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 513/1061] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:35.914233 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47db1a-2baa-4c46-8165-b5ee74cf61c3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.022629 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:36.049086 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47db1a-2baa-4c46-8165-b5ee74cf61c3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:36.086871 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f01622-7787-4f4a-9e90-4f2706aad46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:36.087452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f01622-7787-4f4a-9e90-4f2706aad46c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.087740 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f01622-7787-4f4a-9e90-4f2706aad46c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.088020 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f01622-7787-4f4a-9e90-4f2706aad46c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.100138 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f01622-7787-4f4a-9e90-4f2706aad46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-a2f01622-7787-4f4a-9e90-4f2706aad46c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:36.105291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:36.107590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:36.109627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 549/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:40:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:36.117620 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ede9-7634-40aa-a814-153dc9eef52f None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.239388 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ede9-7634-40aa-a814-153dc9eef52f None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:36.277792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922c84-f58c-4eb2-b2e8-0513f6b35eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:36.278474 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922c84-f58c-4eb2-b2e8-0513f6b35eb9 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.278759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922c84-f58c-4eb2-b2e8-0513f6b35eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.279032 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922c84-f58c-4eb2-b2e8-0513f6b35eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.279766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922c84-f58c-4eb2-b2e8-0513f6b35eb9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:36.280068 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922c84-f58c-4eb2-b2e8-0513f6b35eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:36.281513 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922c84-f58c-4eb2-b2e8-0513f6b35eb9 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:36.291238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 514/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:40:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:36.296926 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f01622-7787-4f4a-9e90-4f2706aad46c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.345091 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a914-db91-4808-a49a-d73a1a7b3a25 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.345310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a914-db91-4808-a49a-d73a1a7b3a25 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.345561 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a914-db91-4808-a49a-d73a1a7b3a25 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.347295 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 515/1064] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:36.353038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e3a95-06d0-49b0-be55-e4d172093e36 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.353318 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e3a95-06d0-49b0-be55-e4d172093e36 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.353597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e3a95-06d0-49b0-be55-e4d172093e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.374335 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-d94e3a95-06d0-49b0-be55-e4d172093e36 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:36.375576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-d94e3a95-06d0-49b0-be55-e4d172093e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:36.404224 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:36.412725 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f01622-7787-4f4a-9e90-4f2706aad46c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:36.459677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:36.460368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.460652 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.460928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.471619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 516/1065] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:36.476386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:36.478288 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:36.480726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:36.494059 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e3a95-06d0-49b0-be55-e4d172093e36 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:36.497078 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:36.498654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:36.500077 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:36.500077 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:36.500077 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:36.502216 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 550/1066] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:36.507841 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.615652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e3a95-06d0-49b0-be55-e4d172093e36 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:36.620786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a054f-d16f-4931-bf0b-30a9c1fdc37a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:36.660581 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac829-c673-4879-81ad-8c08be0b64a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:36.660926 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:36.660926 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.662319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac829-c673-4879-81ad-8c08be0b64a1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.662486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.662636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac829-c673-4879-81ad-8c08be0b64a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.662842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.663008 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac829-c673-4879-81ad-8c08be0b64a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.663110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:36.663110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:36.663723 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:36.675423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bac829-c673-4879-81ad-8c08be0b64a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:36.677499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 551/1067] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-10bac829-c673-4879-81ad-8c08be0b64a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:36.681258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:36.682871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:40:36.684602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:36.685405 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.688638 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 517/1068] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:36.689611 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac829-c673-4879-81ad-8c08be0b64a1 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.715987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:36.757197 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:36.813586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac829-c673-4879-81ad-8c08be0b64a1 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:36.817449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb54c0f-7f24-4198-9c35-3bc5e8aa0a7c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:36.863202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:36.864187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.864187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.864376 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.865114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:36.865422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:36.866154 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:36.866853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:36.867043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:36.867206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:36.867455 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:36.881341 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 518/1069] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:36.881588 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:36.881588 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:36.884323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:36.892865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:36.894319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:36.895537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 552/1070] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:40:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:36.900437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:36.998515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:37.030174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c9b3d-e211-458f-b39b-d4bfd5e9d6b8 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:37.031931 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1401e-7f67-4777-9dc3-6c3775bbff02 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:37.048854 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:37.073244 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278149d-e254-467b-a805-21b0bd4a942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:37.073915 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278149d-e254-467b-a805-21b0bd4a942a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.074209 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278149d-e254-467b-a805-21b0bd4a942a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.074484 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278149d-e254-467b-a805-21b0bd4a942a None admin] PATH_INFO: `/v3/users/5986d7da093b41daa0ec9045dc3d8cd3` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.075735 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278149d-e254-467b-a805-21b0bd4a942a None admin] RBAC: Authorizing `identity:get_user(user_id=5986d7da093b41daa0ec9045dc3d8cd3)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:37.075994 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:37.076643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.076937 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.077202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278149d-e254-467b-a805-21b0bd4a942a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:37.077317 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.078024 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:37.078377 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:37.079941 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:37.082302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 519/1071] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:36 2022] GET /identity/v3/users/5986d7da093b41daa0ec9045dc3d8cd3 => generated 285 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:37.085641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278149d-e254-467b-a805-21b0bd4a942a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:37.108665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 553/1072] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:40:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:37.114462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:37.219987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278149d-e254-467b-a805-21b0bd4a942a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:37.237811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a7dc-4153-4791-9228-3d9ba55c782a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:37.264595 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddfd65-cdcb-47d8-ac9a-65b44bc06383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:37.265618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddfd65-cdcb-47d8-ac9a-65b44bc06383 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.265979 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddfd65-cdcb-47d8-ac9a-65b44bc06383 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.266211 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddfd65-cdcb-47d8-ac9a-65b44bc06383 None admin] PATH_INFO: `/v3/projects/cac2af9e3d3d4a8688161ec1886a3895` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.267526 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddfd65-cdcb-47d8-ac9a-65b44bc06383 None admin] RBAC: Authorizing `identity:get_project(project_id=cac2af9e3d3d4a8688161ec1886a3895)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:37.269103 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddfd65-cdcb-47d8-ac9a-65b44bc06383 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:37.271062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 520/1073] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:40:37 2022] GET /identity/v3/projects/cac2af9e3d3d4a8688161ec1886a3895 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:37.277472 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddfd65-cdcb-47d8-ac9a-65b44bc06383 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:37.284270 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:37.284962 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.285432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.285832 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.295533 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:37.297377 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:37.298769 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:37.315450 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:37.316707 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:37.317836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:37.317836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:37.317836 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:37.322540 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 554/1074] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:37.323451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:37.359833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:37.418690 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddfd65-cdcb-47d8-ac9a-65b44bc06383 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:37.469764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38604de9-902f-4ae1-a202-3b03d96b507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:37.470700 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604de9-902f-4ae1-a202-3b03d96b507d None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.471108 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604de9-902f-4ae1-a202-3b03d96b507d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.471555 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604de9-902f-4ae1-a202-3b03d96b507d None admin] PATH_INFO: `/v3/projects/cac2af9e3d3d4a8688161ec1886a3895/users/5986d7da093b41daa0ec9045dc3d8cd3/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.473881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38604de9-902f-4ae1-a202-3b03d96b507d None admin] RBAC: Authorizing `identity:create_grant(project_id=cac2af9e3d3d4a8688161ec1886a3895, user_id=5986d7da093b41daa0ec9045dc3d8cd3, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:37.475418 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38604de9-902f-4ae1-a202-3b03d96b507d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:37.475704 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac415f5-3cac-493f-9396-70df89ff63d5 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:37.492802 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 521/1075] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:40:37 2022] PUT /identity/v3/projects/cac2af9e3d3d4a8688161ec1886a3895/users/5986d7da093b41daa0ec9045dc3d8cd3/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:40:37.529811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:37.530811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.531412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.531859 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.532824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:37.533320 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:37.535049 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:37.556167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 555/1076] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:37.565312 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38604de9-902f-4ae1-a202-3b03d96b507d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:37.688296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:37.730969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38604de9-902f-4ae1-a202-3b03d96b507d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:37.768695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8e251-4211-4a6a-be77-75eae1f53ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:37.769566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e251-4211-4a6a-be77-75eae1f53ea1 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.769857 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e251-4211-4a6a-be77-75eae1f53ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.770259 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e251-4211-4a6a-be77-75eae1f53ea1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.781851 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8e251-4211-4a6a-be77-75eae1f53ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:37.784357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8e251-4211-4a6a-be77-75eae1f53ea1 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-27d8e251-4211-4a6a-be77-75eae1f53ea1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:37.801781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:37.803866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:37.805694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 522/1077] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:37.812298 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:37.894421 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee103b-daf2-44a6-9327-dc7af5d9c47d None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.894696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee103b-daf2-44a6-9327-dc7af5d9c47d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.894996 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee103b-daf2-44a6-9327-dc7af5d9c47d None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.896510 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 523/1078] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:37.901801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939ab2-9ec3-47d1-af9d-6024c93653bd None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.902151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939ab2-9ec3-47d1-af9d-6024c93653bd None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.902394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939ab2-9ec3-47d1-af9d-6024c93653bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.919395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:37.921698 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-e8939ab2-9ec3-47d1-af9d-6024c93653bd None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:37.922965 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-e8939ab2-9ec3-47d1-af9d-6024c93653bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:37.944984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b126ba4-8589-4470-afa2-f9cf1f9b726e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:37.985549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:37.986158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:37.986441 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:37.986749 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:37.987579 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:37.987973 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:37.989592 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:38.011033 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 556/1079] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:38.013205 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:38.018452 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 524/1080] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:38.045747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8939ab2-9ec3-47d1-af9d-6024c93653bd None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:38.122390 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:38.150160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1fc6-03f6-4b2c-aa25-4ba49ba769b2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:38.173875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8939ab2-9ec3-47d1-af9d-6024c93653bd None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:38.208499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d7bc-464a-428b-a9da-0b856cfdc58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:38.209677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d7bc-464a-428b-a9da-0b856cfdc58c None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:38.210225 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d7bc-464a-428b-a9da-0b856cfdc58c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:38.210726 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d7bc-464a-428b-a9da-0b856cfdc58c None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/90103f3776ce4eeb90bde66ed546aa87/roles/4b13ca2dcb004c4891a205a6f8d64042` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:38.214951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92d7bc-464a-428b-a9da-0b856cfdc58c None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=90103f3776ce4eeb90bde66ed546aa87, role_id=4b13ca2dcb004c4891a205a6f8d64042)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:38.217613 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92d7bc-464a-428b-a9da-0b856cfdc58c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:38.236300 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 557/1081] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:40:38 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/90103f3776ce4eeb90bde66ed546aa87/roles/4b13ca2dcb004c4891a205a6f8d64042 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:40:38.238476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c393a-67a9-47aa-ac97-24081423832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:38.238476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c393a-67a9-47aa-ac97-24081423832f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:38.238476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c393a-67a9-47aa-ac97-24081423832f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:38.239494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c393a-67a9-47aa-ac97-24081423832f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:38.382440 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c393a-67a9-47aa-ac97-24081423832f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:38.385650 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c393a-67a9-47aa-ac97-24081423832f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:38.502891 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:38.553214 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 525/1082] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:40:38.560167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d7bc-464a-428b-a9da-0b856cfdc58c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:38.566632 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:38.692178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d7bc-464a-428b-a9da-0b856cfdc58c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:38.732747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f7da0-00a7-4331-a4a5-f5266d740c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:38.733441 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f7da0-00a7-4331-a4a5-f5266d740c1e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:38.733742 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f7da0-00a7-4331-a4a5-f5266d740c1e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:38.733980 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f7da0-00a7-4331-a4a5-f5266d740c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:38.746527 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f7da0-00a7-4331-a4a5-f5266d740c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:38.748018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f7da0-00a7-4331-a4a5-f5266d740c1e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-0f2f7da0-00a7-4331-a4a5-f5266d740c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:38.765380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:38.766979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:38.768375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:38.768375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:38.768375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:38.768375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 558/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:38.773147 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c393a-67a9-47aa-ac97-24081423832f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:38.957749 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c393a-67a9-47aa-ac97-24081423832f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:39.004744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e8840-8080-4fe7-97be-6144fc67d303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:39.005504 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e8840-8080-4fe7-97be-6144fc67d303 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:39.005938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e8840-8080-4fe7-97be-6144fc67d303 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:39.006282 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e8840-8080-4fe7-97be-6144fc67d303 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:39.006843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e8840-8080-4fe7-97be-6144fc67d303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:39.007165 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e8840-8080-4fe7-97be-6144fc67d303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:39.008685 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e8840-8080-4fe7-97be-6144fc67d303 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:39.022854 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 526/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:39.813827 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e0e9-5e36-42f5-9eef-a9808a5b4528 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:39.814577 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e0e9-5e36-42f5-9eef-a9808a5b4528 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:39.814577 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e0e9-5e36-42f5-9eef-a9808a5b4528 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:39.815873 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 559/1085] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:39.823169 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d88d3-6010-463e-b06c-ab747fc7a9c1 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:39.823777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d88d3-6010-463e-b06c-ab747fc7a9c1 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:39.824281 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d88d3-6010-463e-b06c-ab747fc7a9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:39.844231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-ff3d88d3-6010-463e-b06c-ab747fc7a9c1 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:39.845434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-ff3d88d3-6010-463e-b06c-ab747fc7a9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:39.865034 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:39.916597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 527/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:39.939693 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e0e9-5e36-42f5-9eef-a9808a5b4528 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:40.004347 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:40.051808 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e0e9-5e36-42f5-9eef-a9808a5b4528 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:40.088127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f8dc7-b755-4664-85b6-25f683003707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:40.088802 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f8dc7-b755-4664-85b6-25f683003707 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:40.089085 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f8dc7-b755-4664-85b6-25f683003707 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:40.089361 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f8dc7-b755-4664-85b6-25f683003707 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:40.102300 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f8dc7-b755-4664-85b6-25f683003707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-a28f8dc7-b755-4664-85b6-25f683003707 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:40.107345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:40.109451 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:40:40.111798 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:40.113894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 560/1087] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:40:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:40.115017 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d88d3-6010-463e-b06c-ab747fc7a9c1 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:40.223374 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d88d3-6010-463e-b06c-ab747fc7a9c1 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:40.260518 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:40.261073 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:40.261348 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:40.261619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:40.262386 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:40.262695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:40.264107 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:40.273562 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 528/1088] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:40:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:40.279343 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f8dc7-b755-4664-85b6-25f683003707 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:40.386070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f8dc7-b755-4664-85b6-25f683003707 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:40.406499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:40.426310 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffbe55-1545-49b1-9657-f21fe5ef3bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:40.426979 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffbe55-1545-49b1-9657-f21fe5ef3bf7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:40.427257 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffbe55-1545-49b1-9657-f21fe5ef3bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:40.427532 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffbe55-1545-49b1-9657-f21fe5ef3bf7 None admin] PATH_INFO: `/v3/users/5986d7da093b41daa0ec9045dc3d8cd3` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:40.428728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffbe55-1545-49b1-9657-f21fe5ef3bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=5986d7da093b41daa0ec9045dc3d8cd3)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:40.430204 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffbe55-1545-49b1-9657-f21fe5ef3bf7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:40.431876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 561/1089] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:40 2022] GET /identity/v3/users/5986d7da093b41daa0ec9045dc3d8cd3 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:40.437814 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:40.457007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7da82-57c2-4c4c-bb27-981b4fcc23f9 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:40.457325 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7da82-57c2-4c4c-bb27-981b4fcc23f9 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:40.457554 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7da82-57c2-4c4c-bb27-981b4fcc23f9 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:40.458395 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:40.459196 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 562/1090] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:40:40.465355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a5514-6bab-4cea-b408-1e43a748ca05 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:40.465677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a5514-6bab-4cea-b408-1e43a748ca05 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:40.465909 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a5514-6bab-4cea-b408-1e43a748ca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:40.484845 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-5a4a5514-6bab-4cea-b408-1e43a748ca05 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:40.486084 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-5a4a5514-6bab-4cea-b408-1e43a748ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:40.558220 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bbce-9ce6-4c71-aa53-ec191c64429d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:40.562581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 563/1091] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:40:40.586300 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a5514-6bab-4cea-b408-1e43a748ca05 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:40.603754 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5c2b9-994e-44b7-bdb6-dab9cced8618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:40.604937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c2b9-994e-44b7-bdb6-dab9cced8618 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:40.605324 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c2b9-994e-44b7-bdb6-dab9cced8618 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:40.605703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c2b9-994e-44b7-bdb6-dab9cced8618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:40.692745 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5c2b9-994e-44b7-bdb6-dab9cced8618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:40.695294 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5c2b9-994e-44b7-bdb6-dab9cced8618 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:40.715002 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a5514-6bab-4cea-b408-1e43a748ca05 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:40.717035 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:40.736168 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:40.771471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861941da-6a2f-4b10-8290-d72c08397654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:40.772590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861941da-6a2f-4b10-8290-d72c08397654 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:40.773070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861941da-6a2f-4b10-8290-d72c08397654 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:40.773547 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861941da-6a2f-4b10-8290-d72c08397654 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:40.791913 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861941da-6a2f-4b10-8290-d72c08397654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-861941da-6a2f-4b10-8290-d72c08397654 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:40:40.802123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:40.804181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:40.810345 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 564/1092] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:40:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:40.812135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 529/1093] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:40:40.812806 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5c2b9-994e-44b7-bdb6-dab9cced8618 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:40.812990 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861941da-6a2f-4b10-8290-d72c08397654 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:40.941871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5c2b9-994e-44b7-bdb6-dab9cced8618 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:40.950348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861941da-6a2f-4b10-8290-d72c08397654 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:40.964084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:40.991081 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b0c4a-72b7-407b-b6b8-ed072d73cdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:40.992542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0c4a-72b7-407b-b6b8-ed072d73cdbd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:40.992934 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0c4a-72b7-407b-b6b8-ed072d73cdbd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:40.993371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0c4a-72b7-407b-b6b8-ed072d73cdbd None admin] PATH_INFO: `/v3/projects/cac2af9e3d3d4a8688161ec1886a3895` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:40.995221 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b0c4a-72b7-407b-b6b8-ed072d73cdbd None admin] RBAC: Authorizing `identity:get_project(project_id=cac2af9e3d3d4a8688161ec1886a3895)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:40.998223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b0c4a-72b7-407b-b6b8-ed072d73cdbd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:40.998962 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:40.999950 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:41.000397 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:41.000853 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:41.001174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 530/1094] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:40:40 2022] GET /identity/v3/projects/cac2af9e3d3d4a8688161ec1886a3895 => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:41.003266 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:41.003810 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:41.006399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:41.010570 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b0c4a-72b7-407b-b6b8-ed072d73cdbd None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:41.018985 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 565/1095] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:40:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:41.025648 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:41.074046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:41.142884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b0c4a-72b7-407b-b6b8-ed072d73cdbd None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:41.158801 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f06c1-8bfe-4023-b22d-3ae9e04fdba3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:41.185327 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:41.185967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:41.186267 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:41.186544 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:41.187345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=cac2af9e3d3d4a8688161ec1886a3895, user.id=5986d7da093b41daa0ec9045dc3d8cd3) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:41.187655 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:41.189218 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:41.195997 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 531/1096] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:41 2022] GET /identity/v3/role_assignments?user.id=5986d7da093b41daa0ec9045dc3d8cd3&scope.project.id=cac2af9e3d3d4a8688161ec1886a3895&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:40:41.198909 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:41.199595 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:41.199897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:41.200145 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:41.212741 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:41.214322 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:41.232113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:41.234294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:41.236382 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:41.236382 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:41.236382 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:41.236382 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 566/1097] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:40:41.241070 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:41.300509 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:41.360467 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4ed03-092c-45aa-add1-8496a9cbfc50 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:41.402527 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:41.403148 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:41.403424 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:41.403705 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:41.404426 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:41.404733 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:41.406190 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:41.419945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 532/1098] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:41.427692 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:41.590471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:41.612822 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75575094-8541-4e33-969c-5061c2e03a3e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:41.681076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:41.682293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:41.682786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:41.683294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:41.696344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:41.697848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:40:41.711469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:41.713760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:40:41.716068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 567/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:40:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:41.718734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:41.830485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65caff0-08c2-43a2-bea7-cf4a4568f380 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:41.842435 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:41.871408 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:41.871948 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:41.872225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:41.872502 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:41.873309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:41.873620 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:41.875121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:41.893895 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 533/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:40:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:41.899977 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:42.031840 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046cdd6-542a-4037-8b5e-5f9fe15e6235 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:42.067369 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306ec7e-cc94-4257-af7f-20d2d3225fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:42.068013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306ec7e-cc94-4257-af7f-20d2d3225fa2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:42.068294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306ec7e-cc94-4257-af7f-20d2d3225fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:42.068566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306ec7e-cc94-4257-af7f-20d2d3225fa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:42.087876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:42.140481 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306ec7e-cc94-4257-af7f-20d2d3225fa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:42.141955 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306ec7e-cc94-4257-af7f-20d2d3225fa2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:42.254522 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 568/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:40:42.260662 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:42.392006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226932-668d-4ff2-afdb-fed158178874 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:42.444912 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:42.445649 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:42.445958 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:42.446317 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:42.459661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:42.461210 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:40:42.479164 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:42.480903 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:40:42.482741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:40:42.482741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:40:42.482741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:42.482741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 534/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:42.491499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306ec7e-cc94-4257-af7f-20d2d3225fa2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:42.547003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:42.603858 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306ec7e-cc94-4257-af7f-20d2d3225fa2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:42.648340 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:42.649086 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:42.649377 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:42.649661 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:42.650439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:42.650753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:42.652200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:42.665956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 569/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:42.672308 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:42.701215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:42.820186 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d2af7-104f-49bb-97ed-36cc4897f9ad None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:42.871296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:42.872478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:42.872984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:42.873532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:42.884534 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:42.886064 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:40:42.899884 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:40:42.907569 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:40:42.908747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 535/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:40:42.909457 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:43.050061 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838d263-6550-4583-a296-6bdc4571cfa8 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:43.061203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:43.098874 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c7b23-f2a0-4841-806f-f09194d2a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:43.099710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c7b23-f2a0-4841-806f-f09194d2a0cc None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:43.100058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c7b23-f2a0-4841-806f-f09194d2a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:43.100428 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c7b23-f2a0-4841-806f-f09194d2a0cc None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:43.101662 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c7b23-f2a0-4841-806f-f09194d2a0cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:43.102241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c7b23-f2a0-4841-806f-f09194d2a0cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:43.104463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c7b23-f2a0-4841-806f-f09194d2a0cc None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:43.123380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 570/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:43.132389 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:43.230370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:43.257377 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877299d-eaa2-4d57-b0e2-84c91efd9533 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:43.297621 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eaca9-7f13-4957-b4b8-c8b8e3684efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:43.298445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eaca9-7f13-4957-b4b8-c8b8e3684efd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:43.298756 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eaca9-7f13-4957-b4b8-c8b8e3684efd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:43.299039 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eaca9-7f13-4957-b4b8-c8b8e3684efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:43.300078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09eaca9-7f13-4957-b4b8-c8b8e3684efd None admin] RBAC: Adding query filter params (role.id=4b13ca2dcb004c4891a205a6f8d64042, scope.project.id=813e8db425644f688e926aafe3249721, user.id=90103f3776ce4eeb90bde66ed546aa87) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:43.300483 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09eaca9-7f13-4957-b4b8-c8b8e3684efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:43.302313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09eaca9-7f13-4957-b4b8-c8b8e3684efd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:43.310225 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 536/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:43 2022] GET /identity/v3/role_assignments?user.id=90103f3776ce4eeb90bde66ed546aa87&scope.project.id=813e8db425644f688e926aafe3249721&role.id=4b13ca2dcb004c4891a205a6f8d64042 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:43.332158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f889f-23ef-411c-b51b-61ab4db03a60 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:43.332460 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f889f-23ef-411c-b51b-61ab4db03a60 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:43.332758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f889f-23ef-411c-b51b-61ab4db03a60 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:43.334419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 571/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:43.341066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15b6a-a07f-4992-9f8a-cd7f268615c3 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:43.341387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15b6a-a07f-4992-9f8a-cd7f268615c3 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:43.341749 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15b6a-a07f-4992-9f8a-cd7f268615c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:43.363993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-fda15b6a-a07f-4992-9f8a-cd7f268615c3 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:43.365482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-fda15b6a-a07f-4992-9f8a-cd7f268615c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:43.468365 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 537/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:43.513419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f889f-23ef-411c-b51b-61ab4db03a60 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:43.656394 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f889f-23ef-411c-b51b-61ab4db03a60 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:43.709621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e623d3a-ba28-4bf2-9460-c9459da8c679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:43.710940 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e623d3a-ba28-4bf2-9460-c9459da8c679 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:43.711703 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e623d3a-ba28-4bf2-9460-c9459da8c679 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:43.711978 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e623d3a-ba28-4bf2-9460-c9459da8c679 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:43.714695 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e623d3a-ba28-4bf2-9460-c9459da8c679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:43.717516 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e623d3a-ba28-4bf2-9460-c9459da8c679 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:43.720050 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 572/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:43.729439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda15b6a-a07f-4992-9f8a-cd7f268615c3 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:43.821822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:43.858986 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda15b6a-a07f-4992-9f8a-cd7f268615c3 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:43.903843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16884414-f988-4d3d-a281-bc0c41ed58fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:43.904519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16884414-f988-4d3d-a281-bc0c41ed58fe None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:43.904805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16884414-f988-4d3d-a281-bc0c41ed58fe None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:43.905129 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16884414-f988-4d3d-a281-bc0c41ed58fe None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:43.905948 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16884414-f988-4d3d-a281-bc0c41ed58fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:43.908155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16884414-f988-4d3d-a281-bc0c41ed58fe None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:43.934371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 538/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:40:43 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:40:46.227876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cd013-4534-4469-96b7-4b113cee1ea9 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:46.228459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cd013-4534-4469-96b7-4b113cee1ea9 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:46.228459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cd013-4534-4469-96b7-4b113cee1ea9 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:46.229984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 573/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:46.236646 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ba81f-2b5f-4f81-b527-b11e53a6d227 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:46.237357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ba81f-2b5f-4f81-b527-b11e53a6d227 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:46.237357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ba81f-2b5f-4f81-b527-b11e53a6d227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:46.258058 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-6c0ba81f-2b5f-4f81-b527-b11e53a6d227 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:46.259370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-6c0ba81f-2b5f-4f81-b527-b11e53a6d227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:46.339720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 539/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:46.365728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cd013-4534-4469-96b7-4b113cee1ea9 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:46.409932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:46.493335 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cd013-4534-4469-96b7-4b113cee1ea9 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:46.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d50990-9324-4f45-b195-f9ce17114a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:46.536248 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50990-9324-4f45-b195-f9ce17114a34 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:46.536533 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50990-9324-4f45-b195-f9ce17114a34 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:46.536814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50990-9324-4f45-b195-f9ce17114a34 None admin] PATH_INFO: `/v3/roles/8d072b58656b442f922c3287a5430603` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:46.538505 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d50990-9324-4f45-b195-f9ce17114a34 None admin] RBAC: Authorizing `identity:get_role(role_id=8d072b58656b442f922c3287a5430603)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:46.540177 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d50990-9324-4f45-b195-f9ce17114a34 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:46.541654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 574/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:46 2022] GET /identity/v3/roles/8d072b58656b442f922c3287a5430603 => generated 223 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:46.549204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ba81f-2b5f-4f81-b527-b11e53a6d227 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:46.621839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:46.698010 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ba81f-2b5f-4f81-b527-b11e53a6d227 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:46.737135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9373c2-22f6-48ef-b545-2aaaf6fb1077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:46.737808 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9373c2-22f6-48ef-b545-2aaaf6fb1077 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:46.738093 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9373c2-22f6-48ef-b545-2aaaf6fb1077 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:46.738403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9373c2-22f6-48ef-b545-2aaaf6fb1077 None admin] PATH_INFO: `/v3/users/bd31a9d410f0466890d0b235903f1d42` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:46.757842 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9373c2-22f6-48ef-b545-2aaaf6fb1077 None admin] RBAC: Authorizing `identity:get_user(user_id=bd31a9d410f0466890d0b235903f1d42)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:46.760542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9373c2-22f6-48ef-b545-2aaaf6fb1077 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:46.763604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 540/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:46 2022] GET /identity/v3/users/bd31a9d410f0466890d0b235903f1d42 => generated 286 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:46.770476 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d50990-9324-4f45-b195-f9ce17114a34 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:46.878282 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d50990-9324-4f45-b195-f9ce17114a34 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:46.880855 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:46.920635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00610103-4ccd-4ed9-a9d1-11bfd1379b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:46.921285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00610103-4ccd-4ed9-a9d1-11bfd1379b47 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:46.921568 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00610103-4ccd-4ed9-a9d1-11bfd1379b47 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:46.921845 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00610103-4ccd-4ed9-a9d1-11bfd1379b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:47.002746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00610103-4ccd-4ed9-a9d1-11bfd1379b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:47.004299 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00610103-4ccd-4ed9-a9d1-11bfd1379b47 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:47.117023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 575/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:40:47.124079 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9373c2-22f6-48ef-b545-2aaaf6fb1077 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:47.241512 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9373c2-22f6-48ef-b545-2aaaf6fb1077 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:47.260801 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:47.287176 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea130f7f-5a14-4197-a92f-af3c53a2835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:47.288007 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea130f7f-5a14-4197-a92f-af3c53a2835e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:47.288453 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea130f7f-5a14-4197-a92f-af3c53a2835e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:47.288811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea130f7f-5a14-4197-a92f-af3c53a2835e None admin] PATH_INFO: `/v3/projects/cac2af9e3d3d4a8688161ec1886a3895` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:47.290289 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea130f7f-5a14-4197-a92f-af3c53a2835e None admin] RBAC: Authorizing `identity:get_project(project_id=cac2af9e3d3d4a8688161ec1886a3895)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:47.292177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea130f7f-5a14-4197-a92f-af3c53a2835e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:47.294177 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 541/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:40:47 2022] GET /identity/v3/projects/cac2af9e3d3d4a8688161ec1886a3895 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:47.300951 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00610103-4ccd-4ed9-a9d1-11bfd1379b47 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:47.304456 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:47.414058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00610103-4ccd-4ed9-a9d1-11bfd1379b47 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:47.463377 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf09a0f-4504-4ca9-b0c5-769101c2d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:47.464166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf09a0f-4504-4ca9-b0c5-769101c2d652 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:47.464472 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf09a0f-4504-4ca9-b0c5-769101c2d652 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:47.464890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf09a0f-4504-4ca9-b0c5-769101c2d652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:47.465825 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf09a0f-4504-4ca9-b0c5-769101c2d652 None admin] RBAC: Adding query filter params (role.id=8d072b58656b442f922c3287a5430603, scope.project.id=cac2af9e3d3d4a8688161ec1886a3895, user.id=bd31a9d410f0466890d0b235903f1d42) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:47.466304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf09a0f-4504-4ca9-b0c5-769101c2d652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:47.467783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf09a0f-4504-4ca9-b0c5-769101c2d652 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:47.475318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 576/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:47 2022] GET /identity/v3/role_assignments?user.id=bd31a9d410f0466890d0b235903f1d42&scope.project.id=cac2af9e3d3d4a8688161ec1886a3895&role.id=8d072b58656b442f922c3287a5430603 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:49.482403 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961e322-610d-43e8-b1c3-08ca97106205 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:49.482985 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961e322-610d-43e8-b1c3-08ca97106205 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:49.482985 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961e322-610d-43e8-b1c3-08ca97106205 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:49.484590 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 542/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:49.492201 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3f21b-a7ab-4876-811f-f2fde49d628d None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:49.492789 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3f21b-a7ab-4876-811f-f2fde49d628d None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:49.493076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3f21b-a7ab-4876-811f-f2fde49d628d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:49.516710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-bff3f21b-a7ab-4876-811f-f2fde49d628d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:49.518064 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-bff3f21b-a7ab-4876-811f-f2fde49d628d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:49.598710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:49.605458 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 577/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:49.629846 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961e322-610d-43e8-b1c3-08ca97106205 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:49.761157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961e322-610d-43e8-b1c3-08ca97106205 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:49.803238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c6481-8bc1-4278-8bc6-395a34595ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:49.803984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6481-8bc1-4278-8bc6-395a34595ee2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:49.804269 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6481-8bc1-4278-8bc6-395a34595ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:49.804555 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6481-8bc1-4278-8bc6-395a34595ee2 None admin] PATH_INFO: `/v3/roles/8d072b58656b442f922c3287a5430603` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:49.806302 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c6481-8bc1-4278-8bc6-395a34595ee2 None admin] RBAC: Authorizing `identity:get_role(role_id=8d072b58656b442f922c3287a5430603)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:49.807802 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c6481-8bc1-4278-8bc6-395a34595ee2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:49.809437 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 543/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:49 2022] GET /identity/v3/roles/8d072b58656b442f922c3287a5430603 => generated 223 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:49.816071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3f21b-a7ab-4876-811f-f2fde49d628d None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:49.939627 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3f21b-a7ab-4876-811f-f2fde49d628d None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:49.990007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8877-a6e9-402f-b152-48dff1bcf38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:49.990833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8877-a6e9-402f-b152-48dff1bcf38a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:49.991304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8877-a6e9-402f-b152-48dff1bcf38a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:49.991614 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8877-a6e9-402f-b152-48dff1bcf38a None admin] PATH_INFO: `/v3/users/bd31a9d410f0466890d0b235903f1d42` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:49.993463 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc8877-a6e9-402f-b152-48dff1bcf38a None admin] RBAC: Authorizing `identity:get_user(user_id=bd31a9d410f0466890d0b235903f1d42)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:49.995700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc8877-a6e9-402f-b152-48dff1bcf38a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:49.998751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 578/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:49 2022] GET /identity/v3/users/bd31a9d410f0466890d0b235903f1d42 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:50.004932 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c6481-8bc1-4278-8bc6-395a34595ee2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:50.068491 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:50.160827 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c6481-8bc1-4278-8bc6-395a34595ee2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:50.209319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25259096-014c-4468-863f-4e2fd7cd560f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:50.209997 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25259096-014c-4468-863f-4e2fd7cd560f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:50.210328 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25259096-014c-4468-863f-4e2fd7cd560f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:50.210605 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25259096-014c-4468-863f-4e2fd7cd560f None admin] PATH_INFO: `/v3/projects/cac2af9e3d3d4a8688161ec1886a3895` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:50.211897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25259096-014c-4468-863f-4e2fd7cd560f None admin] RBAC: Authorizing `identity:get_project(project_id=cac2af9e3d3d4a8688161ec1886a3895)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:50.213925 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25259096-014c-4468-863f-4e2fd7cd560f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:50.216313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 544/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:40:50 2022] GET /identity/v3/projects/cac2af9e3d3d4a8688161ec1886a3895 => generated 311 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:50.223135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8877-a6e9-402f-b152-48dff1bcf38a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:50.302302 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:50.345423 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8877-a6e9-402f-b152-48dff1bcf38a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:50.384359 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f4384-80fe-4ca0-8d11-e4a60edb2b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:50.385151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4384-80fe-4ca0-8d11-e4a60edb2b39 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:50.385467 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4384-80fe-4ca0-8d11-e4a60edb2b39 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:50.385773 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4384-80fe-4ca0-8d11-e4a60edb2b39 None admin] PATH_INFO: `/v3/projects/cac2af9e3d3d4a8688161ec1886a3895/users/bd31a9d410f0466890d0b235903f1d42/roles/8d072b58656b442f922c3287a5430603` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:50.388487 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f4384-80fe-4ca0-8d11-e4a60edb2b39 None admin] RBAC: Authorizing `identity:create_grant(project_id=cac2af9e3d3d4a8688161ec1886a3895, user_id=bd31a9d410f0466890d0b235903f1d42, role_id=8d072b58656b442f922c3287a5430603)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:50.390265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f4384-80fe-4ca0-8d11-e4a60edb2b39 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:50.405513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 579/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:40:50 2022] PUT /identity/v3/projects/cac2af9e3d3d4a8688161ec1886a3895/users/bd31a9d410f0466890d0b235903f1d42/roles/8d072b58656b442f922c3287a5430603 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:40:52.473776 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1725cb-560a-46ba-af7d-d0a2ac573dd9 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:52.474456 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1725cb-560a-46ba-af7d-d0a2ac573dd9 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:52.474456 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1725cb-560a-46ba-af7d-d0a2ac573dd9 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:52.475816 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 545/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:52.481600 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7bd2-2319-41d4-b3be-f6cd3c61f6b9 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:52.482070 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7bd2-2319-41d4-b3be-f6cd3c61f6b9 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:52.482235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7bd2-2319-41d4-b3be-f6cd3c61f6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:52.503336 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-e3da7bd2-2319-41d4-b3be-f6cd3c61f6b9 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:52.504572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-e3da7bd2-2319-41d4-b3be-f6cd3c61f6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:52.582773 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 580/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:52.606257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1725cb-560a-46ba-af7d-d0a2ac573dd9 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:52.717489 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1725cb-560a-46ba-af7d-d0a2ac573dd9 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:52.742807 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:52.758940 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b960d3c-f7ea-426a-8660-354ea330b08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:52.759599 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960d3c-f7ea-426a-8660-354ea330b08c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:52.759886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960d3c-f7ea-426a-8660-354ea330b08c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:52.760160 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960d3c-f7ea-426a-8660-354ea330b08c None admin] PATH_INFO: `/v3/roles/8d072b58656b442f922c3287a5430603` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:52.761694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b960d3c-f7ea-426a-8660-354ea330b08c None admin] RBAC: Authorizing `identity:get_role(role_id=8d072b58656b442f922c3287a5430603)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:52.763137 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b960d3c-f7ea-426a-8660-354ea330b08c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:52.764527 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 546/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:52 2022] GET /identity/v3/roles/8d072b58656b442f922c3287a5430603 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:52.770787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da7bd2-2319-41d4-b3be-f6cd3c61f6b9 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:52.833814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:52.880068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da7bd2-2319-41d4-b3be-f6cd3c61f6b9 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:52.917964 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ebcfd-50b3-4cc1-8f8e-cbb1dc4cc8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:52.918665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ebcfd-50b3-4cc1-8f8e-cbb1dc4cc8be None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:52.918955 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ebcfd-50b3-4cc1-8f8e-cbb1dc4cc8be None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:52.919229 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ebcfd-50b3-4cc1-8f8e-cbb1dc4cc8be None admin] PATH_INFO: `/v3/users/bd31a9d410f0466890d0b235903f1d42` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:52.920461 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ebcfd-50b3-4cc1-8f8e-cbb1dc4cc8be None admin] RBAC: Authorizing `identity:get_user(user_id=bd31a9d410f0466890d0b235903f1d42)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:52.921919 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ebcfd-50b3-4cc1-8f8e-cbb1dc4cc8be None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:52.923632 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 581/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:40:52 2022] GET /identity/v3/users/bd31a9d410f0466890d0b235903f1d42 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:52.930149 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b960d3c-f7ea-426a-8660-354ea330b08c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:53.040246 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b960d3c-f7ea-426a-8660-354ea330b08c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:53.076061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450ed8-4908-4f79-b094-42f88a33cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:53.076689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450ed8-4908-4f79-b094-42f88a33cd9d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:53.076965 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450ed8-4908-4f79-b094-42f88a33cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:53.077242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450ed8-4908-4f79-b094-42f88a33cd9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:53.133892 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:53.148119 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450ed8-4908-4f79-b094-42f88a33cd9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:53.149633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450ed8-4908-4f79-b094-42f88a33cd9d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:53.254186 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 547/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:40:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:40:53.260480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ebcfd-50b3-4cc1-8f8e-cbb1dc4cc8be None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:53.366038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ebcfd-50b3-4cc1-8f8e-cbb1dc4cc8be None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:53.389808 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:53.401676 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec008fd-9403-4079-b933-b57ca929cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:53.402410 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec008fd-9403-4079-b933-b57ca929cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:53.402642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec008fd-9403-4079-b933-b57ca929cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:53.402924 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec008fd-9403-4079-b933-b57ca929cfc9 None admin] PATH_INFO: `/v3/projects/cac2af9e3d3d4a8688161ec1886a3895` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:53.404095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec008fd-9403-4079-b933-b57ca929cfc9 None admin] RBAC: Authorizing `identity:get_project(project_id=cac2af9e3d3d4a8688161ec1886a3895)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:53.405545 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec008fd-9403-4079-b933-b57ca929cfc9 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:53.407292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 582/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:40:53 2022] GET /identity/v3/projects/cac2af9e3d3d4a8688161ec1886a3895 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:53.413785 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450ed8-4908-4f79-b094-42f88a33cd9d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:53.506340 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:53.520699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450ed8-4908-4f79-b094-42f88a33cd9d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:53.557494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e22103-f8ab-4de8-801b-faa4a7f69d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:53.558150 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e22103-f8ab-4de8-801b-faa4a7f69d18 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:53.558433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e22103-f8ab-4de8-801b-faa4a7f69d18 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:53.558703 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e22103-f8ab-4de8-801b-faa4a7f69d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:53.559489 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e22103-f8ab-4de8-801b-faa4a7f69d18 None admin] RBAC: Adding query filter params (role.id=8d072b58656b442f922c3287a5430603, scope.project.id=cac2af9e3d3d4a8688161ec1886a3895, user.id=bd31a9d410f0466890d0b235903f1d42) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:40:53.559796 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e22103-f8ab-4de8-801b-faa4a7f69d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:53.561242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e22103-f8ab-4de8-801b-faa4a7f69d18 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:53.567054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 548/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:40:53 2022] GET /identity/v3/role_assignments?user.id=bd31a9d410f0466890d0b235903f1d42&scope.project.id=cac2af9e3d3d4a8688161ec1886a3895&role.id=8d072b58656b442f922c3287a5430603 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:55.592425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0761-ab28-4ea0-9a0a-a97e933d86f7 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:55.593039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0761-ab28-4ea0-9a0a-a97e933d86f7 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:55.593039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0761-ab28-4ea0-9a0a-a97e933d86f7 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:55.594368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 583/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:55.600270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b7895-20c1-4dc3-a74b-2208c9f7c77e None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:55.600714 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b7895-20c1-4dc3-a74b-2208c9f7c77e None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:55.600816 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b7895-20c1-4dc3-a74b-2208c9f7c77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:55.620084 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-7f4b7895-20c1-4dc3-a74b-2208c9f7c77e None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:55.621313 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-7f4b7895-20c1-4dc3-a74b-2208c9f7c77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:55.698601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 549/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:55.722491 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b0761-ab28-4ea0-9a0a-a97e933d86f7 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:55.840418 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b0761-ab28-4ea0-9a0a-a97e933d86f7 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:55.876896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac2152-a68b-4ab3-bfe0-8abd166d5dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:55.877568 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2152-a68b-4ab3-bfe0-8abd166d5dc3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:55.877835 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2152-a68b-4ab3-bfe0-8abd166d5dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:55.878127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2152-a68b-4ab3-bfe0-8abd166d5dc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:55.879377 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac2152-a68b-4ab3-bfe0-8abd166d5dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:55.880842 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac2152-a68b-4ab3-bfe0-8abd166d5dc3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:55.882538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 584/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:55.888981 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b7895-20c1-4dc3-a74b-2208c9f7c77e None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:55.976523 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:55.996312 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b7895-20c1-4dc3-a74b-2208c9f7c77e None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:56.029879 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49add26d-58b0-4a8f-8e09-61f95c9eef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:56.030545 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49add26d-58b0-4a8f-8e09-61f95c9eef45 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:56.030825 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49add26d-58b0-4a8f-8e09-61f95c9eef45 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:56.031098 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49add26d-58b0-4a8f-8e09-61f95c9eef45 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:56.031788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49add26d-58b0-4a8f-8e09-61f95c9eef45 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:56.033249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49add26d-58b0-4a8f-8e09-61f95c9eef45 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:56.062475 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 550/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:40:55 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:40:58.100498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abca40a-c4d6-4d51-8643-795bc4488e50 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:58.101151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abca40a-c4d6-4d51-8643-795bc4488e50 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:58.101151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abca40a-c4d6-4d51-8643-795bc4488e50 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:58.102348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 585/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:40:58.107916 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc8293-b830-42b8-b8ad-518eb46b4793 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:58.108337 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc8293-b830-42b8-b8ad-518eb46b4793 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:58.108482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc8293-b830-42b8-b8ad-518eb46b4793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:58.128819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-02fc8293-b830-42b8-b8ad-518eb46b4793 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:40:58.130070 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-02fc8293-b830-42b8-b8ad-518eb46b4793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:40:58.206078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 551/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:40:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:40:58.230915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abca40a-c4d6-4d51-8643-795bc4488e50 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:58.297167 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:58.343172 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abca40a-c4d6-4d51-8643-795bc4488e50 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:58.379265 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905cb0e-40ec-4687-a0b5-6fbb17e9418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:58.379932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905cb0e-40ec-4687-a0b5-6fbb17e9418a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:58.380485 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905cb0e-40ec-4687-a0b5-6fbb17e9418a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:58.380761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905cb0e-40ec-4687-a0b5-6fbb17e9418a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:58.382141 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905cb0e-40ec-4687-a0b5-6fbb17e9418a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:58.383761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905cb0e-40ec-4687-a0b5-6fbb17e9418a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:58.385609 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 586/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:40:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:40:58.393450 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc8293-b830-42b8-b8ad-518eb46b4793 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:40:58.464961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:40:58.505803 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc8293-b830-42b8-b8ad-518eb46b4793 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:40:58.541876 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2ac76-ecf2-4efc-aca5-84b32731b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:40:58.542543 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2ac76-ecf2-4efc-aca5-84b32731b146 None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:40:58.542821 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2ac76-ecf2-4efc-aca5-84b32731b146 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:40:58.543095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2ac76-ecf2-4efc-aca5-84b32731b146 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:40:58.543795 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2ac76-ecf2-4efc-aca5-84b32731b146 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:40:58.545274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2ac76-ecf2-4efc-aca5-84b32731b146 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:40:58.571984 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 552/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:40:58 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:41:00.674304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d050-ae81-4560-ae41-ada1b896245b None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:00.674900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d050-ae81-4560-ae41-ada1b896245b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:00.674900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d050-ae81-4560-ae41-ada1b896245b None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:00.676214 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 587/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:00.682200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1262d1-aba5-4f92-8bf0-8fec48557c13 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:00.682783 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1262d1-aba5-4f92-8bf0-8fec48557c13 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:00.682925 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1262d1-aba5-4f92-8bf0-8fec48557c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:00.702325 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-3a1262d1-aba5-4f92-8bf0-8fec48557c13 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:00.703549 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-3a1262d1-aba5-4f92-8bf0-8fec48557c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:00.778208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 553/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:00.804755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12d050-ae81-4560-ae41-ada1b896245b None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:00.917905 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12d050-ae81-4560-ae41-ada1b896245b None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:00.947158 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:00.956143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e37d7-03a5-4e2f-80d5-a5ac91b44c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:00.956936 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e37d7-03a5-4e2f-80d5-a5ac91b44c4f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:00.957317 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e37d7-03a5-4e2f-80d5-a5ac91b44c4f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:00.957704 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e37d7-03a5-4e2f-80d5-a5ac91b44c4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:00.968868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e37d7-03a5-4e2f-80d5-a5ac91b44c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-f47e37d7-03a5-4e2f-80d5-a5ac91b44c4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:41:00.974383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:41:00.975699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:41:00.976976 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 588/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:41:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:41:00.982021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1262d1-aba5-4f92-8bf0-8fec48557c13 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:01.039848 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:01.099727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1262d1-aba5-4f92-8bf0-8fec48557c13 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:01.136729 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:01.137367 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:01.137648 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:01.137913 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:01.138755 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:01.139065 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:01.140469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:01.149215 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 554/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:41:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:01.158562 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e37d7-03a5-4e2f-80d5-a5ac91b44c4f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:01.277511 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e37d7-03a5-4e2f-80d5-a5ac91b44c4f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:01.318028 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ea292-3f8a-46da-a0e6-226017fcb5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:01.318736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ea292-3f8a-46da-a0e6-226017fcb5bf None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:01.319393 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ea292-3f8a-46da-a0e6-226017fcb5bf None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:01.319679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ea292-3f8a-46da-a0e6-226017fcb5bf None admin] PATH_INFO: `/v3/users/fb9a0b79e88d40ca812c6635b349b0ab` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:01.334547 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ea292-3f8a-46da-a0e6-226017fcb5bf None admin] RBAC: Authorizing `identity:get_user(user_id=fb9a0b79e88d40ca812c6635b349b0ab)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:01.336045 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ea292-3f8a-46da-a0e6-226017fcb5bf None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:01.337821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 589/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:41:01 2022] GET /identity/v3/users/fb9a0b79e88d40ca812c6635b349b0ab => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:01.344223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:01.448040 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c9044-5c13-4146-9c9c-f2b263cf792c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:01.461738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:01.484562 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91fe0d-0c97-4305-9c2d-63a884389371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:01.485194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91fe0d-0c97-4305-9c2d-63a884389371 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:01.485470 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91fe0d-0c97-4305-9c2d-63a884389371 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:01.485742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91fe0d-0c97-4305-9c2d-63a884389371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:01.555238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91fe0d-0c97-4305-9c2d-63a884389371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:01.556633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91fe0d-0c97-4305-9c2d-63a884389371 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:01.667938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 555/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:41:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:41:01.674336 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ea292-3f8a-46da-a0e6-226017fcb5bf None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:01.724118 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:01.782688 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ea292-3f8a-46da-a0e6-226017fcb5bf None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:01.822865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0479d-7796-4822-84d3-764c72e93163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:01.823436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0479d-7796-4822-84d3-764c72e93163 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:01.823716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0479d-7796-4822-84d3-764c72e93163 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:01.823989 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0479d-7796-4822-84d3-764c72e93163 None admin] PATH_INFO: `/v3/projects/fd9646b66dad4f42b2e9fcc4bc47c673` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:01.825228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0479d-7796-4822-84d3-764c72e93163 None admin] RBAC: Authorizing `identity:get_project(project_id=fd9646b66dad4f42b2e9fcc4bc47c673)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:01.826706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0479d-7796-4822-84d3-764c72e93163 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:01.828397 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 590/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:41:01 2022] GET /identity/v3/projects/fd9646b66dad4f42b2e9fcc4bc47c673 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:01.835042 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91fe0d-0c97-4305-9c2d-63a884389371 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:01.850405 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:01.946252 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91fe0d-0c97-4305-9c2d-63a884389371 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:01.984509 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28111e75-e9c8-4207-936b-dc8cb04bbf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:01.985197 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28111e75-e9c8-4207-936b-dc8cb04bbf43 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:01.985469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28111e75-e9c8-4207-936b-dc8cb04bbf43 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:01.985750 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28111e75-e9c8-4207-936b-dc8cb04bbf43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:01.986596 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28111e75-e9c8-4207-936b-dc8cb04bbf43 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=fd9646b66dad4f42b2e9fcc4bc47c673, user.id=fb9a0b79e88d40ca812c6635b349b0ab) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:01.986913 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28111e75-e9c8-4207-936b-dc8cb04bbf43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:01.988518 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28111e75-e9c8-4207-936b-dc8cb04bbf43 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:01.994674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 556/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:41:01 2022] GET /identity/v3/role_assignments?user.id=fb9a0b79e88d40ca812c6635b349b0ab&scope.project.id=fd9646b66dad4f42b2e9fcc4bc47c673&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:04.049291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438cf4-a9ed-482e-b731-818db5d7f173 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:04.049862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438cf4-a9ed-482e-b731-818db5d7f173 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:04.049862 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438cf4-a9ed-482e-b731-818db5d7f173 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:04.051251 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 591/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:04.057249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee854-ce09-414f-bc3b-0caef727331a None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:04.057759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee854-ce09-414f-bc3b-0caef727331a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:04.057759 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee854-ce09-414f-bc3b-0caef727331a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:04.077716 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-807ee854-ce09-414f-bc3b-0caef727331a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:04.078975 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-807ee854-ce09-414f-bc3b-0caef727331a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:04.150743 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 557/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:04.173314 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a438cf4-a9ed-482e-b731-818db5d7f173 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:04.289625 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a438cf4-a9ed-482e-b731-818db5d7f173 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:04.305150 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:04.332916 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d565c6-8034-4797-a0fa-a2da350b2eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:04.333581 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d565c6-8034-4797-a0fa-a2da350b2eda None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:04.333869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d565c6-8034-4797-a0fa-a2da350b2eda None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:04.334171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d565c6-8034-4797-a0fa-a2da350b2eda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:04.345549 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d565c6-8034-4797-a0fa-a2da350b2eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-23d565c6-8034-4797-a0fa-a2da350b2eda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:41:04.350718 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:41:04.352781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:41:04.354719 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 592/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:41:04.358781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ee854-ce09-414f-bc3b-0caef727331a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:04.369011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:04.485138 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ee854-ce09-414f-bc3b-0caef727331a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:04.525422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:04.526092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:04.526447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:04.526760 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:04.527587 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:04.527951 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:04.529484 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:04.538692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 558/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:41:04.545010 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d565c6-8034-4797-a0fa-a2da350b2eda None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:04.649955 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d565c6-8034-4797-a0fa-a2da350b2eda None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:04.684919 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767d489-aee8-4cae-8fbf-07f92e364963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:04.685583 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767d489-aee8-4cae-8fbf-07f92e364963 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:04.685864 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767d489-aee8-4cae-8fbf-07f92e364963 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:04.686161 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767d489-aee8-4cae-8fbf-07f92e364963 None admin] PATH_INFO: `/v3/users/fb9a0b79e88d40ca812c6635b349b0ab` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:04.687418 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767d489-aee8-4cae-8fbf-07f92e364963 None admin] RBAC: Authorizing `identity:get_user(user_id=fb9a0b79e88d40ca812c6635b349b0ab)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:04.688874 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767d489-aee8-4cae-8fbf-07f92e364963 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:04.690572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 593/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:41:04 2022] GET /identity/v3/users/fb9a0b79e88d40ca812c6635b349b0ab => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:04.697326 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:04.770054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:04.810991 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03823842-c8c8-4f7f-83d6-bcf681181b63 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:04.846680 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d508b4-59d7-4062-80fa-47877addc9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:04.847343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d508b4-59d7-4062-80fa-47877addc9ca None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:04.847615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d508b4-59d7-4062-80fa-47877addc9ca None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:04.847947 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d508b4-59d7-4062-80fa-47877addc9ca None admin] PATH_INFO: `/v3/projects/fd9646b66dad4f42b2e9fcc4bc47c673` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:04.849268 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d508b4-59d7-4062-80fa-47877addc9ca None admin] RBAC: Authorizing `identity:get_project(project_id=fd9646b66dad4f42b2e9fcc4bc47c673)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:04.850800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d508b4-59d7-4062-80fa-47877addc9ca None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:04.852546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 559/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:41:04 2022] GET /identity/v3/projects/fd9646b66dad4f42b2e9fcc4bc47c673 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:04.858921 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767d489-aee8-4cae-8fbf-07f92e364963 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:04.904479 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:04.977195 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767d489-aee8-4cae-8fbf-07f92e364963 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:05.018756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e4009-2982-45f7-b893-9aceb2830a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:05.019482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4009-2982-45f7-b893-9aceb2830a84 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:05.019788 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4009-2982-45f7-b893-9aceb2830a84 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:05.020069 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4009-2982-45f7-b893-9aceb2830a84 None admin] PATH_INFO: `/v3/projects/fd9646b66dad4f42b2e9fcc4bc47c673/users/fb9a0b79e88d40ca812c6635b349b0ab/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:05.022163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e4009-2982-45f7-b893-9aceb2830a84 None admin] RBAC: Authorizing `identity:create_grant(project_id=fd9646b66dad4f42b2e9fcc4bc47c673, user_id=fb9a0b79e88d40ca812c6635b349b0ab, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:05.023632 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e4009-2982-45f7-b893-9aceb2830a84 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:05.038838 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 594/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:41:04 2022] PUT /identity/v3/projects/fd9646b66dad4f42b2e9fcc4bc47c673/users/fb9a0b79e88d40ca812c6635b349b0ab/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:41:07.098009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0035b-7d44-4070-ae2f-a262dea93990 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:07.098693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0035b-7d44-4070-ae2f-a262dea93990 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:07.098693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0035b-7d44-4070-ae2f-a262dea93990 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:07.099937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 560/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:07.105796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910915b-27a3-4cc8-a0f8-0566e106384f None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:07.106395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910915b-27a3-4cc8-a0f8-0566e106384f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:07.106395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910915b-27a3-4cc8-a0f8-0566e106384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:07.126431 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-5910915b-27a3-4cc8-a0f8-0566e106384f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:07.127668 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-5910915b-27a3-4cc8-a0f8-0566e106384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:07.200088 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 595/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:07.223644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0035b-7d44-4070-ae2f-a262dea93990 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:07.335886 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0035b-7d44-4070-ae2f-a262dea93990 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:07.372881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7a904-b139-44da-99b0-917178a4eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:07.373462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7a904-b139-44da-99b0-917178a4eda5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:07.373747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7a904-b139-44da-99b0-917178a4eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:07.374027 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7a904-b139-44da-99b0-917178a4eda5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:07.377077 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:07.386648 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7a904-b139-44da-99b0-917178a4eda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-7ae7a904-b139-44da-99b0-917178a4eda5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:41:07.391656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:41:07.393309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:41:07.394875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 561/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:41:07.399308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910915b-27a3-4cc8-a0f8-0566e106384f None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:07.431916 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:07.521304 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910915b-27a3-4cc8-a0f8-0566e106384f None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:07.572498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:07.573156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:07.573437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:07.573712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:07.574479 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:07.574790 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:07.576164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:07.589787 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 596/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:07.591371 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7a904-b139-44da-99b0-917178a4eda5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:07.707345 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7a904-b139-44da-99b0-917178a4eda5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:07.741793 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c771e-d047-4aba-8c28-4cb1db6c5b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:07.742501 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c771e-d047-4aba-8c28-4cb1db6c5b79 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:07.742788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c771e-d047-4aba-8c28-4cb1db6c5b79 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:07.743065 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c771e-d047-4aba-8c28-4cb1db6c5b79 None admin] PATH_INFO: `/v3/users/fb9a0b79e88d40ca812c6635b349b0ab` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:07.744257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c771e-d047-4aba-8c28-4cb1db6c5b79 None admin] RBAC: Authorizing `identity:get_user(user_id=fb9a0b79e88d40ca812c6635b349b0ab)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:07.745742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c771e-d047-4aba-8c28-4cb1db6c5b79 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:07.747431 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 562/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:41:07 2022] GET /identity/v3/users/fb9a0b79e88d40ca812c6635b349b0ab => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:07.753879 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:07.856893 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:07.864604 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685f14f-e85c-47bc-a07e-e3f168d4af30 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:07.901833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8e38d-efa3-4cd1-a8e0-2393c353b0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:07.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8e38d-efa3-4cd1-a8e0-2393c353b0be None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:07.902804 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8e38d-efa3-4cd1-a8e0-2393c353b0be None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:07.903059 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8e38d-efa3-4cd1-a8e0-2393c353b0be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:07.976814 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8e38d-efa3-4cd1-a8e0-2393c353b0be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:07.978644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8e38d-efa3-4cd1-a8e0-2393c353b0be None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:08.074572 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:08.083524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 597/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:41:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:41:08.090300 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c771e-d047-4aba-8c28-4cb1db6c5b79 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:08.156754 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:08.197867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c771e-d047-4aba-8c28-4cb1db6c5b79 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:08.243538 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7cb6-a977-4fbf-802b-4033de2675de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:08.244107 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7cb6-a977-4fbf-802b-4033de2675de None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:08.244388 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7cb6-a977-4fbf-802b-4033de2675de None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:08.244665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7cb6-a977-4fbf-802b-4033de2675de None admin] PATH_INFO: `/v3/projects/fd9646b66dad4f42b2e9fcc4bc47c673` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:08.245914 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7cb6-a977-4fbf-802b-4033de2675de None admin] RBAC: Authorizing `identity:get_project(project_id=fd9646b66dad4f42b2e9fcc4bc47c673)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:08.247422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7cb6-a977-4fbf-802b-4033de2675de None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:08.249154 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 563/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:41:08 2022] GET /identity/v3/projects/fd9646b66dad4f42b2e9fcc4bc47c673 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:08.255894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8e38d-efa3-4cd1-a8e0-2393c353b0be None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:08.364242 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8e38d-efa3-4cd1-a8e0-2393c353b0be None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:08.401558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe5c22-296b-40b4-b0db-a979759a0495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:08.402222 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe5c22-296b-40b4-b0db-a979759a0495 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:08.402517 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe5c22-296b-40b4-b0db-a979759a0495 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:08.402799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe5c22-296b-40b4-b0db-a979759a0495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:08.403603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe5c22-296b-40b4-b0db-a979759a0495 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=fd9646b66dad4f42b2e9fcc4bc47c673, user.id=fb9a0b79e88d40ca812c6635b349b0ab) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:08.403918 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe5c22-296b-40b4-b0db-a979759a0495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:08.405380 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe5c22-296b-40b4-b0db-a979759a0495 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:08.411868 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 598/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:41:08 2022] GET /identity/v3/role_assignments?user.id=fb9a0b79e88d40ca812c6635b349b0ab&scope.project.id=fd9646b66dad4f42b2e9fcc4bc47c673&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:10.426950 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f5f10-ffa2-4e57-8cfd-c43c9ee2ce49 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:10.427562 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f5f10-ffa2-4e57-8cfd-c43c9ee2ce49 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:10.427562 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f5f10-ffa2-4e57-8cfd-c43c9ee2ce49 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:10.428904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 564/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:10.434606 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2952-9483-4b9c-ac9b-2d0ee8b3dd3b None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:10.435110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2952-9483-4b9c-ac9b-2d0ee8b3dd3b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:10.435330 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2952-9483-4b9c-ac9b-2d0ee8b3dd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:10.455137 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-c19f2952-9483-4b9c-ac9b-2d0ee8b3dd3b None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:10.456373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-c19f2952-9483-4b9c-ac9b-2d0ee8b3dd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:10.514826 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:10.529496 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 599/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:10.552055 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f5f10-ffa2-4e57-8cfd-c43c9ee2ce49 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:10.663656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f5f10-ffa2-4e57-8cfd-c43c9ee2ce49 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:10.695306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:10.701040 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b7bb7-d8df-4086-bc47-91468227095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:10.701677 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b7bb7-d8df-4086-bc47-91468227095c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:10.701961 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b7bb7-d8df-4086-bc47-91468227095c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:10.702274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b7bb7-d8df-4086-bc47-91468227095c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:10.775663 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b7bb7-d8df-4086-bc47-91468227095c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:10.777162 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b7bb7-d8df-4086-bc47-91468227095c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:10.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 565/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:41:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:41:10.887264 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f2952-9483-4b9c-ac9b-2d0ee8b3dd3b None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:10.995652 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f2952-9483-4b9c-ac9b-2d0ee8b3dd3b None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:11.031625 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ed756-fe26-4d31-a420-187ed2fa4c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:11.032282 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ed756-fe26-4d31-a420-187ed2fa4c67 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:11.032570 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ed756-fe26-4d31-a420-187ed2fa4c67 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:11.032854 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ed756-fe26-4d31-a420-187ed2fa4c67 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:11.040055 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ed756-fe26-4d31-a420-187ed2fa4c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:11.041517 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ed756-fe26-4d31-a420-187ed2fa4c67 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-f88ed756-fe26-4d31-a420-187ed2fa4c67 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:41:11.053745 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:41:11.055331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:41:11.056753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 600/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:41:10 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:41:11.061356 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b7bb7-d8df-4086-bc47-91468227095c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:11.094797 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:11.167791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b7bb7-d8df-4086-bc47-91468227095c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:11.204243 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778474b-885e-4cba-8b6f-f4997a75a762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:11.204879 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778474b-885e-4cba-8b6f-f4997a75a762 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:11.205163 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778474b-885e-4cba-8b6f-f4997a75a762 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:11.205441 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778474b-885e-4cba-8b6f-f4997a75a762 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:11.206162 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778474b-885e-4cba-8b6f-f4997a75a762 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:11.206482 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778474b-885e-4cba-8b6f-f4997a75a762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:11.207926 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778474b-885e-4cba-8b6f-f4997a75a762 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:11.215550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 566/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:41:11 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:13.204015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e09b3-7019-4a57-a2ac-c32bb9ac4b78 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:13.204644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e09b3-7019-4a57-a2ac-c32bb9ac4b78 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:13.204644 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e09b3-7019-4a57-a2ac-c32bb9ac4b78 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:13.205955 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 601/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:13.211662 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfdf7-8ed9-4cb4-bae9-1706567a2208 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:13.212154 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfdf7-8ed9-4cb4-bae9-1706567a2208 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:13.212281 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfdf7-8ed9-4cb4-bae9-1706567a2208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:13.231905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-b6bbfdf7-8ed9-4cb4-bae9-1706567a2208 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:13.233165 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-b6bbfdf7-8ed9-4cb4-bae9-1706567a2208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:13.310030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 567/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:13.333437 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e09b3-7019-4a57-a2ac-c32bb9ac4b78 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:13.390165 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:13.455608 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e09b3-7019-4a57-a2ac-c32bb9ac4b78 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:13.497126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3f05c-f0cb-4c44-8b4b-f6e2f0d8415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:13.497765 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f05c-f0cb-4c44-8b4b-f6e2f0d8415d None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:13.498052 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f05c-f0cb-4c44-8b4b-f6e2f0d8415d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:13.498358 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f05c-f0cb-4c44-8b4b-f6e2f0d8415d None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:13.499096 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3f05c-f0cb-4c44-8b4b-f6e2f0d8415d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:13.500484 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3f05c-f0cb-4c44-8b4b-f6e2f0d8415d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:13.520095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 602/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jul 28 08:41:13 2022] POST /identity/v3/domains => generated 267 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:41:15.607727 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25bc47-48cf-437f-9073-bc32d122160f None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:15.608343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25bc47-48cf-437f-9073-bc32d122160f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:15.608343 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25bc47-48cf-437f-9073-bc32d122160f None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:15.609679 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 568/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:15.615290 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fb9b0-25c9-4150-8e7e-77b6d25fc757 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:15.615775 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fb9b0-25c9-4150-8e7e-77b6d25fc757 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:15.615889 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fb9b0-25c9-4150-8e7e-77b6d25fc757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:15.636261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-2d3fb9b0-25c9-4150-8e7e-77b6d25fc757 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:15.637493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-2d3fb9b0-25c9-4150-8e7e-77b6d25fc757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:15.709162 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 603/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:15.732385 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25bc47-48cf-437f-9073-bc32d122160f None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:15.768110 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:15.851774 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25bc47-48cf-437f-9073-bc32d122160f None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:15.888092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e59b9c-f00c-4a20-a4e0-3683929ac885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:15.888754 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e59b9c-f00c-4a20-a4e0-3683929ac885 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:15.889017 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e59b9c-f00c-4a20-a4e0-3683929ac885 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:15.889280 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e59b9c-f00c-4a20-a4e0-3683929ac885 None admin] PATH_INFO: `/v3/domains/21f2820b481e40abad3b399427e29288` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:15.900311 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e59b9c-f00c-4a20-a4e0-3683929ac885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=21f2820b481e40abad3b399427e29288)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:15.901782 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e59b9c-f00c-4a20-a4e0-3683929ac885 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:15.903473 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 569/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:41:15 2022] GET /identity/v3/domains/21f2820b481e40abad3b399427e29288 => generated 267 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:15.909923 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fb9b0-25c9-4150-8e7e-77b6d25fc757 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:15.950047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:16.013724 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fb9b0-25c9-4150-8e7e-77b6d25fc757 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:16.050698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32118606-8831-4b9e-b32a-e9b2e8107f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:16.051329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32118606-8831-4b9e-b32a-e9b2e8107f90 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:16.051607 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32118606-8831-4b9e-b32a-e9b2e8107f90 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:16.051883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32118606-8831-4b9e-b32a-e9b2e8107f90 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:16.052578 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32118606-8831-4b9e-b32a-e9b2e8107f90 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:16.054018 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32118606-8831-4b9e-b32a-e9b2e8107f90 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:16.082334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 604/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:41:15 2022] POST /identity/v3/projects => generated 361 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:41:18.135880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6df6d-ebeb-413b-8014-91e207947e5d None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:18.136463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6df6d-ebeb-413b-8014-91e207947e5d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:18.136463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6df6d-ebeb-413b-8014-91e207947e5d None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:18.137803 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 570/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:18.143419 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55598-82fe-45f6-bf43-f66fac538667 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:18.143843 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55598-82fe-45f6-bf43-f66fac538667 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:18.143960 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55598-82fe-45f6-bf43-f66fac538667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:18.163404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-8ff55598-82fe-45f6-bf43-f66fac538667 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:18.164620 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-8ff55598-82fe-45f6-bf43-f66fac538667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:18.237318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 605/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:18.263231 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6df6d-ebeb-413b-8014-91e207947e5d None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:18.351465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:18.368315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6df6d-ebeb-413b-8014-91e207947e5d None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:18.402917 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bb0a4-8449-4431-b8d1-cc1d32f51385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:18.403515 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bb0a4-8449-4431-b8d1-cc1d32f51385 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:18.403799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bb0a4-8449-4431-b8d1-cc1d32f51385 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:18.404082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bb0a4-8449-4431-b8d1-cc1d32f51385 None admin] PATH_INFO: `/v3/domains/21f2820b481e40abad3b399427e29288` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:18.405285 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bb0a4-8449-4431-b8d1-cc1d32f51385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=21f2820b481e40abad3b399427e29288)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:18.406798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bb0a4-8449-4431-b8d1-cc1d32f51385 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:18.408437 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 571/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:41:18 2022] GET /identity/v3/domains/21f2820b481e40abad3b399427e29288 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:18.414512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff55598-82fe-45f6-bf43-f66fac538667 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:18.435669 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:18.517033 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff55598-82fe-45f6-bf43-f66fac538667 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:18.553551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a063c22-351a-46dc-820a-ae010711f4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:18.554221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063c22-351a-46dc-820a-ae010711f4a3 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:18.554506 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063c22-351a-46dc-820a-ae010711f4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:18.554785 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063c22-351a-46dc-820a-ae010711f4a3 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:18.555485 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a063c22-351a-46dc-820a-ae010711f4a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:18.556944 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a063c22-351a-46dc-820a-ae010711f4a3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:18.586048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 606/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:41:18 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:41:20.654088 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2da2-9587-4c12-b351-17f315b6025a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:20.654720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2da2-9587-4c12-b351-17f315b6025a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:20.654720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2da2-9587-4c12-b351-17f315b6025a None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:20.656083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 572/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:20.661755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5ecb-5498-4c29-93d9-25d871c7796f None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:20.662237 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5ecb-5498-4c29-93d9-25d871c7796f None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:20.662344 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5ecb-5498-4c29-93d9-25d871c7796f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:20.686254 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-84ea5ecb-5498-4c29-93d9-25d871c7796f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:20.687508 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-84ea5ecb-5498-4c29-93d9-25d871c7796f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:20.772565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 607/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:20.797062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b2da2-9587-4c12-b351-17f315b6025a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:20.910867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b2da2-9587-4c12-b351-17f315b6025a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:20.946393 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca32e-c959-43ac-a99e-a50ab85b6aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:20.947048 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca32e-c959-43ac-a99e-a50ab85b6aa8 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:20.947330 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca32e-c959-43ac-a99e-a50ab85b6aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:20.947606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca32e-c959-43ac-a99e-a50ab85b6aa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:20.959386 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ca32e-c959-43ac-a99e-a50ab85b6aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-e07ca32e-c959-43ac-a99e-a50ab85b6aa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:41:20.964443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:41:20.965766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:41:20.966937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 573/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:41:20.973449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea5ecb-5498-4c29-93d9-25d871c7796f None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:20.982310 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:21.084186 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea5ecb-5498-4c29-93d9-25d871c7796f None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:21.120642 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:21.121287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:21.121584 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:21.121851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:21.122632 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:21.122941 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:21.124321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:21.133281 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 608/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:21.139592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca32e-c959-43ac-a99e-a50ab85b6aa8 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:21.163691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:21.255097 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca32e-c959-43ac-a99e-a50ab85b6aa8 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:21.290474 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039895d-38e0-4f55-a58d-0955e75c0a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:21.291136 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039895d-38e0-4f55-a58d-0955e75c0a1e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:21.291418 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039895d-38e0-4f55-a58d-0955e75c0a1e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:21.291680 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039895d-38e0-4f55-a58d-0955e75c0a1e None admin] PATH_INFO: `/v3/users/0c0285da37da4faa905d6f8527a046dd` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:21.306373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039895d-38e0-4f55-a58d-0955e75c0a1e None admin] RBAC: Authorizing `identity:get_user(user_id=0c0285da37da4faa905d6f8527a046dd)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:21.307870 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039895d-38e0-4f55-a58d-0955e75c0a1e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:21.309604 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 574/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:41:21 2022] GET /identity/v3/users/0c0285da37da4faa905d6f8527a046dd => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:21.315770 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:21.387217 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:21.426647 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d028b-ba88-4b49-b5ce-0100afb4177b None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:21.462300 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a0efc-68b9-43af-aaf1-a298932366d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:21.462932 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a0efc-68b9-43af-aaf1-a298932366d3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:21.463215 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a0efc-68b9-43af-aaf1-a298932366d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:21.463489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a0efc-68b9-43af-aaf1-a298932366d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:21.555013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a0efc-68b9-43af-aaf1-a298932366d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:21.556515 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a0efc-68b9-43af-aaf1-a298932366d3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:21.626901 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:21.673168 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 609/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:41:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:41:21.679652 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039895d-38e0-4f55-a58d-0955e75c0a1e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:21.756478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:21.789706 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039895d-38e0-4f55-a58d-0955e75c0a1e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:21.827738 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e570b7c-36a5-4423-946e-0bb4e0ec38c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:21.828410 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570b7c-36a5-4423-946e-0bb4e0ec38c2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:21.828691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570b7c-36a5-4423-946e-0bb4e0ec38c2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:21.828969 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570b7c-36a5-4423-946e-0bb4e0ec38c2 None admin] PATH_INFO: `/v3/projects/f3aa360ee25041e38db482cf6b84d025` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:21.830249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570b7c-36a5-4423-946e-0bb4e0ec38c2 None admin] RBAC: Authorizing `identity:get_project(project_id=f3aa360ee25041e38db482cf6b84d025)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:21.831783 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570b7c-36a5-4423-946e-0bb4e0ec38c2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:21.833519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 575/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:41:21 2022] GET /identity/v3/projects/f3aa360ee25041e38db482cf6b84d025 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:21.840074 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a0efc-68b9-43af-aaf1-a298932366d3 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:22.001131 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a0efc-68b9-43af-aaf1-a298932366d3 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:22.037910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987949b-4164-46af-8387-13b103b8ca77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:22.038575 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987949b-4164-46af-8387-13b103b8ca77 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:22.038863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987949b-4164-46af-8387-13b103b8ca77 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:22.039143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987949b-4164-46af-8387-13b103b8ca77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:22.039946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987949b-4164-46af-8387-13b103b8ca77 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=f3aa360ee25041e38db482cf6b84d025, user.id=0c0285da37da4faa905d6f8527a046dd) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:22.040259 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987949b-4164-46af-8387-13b103b8ca77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:22.041714 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987949b-4164-46af-8387-13b103b8ca77 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:22.047440 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 610/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:41:21 2022] GET /identity/v3/role_assignments?user.id=0c0285da37da4faa905d6f8527a046dd&scope.project.id=f3aa360ee25041e38db482cf6b84d025&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:24.031354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a3f9-d5ae-42d5-9f6b-030502f516e0 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:24.031919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a3f9-d5ae-42d5-9f6b-030502f516e0 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:24.031919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a3f9-d5ae-42d5-9f6b-030502f516e0 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:24.033200 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 576/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:24.038721 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac21f1c-f8e7-41ff-99b8-4e137dd456c4 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:24.039168 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac21f1c-f8e7-41ff-99b8-4e137dd456c4 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:24.039289 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac21f1c-f8e7-41ff-99b8-4e137dd456c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:24.057982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-cac21f1c-f8e7-41ff-99b8-4e137dd456c4 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:24.059232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-cac21f1c-f8e7-41ff-99b8-4e137dd456c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:24.086786 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:24.131802 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 611/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:24.154626 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624a3f9-d5ae-42d5-9f6b-030502f516e0 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:24.271909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624a3f9-d5ae-42d5-9f6b-030502f516e0 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:24.309991 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5c3a8-ace4-4bcf-904c-0adfc243a9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:24.310599 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5c3a8-ace4-4bcf-904c-0adfc243a9c0 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:24.310881 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5c3a8-ace4-4bcf-904c-0adfc243a9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:24.311155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5c3a8-ace4-4bcf-904c-0adfc243a9c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:24.319391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:24.322745 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5c3a8-ace4-4bcf-904c-0adfc243a9c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-cda5c3a8-ace4-4bcf-904c-0adfc243a9c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:41:24.328043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:41:24.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:41:24.330585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 577/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:41:24.335714 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac21f1c-f8e7-41ff-99b8-4e137dd456c4 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:24.441988 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac21f1c-f8e7-41ff-99b8-4e137dd456c4 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:24.481429 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:24.482076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:24.482396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:24.482657 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:24.483402 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:24.483712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:24.485143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:24.494372 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 612/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:41:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:24.500623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5c3a8-ace4-4bcf-904c-0adfc243a9c0 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:24.606384 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5c3a8-ace4-4bcf-904c-0adfc243a9c0 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:24.642121 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56c29-1d3b-490f-86e5-deb6be93f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:24.642800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56c29-1d3b-490f-86e5-deb6be93f444 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:24.643085 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56c29-1d3b-490f-86e5-deb6be93f444 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:24.643363 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56c29-1d3b-490f-86e5-deb6be93f444 None admin] PATH_INFO: `/v3/users/0c0285da37da4faa905d6f8527a046dd` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:24.644644 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56c29-1d3b-490f-86e5-deb6be93f444 None admin] RBAC: Authorizing `identity:get_user(user_id=0c0285da37da4faa905d6f8527a046dd)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:24.646263 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56c29-1d3b-490f-86e5-deb6be93f444 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:24.647935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 578/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:41:24 2022] GET /identity/v3/users/0c0285da37da4faa905d6f8527a046dd => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:24.654210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:24.683002 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:24.762071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc7340-3930-4289-a5e8-ccff86b098e4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:24.797401 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38226c9-a147-461b-b323-d5fd925907f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:24.798060 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38226c9-a147-461b-b323-d5fd925907f5 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:24.798370 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38226c9-a147-461b-b323-d5fd925907f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:24.798646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38226c9-a147-461b-b323-d5fd925907f5 None admin] PATH_INFO: `/v3/projects/f3aa360ee25041e38db482cf6b84d025` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:24.799954 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38226c9-a147-461b-b323-d5fd925907f5 None admin] RBAC: Authorizing `identity:get_project(project_id=f3aa360ee25041e38db482cf6b84d025)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:24.801325 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38226c9-a147-461b-b323-d5fd925907f5 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:24.803056 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 613/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:41:24 2022] GET /identity/v3/projects/f3aa360ee25041e38db482cf6b84d025 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:24.810098 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56c29-1d3b-490f-86e5-deb6be93f444 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:24.878514 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:24.914442 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56c29-1d3b-490f-86e5-deb6be93f444 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:24.951054 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020a50c-2625-4085-a3f3-df0b4190b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:24.951743 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a50c-2625-4085-a3f3-df0b4190b37e None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:24.952027 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a50c-2625-4085-a3f3-df0b4190b37e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:24.952308 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a50c-2625-4085-a3f3-df0b4190b37e None admin] PATH_INFO: `/v3/projects/f3aa360ee25041e38db482cf6b84d025/users/0c0285da37da4faa905d6f8527a046dd/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:24.954467 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020a50c-2625-4085-a3f3-df0b4190b37e None admin] RBAC: Authorizing `identity:create_grant(project_id=f3aa360ee25041e38db482cf6b84d025, user_id=0c0285da37da4faa905d6f8527a046dd, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:24.955964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020a50c-2625-4085-a3f3-df0b4190b37e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:24.971314 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 579/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:41:24 2022] PUT /identity/v3/projects/f3aa360ee25041e38db482cf6b84d025/users/0c0285da37da4faa905d6f8527a046dd/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:41:26.965201 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3da36-3f92-48db-829a-f282b158f570 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:26.965767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3da36-3f92-48db-829a-f282b158f570 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:26.965767 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3da36-3f92-48db-829a-f282b158f570 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:26.967163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 614/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:41:26.973934 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b588936-fe63-4283-8d3e-326de7a804f3 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:26.974708 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b588936-fe63-4283-8d3e-326de7a804f3 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:26.975451 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b588936-fe63-4283-8d3e-326de7a804f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:26.996409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-2b588936-fe63-4283-8d3e-326de7a804f3 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:41:26.997740 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-2b588936-fe63-4283-8d3e-326de7a804f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:41:27.077625 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 580/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:41:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:41:27.101946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3da36-3f92-48db-829a-f282b158f570 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:27.201538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:27.212409 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3da36-3f92-48db-829a-f282b158f570 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:27.249352 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86e369-2f1b-4c99-98ff-b11230d76e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:27.249940 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86e369-2f1b-4c99-98ff-b11230d76e72 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:27.250329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86e369-2f1b-4c99-98ff-b11230d76e72 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:27.250605 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86e369-2f1b-4c99-98ff-b11230d76e72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:27.261558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86e369-2f1b-4c99-98ff-b11230d76e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-6c86e369-2f1b-4c99-98ff-b11230d76e72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:41:27.266699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:41:27.268903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:41:27.270480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 615/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:41:27.274696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b588936-fe63-4283-8d3e-326de7a804f3 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:27.334114 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:27.380797 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b588936-fe63-4283-8d3e-326de7a804f3 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:27.417879 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:27.418566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:27.418867 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:27.419148 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:27.419872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:27.420186 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:27.421602 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:27.430228 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 581/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:41:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:27.437780 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86e369-2f1b-4c99-98ff-b11230d76e72 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:27.542093 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86e369-2f1b-4c99-98ff-b11230d76e72 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:27.577876 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf013c8-a106-4e31-b0de-80331486a0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:27.578565 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf013c8-a106-4e31-b0de-80331486a0fa None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:27.578847 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf013c8-a106-4e31-b0de-80331486a0fa None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:27.579127 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf013c8-a106-4e31-b0de-80331486a0fa None admin] PATH_INFO: `/v3/users/0c0285da37da4faa905d6f8527a046dd` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:27.580396 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf013c8-a106-4e31-b0de-80331486a0fa None admin] RBAC: Authorizing `identity:get_user(user_id=0c0285da37da4faa905d6f8527a046dd)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:27.581910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf013c8-a106-4e31-b0de-80331486a0fa None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:27.583617 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 616/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:41:27 2022] GET /identity/v3/users/0c0285da37da4faa905d6f8527a046dd => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:41:27.589755 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:27.712797 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd001-3425-4e80-954b-a239f796f130 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:27.734180 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:27.750476 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16293f1-1349-41cd-8693-d6420fd71126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:27.751125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16293f1-1349-41cd-8693-d6420fd71126 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:27.751409 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16293f1-1349-41cd-8693-d6420fd71126 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:27.751690 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16293f1-1349-41cd-8693-d6420fd71126 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:27.823743 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16293f1-1349-41cd-8693-d6420fd71126 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:27.825162 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16293f1-1349-41cd-8693-d6420fd71126 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:27.932863 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 582/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:41:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:41:27.939489 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf013c8-a106-4e31-b0de-80331486a0fa None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:27.963208 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:28.048053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf013c8-a106-4e31-b0de-80331486a0fa None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:28.085816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6e81b-7e2e-4b2d-a677-92dfa2da1fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:28.086425 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e81b-7e2e-4b2d-a677-92dfa2da1fb3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:28.086711 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e81b-7e2e-4b2d-a677-92dfa2da1fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:28.086995 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e81b-7e2e-4b2d-a677-92dfa2da1fb3 None admin] PATH_INFO: `/v3/projects/f3aa360ee25041e38db482cf6b84d025` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:28.088192 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6e81b-7e2e-4b2d-a677-92dfa2da1fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=f3aa360ee25041e38db482cf6b84d025)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:28.089656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6e81b-7e2e-4b2d-a677-92dfa2da1fb3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:28.091431 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 617/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:41:27 2022] GET /identity/v3/projects/f3aa360ee25041e38db482cf6b84d025 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:28.098934 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16293f1-1349-41cd-8693-d6420fd71126 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:41:28.125425 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:28.222161 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16293f1-1349-41cd-8693-d6420fd71126 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:41:28.264344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f9cdc-d3f4-4094-8caa-87a896210fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:41:28.264997 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f9cdc-d3f4-4094-8caa-87a896210fbd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:41:28.265284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f9cdc-d3f4-4094-8caa-87a896210fbd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:41:28.265565 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f9cdc-d3f4-4094-8caa-87a896210fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:41:28.266405 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f9cdc-d3f4-4094-8caa-87a896210fbd None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=f3aa360ee25041e38db482cf6b84d025, user.id=0c0285da37da4faa905d6f8527a046dd) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:41:28.266720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f9cdc-d3f4-4094-8caa-87a896210fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:41:28.268227 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f9cdc-d3f4-4094-8caa-87a896210fbd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:41:28.275102 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 583/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:41:28 2022] GET /identity/v3/role_assignments?user.id=0c0285da37da4faa905d6f8527a046dd&scope.project.id=f3aa360ee25041e38db482cf6b84d025&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:41:38.080698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=56 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:41:38.272136 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=58 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:23.900684 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43cae7-5afc-49d5-a69b-7e964216cb87 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:23.901309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43cae7-5afc-49d5-a69b-7e964216cb87 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:23.901309 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43cae7-5afc-49d5-a69b-7e964216cb87 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:23.902673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 618/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:42:23.909934 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbab942-376c-43d7-8ab7-2c121c8c3ff8 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:23.910799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbab942-376c-43d7-8ab7-2c121c8c3ff8 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:23.910799 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbab942-376c-43d7-8ab7-2c121c8c3ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:23.932066 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-cfbab942-376c-43d7-8ab7-2c121c8c3ff8 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:23.933346 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-cfbab942-376c-43d7-8ab7-2c121c8c3ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:24.028386 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 584/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:42:24.053368 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43cae7-5afc-49d5-a69b-7e964216cb87 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:24.196386 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43cae7-5afc-49d5-a69b-7e964216cb87 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:24.246900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:24.247783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:24.248128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:24.248612 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:24.263287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:24.265878 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:24.292469 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:24.294155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:24.295735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:24.295735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:24.295735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:24.295735 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 619/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:42:24.303817 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbab942-376c-43d7-8ab7-2c121c8c3ff8 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:24.476627 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbab942-376c-43d7-8ab7-2c121c8c3ff8 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:24.510069 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:24.519090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b3da8-db5b-426f-9492-063dc0b81a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:24.519758 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b3da8-db5b-426f-9492-063dc0b81a09 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:24.520041 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b3da8-db5b-426f-9492-063dc0b81a09 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:24.520383 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b3da8-db5b-426f-9492-063dc0b81a09 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:24.521118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b3da8-db5b-426f-9492-063dc0b81a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:24.521420 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b3da8-db5b-426f-9492-063dc0b81a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:24.522904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b3da8-db5b-426f-9492-063dc0b81a09 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:24.546208 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 585/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:24.547355 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:24.621271 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:24.682471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5250-8265-43bd-babd-18a2a8fa8537 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:24.748247 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7e646-cb64-40b9-a3ee-e9520eed039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:24.748881 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7e646-cb64-40b9-a3ee-e9520eed039b None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:24.749163 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7e646-cb64-40b9-a3ee-e9520eed039b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:24.749439 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7e646-cb64-40b9-a3ee-e9520eed039b None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:24.750200 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7e646-cb64-40b9-a3ee-e9520eed039b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:24.751809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7e646-cb64-40b9-a3ee-e9520eed039b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:24.779305 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 620/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:42:24 2022] POST /identity/v3/users => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:42:27.190325 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403846c-8b20-4094-883d-ea26b502caef None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:27.190957 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403846c-8b20-4094-883d-ea26b502caef None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:27.191081 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403846c-8b20-4094-883d-ea26b502caef None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:27.193022 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 586/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:42:27.200121 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1d59f-ee6a-47bf-82a1-be844d30c48c None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:27.200710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1d59f-ee6a-47bf-82a1-be844d30c48c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:27.200710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1d59f-ee6a-47bf-82a1-be844d30c48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:27.223306 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-b5f1d59f-ee6a-47bf-82a1-be844d30c48c None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:27.224232 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-b5f1d59f-ee6a-47bf-82a1-be844d30c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:27.343399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 621/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:42:27.379994 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403846c-8b20-4094-883d-ea26b502caef None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:27.556905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403846c-8b20-4094-883d-ea26b502caef None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:27.613782 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00f0c4-3e50-44d2-8fa3-ec19634f10c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:27.614519 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00f0c4-3e50-44d2-8fa3-ec19634f10c4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:27.614834 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00f0c4-3e50-44d2-8fa3-ec19634f10c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:27.615113 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00f0c4-3e50-44d2-8fa3-ec19634f10c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:27.628901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00f0c4-3e50-44d2-8fa3-ec19634f10c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-3b00f0c4-3e50-44d2-8fa3-ec19634f10c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:42:27.634465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:27.636764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:27.638813 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 587/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:42:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:27.642524 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1d59f-ee6a-47bf-82a1-be844d30c48c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:27.697025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:27.797620 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1d59f-ee6a-47bf-82a1-be844d30c48c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:27.858480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:27.858480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:27.858480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:27.858480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:27.859445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:27.859445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:27.860887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:27.878025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 622/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:42:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:27.879057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00f0c4-3e50-44d2-8fa3-ec19634f10c4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:27.962078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:28.043109 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00f0c4-3e50-44d2-8fa3-ec19634f10c4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:28.090485 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:28.091491 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:28.091851 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:28.092301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:28.115259 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:28.116695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:28.135463 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:28.137694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:28.139816 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:28.139816 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:28.139816 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:28.141854 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 588/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:28.145466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:28.303431 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:28.308164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c880fc-33d9-4b7c-8e72-f39b0cdbf763 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:28.357294 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:28.358297 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:28.358712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:28.359221 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:28.360285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:28.360820 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:28.362905 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:28.379907 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 623/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:28.386090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:28.510311 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dd1da-4653-41dd-b262-50abf919377c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:28.512009 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:28.551337 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:28.552021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:28.552308 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:28.552657 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:28.561680 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:28.563731 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:42:28.578372 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:28.584987 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:28.586616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 589/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:28.587548 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:28.774454 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08914ab9-9891-4ecb-a66c-c6582ee9e53f None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:28.834486 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:28.835519 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:28.835974 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:28.836512 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:28.838030 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:28.838611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:28.840722 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:28.869020 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 624/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:42:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:28.876216 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:29.028179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37e109-53a4-4fac-961d-9d907bd61f10 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:29.076777 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6810a-7c39-486c-96b1-50856fdb75a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:29.077684 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6810a-7c39-486c-96b1-50856fdb75a2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:29.078223 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6810a-7c39-486c-96b1-50856fdb75a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:29.078646 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6810a-7c39-486c-96b1-50856fdb75a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:29.132465 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:29.193204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6810a-7c39-486c-96b1-50856fdb75a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:29.196498 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6810a-7c39-486c-96b1-50856fdb75a2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:29.358522 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 590/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:42:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:42:29.365019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:29.400392 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:29.541366 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1152312-3d6d-40b6-abf0-da24f4e1b5a2 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:29.602556 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:29.604111 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:29.604771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:29.605169 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:29.624361 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:29.626331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:29.651062 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:29.653191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:29.655293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:29.655293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:29.655293 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:29.658412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 625/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:29.661140 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6810a-7c39-486c-96b1-50856fdb75a2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:29.768323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:29.828936 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6810a-7c39-486c-96b1-50856fdb75a2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:29.890303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:29.891351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:29.891822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:29.892287 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:29.893603 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:29.894154 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:29.896870 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:29.923852 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 591/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:42:29.933178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:30.058518 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:30.108397 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f5398-be31-4326-8d46-52bf9e61e0ae None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:30.151543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:30.152147 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:30.152442 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:30.152768 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:30.162064 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:30.163729 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:42:30.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:30.180587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:30.183287 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 626/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:30.192504 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:30.342050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ffff7-246f-445f-978d-26986994bb01 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:30.384672 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:30.401021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e171a2d-77ac-4a16-81bd-57cb603dfd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:30.402058 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e171a2d-77ac-4a16-81bd-57cb603dfd96 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:30.402446 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e171a2d-77ac-4a16-81bd-57cb603dfd96 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:30.402909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e171a2d-77ac-4a16-81bd-57cb603dfd96 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:30.403945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e171a2d-77ac-4a16-81bd-57cb603dfd96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:30.404460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e171a2d-77ac-4a16-81bd-57cb603dfd96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:30.406596 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e171a2d-77ac-4a16-81bd-57cb603dfd96 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:30.422251 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 592/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:42:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:30.430663 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:30.602733 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d25ec-8e37-4dd8-9818-4a1ff6e90cbb None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:30.658332 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:30.666392 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca3347-922a-4f54-8d5a-a8b40c324591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:30.667299 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3347-922a-4f54-8d5a-a8b40c324591 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:30.667558 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3347-922a-4f54-8d5a-a8b40c324591 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:30.667869 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3347-922a-4f54-8d5a-a8b40c324591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:30.668970 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca3347-922a-4f54-8d5a-a8b40c324591 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=bca9fa56ced04b91b67f2c802dc90a21) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:30.669319 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca3347-922a-4f54-8d5a-a8b40c324591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:30.671225 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca3347-922a-4f54-8d5a-a8b40c324591 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:30.694159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 627/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:42:30 2022] GET /identity/v3/role_assignments?user.id=bca9fa56ced04b91b67f2c802dc90a21&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:32.914458 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2e6e-ecf5-470f-8627-498a61994511 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:32.915438 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2e6e-ecf5-470f-8627-498a61994511 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:32.915438 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2e6e-ecf5-470f-8627-498a61994511 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:32.917427 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 593/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:42:32.923709 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c17323-d3f5-40a6-8a59-8cf9d346eeee None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:32.924424 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c17323-d3f5-40a6-8a59-8cf9d346eeee None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:32.924424 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c17323-d3f5-40a6-8a59-8cf9d346eeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:32.944624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-25c17323-d3f5-40a6-8a59-8cf9d346eeee None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:32.945812 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-25c17323-d3f5-40a6-8a59-8cf9d346eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:33.024968 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 628/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:42:33.049643 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2e6e-ecf5-470f-8627-498a61994511 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:33.214591 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2e6e-ecf5-470f-8627-498a61994511 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:33.269490 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd6b8-f9d9-499e-9bc3-027ab34a5d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:33.270428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd6b8-f9d9-499e-9bc3-027ab34a5d21 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:33.270825 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd6b8-f9d9-499e-9bc3-027ab34a5d21 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:33.271187 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd6b8-f9d9-499e-9bc3-027ab34a5d21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:33.285002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbd6b8-f9d9-499e-9bc3-027ab34a5d21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-64dbd6b8-f9d9-499e-9bc3-027ab34a5d21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:42:33.291847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:33.304576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:33.306018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 594/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:42:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:33.306887 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c17323-d3f5-40a6-8a59-8cf9d346eeee None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:33.473156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c17323-d3f5-40a6-8a59-8cf9d346eeee None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:33.508796 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:33.527799 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:33.528501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:33.528941 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:33.529361 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:33.530271 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] RBAC: Adding query filter params (name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:33.530587 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:33.532330 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:33.542610 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 629/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:42:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:33.551060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd6b8-f9d9-499e-9bc3-027ab34a5d21 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:33.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:33.703449 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd6b8-f9d9-499e-9bc3-027ab34a5d21 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:33.754067 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:33.755018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:33.755432 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:33.755977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:33.770592 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:33.772387 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:33.791580 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:33.794072 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:33.796050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:33.796050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:33.796050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:33.796050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 595/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:33.801452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:33.959912 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58938ba1-bc38-403f-a269-51b1712c2d37 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:34.008764 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:34.009538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:34.009925 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:34.010466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:34.011554 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:34.012048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:34.013716 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:34.030820 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 630/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:34.038310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:34.162910 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:34.170721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ae98b-a05f-410c-bc41-2fc847032b97 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:34.221671 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:34.222460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:34.224181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:34.224181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:34.233610 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:34.235156 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:42:34.247106 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:34.249249 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:34.251469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 596/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:42:34.258883 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:34.312263 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:34.405331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013ff9f-e588-41d2-9e24-a0094ae86d73 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:34.448042 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:34.448915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:34.449318 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:34.449761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:34.450985 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:34.452651 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:34.453866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:34.476243 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 631/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:42:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:34.485633 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:34.627416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53625fd6-a821-48ab-96ab-b91a905effb2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:34.667430 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:34.668258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:34.668540 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:34.668859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:34.682505 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:34.684167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:34.707866 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:34.709242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:34.710831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:34.710831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:34.710831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:34.710831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 597/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:42:34.716742 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:34.854914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a2c0a-f9b4-47d9-8878-869ca19b10f0 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:34.859983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:34.907882 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:34.908500 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:34.908783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:34.909238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:34.910697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:34.911020 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:34.912636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:34.929143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 632/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:34.935761 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:34.947305 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:35.096191 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae99f1f-7dab-48b9-a9ab-35c010619a3d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:35.136699 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:35.137394 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:35.137701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:35.138030 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:35.146373 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:35.148019 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:42:35.161174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:35.163588 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:35.166462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 598/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:35.171068 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:35.334902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be45f-fa2e-4bbf-a9c9-41ea28b41101 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:35.384611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4fb87-28d5-4106-ad7a-e7eb4491dc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:35.385538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4fb87-28d5-4106-ad7a-e7eb4491dc15 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:35.385933 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4fb87-28d5-4106-ad7a-e7eb4491dc15 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:35.386375 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4fb87-28d5-4106-ad7a-e7eb4491dc15 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:35.387303 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4fb87-28d5-4106-ad7a-e7eb4491dc15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:35.387734 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4fb87-28d5-4106-ad7a-e7eb4491dc15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:35.389664 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4fb87-28d5-4106-ad7a-e7eb4491dc15 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:35.408001 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 633/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:35.416191 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:35.510610 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:35.558459 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4a16f-ebbb-44c2-bb72-d9434f0c693b None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:35.608701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004b6d3-e782-486f-8dbb-9a90cce45e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:35.609620 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b6d3-e782-486f-8dbb-9a90cce45e8e None admin] REQUEST_METHOD: `PUT` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:35.610042 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b6d3-e782-486f-8dbb-9a90cce45e8e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:35.610521 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b6d3-e782-486f-8dbb-9a90cce45e8e None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/bca9fa56ced04b91b67f2c802dc90a21/roles/cc9e44cb256e4496ba5fd23548c12437` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:35.632489 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004b6d3-e782-486f-8dbb-9a90cce45e8e None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=bca9fa56ced04b91b67f2c802dc90a21, role_id=cc9e44cb256e4496ba5fd23548c12437)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:35.635070 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004b6d3-e782-486f-8dbb-9a90cce45e8e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:35.656244 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 599/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:42:35 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/bca9fa56ced04b91b67f2c802dc90a21/roles/cc9e44cb256e4496ba5fd23548c12437 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:42:37.708614 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428779ab-8300-485b-ab31-2aac97f213e6 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:37.709385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428779ab-8300-485b-ab31-2aac97f213e6 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:37.709385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428779ab-8300-485b-ab31-2aac97f213e6 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:37.711306 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 634/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:42:37.719331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c5088-97bd-4258-afc6-f56e2bdc1510 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:37.720270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c5088-97bd-4258-afc6-f56e2bdc1510 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:37.720546 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c5088-97bd-4258-afc6-f56e2bdc1510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:37.740298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-d35c5088-97bd-4258-afc6-f56e2bdc1510 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:37.741747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-d35c5088-97bd-4258-afc6-f56e2bdc1510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:37.822525 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 600/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:42:37.849024 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428779ab-8300-485b-ab31-2aac97f213e6 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:37.881965 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:37.982094 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428779ab-8300-485b-ab31-2aac97f213e6 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:38.027350 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe4059-8578-422f-b9fa-a90ca626bbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:38.028067 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe4059-8578-422f-b9fa-a90ca626bbf0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:38.028391 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe4059-8578-422f-b9fa-a90ca626bbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:38.028667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe4059-8578-422f-b9fa-a90ca626bbf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:38.042201 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe4059-8578-422f-b9fa-a90ca626bbf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-ddfe4059-8578-422f-b9fa-a90ca626bbf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:42:38.047389 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:38.049877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:38.052667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 635/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:38.056154 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c5088-97bd-4258-afc6-f56e2bdc1510 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:38.132184 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:38.198754 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c5088-97bd-4258-afc6-f56e2bdc1510 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:38.243581 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:38.244135 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:38.244414 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:38.244744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:38.245656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] RBAC: Adding query filter params (name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:38.245971 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:38.247434 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:38.256621 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 601/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:38.262806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe4059-8578-422f-b9fa-a90ca626bbf0 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:38.363984 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:38.387535 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe4059-8578-422f-b9fa-a90ca626bbf0 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:38.444791 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:38.445518 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:38.445826 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:38.446171 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:38.465790 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:38.468755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:38.491349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:38.492771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:38.493956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:38.493956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:38.493956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:38.494746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 636/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:38.503204 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:38.635507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ef64-c3db-43a7-86dd-352f0d859a81 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:38.648363 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:38.681735 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:38.683357 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:38.684002 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:38.684507 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:38.685841 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:38.686354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:38.688791 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:38.717024 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 602/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:42:38.725662 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:38.851759 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60219bc1-2ede-44bf-a371-abf384698f2c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:38.892013 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:38.892673 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:38.892959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:38.893233 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:38.896713 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:38.904520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:38.906292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:42:38.921851 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:38.923499 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:38.925174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 637/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:38.930167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:39.040364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554854c-8708-451e-8a7e-1328330956b5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:39.081229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:39.081900 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:39.082351 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:39.082696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:39.083731 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:39.084083 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:39.085561 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:39.117419 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 603/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:42:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:39.118850 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:39.243082 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2e4ef-d8c3-4a25-89fc-a8d89c5eafb0 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:39.285071 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99327a4f-5d3c-4c91-98b4-38ff1f7330c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:39.285807 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99327a4f-5d3c-4c91-98b4-38ff1f7330c7 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:39.286032 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99327a4f-5d3c-4c91-98b4-38ff1f7330c7 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:39.286351 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99327a4f-5d3c-4c91-98b4-38ff1f7330c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:39.363292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:39.373028 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99327a4f-5d3c-4c91-98b4-38ff1f7330c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:39.374617 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99327a4f-5d3c-4c91-98b4-38ff1f7330c7 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:39.507367 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 638/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:42:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:42:39.512828 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:39.546530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:39.637273 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714f21a-738b-422d-9a71-584cb520dd1c None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:39.682167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:39.682847 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:39.683822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:39.683822 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:39.699491 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:39.701235 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:39.725725 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:39.727674 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:39.729214 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:39.729214 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:39.729214 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:39.729214 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 604/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:39.737427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99327a4f-5d3c-4c91-98b4-38ff1f7330c7 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:39.881875 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:39.889117 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99327a4f-5d3c-4c91-98b4-38ff1f7330c7 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:39.937466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:39.938621 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:39.939112 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:39.939833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:39.941022 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:39.941442 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:39.944164 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:39.966744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 639/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:42:39.968290 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:40.066615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:40.094798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c8280-3382-4daf-bc3a-7c7a819e32a6 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:40.134311 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:40.135129 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:40.135675 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:40.136034 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:40.144924 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:40.146424 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:42:40.161665 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:40.163464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:40.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 605/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:40.169403 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:40.291052 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16f03-9ef0-4d82-889f-51db31f8f2d4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:40.337498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791aea85-65ed-4cf4-9b86-6480d5493be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:40.338036 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aea85-65ed-4cf4-9b86-6480d5493be0 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:40.339067 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aea85-65ed-4cf4-9b86-6480d5493be0 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:40.339067 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aea85-65ed-4cf4-9b86-6480d5493be0 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:40.339348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791aea85-65ed-4cf4-9b86-6480d5493be0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:40.340089 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791aea85-65ed-4cf4-9b86-6480d5493be0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:40.341128 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791aea85-65ed-4cf4-9b86-6480d5493be0 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:40.356046 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 640/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:40.364055 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:40.512464 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e684ea-edea-419c-a112-dfa70906ccd7 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:40.553253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23647e-ae05-4926-9779-bce9f3fd2983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:40.554468 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23647e-ae05-4926-9779-bce9f3fd2983 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:40.554971 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23647e-ae05-4926-9779-bce9f3fd2983 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:40.555558 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23647e-ae05-4926-9779-bce9f3fd2983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:40.556820 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23647e-ae05-4926-9779-bce9f3fd2983 None admin] RBAC: Adding query filter params (role.id=cc9e44cb256e4496ba5fd23548c12437, scope.project.id=813e8db425644f688e926aafe3249721, user.id=bca9fa56ced04b91b67f2c802dc90a21) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:40.557447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23647e-ae05-4926-9779-bce9f3fd2983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:40.559169 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23647e-ae05-4926-9779-bce9f3fd2983 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:40.568927 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 606/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:42:40 2022] GET /identity/v3/role_assignments?user.id=bca9fa56ced04b91b67f2c802dc90a21&scope.project.id=813e8db425644f688e926aafe3249721&role.id=cc9e44cb256e4496ba5fd23548c12437 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:42.807501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1d4de-c78c-4f48-a8cc-c6ac0bd1ac91 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:42.808384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1d4de-c78c-4f48-a8cc-c6ac0bd1ac91 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:42.808384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1d4de-c78c-4f48-a8cc-c6ac0bd1ac91 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:42.809404 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 641/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:42:42.816731 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbb4ff-1360-49a2-b684-64f54271c6c5 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:42.817443 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbb4ff-1360-49a2-b684-64f54271c6c5 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:42.817757 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbb4ff-1360-49a2-b684-64f54271c6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:42.831181 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:42.845996 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-39fbb4ff-1360-49a2-b684-64f54271c6c5 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:42.847347 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-39fbb4ff-1360-49a2-b684-64f54271c6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:42.929989 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 607/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:42:42.960980 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1d4de-c78c-4f48-a8cc-c6ac0bd1ac91 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:42.983611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:43.098037 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1d4de-c78c-4f48-a8cc-c6ac0bd1ac91 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:43.149147 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14af1e-c926-4a68-8fb3-5622f7bf598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:43.150140 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14af1e-c926-4a68-8fb3-5622f7bf598e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:43.150567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14af1e-c926-4a68-8fb3-5622f7bf598e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:43.150982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14af1e-c926-4a68-8fb3-5622f7bf598e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:43.166580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14af1e-c926-4a68-8fb3-5622f7bf598e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8f14af1e-c926-4a68-8fb3-5622f7bf598e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:42:43.172202 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:43.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:43.177834 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 642/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:42:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:43.179986 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbb4ff-1360-49a2-b684-64f54271c6c5 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:43.296155 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbb4ff-1360-49a2-b684-64f54271c6c5 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:43.335668 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:43.342087 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:43.343018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:43.343447 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:43.343779 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:43.344819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:43.345141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:43.346824 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:43.359391 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 608/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:42:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:43.366497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14af1e-c926-4a68-8fb3-5622f7bf598e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:43.492566 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:43.531994 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14af1e-c926-4a68-8fb3-5622f7bf598e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:43.581554 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:43.582371 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:43.582801 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:43.583324 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:43.598712 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:43.601383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:43.624206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:43.626666 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:43.628784 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:43.628784 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:43.628784 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:43.628784 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 643/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:43.634296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:43.783859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d231f-552d-44ae-b479-1480da8db7a0 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:43.828783 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:43.829883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:43.829883 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:43.830303 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:43.830779 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:43.831090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:43.832611 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:43.846734 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 609/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:43.853899 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:44.026865 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52a289-d463-4e47-b0a1-3ec9af982863 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:44.064793 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:44.076982 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:44.077861 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:44.078250 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:44.078528 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:44.093092 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:44.095082 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:42:44.109961 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:44.119135 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:44.121717 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 644/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:42:44.123400 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:44.213319 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:44.274601 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45b406-455d-4b2b-8938-4df9a69939c8 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:44.315899 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:44.316566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:44.316825 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:44.317092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:44.317956 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:44.318299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:44.320193 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:44.350664 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 610/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:42:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:44.352028 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:44.499416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30417c-3cf1-4b55-aae7-00b9f2a63d60 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:44.544407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d9dec-22ed-4a50-b9dc-c7f605ece297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:44.545700 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d9dec-22ed-4a50-b9dc-c7f605ece297 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:44.546238 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d9dec-22ed-4a50-b9dc-c7f605ece297 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:44.546704 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d9dec-22ed-4a50-b9dc-c7f605ece297 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:44.636197 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:44.653012 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d9dec-22ed-4a50-b9dc-c7f605ece297 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:44.654679 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d9dec-22ed-4a50-b9dc-c7f605ece297 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:44.796603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 645/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:42:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:42:44.798923 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:44.923492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113cb3c-dd1a-4d05-b2e9-c6b586796725 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:44.941292 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:44.972163 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:44.972849 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:44.973130 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:44.973462 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:44.986370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:44.988080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:45.004640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:45.006354 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:45.008074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:45.008074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:45.008074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:45.008074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 611/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:45.012106 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d9dec-22ed-4a50-b9dc-c7f605ece297 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:45.139503 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:45.161444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d9dec-22ed-4a50-b9dc-c7f605ece297 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:45.207935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:45.208513 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:45.208863 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:45.209076 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:45.209825 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:45.210170 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:45.211628 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:45.225934 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 646/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:45.232344 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:45.400194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc36a3-6f74-4d31-8187-8c240a4532d5 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:45.460176 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:45.460787 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:45.461071 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:45.461349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] PATH_INFO: `/v3/projects/service` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:45.469880 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:45.471584 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:42:45.485242 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:45.487258 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:45.488429 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 612/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:42:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:42:45.493418 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:45.637636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98658bd-80c1-4b45-ae32-0d6fa0b56630 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:45.683959 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee27c36-a8b8-4787-b44f-a3f070819997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:45.684792 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee27c36-a8b8-4787-b44f-a3f070819997 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:45.689057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee27c36-a8b8-4787-b44f-a3f070819997 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:45.689057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee27c36-a8b8-4787-b44f-a3f070819997 None admin] PATH_INFO: `/v3/projects` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:45.689057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee27c36-a8b8-4787-b44f-a3f070819997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:45.689057 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee27c36-a8b8-4787-b44f-a3f070819997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:45.690559 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee27c36-a8b8-4787-b44f-a3f070819997 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:45.717132 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 647/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:42:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:45.718671 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:45.761268 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:45.889647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482446f-df4d-4e58-817c-d130b75d3dde None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:45.947838 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c969e83-8de7-4730-b739-669320f327e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:45.948874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c969e83-8de7-4730-b739-669320f327e4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:45.948874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c969e83-8de7-4730-b739-669320f327e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:45.949202 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c969e83-8de7-4730-b739-669320f327e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:45.949897 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c969e83-8de7-4730-b739-669320f327e4 None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=813e8db425644f688e926aafe3249721, user.id=bca9fa56ced04b91b67f2c802dc90a21) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:45.950239 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c969e83-8de7-4730-b739-669320f327e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:45.951744 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c969e83-8de7-4730-b739-669320f327e4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:45.958552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 613/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:42:45 2022] GET /identity/v3/role_assignments?user.id=bca9fa56ced04b91b67f2c802dc90a21&scope.project.id=813e8db425644f688e926aafe3249721&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:48.119956 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3a1c7-ab9c-4e28-a66c-eb775dacf2d9 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:48.120611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3a1c7-ab9c-4e28-a66c-eb775dacf2d9 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:48.120611 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3a1c7-ab9c-4e28-a66c-eb775dacf2d9 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:48.122004 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 648/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:42:48.127618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c142666-5872-4457-b978-56b66b9f9578 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:48.128336 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c142666-5872-4457-b978-56b66b9f9578 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:48.128336 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c142666-5872-4457-b978-56b66b9f9578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:48.147893 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-5c142666-5872-4457-b978-56b66b9f9578 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:48.149133 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-5c142666-5872-4457-b978-56b66b9f9578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:48.229578 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 614/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:42:48.252595 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3a1c7-ab9c-4e28-a66c-eb775dacf2d9 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:48.269635 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:48.380029 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3a1c7-ab9c-4e28-a66c-eb775dacf2d9 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:48.427276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2db5a4-1b47-4edb-92ef-e0a00977f583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:48.428178 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2db5a4-1b47-4edb-92ef-e0a00977f583 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:48.428591 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2db5a4-1b47-4edb-92ef-e0a00977f583 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:48.428930 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2db5a4-1b47-4edb-92ef-e0a00977f583 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:48.445777 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2db5a4-1b47-4edb-92ef-e0a00977f583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-de2db5a4-1b47-4edb-92ef-e0a00977f583 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:42:48.451053 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:48.459462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:48.461159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 649/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:48.462383 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c142666-5872-4457-b978-56b66b9f9578 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:48.490597 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:48.591513 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c142666-5872-4457-b978-56b66b9f9578 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:48.634661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:48.635541 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:48.635930 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:48.636350 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] PATH_INFO: `/v3/roles` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:48.637579 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:48.637956 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:48.640334 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:48.650861 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 615/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:42:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:42:48.658543 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2db5a4-1b47-4edb-92ef-e0a00977f583 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:48.770422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:48.818339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2db5a4-1b47-4edb-92ef-e0a00977f583 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:48.859271 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:48.859987 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:48.860331 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:48.860585 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:48.872636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:48.874348 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:48.893115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:48.894487 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:48.894487 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:48.894487 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:48.894487 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 650/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:48.898689 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:49.020938 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:49.047530 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41db220-ab98-482a-8fc8-80b55e8757d4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:49.088824 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:49.089503 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:49.089782 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:49.090057 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:49.090835 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:49.091144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:49.092609 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:49.108407 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 616/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:49.114761 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:49.240816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c4c67-365d-4cfa-ada2-debec673c97a None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:49.265923 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:49.284751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:49.285656 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:49.286007 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:49.286384 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:49.295559 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:49.297101 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:42:49.310947 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:49.313493 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:49.316185 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:49.318048 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 651/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:49.318935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:49.453296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86a223-a60b-44d8-ad0f-0fc8f2d364f3 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:49.495019 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:49.495576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:49.495862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:49.496140 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] PATH_INFO: `/v3/users` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:49.496966 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:49.497277 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:49.498805 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:49.509919 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:49.521640 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 617/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:42:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:49.527806 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:49.657002 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0959ab1-bc45-4293-8921-e5ed5f142752 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:49.698637 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:49.699330 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:49.699618 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:49.699899 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:49.719475 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:49.723996 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:49.747946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:49.751026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:49.754291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:49.754291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:49.754291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:49.754291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 652/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:42:49.760536 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:49.940898 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c206d-1be7-4fe0-93e2-c5e0c9e5bdbd None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:50.001934 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:50.002909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:50.003405 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:50.003908 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:50.005237 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:50.005798 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:50.008316 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:50.021484 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 618/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:50.028654 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:50.066715 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:50.141983 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861dced-5156-4f09-8d07-1ed8e3559b85 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:50.178364 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:50.179039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:50.179324 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:50.179604 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:50.187040 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:50.188536 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:42:50.201427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:50.203436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:50.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 653/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:50.211450 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:50.277091 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:50.324250 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb284b82-199b-4bc9-9e2d-47a50f9726eb None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:50.361282 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbb248-c56d-40b9-aedc-b28ddfa31732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:50.361922 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbb248-c56d-40b9-aedc-b28ddfa31732 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:50.362197 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbb248-c56d-40b9-aedc-b28ddfa31732 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:50.362497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbb248-c56d-40b9-aedc-b28ddfa31732 None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:50.363257 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbb248-c56d-40b9-aedc-b28ddfa31732 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:50.363564 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbb248-c56d-40b9-aedc-b28ddfa31732 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:50.365045 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbb248-c56d-40b9-aedc-b28ddfa31732 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:50.378532 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 619/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:42:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:50.385340 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:50.503414 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:50.520110 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3b70-9124-4754-adb0-8b51dbc82a0e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:50.561430 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521e2ed-a285-4efd-a68f-ca008d2596f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:50.562055 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e2ed-a285-4efd-a68f-ca008d2596f6 None admin] REQUEST_METHOD: `PUT` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:50.562388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e2ed-a285-4efd-a68f-ca008d2596f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:50.562675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e2ed-a285-4efd-a68f-ca008d2596f6 None admin] PATH_INFO: `/v3/projects/813e8db425644f688e926aafe3249721/users/bca9fa56ced04b91b67f2c802dc90a21/roles/1bac563d88ad45aeab025bacff7724d2` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:50.564897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521e2ed-a285-4efd-a68f-ca008d2596f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=813e8db425644f688e926aafe3249721, user_id=bca9fa56ced04b91b67f2c802dc90a21, role_id=1bac563d88ad45aeab025bacff7724d2)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:50.566443 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521e2ed-a285-4efd-a68f-ca008d2596f6 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:50.582972 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 654/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:42:50 2022] PUT /identity/v3/projects/813e8db425644f688e926aafe3249721/users/bca9fa56ced04b91b67f2c802dc90a21/roles/1bac563d88ad45aeab025bacff7724d2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:42:52.677585 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6480-41f3-4cb9-be74-933139d069dd None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:52.678299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6480-41f3-4cb9-be74-933139d069dd None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:52.678299 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6480-41f3-4cb9-be74-933139d069dd None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:52.679574 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 620/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:42:52.685902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e8e9-7d74-4c81-8e16-50ada381f102 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:52.686505 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e8e9-7d74-4c81-8e16-50ada381f102 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:52.686619 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e8e9-7d74-4c81-8e16-50ada381f102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:52.708438 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-1240e8e9-7d74-4c81-8e16-50ada381f102 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:52.709797 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-1240e8e9-7d74-4c81-8e16-50ada381f102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:52.805101 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 655/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:42:52.847086 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c6480-41f3-4cb9-be74-933139d069dd None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:53.020253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c6480-41f3-4cb9-be74-933139d069dd None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:53.058268 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:53.079974 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e5d94-0a69-4eb5-bd0a-37f9ec230d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:53.080653 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5d94-0a69-4eb5-bd0a-37f9ec230d2d None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:53.080942 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5d94-0a69-4eb5-bd0a-37f9ec230d2d None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:53.081263 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5d94-0a69-4eb5-bd0a-37f9ec230d2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:53.092074 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e5d94-0a69-4eb5-bd0a-37f9ec230d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-373e5d94-0a69-4eb5-bd0a-37f9ec230d2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:42:53.097099 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:53.099056 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:53.100945 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 621/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:42:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:53.104793 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e8e9-7d74-4c81-8e16-50ada381f102 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:53.187206 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:53.206873 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e8e9-7d74-4c81-8e16-50ada381f102 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:53.243760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:53.244407 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:53.244691 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:53.244969 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] PATH_INFO: `/v3/roles` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:53.245771 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] RBAC: Adding query filter params (name=admin) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:53.246083 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:53.247510 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:53.256482 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 656/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:42:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:42:53.262661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e5d94-0a69-4eb5-bd0a-37f9ec230d2d None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:53.371904 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e5d94-0a69-4eb5-bd0a-37f9ec230d2d None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:53.409636 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:53.410324 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:53.410617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:53.410894 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:53.421917 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:53.423457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:53.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:53.441011 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:53.442548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:53.442548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:53.442548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:53.442548 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 622/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:53.447270 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:53.567373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344a550-c060-4851-870c-ed46bf4e023c None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:53.604399 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:53.605030 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:53.605315 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:53.605590 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] PATH_INFO: `/v3/domains` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:53.606328 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:53.606639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:53.608023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:53.620589 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 657/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:53.626655 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:53.658946 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:53.734229 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de02342-f4e7-456a-b63e-6aa71e2fd61e None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:53.769416 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:53.770060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:53.770362 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:53.770635 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:53.778355 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:53.779811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:42:53.791212 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:53.792550 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:53.794038 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 623/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:53.799197 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:53.803466 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:53.907102 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f86a3-3dbf-4d96-a47b-3d2a98a5d832 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:53.942545 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:53.943174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:53.943456 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:53.944100 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] PATH_INFO: `/v3/users` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:53.944903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:53.945213 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:53.946691 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:53.964058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 658/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:42:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:53.972723 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:54.073270 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:54.085401 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fc21c-19a3-461a-8388-2ad32f1a8422 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:54.123751 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dcf46-a84f-4e8c-8365-fc8701d1b9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:54.124377 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dcf46-a84f-4e8c-8365-fc8701d1b9c4 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:54.124661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dcf46-a84f-4e8c-8365-fc8701d1b9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:54.124937 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dcf46-a84f-4e8c-8365-fc8701d1b9c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:54.201166 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dcf46-a84f-4e8c-8365-fc8701d1b9c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:54.202618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dcf46-a84f-4e8c-8365-fc8701d1b9c4 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:54.296750 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:54.308557 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 624/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:42:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:42:54.315126 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:54.380503 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:54.420452 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b6b41-6a1e-4690-9fa8-04f7c03107a4 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:54.457894 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:54.458498 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:54.458776 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:54.459050 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:54.469897 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:54.471417 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:42:54.487291 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:54.488603 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:42:54.490133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:42:54.490133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:42:54.490133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:54.490133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 659/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:54.494575 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dcf46-a84f-4e8c-8365-fc8701d1b9c4 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:54.612374 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dcf46-a84f-4e8c-8365-fc8701d1b9c4 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:54.648090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:54.648723 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:54.649006 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:54.649284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] PATH_INFO: `/v3/domains` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:54.649989 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:54.650331 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:54.651755 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:54.663293 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 625/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:54.669349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:54.792508 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4c12-5e07-4b40-a348-c1c6559c6908 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:54.806994 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:54.832820 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:54.833473 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:54.833755 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:54.834030 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:54.841506 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:54.843058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:42:54.855261 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:54.856520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: ERROR keystone.server.flask.application Jul 28 08:42:54.857667 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 660/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:42:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:54.862654 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:54.914797 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:54.968721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f5e7d-d312-4ab7-a45c-70c4fd454e0f None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:55.005944 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf23b51-9686-4bc5-a190-dce538883f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:55.006623 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23b51-9686-4bc5-a190-dce538883f6f None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:55.006905 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23b51-9686-4bc5-a190-dce538883f6f None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:55.007175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23b51-9686-4bc5-a190-dce538883f6f None admin] PATH_INFO: `/v3/projects` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:55.007936 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf23b51-9686-4bc5-a190-dce538883f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:55.008243 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf23b51-9686-4bc5-a190-dce538883f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:55.009712 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf23b51-9686-4bc5-a190-dce538883f6f None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:55.023760 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 626/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:42:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:55.031151 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:55.148422 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e285d8b-5b02-4263-a066-3590bcb74e90 None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:55.187114 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a2af7-99a8-44d8-8be0-47008f896c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:55.187744 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a2af7-99a8-44d8-8be0-47008f896c1b None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:55.188030 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a2af7-99a8-44d8-8be0-47008f896c1b None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:55.188310 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a2af7-99a8-44d8-8be0-47008f896c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:55.189159 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a2af7-99a8-44d8-8be0-47008f896c1b None admin] RBAC: Adding query filter params (role.id=1bac563d88ad45aeab025bacff7724d2, scope.project.id=813e8db425644f688e926aafe3249721, user.id=bca9fa56ced04b91b67f2c802dc90a21) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:55.189455 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a2af7-99a8-44d8-8be0-47008f896c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:55.190935 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a2af7-99a8-44d8-8be0-47008f896c1b None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:55.197336 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 661/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:42:55 2022] GET /identity/v3/role_assignments?user.id=bca9fa56ced04b91b67f2c802dc90a21&scope.project.id=813e8db425644f688e926aafe3249721&role.id=1bac563d88ad45aeab025bacff7724d2 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:57.234953 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd87c-67a8-43bc-b60f-d9072e808c4a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:57.235902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd87c-67a8-43bc-b60f-d9072e808c4a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:57.235902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd87c-67a8-43bc-b60f-d9072e808c4a None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:57.237469 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 627/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:42:57.243903 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4668d0-eb3b-496c-ab26-a98d704df24e None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:57.244520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4668d0-eb3b-496c-ab26-a98d704df24e None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:57.244646 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4668d0-eb3b-496c-ab26-a98d704df24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:57.272095 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-ee4668d0-eb3b-496c-ab26-a98d704df24e None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:57.273296 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-ee4668d0-eb3b-496c-ab26-a98d704df24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:57.304978 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:57.348274 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 662/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:42:57.371050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fd87c-67a8-43bc-b60f-d9072e808c4a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:57.489978 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fd87c-67a8-43bc-b60f-d9072e808c4a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:57.502157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:57.527370 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:57.528023 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:57.528306 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:57.528584 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:57.529251 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:57.530696 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:42:57.540075 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:42:57.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:42:57.543656 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 628/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:42:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:42:57.547537 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4668d0-eb3b-496c-ab26-a98d704df24e None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:57.660564 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4668d0-eb3b-496c-ab26-a98d704df24e None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:57.698823 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7c808-1da8-42b0-9f51-1de311e3c9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:57.699471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7c808-1da8-42b0-9f51-1de311e3c9b3 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:57.699758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7c808-1da8-42b0-9f51-1de311e3c9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:57.700039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7c808-1da8-42b0-9f51-1de311e3c9b3 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:57.700733 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7c808-1da8-42b0-9f51-1de311e3c9b3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:57.701039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7c808-1da8-42b0-9f51-1de311e3c9b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:57.702459 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7c808-1da8-42b0-9f51-1de311e3c9b3 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:57.707946 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 663/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:42:57 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:42:57.714785 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:42:57.839183 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def21060-f2b9-4cef-9566-1494d4a91aa2 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:42:57.877067 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d9174-6639-439d-a191-403e9a5e6ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:42:57.877692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d9174-6639-439d-a191-403e9a5e6ce7 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:57.877967 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d9174-6639-439d-a191-403e9a5e6ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:57.878276 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d9174-6639-439d-a191-403e9a5e6ce7 None admin] PATH_INFO: `/v3/services` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:57.878956 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d9174-6639-439d-a191-403e9a5e6ce7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:42:57.879255 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d9174-6639-439d-a191-403e9a5e6ce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:42:57.880659 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d9174-6639-439d-a191-403e9a5e6ce7 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:42:57.885352 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 629/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:42:57 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:42:59.880019 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c42fae-bf90-4441-814f-2adc6b534d39 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:59.880910 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c42fae-bf90-4441-814f-2adc6b534d39 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:59.881639 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c42fae-bf90-4441-814f-2adc6b534d39 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:59.885736 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 664/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:42:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:42:59.892420 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1d07d-cedb-4a7a-8a07-ccb611605e1d None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:42:59.893037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1d07d-cedb-4a7a-8a07-ccb611605e1d None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:42:59.893037 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1d07d-cedb-4a7a-8a07-ccb611605e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:42:59.917878 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-beb1d07d-cedb-4a7a-8a07-ccb611605e1d None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:42:59.919157 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-beb1d07d-cedb-4a7a-8a07-ccb611605e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:42:59.950363 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:42:59.992918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 630/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:42:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:43:00.015614 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c42fae-bf90-4441-814f-2adc6b534d39 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:43:00.052698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:43:00.133102 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c42fae-bf90-4441-814f-2adc6b534d39 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:43:00.173235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede973b-b45d-450d-ac0a-6cff6d2065f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:43:00.173900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede973b-b45d-450d-ac0a-6cff6d2065f6 None admin] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:00.174224 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede973b-b45d-450d-ac0a-6cff6d2065f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:00.174501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede973b-b45d-450d-ac0a-6cff6d2065f6 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:00.175205 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede973b-b45d-450d-ac0a-6cff6d2065f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:43:00.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede973b-b45d-450d-ac0a-6cff6d2065f6 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:43:00.194929 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 665/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:43:00 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:43:02.233859 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c1944-abc3-4a8e-8367-3a454f045aa9 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:02.234516 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c1944-abc3-4a8e-8367-3a454f045aa9 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:02.234516 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c1944-abc3-4a8e-8367-3a454f045aa9 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:02.235871 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 631/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:43:02.242354 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def91c46-a562-4027-97e6-d81af7af0a67 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:02.243143 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def91c46-a562-4027-97e6-d81af7af0a67 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:02.243334 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def91c46-a562-4027-97e6-d81af7af0a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:02.266926 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-def91c46-a562-4027-97e6-d81af7af0a67 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:43:02.268139 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-def91c46-a562-4027-97e6-d81af7af0a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:43:02.355219 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 666/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:43:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:43:02.387021 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c1944-abc3-4a8e-8367-3a454f045aa9 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:43:02.505800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c1944-abc3-4a8e-8367-3a454f045aa9 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:43:02.544624 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:43:02.545274 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:02.545557 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:02.545830 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] PATH_INFO: `/v3/services/placement` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:02.546492 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:43:02.547901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:43:02.557198 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:43:02.558721 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:43:02.559977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:43:02.562296 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 632/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:43:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:43:02.565003 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def91c46-a562-4027-97e6-d81af7af0a67 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:43:02.586690 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:43:02.681038 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def91c46-a562-4027-97e6-d81af7af0a67 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:43:02.723392 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:43:02.724039 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:02.724321 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:02.724597 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:02.725292 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] RBAC: Adding query filter params (name=placement) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:43:02.725597 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:43:02.727012 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:43:02.732546 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 667/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:43:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:43:02.740742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:43:02.779747 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:43:02.855911 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452afd8-4928-490e-b796-7647e80a0956 None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:43:02.892438 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c151f-d0b0-433d-971b-afb36abfcecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:43:02.893081 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c151f-d0b0-433d-971b-afb36abfcecf None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:02.893365 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c151f-d0b0-433d-971b-afb36abfcecf None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:02.893645 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c151f-d0b0-433d-971b-afb36abfcecf None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:02.894459 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c151f-d0b0-433d-971b-afb36abfcecf None admin] RBAC: Adding query filter params (interface=public, service_id=e89e482a25b042e3a8fd2581b24996fe, region_id=RegionOne) {{(pid=277713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:43:02.894773 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c151f-d0b0-433d-971b-afb36abfcecf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:43:02.896179 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c151f-d0b0-433d-971b-afb36abfcecf None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:43:02.901651 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 633/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:43:02 2022] GET /identity/v3/endpoints?service_id=e89e482a25b042e3a8fd2581b24996fe&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:43:02.907706 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:43:02.996574 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:43:03.018890 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4a874-96b8-4122-8241-3c794b3fce4e None admin] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:43:03.064832 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f3552-5bb9-4ab3-af42-13bc04a6f235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:43:03.065395 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f3552-5bb9-4ab3-af42-13bc04a6f235 None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:03.065675 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f3552-5bb9-4ab3-af42-13bc04a6f235 None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:03.065949 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f3552-5bb9-4ab3-af42-13bc04a6f235 None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:03.066640 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f3552-5bb9-4ab3-af42-13bc04a6f235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:43:03.068024 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f3552-5bb9-4ab3-af42-13bc04a6f235 None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:43:03.075337 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 668/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:43:02 2022] GET /identity/v3/services => generated 2191 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:43:05.082757 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d9ae-8b95-45b9-971e-6615887cf7f3 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:05.083315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d9ae-8b95-45b9-971e-6615887cf7f3 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:05.083315 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d9ae-8b95-45b9-971e-6615887cf7f3 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:05.084615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 634/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:43:05.090551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acaa22-6dc6-44ce-867c-5d82afaf3d90 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:05.091012 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acaa22-6dc6-44ce-867c-5d82afaf3d90 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:05.091113 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acaa22-6dc6-44ce-867c-5d82afaf3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:05.112447 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-79acaa22-6dc6-44ce-867c-5d82afaf3d90 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:43:05.113699 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-79acaa22-6dc6-44ce-867c-5d82afaf3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:43:05.189111 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 669/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:43:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:43:05.213576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494d9ae-8b95-45b9-971e-6615887cf7f3 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:43:05.325639 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:43:05.334573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494d9ae-8b95-45b9-971e-6615887cf7f3 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:43:05.369151 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:43:05.369811 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:05.370095 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:05.370404 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] PATH_INFO: `/v3/services/placement` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:05.371046 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:43:05.372457 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:43:05.381573 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:43:05.383051 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application with Lock( Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: ERROR keystone.server.flask.application Jul 28 08:43:05.384323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 635/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:43:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:43:05.389235 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acaa22-6dc6-44ce-867c-5d82afaf3d90 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:43:05.471326 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:43:05.496210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acaa22-6dc6-44ce-867c-5d82afaf3d90 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:43:05.533099 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51244bb3-8293-4f81-b1aa-cb8e9217ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:43:05.533739 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51244bb3-8293-4f81-b1aa-cb8e9217ec5d None admin] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:05.534026 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51244bb3-8293-4f81-b1aa-cb8e9217ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:05.534339 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51244bb3-8293-4f81-b1aa-cb8e9217ec5d None admin] PATH_INFO: `/v3/services` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:05.535029 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51244bb3-8293-4f81-b1aa-cb8e9217ec5d None admin] RBAC: Adding query filter params (name=placement) {{(pid=277712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:43:05.535341 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51244bb3-8293-4f81-b1aa-cb8e9217ec5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:43:05.536734 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51244bb3-8293-4f81-b1aa-cb8e9217ec5d None admin] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:43:05.542582 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 670/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:43:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:43:05.549130 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:43:05.664236 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db131427-9f1d-4a86-aaaa-9e549e5cfa2a None admin] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:43:05.704018 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fcd04-09dd-4b6c-9044-a8ae1b74c3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b56a4c63e0f64f7182c22666f3fab6a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:43:05.704736 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fcd04-09dd-4b6c-9044-a8ae1b74c3cc None admin] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:43:05.704993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fcd04-09dd-4b6c-9044-a8ae1b74c3cc None admin] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:43:05.705349 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fcd04-09dd-4b6c-9044-a8ae1b74c3cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:43:05.706184 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fcd04-09dd-4b6c-9044-a8ae1b74c3cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:43:05.707598 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fcd04-09dd-4b6c-9044-a8ae1b74c3cc None admin] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:43:05.730497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 636/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:43:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:43:15.539027 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=30 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:43:15.725749 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:14.136427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7527c-9664-43c3-a0a1-f1e10b2e6454 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:14.137025 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7527c-9664-43c3-a0a1-f1e10b2e6454 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:14.137858 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7527c-9664-43c3-a0a1-f1e10b2e6454 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:14.139758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 671/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:14.145238 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53051b1d-4e3b-450f-8df9-c45930e5b30f None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:14.145812 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53051b1d-4e3b-450f-8df9-c45930e5b30f None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:14.145812 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53051b1d-4e3b-450f-8df9-c45930e5b30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:14.313584 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-53051b1d-4e3b-450f-8df9-c45930e5b30f None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:14.315020 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-53051b1d-4e3b-450f-8df9-c45930e5b30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:14.545911 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 637/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:14.626623 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442201a-07b5-4064-958d-8b634a43846a None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:14.626911 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442201a-07b5-4064-958d-8b634a43846a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:14.627191 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442201a-07b5-4064-958d-8b634a43846a None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:14.628881 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 672/1309] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 08:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:14.635501 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408b728-0cb5-481b-959b-3c9be51280b4 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:14.635818 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408b728-0cb5-481b-959b-3c9be51280b4 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:14.636100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408b728-0cb5-481b-959b-3c9be51280b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:14.792648 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-9408b728-0cb5-481b-959b-3c9be51280b4 None None] MFA Rules not processed for user `a1abd448d02d41cf9d969fe9b403259e`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:14.793879 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-9408b728-0cb5-481b-959b-3c9be51280b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:14.839572 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:14.966298 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 638/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 08:44:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:14.975847 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442201a-07b5-4064-958d-8b634a43846a None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:14.977710 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-0442201a-07b5-4064-958d-8b634a43846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:15.241687 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442201a-07b5-4064-958d-8b634a43846a None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:15.400542 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa747f6-986b-4c68-a781-e17634bfc525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:15.401203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa747f6-986b-4c68-a781-e17634bfc525 service neutron] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:15.401483 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa747f6-986b-4c68-a781-e17634bfc525 service neutron] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:15.401753 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa747f6-986b-4c68-a781-e17634bfc525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:15.403561 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-7fa747f6-986b-4c68-a781-e17634bfc525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:15.504449 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:15.586047 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa747f6-986b-4c68-a781-e17634bfc525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:15.588125 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa747f6-986b-4c68-a781-e17634bfc525 service neutron] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:15.622551 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 673/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:18.082034 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cdf33-da50-4c51-992b-222519b389f8 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:18.082647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cdf33-da50-4c51-992b-222519b389f8 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:18.082647 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cdf33-da50-4c51-992b-222519b389f8 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:18.083935 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 639/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:18.089550 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e76b6c-d7c1-474e-8391-acf4b64426c6 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:18.090017 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e76b6c-d7c1-474e-8391-acf4b64426c6 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:18.090160 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e76b6c-d7c1-474e-8391-acf4b64426c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:18.176797 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-d5e76b6c-d7c1-474e-8391-acf4b64426c6 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:18.178015 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-d5e76b6c-d7c1-474e-8391-acf4b64426c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:18.278222 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 674/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:18.316660 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4cdf33-da50-4c51-992b-222519b389f8 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:18.318267 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-8d4cdf33-da50-4c51-992b-222519b389f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:18.446080 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4cdf33-da50-4c51-992b-222519b389f8 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:18.527694 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:18.530976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad89bee-631c-4edc-9a97-97f014bba128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:18.531976 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad89bee-631c-4edc-9a97-97f014bba128 service neutron] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:18.532259 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad89bee-631c-4edc-9a97-97f014bba128 service neutron] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:18.532908 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad89bee-631c-4edc-9a97-97f014bba128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:18.534896 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-4ad89bee-631c-4edc-9a97-97f014bba128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:18.650852 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad89bee-631c-4edc-9a97-97f014bba128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:18.652323 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad89bee-631c-4edc-9a97-97f014bba128 service neutron] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:18.671970 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 640/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 08:44:20.999289 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d1a7-f177-403e-a42f-5b63cead8911 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:20.999843 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d1a7-f177-403e-a42f-5b63cead8911 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:20.999843 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d1a7-f177-403e-a42f-5b63cead8911 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:21.001176 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 675/1315] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:21.006557 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadaaf6-605a-4ecb-b0b3-bfe9ac901f80 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:21.006997 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadaaf6-605a-4ecb-b0b3-bfe9ac901f80 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:21.007125 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadaaf6-605a-4ecb-b0b3-bfe9ac901f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:21.078423 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-5eadaaf6-605a-4ecb-b0b3-bfe9ac901f80 None None] MFA Rules not processed for user `20f19629679940f0b5f79bf0614b5da6`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:21.079687 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-5eadaaf6-605a-4ecb-b0b3-bfe9ac901f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:21.135213 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:21.182873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 641/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:21.225065 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2d1a7-f177-403e-a42f-5b63cead8911 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:21.226728 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-47f2d1a7-f177-403e-a42f-5b63cead8911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:21.237198 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:21.337134 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2d1a7-f177-403e-a42f-5b63cead8911 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:21.415824 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857a51e-30ba-4767-9432-ecd860878189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:21.416471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857a51e-30ba-4767-9432-ecd860878189 service neutron] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:21.416763 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857a51e-30ba-4767-9432-ecd860878189 service neutron] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:21.417044 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857a51e-30ba-4767-9432-ecd860878189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:21.418760 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-3857a51e-30ba-4767-9432-ecd860878189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:21.488101 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:21.537255 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857a51e-30ba-4767-9432-ecd860878189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:21.538756 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857a51e-30ba-4767-9432-ecd860878189 service neutron] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:21.557852 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 676/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:24.146301 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfbc3-d8d7-435c-9315-2b64d0ad7387 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:24.146902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfbc3-d8d7-435c-9315-2b64d0ad7387 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:24.146902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfbc3-d8d7-435c-9315-2b64d0ad7387 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:24.148253 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 642/1318] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:24.155211 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4124c-bdd4-486d-9066-4cb2bb0df3c4 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:24.155715 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4124c-bdd4-486d-9066-4cb2bb0df3c4 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:24.155833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4124c-bdd4-486d-9066-4cb2bb0df3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:24.237133 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-48a4124c-bdd4-486d-9066-4cb2bb0df3c4 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:24.238866 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-48a4124c-bdd4-486d-9066-4cb2bb0df3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:24.353109 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 677/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:24.391756 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecfbc3-d8d7-435c-9315-2b64d0ad7387 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:24.393428 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-efecfbc3-d8d7-435c-9315-2b64d0ad7387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:24.504292 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecfbc3-d8d7-435c-9315-2b64d0ad7387 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:24.582586 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea989f6-7c65-4b19-88f6-b95f2af5aac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:24.583226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea989f6-7c65-4b19-88f6-b95f2af5aac5 service neutron] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:24.583511 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea989f6-7c65-4b19-88f6-b95f2af5aac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:24.583788 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea989f6-7c65-4b19-88f6-b95f2af5aac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:24.585600 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-bea989f6-7c65-4b19-88f6-b95f2af5aac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:24.593556 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:24.712117 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea989f6-7c65-4b19-88f6-b95f2af5aac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:24.714751 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea989f6-7c65-4b19-88f6-b95f2af5aac5 service neutron] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:24.740528 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 643/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:27.870628 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d430fae-17d0-4793-9dff-1f0905224523 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:27.871226 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d430fae-17d0-4793-9dff-1f0905224523 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:27.871636 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d430fae-17d0-4793-9dff-1f0905224523 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:27.874308 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 678/1321] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:27.880954 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e85ecb-9e20-4062-9978-e2c9bb1c89a1 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:27.881564 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e85ecb-9e20-4062-9978-e2c9bb1c89a1 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:27.882146 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e85ecb-9e20-4062-9978-e2c9bb1c89a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:27.967741 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-76e85ecb-9e20-4062-9978-e2c9bb1c89a1 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:27.968720 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-76e85ecb-9e20-4062-9978-e2c9bb1c89a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:28.033453 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:28.074872 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 644/1322] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:28.111881 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d430fae-17d0-4793-9dff-1f0905224523 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:28.113470 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-9d430fae-17d0-4793-9dff-1f0905224523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:28.135649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:28.228329 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d430fae-17d0-4793-9dff-1f0905224523 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:28.316456 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770afac-8d48-45df-b90a-123c81e5952d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:28.317098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770afac-8d48-45df-b90a-123c81e5952d service neutron] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:28.317385 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770afac-8d48-45df-b90a-123c81e5952d service neutron] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:28.317665 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770afac-8d48-45df-b90a-123c81e5952d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:28.319697 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-3770afac-8d48-45df-b90a-123c81e5952d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:28.399325 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:28.442659 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770afac-8d48-45df-b90a-123c81e5952d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:28.444481 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770afac-8d48-45df-b90a-123c81e5952d service neutron] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:28.464971 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 679/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:30.545608 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4145661-75af-4e40-bc78-41d899f61a6c None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:30.546194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4145661-75af-4e40-bc78-41d899f61a6c None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:30.546194 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4145661-75af-4e40-bc78-41d899f61a6c None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:30.547606 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 645/1324] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:30.553236 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63894d-4afb-412a-9572-235d0c87276a None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:30.553698 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63894d-4afb-412a-9572-235d0c87276a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:30.553816 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63894d-4afb-412a-9572-235d0c87276a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:30.622833 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-ba63894d-4afb-412a-9572-235d0c87276a None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:30.624056 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-ba63894d-4afb-412a-9572-235d0c87276a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:30.728797 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 680/1325] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:30.765328 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4145661-75af-4e40-bc78-41d899f61a6c None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:30.767092 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-c4145661-75af-4e40-bc78-41d899f61a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:30.874047 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4145661-75af-4e40-bc78-41d899f61a6c None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:30.953511 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de58a6-5835-4301-ba40-b86694974dbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:30.954163 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de58a6-5835-4301-ba40-b86694974dbc service neutron] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:30.954576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de58a6-5835-4301-ba40-b86694974dbc service neutron] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:30.954861 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de58a6-5835-4301-ba40-b86694974dbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:30.956566 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-62de58a6-5835-4301-ba40-b86694974dbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:30.973733 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:31.084615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de58a6-5835-4301-ba40-b86694974dbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:31.086000 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de58a6-5835-4301-ba40-b86694974dbc service neutron] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:31.104858 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 646/1326] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:33.173137 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a9cf7-35e6-467f-aa3b-b0743e866cbb None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:33.173762 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a9cf7-35e6-467f-aa3b-b0743e866cbb None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:33.173762 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a9cf7-35e6-467f-aa3b-b0743e866cbb None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:33.175123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 681/1327] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:33.181898 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd9c8d-87ce-4e44-a199-91706e9618dd None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:33.182619 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd9c8d-87ce-4e44-a199-91706e9618dd None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:33.182780 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd9c8d-87ce-4e44-a199-91706e9618dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:33.264902 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-5bbd9c8d-87ce-4e44-a199-91706e9618dd None None] MFA Rules not processed for user `20f19629679940f0b5f79bf0614b5da6`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:33.266769 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-5bbd9c8d-87ce-4e44-a199-91706e9618dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:33.344043 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:33.380359 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 647/1328] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:33 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:33.436523 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bd92c-4dd2-4b31-a541-bb7af7636e2c None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:33.436848 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bd92c-4dd2-4b31-a541-bb7af7636e2c None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:33.437241 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bd92c-4dd2-4b31-a541-bb7af7636e2c None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:33.439520 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 682/1329] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 08:44:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:33.445819 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785988-11e6-4fcc-9db6-25b751028140 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:33.446144 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785988-11e6-4fcc-9db6-25b751028140 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:33.446438 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785988-11e6-4fcc-9db6-25b751028140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:33.527831 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-a5785988-11e6-4fcc-9db6-25b751028140 None None] MFA Rules not processed for user `a1abd448d02d41cf9d969fe9b403259e`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:33.529060 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-a5785988-11e6-4fcc-9db6-25b751028140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:33.614973 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 648/1330] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 08:44:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:44:33.624480 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bd92c-4dd2-4b31-a541-bb7af7636e2c None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:33.626049 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-dc8bd92c-4dd2-4b31-a541-bb7af7636e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:33.659731 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:33.745223 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bd92c-4dd2-4b31-a541-bb7af7636e2c None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:33.823896 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568232e-9fa5-4922-aa69-1c61573264ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:33.824538 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568232e-9fa5-4922-aa69-1c61573264ed service neutron] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:33.824821 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568232e-9fa5-4922-aa69-1c61573264ed service neutron] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:33.825098 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568232e-9fa5-4922-aa69-1c61573264ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:33.826809 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-7568232e-9fa5-4922-aa69-1c61573264ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:33.931660 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:33.970900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568232e-9fa5-4922-aa69-1c61573264ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:33.972659 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568232e-9fa5-4922-aa69-1c61573264ed service neutron] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:33.991764 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 683/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:33 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:36.641930 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4fe91-16c7-4159-a0fe-54c2e62e6e3a None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:36.642909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4fe91-16c7-4159-a0fe-54c2e62e6e3a None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:36.642909 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4fe91-16c7-4159-a0fe-54c2e62e6e3a None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:36.645107 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 649/1332] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:36.651481 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e2b47-a4f7-4711-84ff-d38916dbd8b9 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:36.652210 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e2b47-a4f7-4711-84ff-d38916dbd8b9 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:36.652416 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e2b47-a4f7-4711-84ff-d38916dbd8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:36.729306 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-f84e2b47-a4f7-4711-84ff-d38916dbd8b9 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:36.730593 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-f84e2b47-a4f7-4711-84ff-d38916dbd8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:36.830900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 684/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:36.873615 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4fe91-16c7-4159-a0fe-54c2e62e6e3a None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:36.875228 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-55c4fe91-16c7-4159-a0fe-54c2e62e6e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:36.961174 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:36.997660 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4fe91-16c7-4159-a0fe-54c2e62e6e3a None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:37.078733 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237286c6-f4ab-48dd-8ad2-b7fe2aabd568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:37.079430 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237286c6-f4ab-48dd-8ad2-b7fe2aabd568 service neutron] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:37.079713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237286c6-f4ab-48dd-8ad2-b7fe2aabd568 service neutron] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:37.079993 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237286c6-f4ab-48dd-8ad2-b7fe2aabd568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:37.081661 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-237286c6-f4ab-48dd-8ad2-b7fe2aabd568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:37.220284 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237286c6-f4ab-48dd-8ad2-b7fe2aabd568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:37.221713 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237286c6-f4ab-48dd-8ad2-b7fe2aabd568 service neutron] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:37.232050 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:37.241929 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 650/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:39.313704 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160628de-dadb-446d-8a33-99be88dc011a None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:39.314365 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160628de-dadb-446d-8a33-99be88dc011a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:39.314365 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160628de-dadb-446d-8a33-99be88dc011a None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:39.315671 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 685/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:39.321437 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e1c02-f9b8-49dc-8223-b37290dd80b7 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:39.321901 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e1c02-f9b8-49dc-8223-b37290dd80b7 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:39.322014 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e1c02-f9b8-49dc-8223-b37290dd80b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:39.391061 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-a63e1c02-f9b8-49dc-8223-b37290dd80b7 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:39.392460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-a63e1c02-f9b8-49dc-8223-b37290dd80b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:39.498874 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 651/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:39.539149 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160628de-dadb-446d-8a33-99be88dc011a None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:39.540722 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-160628de-dadb-446d-8a33-99be88dc011a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:39.583388 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:39.652577 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160628de-dadb-446d-8a33-99be88dc011a None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:39.729471 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99492cb-681a-4a1d-b42e-8f12d5a37802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:39.730205 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99492cb-681a-4a1d-b42e-8f12d5a37802 service neutron] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:39.730414 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99492cb-681a-4a1d-b42e-8f12d5a37802 service neutron] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:39.730689 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99492cb-681a-4a1d-b42e-8f12d5a37802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:39.732341 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-f99492cb-681a-4a1d-b42e-8f12d5a37802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:39.827817 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:39.856360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99492cb-681a-4a1d-b42e-8f12d5a37802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:39.857733 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99492cb-681a-4a1d-b42e-8f12d5a37802 service neutron] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:39.877915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 686/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:42.379100 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c42374-c36a-47ab-b2dd-b98010a64fbd None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:42.379809 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c42374-c36a-47ab-b2dd-b98010a64fbd None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:42.379809 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c42374-c36a-47ab-b2dd-b98010a64fbd None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:42.381062 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 652/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:42.387643 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e618d-66d0-467e-b176-72bf39ad6843 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:42.388444 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e618d-66d0-467e-b176-72bf39ad6843 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:42.388634 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e618d-66d0-467e-b176-72bf39ad6843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:42.461545 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-223e618d-66d0-467e-b176-72bf39ad6843 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:42.462813 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-223e618d-66d0-467e-b176-72bf39ad6843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:42.562155 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 687/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:42.600210 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c42374-c36a-47ab-b2dd-b98010a64fbd None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:42.601781 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-f1c42374-c36a-47ab-b2dd-b98010a64fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:42.683873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:42.715421 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c42374-c36a-47ab-b2dd-b98010a64fbd None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:42.805180 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c607b6-c00b-41ed-a83f-bfdb3121f5c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:42.805816 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c607b6-c00b-41ed-a83f-bfdb3121f5c4 service neutron] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:42.806086 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c607b6-c00b-41ed-a83f-bfdb3121f5c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:42.806422 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c607b6-c00b-41ed-a83f-bfdb3121f5c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:42.808089 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-99c607b6-c00b-41ed-a83f-bfdb3121f5c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:42.920911 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c607b6-c00b-41ed-a83f-bfdb3121f5c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:42.922445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c607b6-c00b-41ed-a83f-bfdb3121f5c4 service neutron] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:42.947254 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 653/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:46.946496 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5c0cc-9ca8-4661-8a82-93ed2c96675a None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:46.947043 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5c0cc-9ca8-4661-8a82-93ed2c96675a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:46.947360 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5c0cc-9ca8-4661-8a82-93ed2c96675a None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:46.949546 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 688/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:44:46.955141 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93e4ea-4e8d-44d5-864c-4f27fc86e544 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:46.955553 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93e4ea-4e8d-44d5-864c-4f27fc86e544 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:46.955701 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93e4ea-4e8d-44d5-864c-4f27fc86e544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:46.961858 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:47.027620 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-be93e4ea-4e8d-44d5-864c-4f27fc86e544 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:47.028875 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-be93e4ea-4e8d-44d5-864c-4f27fc86e544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:47.132764 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 654/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:47.171681 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5c0cc-9ca8-4661-8a82-93ed2c96675a None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:47.173263 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-dac5c0cc-9ca8-4661-8a82-93ed2c96675a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:47.226180 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:47.283446 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5c0cc-9ca8-4661-8a82-93ed2c96675a None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:47.358886 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997365e-eef5-4a35-b869-044b7df6acb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:47.359528 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997365e-eef5-4a35-b869-044b7df6acb7 service neutron] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:47.359811 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997365e-eef5-4a35-b869-044b7df6acb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:47.360086 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997365e-eef5-4a35-b869-044b7df6acb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:47.361768 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-f997365e-eef5-4a35-b869-044b7df6acb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:47.459823 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:47.478186 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997365e-eef5-4a35-b869-044b7df6acb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:47.479580 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997365e-eef5-4a35-b869-044b7df6acb7 service neutron] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:47.501138 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 689/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:50.296843 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a298c-901f-46dc-8699-8f5f660164ab None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:50.297496 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a298c-901f-46dc-8699-8f5f660164ab None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:50.297496 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a298c-901f-46dc-8699-8f5f660164ab None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:50.298848 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 655/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:44:50.304781 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1445c-5cb9-49d2-a21e-62af23f35472 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:50.305349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1445c-5cb9-49d2-a21e-62af23f35472 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:50.305349 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1445c-5cb9-49d2-a21e-62af23f35472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:50.381825 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-75e1445c-5cb9-49d2-a21e-62af23f35472 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:50.383115 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-75e1445c-5cb9-49d2-a21e-62af23f35472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:50.491747 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 690/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:50.529383 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a298c-901f-46dc-8699-8f5f660164ab None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:50.531013 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-bb7a298c-901f-46dc-8699-8f5f660164ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:50.623584 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:50.642766 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a298c-901f-46dc-8699-8f5f660164ab None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:50.725162 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29e4ea-b547-4994-9d27-005179f7c303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:50.725918 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29e4ea-b547-4994-9d27-005179f7c303 service neutron] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:50.726289 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29e4ea-b547-4994-9d27-005179f7c303 service neutron] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:50.726567 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29e4ea-b547-4994-9d27-005179f7c303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:50.728460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-fe29e4ea-b547-4994-9d27-005179f7c303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:50.853025 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29e4ea-b547-4994-9d27-005179f7c303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:50.854552 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29e4ea-b547-4994-9d27-005179f7c303 service neutron] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:50.875523 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 656/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:44:54.919893 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d093244-d438-4dd5-b59e-9929568c686a None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:54.920479 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d093244-d438-4dd5-b59e-9929568c686a None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:54.920479 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d093244-d438-4dd5-b59e-9929568c686a None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:54.921828 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 691/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:44:54.927616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428faff-e030-479d-9206-a6d5a83f8173 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:54.928118 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428faff-e030-479d-9206-a6d5a83f8173 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:54.928234 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428faff-e030-479d-9206-a6d5a83f8173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:54.945116 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:55.004841 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-9428faff-e030-479d-9206-a6d5a83f8173 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:44:55.006089 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-9428faff-e030-479d-9206-a6d5a83f8173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:55.112681 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 657/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:44:55.151105 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d093244-d438-4dd5-b59e-9929568c686a None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:44:55.152691 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-5d093244-d438-4dd5-b59e-9929568c686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:55.214432 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:55.264914 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d093244-d438-4dd5-b59e-9929568c686a None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:44:55.343484 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c02eb-d0e7-44af-902a-0f66fb453359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1abd448d02d41cf9d969fe9b403259e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:44:55.344196 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c02eb-d0e7-44af-902a-0f66fb453359 service neutron] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:44:55.344529 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c02eb-d0e7-44af-902a-0f66fb453359 service neutron] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:44:55.344829 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c02eb-d0e7-44af-902a-0f66fb453359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:44:55.346647 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-3a8c02eb-d0e7-44af-902a-0f66fb453359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:44:55.457132 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:44:55.461436 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c02eb-d0e7-44af-902a-0f66fb453359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:44:55.463326 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c02eb-d0e7-44af-902a-0f66fb453359 service neutron] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:44:55.483705 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 692/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:45:00.556175 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077a29-4ea6-4661-ad5b-a8cb790b8fd5 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:00.556692 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077a29-4ea6-4661-ad5b-a8cb790b8fd5 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:00.556800 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077a29-4ea6-4661-ad5b-a8cb790b8fd5 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:00.558358 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 658/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:00.564390 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0bf6f-34b7-4b53-b7d9-1114cf5d96f8 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:00.568900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0bf6f-34b7-4b53-b7d9-1114cf5d96f8 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:00.568900 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0bf6f-34b7-4b53-b7d9-1114cf5d96f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:00.684855 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-c8a0bf6f-34b7-4b53-b7d9-1114cf5d96f8 None None] MFA Rules not processed for user `bca9fa56ced04b91b67f2c802dc90a21`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:00.686682 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-c8a0bf6f-34b7-4b53-b7d9-1114cf5d96f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:00.886545 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 693/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:45:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:05.028760 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065dbb7-2483-4e63-8bb5-8a434772d5e0 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:05.029676 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065dbb7-2483-4e63-8bb5-8a434772d5e0 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:05.029977 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065dbb7-2483-4e63-8bb5-8a434772d5e0 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:05.031607 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 659/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:05.036928 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549b79-c0ec-40a1-a280-f5785029c324 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:05.037373 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549b79-c0ec-40a1-a280-f5785029c324 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:05.037465 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549b79-c0ec-40a1-a280-f5785029c324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:05.091427 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:05.110797 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=59 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:05.118462 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-bf549b79-c0ec-40a1-a280-f5785029c324 None None] MFA Rules not processed for user `bca9fa56ced04b91b67f2c802dc90a21`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:05.120161 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-bf549b79-c0ec-40a1-a280-f5785029c324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:05.239056 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 694/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:45:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:06.655617 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1143-61e4-4f51-b05a-1e38d0bed1c7 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:06.656226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1143-61e4-4f51-b05a-1e38d0bed1c7 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:06.656226 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1143-61e4-4f51-b05a-1e38d0bed1c7 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:06.657533 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 660/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 08:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:06.663058 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3704f-8a48-43a0-839a-30d0b31ff4b0 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:06.663871 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3704f-8a48-43a0-839a-30d0b31ff4b0 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:06.664156 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3704f-8a48-43a0-839a-30d0b31ff4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:06.750993 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-dba3704f-8a48-43a0-839a-30d0b31ff4b0 None None] MFA Rules not processed for user `bca9fa56ced04b91b67f2c802dc90a21`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:06.752408 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-dba3704f-8a48-43a0-839a-30d0b31ff4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:06.797271 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:06.867284 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 695/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 08:45:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:10.353364 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01865-a281-4a84-81f4-48d8fac34802 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:10.354385 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01865-a281-4a84-81f4-48d8fac34802 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:10.354664 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01865-a281-4a84-81f4-48d8fac34802 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:10.356068 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 661/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:10.362181 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22c07-985e-44f1-9c64-ac0520eab789 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:10.362632 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22c07-985e-44f1-9c64-ac0520eab789 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:10.362758 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22c07-985e-44f1-9c64-ac0520eab789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:10.460497 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-69c22c07-985e-44f1-9c64-ac0520eab789 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:10.462228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-69c22c07-985e-44f1-9c64-ac0520eab789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:10.610000 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 696/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:10.670630 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728392-a961-465f-a4c1-ce5cfa57f5eb None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:10.671165 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728392-a961-465f-a4c1-ce5cfa57f5eb None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:10.671641 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728392-a961-465f-a4c1-ce5cfa57f5eb None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:10.673565 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 662/1358] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 08:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:10.684393 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42dad6-d165-48ce-91c0-264c4d910a17 None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:10.684945 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42dad6-d165-48ce-91c0-264c4d910a17 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:10.685598 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42dad6-d165-48ce-91c0-264c4d910a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:10.710684 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:10.778027 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-ac42dad6-d165-48ce-91c0-264c4d910a17 None None] MFA Rules not processed for user `90103f3776ce4eeb90bde66ed546aa87`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:10.779456 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-ac42dad6-d165-48ce-91c0-264c4d910a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:10.929086 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 697/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 08:45:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:10.937542 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728392-a961-465f-a4c1-ce5cfa57f5eb None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:45:10.939433 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-5a728392-a961-465f-a4c1-ce5cfa57f5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:11.097090 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728392-a961-465f-a4c1-ce5cfa57f5eb None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:45:11.174369 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:11.217497 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf1dbd-b763-4422-86e3-3e555770a332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90103f3776ce4eeb90bde66ed546aa87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:45:11.218167 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf1dbd-b763-4422-86e3-3e555770a332 service nova] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:11.218440 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf1dbd-b763-4422-86e3-3e555770a332 service nova] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:11.218742 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf1dbd-b763-4422-86e3-3e555770a332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:11.220439 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-82bf1dbd-b763-4422-86e3-3e555770a332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:11.314695 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf1dbd-b763-4422-86e3-3e555770a332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:45:11.316576 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf1dbd-b763-4422-86e3-3e555770a332 service nova] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:45:11.337230 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 663/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 08:45:13.562504 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5722835-5b28-454b-b6a3-a25f0af8b712 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:13.563080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5722835-5b28-454b-b6a3-a25f0af8b712 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:13.563080 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5722835-5b28-454b-b6a3-a25f0af8b712 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:13.564751 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 698/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:13.570802 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9127-b9e3-4db8-a9c3-0bc843691b3b None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:13.571309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9127-b9e3-4db8-a9c3-0bc843691b3b None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:13.571309 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9127-b9e3-4db8-a9c3-0bc843691b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:13.643618 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-bb6a9127-b9e3-4db8-a9c3-0bc843691b3b None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:13.644821 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-bb6a9127-b9e3-4db8-a9c3-0bc843691b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:13.695180 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:13.768128 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 664/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:13.813208 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5722835-5b28-454b-b6a3-a25f0af8b712 None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:45:13.814837 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-e5722835-5b28-454b-b6a3-a25f0af8b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:13.889750 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:13.990279 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5722835-5b28-454b-b6a3-a25f0af8b712 None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:45:14.126203 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dd4c9-8afb-4b61-80e9-787ecf415696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90103f3776ce4eeb90bde66ed546aa87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:45:14.127092 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dd4c9-8afb-4b61-80e9-787ecf415696 service nova] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:14.127495 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dd4c9-8afb-4b61-80e9-787ecf415696 service nova] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:14.127812 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dd4c9-8afb-4b61-80e9-787ecf415696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:14.129501 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-823dd4c9-8afb-4b61-80e9-787ecf415696 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:14.171383 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:14.235888 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dd4c9-8afb-4b61-80e9-787ecf415696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:45:14.237521 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dd4c9-8afb-4b61-80e9-787ecf415696 service nova] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:45:14.257772 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 699/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:45:16.625792 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a234d7d-c9f5-4960-9375-f0037e486d55 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:16.626445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a234d7d-c9f5-4960-9375-f0037e486d55 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:16.626445 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a234d7d-c9f5-4960-9375-f0037e486d55 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:16.628116 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 665/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:16.633703 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def2d98-009c-420e-9d8c-ded9df28fc6b None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:16.634229 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def2d98-009c-420e-9d8c-ded9df28fc6b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:16.634338 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def2d98-009c-420e-9d8c-ded9df28fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:16.712166 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-9def2d98-009c-420e-9d8c-ded9df28fc6b None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:16.713941 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-9def2d98-009c-420e-9d8c-ded9df28fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:16.834915 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:16.842445 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 700/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:16.885248 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaca378-ebd1-46c8-8a89-daa96dab4342 None None] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:16.886380 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaca378-ebd1-46c8-8a89-daa96dab4342 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:16.886939 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaca378-ebd1-46c8-8a89-daa96dab4342 None None] PATH_INFO: `/` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:16.895263 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 666/1366] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 08:45:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:16.901677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38f507-282b-4216-b65b-8baba94edbdc None None] REQUEST_METHOD: `POST` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:16.901968 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38f507-282b-4216-b65b-8baba94edbdc None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:16.902285 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38f507-282b-4216-b65b-8baba94edbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:17.141783 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.auth.core [None req-2c38f507-282b-4216-b65b-8baba94edbdc None None] MFA Rules not processed for user `90103f3776ce4eeb90bde66ed546aa87`. Rule list: `[]` (Enabled: `True`). {{(pid=277712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:17.143023 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-2c38f507-282b-4216-b65b-8baba94edbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:17.276123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 701/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 08:45:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:17.284271 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaca378-ebd1-46c8-8a89-daa96dab4342 None None] Authenticating user token {{(pid=277713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:45:17.286616 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-aeaca378-ebd1-46c8-8a89-daa96dab4342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:17.469771 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaca378-ebd1-46c8-8a89-daa96dab4342 None None] Validating token access rules against request {{(pid=277713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:45:17.491877 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:17.590663 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9642508-6865-45d5-9cf2-8acfc110079b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90103f3776ce4eeb90bde66ed546aa87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:45:17.591310 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642508-6865-45d5-9cf2-8acfc110079b service nova] REQUEST_METHOD: `GET` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:17.591589 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642508-6865-45d5-9cf2-8acfc110079b service nova] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:17.591862 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642508-6865-45d5-9cf2-8acfc110079b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:17.593511 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-d9642508-6865-45d5-9cf2-8acfc110079b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:17.694178 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9642508-6865-45d5-9cf2-8acfc110079b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:45:17.695861 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9642508-6865-45d5-9cf2-8acfc110079b service nova] RBAC: Authorization granted {{(pid=277713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:45:17.715660 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 667/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:45:18.316049 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac038d7-3348-40ac-8dfa-071c4463b711 None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:18.316649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac038d7-3348-40ac-8dfa-071c4463b711 None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:18.316649 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac038d7-3348-40ac-8dfa-071c4463b711 None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:18.318514 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 702/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:18.323693 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce313a43-0274-4692-ab98-5d03ae59ce03 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:18.323964 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce313a43-0274-4692-ab98-5d03ae59ce03 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:18.324218 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce313a43-0274-4692-ab98-5d03ae59ce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:18.357494 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:18.394329 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-ce313a43-0274-4692-ab98-5d03ae59ce03 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:18.395691 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-ce313a43-0274-4692-ab98-5d03ae59ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:18.498152 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 668/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 08:45:19.832668 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f272-f6fa-4203-b6c9-5878b1e96c4b None None] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:19.833240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f272-f6fa-4203-b6c9-5878b1e96c4b None None] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:19.833240 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f272-f6fa-4203-b6c9-5878b1e96c4b None None] PATH_INFO: `/` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:19.837877 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 703/1371] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:45:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:45:19.840873 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d39d3-a92f-402f-bcf9-e67f3ba1ceb6 None None] REQUEST_METHOD: `POST` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:19.841460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d39d3-a92f-402f-bcf9-e67f3ba1ceb6 None None] SCRIPT_NAME: `/identity` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:19.841460 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d39d3-a92f-402f-bcf9-e67f3ba1ceb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:19.911839 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.auth.core [None req-6d7d39d3-a92f-402f-bcf9-e67f3ba1ceb6 None None] MFA Rules not processed for user `b56a4c63e0f64f7182c22666f3fab6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=277713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:45:19.913082 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG keystone.common.fernet_utils [None req-6d7d39d3-a92f-402f-bcf9-e67f3ba1ceb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:19.970220 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=100 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:20.018078 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: [pid: 277713|app: 0|req: 669/1372] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 08:45:20.063123 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f272-f6fa-4203-b6c9-5878b1e96c4b None None] Authenticating user token {{(pid=277712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:45:20.064746 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-b6c8f272-f6fa-4203-b6c9-5878b1e96c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:20.130174 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:20.334488 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f272-f6fa-4203-b6c9-5878b1e96c4b None None] Validating token access rules against request {{(pid=277712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:45:20.477672 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a4b35-4d40-4f32-a403-b77a8db1e2aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90103f3776ce4eeb90bde66ed546aa87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813e8db425644f688e926aafe3249721', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:45:20.478342 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4b35-4d40-4f32-a403-b77a8db1e2aa service nova] REQUEST_METHOD: `GET` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:45:20.478624 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4b35-4d40-4f32-a403-b77a8db1e2aa service nova] SCRIPT_NAME: `/identity` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:45:20.478902 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4b35-4d40-4f32-a403-b77a8db1e2aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:45:20.480573 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.fernet_utils [None req-0f3a4b35-4d40-4f32-a403-b77a8db1e2aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:45:20.498228 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=100 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:20.603585 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a4b35-4d40-4f32-a403-b77a8db1e2aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:45:20.605412 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a4b35-4d40-4f32-a403-b77a8db1e2aa service nova] RBAC: Authorization granted {{(pid=277712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:45:20.625276 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: [pid: 277712|app: 0|req: 704/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 08:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 08:45:30.016478 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: DEBUG dbcounter [-] [277713] Writing DB stats keystone:SELECT=19 {{(pid=277713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:30.622515 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: DEBUG dbcounter [-] [277712] Writing DB stats keystone:SELECT=41 {{(pid=277712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:45:45.001919 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: Thu Jul 28 08:45:45 2022 - graceful shutdown triggered... Jul 28 08:45:45.002677 dv-cinder-851351-1-iscsi devstack@keystone.service[277712]: Gracefully killing worker 1 (pid: 277712)... Jul 28 08:45:45.003163 dv-cinder-851351-1-iscsi devstack@keystone.service[277713]: Gracefully killing worker 2 (pid: 277713)... Jul 28 08:45:45.003968 dv-cinder-851351-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Jul 28 08:45:46.003599 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: worker 1 buried after 1 seconds Jul 28 08:45:46.003599 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: worker 2 buried after 1 seconds Jul 28 08:45:46.003599 dv-cinder-851351-1-iscsi devstack@keystone.service[277701]: goodbye to uWSGI. Jul 28 08:45:46.006193 dv-cinder-851351-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Jul 28 08:45:46.007260 dv-cinder-851351-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. Jul 28 08:54:43.284812 dv-cinder-851351-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 08:54:43.295845 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 08:54:43 2022] *** Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: nodename: dv-cinder-851351-1-iscsi Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: machine: x86_64 Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: clock source: unix Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: pcre jit disabled Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: detected number of CPU cores: 4 Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: current working directory: / Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: detected binary path: /usr/bin/uwsgi-core Jul 28 08:54:43.297822 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: your processes number limit is 47680 Jul 28 08:54:43.299191 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: your memory page size is 4096 bytes Jul 28 08:54:43.299191 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: detected max file descriptor number: 2048 Jul 28 08:54:43.299191 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: lock engine: pthread robust mutexes Jul 28 08:54:43.299191 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: thunder lock: enabled Jul 28 08:54:43.299191 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 08:54:43.299191 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 08:54:43.338716 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: Python main interpreter initialized at 0x558453e3d9c0 Jul 28 08:54:43.338716 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: python threads support enabled Jul 28 08:54:43.338716 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: your server socket listen backlog is limited to 100 connections Jul 28 08:54:43.338716 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: your mercy for graceful operations on workers is 90 seconds Jul 28 08:54:43.339256 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 08:54:43.339256 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: *** Operational MODE: preforking *** Jul 28 08:54:43.339438 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: *** uWSGI is running in multiple interpreter mode *** Jul 28 08:54:43.339438 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: spawned uWSGI master process (pid: 347133) Jul 28 08:54:43.339720 dv-cinder-851351-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 08:54:43.340524 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: spawned uWSGI worker 1 (pid: 347146, cores: 1) Jul 28 08:54:43.340692 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: spawned uWSGI worker 2 (pid: 347147, cores: 1) Jul 28 08:54:43.340861 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 08:54:48.842814 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:54:48.842814 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: from cryptography.utils import int_from_bytes Jul 28 08:54:48.844093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:54:48.844093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: from cryptography.utils import int_from_bytes Jul 28 08:54:48.892296 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:54:48.892553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:54:48.892688 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] command line args: [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:54:48.892921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:54:48.893222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ================================================================================ {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:54:48.893489 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] admin_token = **** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.893739 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] config_dir = [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.893928 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.894127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] config_source = [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.894320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] debug = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.894518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.894904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.895076 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] insecure_debug = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.895333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.895587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.895816 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] list_limit = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.895994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] log_config_append = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.896172 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.896333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] log_dir = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.896514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] log_file = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.896751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.896998 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.897271 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.897609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.897785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.897940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.898074 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.898349 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.898584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.898817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.899051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.899282 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.899529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.899778 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] notification_format = cadf {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.900066 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.900581 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.900763 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] publish_errors = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.900996 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.901228 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.901408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.901656 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.901892 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.902151 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] standard_threads = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.902387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] strict_password_check = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.902633 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.902810 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] use_eventlog = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.902986 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] use_journal = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.903162 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] use_json = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.903323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] use_stderr = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.903498 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] use_syslog = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.903730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] watch_log_file = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.903992 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.904232 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.904495 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.904735 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.904987 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.905230 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.905483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.905721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.905970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.906242 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.906497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.906734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.906985 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.907228 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.907480 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.907730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.907969 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.908227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.908485 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.908770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.909011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.909268 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.909534 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.909792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.910245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.910432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.910589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.910803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.911040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.915297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.919088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.920315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.921529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.922819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.922819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.922819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.922819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.922819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.923391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:54:48.923391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: from cryptography.utils import int_from_bytes Jul 28 08:54:48.924081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:54:48.924486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: from cryptography.utils import int_from_bytes Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.enabled = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.924751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.925951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.927201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.927201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.927201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.927201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.927201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.927201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.927201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.927201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.928345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.928345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.928345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.928345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.928899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.929042 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.929439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] auth.external = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.929824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] auth.mapped = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.930164 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.930494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.930928 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] auth.password = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.931362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] auth.token = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.931823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.932206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] catalog.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.934382 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.934740 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.935220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.935700 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.936144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.936466 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] credential.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.936739 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] credential.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.937009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.937284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.937543 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.937800 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.938055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.938362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.938627 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.938875 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.939141 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.939447 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.939697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.940030 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.940310 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.940582 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.940826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] federation.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.941074 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.941664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] federation.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.941932 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.942203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.942455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.942703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.942969 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.943211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.943470 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.943714 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.943969 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.944218 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.944468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.944721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.944971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.945213 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.945460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.945699 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.945945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.946220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.946466 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.946702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.946939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.947188 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.947438 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.947684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.947936 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.948203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.948449 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.948793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.949100 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.949355 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.949589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.949828 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.950068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.950322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.950565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.950815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.951049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.951287 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.951541 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.951796 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.952288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.952539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.952777 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.953020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.953255 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.password = **** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.953497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.953737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.953988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.954258 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.954491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.954722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.954974 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.955211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.955451 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.955741 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.955996 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.956235 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.956473 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.956711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.956945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.957204 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.957459 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.957700 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.957954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.958560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.958560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.958999 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.958999 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.959436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.959436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.959873 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.959873 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.960481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.960481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.960481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.961134 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.961134 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.961549 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.961549 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.962005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.962005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.962551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.962551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.962551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.963180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.963180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.963527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] policy.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.963527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.963977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.963977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.964430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] receipt.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.964430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.964884 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.964884 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.965338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.965338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.965767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] resource.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.966036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.966036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] resource.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.966520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.966520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.966938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.967160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] revoke.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.967396 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.967627 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.967859 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] role.cache_time = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.968111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] role.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.968111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] role.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.968537 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] role.list_limit = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.968892 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.968892 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.969428 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.969428 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.969863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.969863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.970371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.970371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.970799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.970799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.971338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.971338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.971804 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:54:48.971804 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:54:48.972245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.972490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] command line args: [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:54:48.972490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:54:48.972825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.973104 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ================================================================================ {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:54:48.973351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.973586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] admin_token = **** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.973816 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.974052 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] config_dir = [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.974250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.974411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.974643 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.974867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] config_source = [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.974867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] debug = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.975312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.975553 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.975838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.976091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.976326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.976560 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] insecure_debug = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.976787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.977048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.977293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.977527 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.977719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.977941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] list_limit = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.977941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] log_config_append = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.978422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.978641 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.978861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.979077 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] log_dir = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.979460 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] log_file = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.979610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.979759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.979759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.980143 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.980143 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.980464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.980618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.980852 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.981231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] token.cache_time = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.981438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.981576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] token.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.981729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.981884 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.982029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.982202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] token.provider = fernet {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.982351 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.982496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.982654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.982792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.982929 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.983082 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.983228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.983374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.983601 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.983794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.983981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] notification_format = cadf {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.984152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.984300 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.984456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] trust.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.984456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.984753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.984909 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.985099 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] publish_errors = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.985326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.985500 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.985640 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.985778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.985778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.986176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.986355 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.986511 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.986664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.986839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.987050 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] standard_threads = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.987203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.987386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] strict_password_check = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.987539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.connection = **** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.987703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.987893 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.988062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] use_eventlog = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.988062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] use_journal = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.988337 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.988482 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] use_json = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.988622 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.988773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] use_stderr = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.988773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] use_syslog = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.989045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.989188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] watch_log_file = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:54:48.989333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.989478 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.989624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.989624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.989902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.990048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.990217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.990357 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.990517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.990681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.990827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.990971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.991118 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.991262 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.991400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.991551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.991696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.991851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.991992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.992128 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.992265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.992416 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.992562 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.992709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.992856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=347147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.993455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.994701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.995883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.997103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.998418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:48.999565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.000657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.enabled = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.001744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.002893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.002893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.002893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.002893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.002893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.003486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.003486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.003724 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.003864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.004019 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.004264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.004501 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.004752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.004995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.005256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.005500 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.005761 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.006032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.006316 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.006559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] auth.external = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.006795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] auth.mapped = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.007105 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.007344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.007581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] auth.password = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.007817 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] auth.token = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.008065 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.008308 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] catalog.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.008561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.008798 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.009053 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.009306 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.009544 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.009785 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] credential.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.010038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] credential.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.010323 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.010576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.010819 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.011070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.011310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.011603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.011864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.012108 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.012377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.012675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.012928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.013196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.013446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.013950 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] federation.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.014234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.014493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] federation.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.014743 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.014981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.015232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.015479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.015741 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.015984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.016244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.016486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.016750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.017000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.017759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.017759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.017759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.018289 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.018289 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.018585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.018731 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.018981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.019238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.019476 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.019714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.019953 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.020209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.020460 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.020715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.020983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.021239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.021516 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.021766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.022001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.022342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.022555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.022788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.023021 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.023240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.023455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.023674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.023902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.024113 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.024345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.024569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.024797 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.025047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.025285 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.025511 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.025763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.password = **** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.025988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.026262 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.026491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.026716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.026926 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.027160 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.027398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.027644 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.027860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.028087 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.028310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.028543 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.028760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.029011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.029242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.029486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.029757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.029992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.030275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.030537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.030791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.031012 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.031263 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.031488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.031732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.031977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.032215 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.032467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.032720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.032971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.033225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.033490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.033789 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.034041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.034309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.035473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.035473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.035473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.035473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.035473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.036534 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.036534 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] policy.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.036534 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.036534 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.037277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.037277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] receipt.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.037277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.038038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.038038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.038038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.038712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.038712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] resource.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.038712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.039381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] resource.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.039381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.039381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.040025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.040025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] revoke.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.040466 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.040466 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.040909 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] role.cache_time = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.040909 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] role.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.041369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] role.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.041369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] role.list_limit = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.041819 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.041819 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.042282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.042282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.042718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.042718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.043084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.043324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.043560 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.043795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.044010 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.044241 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.044484 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.044766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.045016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.045275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.045514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.045754 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.046018 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.046335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.046573 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.046799 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.047026 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.047260 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.047490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.047711 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.047935 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.048171 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.048391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.048608 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.048855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.049092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] token.cache_time = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.049363 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] token.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.049581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.049813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] token.provider = fernet {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.050060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.050337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.050566 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.050782 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.051026 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.051261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.051520 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] trust.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.051768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.052017 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.052260 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.052563 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.052809 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.053024 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.053228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.053512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.053776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.connection = **** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.054026 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.054287 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.054535 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.054778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.055018 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.055257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.055493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.055732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.055988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.056226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.056471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.056707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.056965 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.057205 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.057454 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.057693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.057936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.058198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:54:49.058372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=347146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:54:49.089750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.090081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.090302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.099078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.102079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.102332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.102563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.111405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.129505 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.129776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.130033 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.130269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.183648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.185018 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.234449 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.234697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.236044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.236280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.301677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.301953 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.302096 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.302345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.302568 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.302786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.302977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.303164 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.303331 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.303559 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.303777 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.303994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.326875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.327177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.327314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.327478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.327649 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.327844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.327991 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.328104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.328208 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.328304 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.328471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.328602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.328719 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.328868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.339838 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.340085 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.353283 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.353493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.353604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.353720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.353829 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.353939 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.354048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.354048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.354269 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.354371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.354480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.354585 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=347147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.354690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.354866 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=347146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:54:49.371897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.372216 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.372408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:54:49.372613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:54:49.372792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.372966 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.373165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:54:49.373363 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:54:49.373554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.373728 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.373897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.374063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.374311 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.374479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.374657 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.374846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.383303 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.383565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.383706 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.383806 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.383920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.384023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.384229 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.384451 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.391221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.391357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.394260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.394544 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.394824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.395101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.395500 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.395756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.396029 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.396221 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.396339 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.396499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.396623 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.396768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.396877 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.397063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.397175 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.397348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.397603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.397871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.398155 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.398432 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.398703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.398971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.419939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.421440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.421707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.421942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.423485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.423768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.426562 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.426845 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.427125 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.428507 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.428741 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.429016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.433579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.433826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.435615 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.435885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.438058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.438370 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.438660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.438938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.439227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.439503 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.439780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.440076 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.440189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.440467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.440752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.441035 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.441307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.441962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.442255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.442531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.456855 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.457123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.458555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.458841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.462738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.462902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.465870 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.466014 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.466203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.466325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.470464 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.470633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.473812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.473976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.476992 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.477171 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.477293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.477396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.477555 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.477671 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.477840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.477956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.478137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.478249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.478437 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.478576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.494512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.494788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.494981 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.495144 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.495302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.495508 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.495670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.495823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.502301 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.502530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.503988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.504267 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.504788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.505214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.507000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.507277 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.507558 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.507959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.508367 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.508543 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.508742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.509031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.509317 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.519709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.520238 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.520238 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.520773 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.521092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.521374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.521481 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.521611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.521729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.523388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.523586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.523586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.523586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.523586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.537291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.538271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.540315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.541263 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.543274 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.544231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.544751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.546407 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.546694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.546931 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.547068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.548436 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.548732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.549043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.553550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.555099 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.555645 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.557262 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.558143 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.558436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.558719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.559009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.560196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.560484 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.560767 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.561042 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.567356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.567641 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.567922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.569530 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.569826 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.570131 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.574918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=347147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.575202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.575484 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.575781 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.576068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.576347 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.576618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.576888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.577062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=347146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:54:49.577201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.577342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.577525 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.577632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.577767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.577938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.578096 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.578342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.578536 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.578714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.578883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=347147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.579025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.579025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.579360 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.579523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.579813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.580039 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.580308 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.580638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=347146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:54:49.613538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558453e3d9c0 pid: 347147 (default app) Jul 28 08:54:49.614081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558453e3d9c0 pid: 347146 (default app) Jul 28 08:54:49.616985 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a3409-0e21-4cd4-a328-f2b74eea4162 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:54:49.617277 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a3409-0e21-4cd4-a328-f2b74eea4162 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:54:49.617552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a3409-0e21-4cd4-a328-f2b74eea4162 None None] PATH_INFO: `/v3` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:54:49.621809 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jul 28 08:54:49 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:54:57.537924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed883d-2b0c-4629-909f-59a314aaf56c None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:54:57.538645 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed883d-2b0c-4629-909f-59a314aaf56c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:54:57.538645 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed883d-2b0c-4629-909f-59a314aaf56c None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:54:57.543321 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:54:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:54:57.549720 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:54:57.550385 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:54:57.550385 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:54:57.621589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: INFO dbcounter [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] Registered counter for database keystone Jul 28 08:54:57.644343 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG oslo_db.sqlalchemy.engines [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=347147) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:54:57.651784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writer thread running {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:54:57.834149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG passlib.handlers.bcrypt [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=347147) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:54:57.834506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG passlib.handlers.bcrypt [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=347147) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:54:57.891551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:54:57.893300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:54:58.035757 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:54:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:54:58.059533 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed883d-2b0c-4629-909f-59a314aaf56c None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:54:58.136509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: INFO dbcounter [None req-aeed883d-2b0c-4629-909f-59a314aaf56c None None] Registered counter for database keystone Jul 28 08:54:58.156777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG oslo_db.sqlalchemy.engines [None req-aeed883d-2b0c-4629-909f-59a314aaf56c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=347146) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:54:58.164408 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writer thread running {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:54:58.285912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed883d-2b0c-4629-909f-59a314aaf56c None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:54:58.291510 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d467f7-deab-487b-af4c-16112f2a6b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9904f995f8d54e70b75420b12eb6d746', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:54:58.292005 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d467f7-deab-487b-af4c-16112f2a6b5d admin admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:54:58.292288 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d467f7-deab-487b-af4c-16112f2a6b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:54:58.292562 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d467f7-deab-487b-af4c-16112f2a6b5d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:54:58.302567 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d467f7-deab-487b-af4c-16112f2a6b5d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:54:58.354617 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c8d467f7-deab-487b-af4c-16112f2a6b5d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:54:58.354942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d467f7-deab-487b-af4c-16112f2a6b5d admin admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:54:58.366143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:54:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:54:58.372859 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:54:58.388467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea873b-4514-40e2-ad1c-4ae8346a0153 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:54:58.393402 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e9bc1-c428-4a8c-88f0-cc75abfeb2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9904f995f8d54e70b75420b12eb6d746', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:54:58.394599 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9bc1-c428-4a8c-88f0-cc75abfeb2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:54:58.394884 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9bc1-c428-4a8c-88f0-cc75abfeb2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:54:58.395161 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9bc1-c428-4a8c-88f0-cc75abfeb2f3 admin admin] PATH_INFO: `/v3/projects/9904f995f8d54e70b75420b12eb6d746` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:54:58.396890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e9bc1-c428-4a8c-88f0-cc75abfeb2f3 admin admin] RBAC: Authorizing `identity:get_project(project_id=9904f995f8d54e70b75420b12eb6d746)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:54:58.450940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: WARNING keystone.common.rbac_enforcer.enforcer [None req-743e9bc1-c428-4a8c-88f0-cc75abfeb2f3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:54:58.451258 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e9bc1-c428-4a8c-88f0-cc75abfeb2f3 admin admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:54:58.456178 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jul 28 08:54:58 2022] GET /identity/v3/projects/9904f995f8d54e70b75420b12eb6d746 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:00.494515 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9ea4b-68f5-4c5c-b530-4f69b5e548cf None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:00.495126 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9ea4b-68f5-4c5c-b530-4f69b5e548cf None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:00.495126 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9ea4b-68f5-4c5c-b530-4f69b5e548cf None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:00.499270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:00.505215 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b891-d9b8-463c-adcb-a19b5ec12383 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:00.505715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b891-d9b8-463c-adcb-a19b5ec12383 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:00.505715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b891-d9b8-463c-adcb-a19b5ec12383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:00.527865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-aa08b891-d9b8-463c-adcb-a19b5ec12383 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:00.529456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-aa08b891-d9b8-463c-adcb-a19b5ec12383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:00.539557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:55:00.561676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9ea4b-68f5-4c5c-b530-4f69b5e548cf None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:00.576773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9ea4b-68f5-4c5c-b530-4f69b5e548cf None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:00.581810 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd4893-269a-49f7-b40f-0ae585a3d5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9904f995f8d54e70b75420b12eb6d746', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:00.582491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd4893-269a-49f7-b40f-0ae585a3d5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:00.582778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd4893-269a-49f7-b40f-0ae585a3d5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:00.583058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd4893-269a-49f7-b40f-0ae585a3d5a9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:00.591315 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd4893-269a-49f7-b40f-0ae585a3d5a9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:00.593167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd4893-269a-49f7-b40f-0ae585a3d5a9 admin admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:00.602540 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 08:55:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 08:55:00.608643 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b891-d9b8-463c-adcb-a19b5ec12383 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:00.618679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b891-d9b8-463c-adcb-a19b5ec12383 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:00.624888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0d06b-cd30-44b6-8715-1fe189e9a438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9904f995f8d54e70b75420b12eb6d746', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:00.625573 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0d06b-cd30-44b6-8715-1fe189e9a438 admin admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:00.625854 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0d06b-cd30-44b6-8715-1fe189e9a438 admin admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:00.626737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0d06b-cd30-44b6-8715-1fe189e9a438 admin admin] PATH_INFO: `/v3/users/27091c013a8c447a81380abde9ae7c70` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:00.628096 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0d06b-cd30-44b6-8715-1fe189e9a438 admin admin] RBAC: Authorizing `identity:get_user(user_id=27091c013a8c447a81380abde9ae7c70)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:00.632322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0d06b-cd30-44b6-8715-1fe189e9a438 admin admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:00.635804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jul 28 08:55:00 2022] GET /identity/v3/users/27091c013a8c447a81380abde9ae7c70 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:02.694712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275138a-4a0e-4690-a635-35436303fccb None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:02.695297 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275138a-4a0e-4690-a635-35436303fccb None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:02.695297 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275138a-4a0e-4690-a635-35436303fccb None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:02.696644 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:02.702042 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2112bd1-83a9-4740-a3c9-8f8b170be0b6 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:02.702560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2112bd1-83a9-4740-a3c9-8f8b170be0b6 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:02.702682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2112bd1-83a9-4740-a3c9-8f8b170be0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:02.722275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-e2112bd1-83a9-4740-a3c9-8f8b170be0b6 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:02.723462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-e2112bd1-83a9-4740-a3c9-8f8b170be0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:02.762678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c537f-d4f0-471e-936b-fae00cd3a514 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:02.762960 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c537f-d4f0-471e-936b-fae00cd3a514 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:02.763223 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c537f-d4f0-471e-936b-fae00cd3a514 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:02.764803 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:02.770088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:02.770410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:02.770690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:02.810075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:02.814555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG passlib.handlers.bcrypt [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=347146) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:55:02.814995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG passlib.handlers.bcrypt [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=347146) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:55:02.833416 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2112bd1-83a9-4740-a3c9-8f8b170be0b6 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:02.858134 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:02.860441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:02.862694 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:02.941808 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2112bd1-83a9-4740-a3c9-8f8b170be0b6 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:02.965639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:02.978780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd1a1-4d98-443a-9bc2-b4dce54c53e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:02.979451 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd1a1-4d98-443a-9bc2-b4dce54c53e3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:02.979746 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd1a1-4d98-443a-9bc2-b4dce54c53e3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:02.980029 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd1a1-4d98-443a-9bc2-b4dce54c53e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:02.992286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cd1a1-4d98-443a-9bc2-b4dce54c53e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:02.993875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1f5cd1a1-4d98-443a-9bc2-b4dce54c53e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:03.001290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:03.002674 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:03.004202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:03.008654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd1a1-4d98-443a-9bc2-b4dce54c53e3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:03.093064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:03.100927 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01c037-98cf-4a3a-a4c0-f371c3c2974e None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:03.103466 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:03.112130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd1a1-4d98-443a-9bc2-b4dce54c53e3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:03.141959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26afbe6-d6f2-4db2-a677-776be672dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:03.142953 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26afbe6-d6f2-4db2-a677-776be672dc38 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:03.143244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26afbe6-d6f2-4db2-a677-776be672dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:03.143526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26afbe6-d6f2-4db2-a677-776be672dc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:03.149678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:03.150355 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:03.150644 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:03.150922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:03.151790 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:03.152093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:03.153546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:03.166579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:03.172266 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:03.213807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26afbe6-d6f2-4db2-a677-776be672dc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:03.215455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26afbe6-d6f2-4db2-a677-776be672dc38 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:03.277732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b905c75-1343-4a5f-b810-907428f7c4d1 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:03.316922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19c01e-98b8-49be-b0a9-ad00f934b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:03.317574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19c01e-98b8-49be-b0a9-ad00f934b587 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:03.317861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19c01e-98b8-49be-b0a9-ad00f934b587 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:03.318163 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19c01e-98b8-49be-b0a9-ad00f934b587 None admin] PATH_INFO: `/v3/users/27091c013a8c447a81380abde9ae7c70` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:03.319383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19c01e-98b8-49be-b0a9-ad00f934b587 None admin] RBAC: Authorizing `identity:get_user(user_id=27091c013a8c447a81380abde9ae7c70)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:03.320311 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:55:03.321004 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19c01e-98b8-49be-b0a9-ad00f934b587 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:03.322771 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:03 2022] GET /identity/v3/users/27091c013a8c447a81380abde9ae7c70 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:03.325847 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26afbe6-d6f2-4db2-a677-776be672dc38 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:03.328110 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19c01e-98b8-49be-b0a9-ad00f934b587 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:03.328977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:03.343737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:03.427097 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26afbe6-d6f2-4db2-a677-776be672dc38 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:03.432790 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19c01e-98b8-49be-b0a9-ad00f934b587 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:03.463670 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:03.464552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:03.464836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:03.465109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:03.468614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ee94-e787-4cc5-ade7-3f14e3421535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:03.469386 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ee94-e787-4cc5-ade7-3f14e3421535 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:03.469671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ee94-e787-4cc5-ade7-3f14e3421535 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:03.469950 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ee94-e787-4cc5-ade7-3f14e3421535 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:03.471206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4ee94-e787-4cc5-ade7-3f14e3421535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:03.472725 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4ee94-e787-4cc5-ade7-3f14e3421535 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:03.474409 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:03.480214 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ee94-e787-4cc5-ade7-3f14e3421535 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:03.484080 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:03.485590 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:55:03.504007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:03.505361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:03.507116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:55:03.507116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:55:03.507116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:55:03.507116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:03.507116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:03.511107 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:03.565864 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:03.569955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:03.588803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ee94-e787-4cc5-ade7-3f14e3421535 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:03.615928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea173e-b6bb-4181-a440-c1256904dec6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:03.624606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219888f-c2ed-4152-9686-cac29c420b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:03.625255 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219888f-c2ed-4152-9686-cac29c420b82 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:03.625561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219888f-c2ed-4152-9686-cac29c420b82 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:03.625838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219888f-c2ed-4152-9686-cac29c420b82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:03.626668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219888f-c2ed-4152-9686-cac29c420b82 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.domain.id=default, user.id=27091c013a8c447a81380abde9ae7c70) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:03.626984 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219888f-c2ed-4152-9686-cac29c420b82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:03.628548 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219888f-c2ed-4152-9686-cac29c420b82 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:03.635650 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:55:03 2022] GET /identity/v3/role_assignments?user.id=27091c013a8c447a81380abde9ae7c70&scope.domain.id=default&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:03.653303 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce5775-2638-44ee-96d2-a8401da38584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:03.653973 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5775-2638-44ee-96d2-a8401da38584 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:03.654304 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5775-2638-44ee-96d2-a8401da38584 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:03.654587 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5775-2638-44ee-96d2-a8401da38584 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:03.655336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce5775-2638-44ee-96d2-a8401da38584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:03.655671 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce5775-2638-44ee-96d2-a8401da38584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:03.657178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce5775-2638-44ee-96d2-a8401da38584 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:03.678277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:05.823360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbd3b3-ff93-4226-bc89-5f60de6984be None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:05.824354 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbd3b3-ff93-4226-bc89-5f60de6984be None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:05.824853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbd3b3-ff93-4226-bc89-5f60de6984be None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:05.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:05.832319 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7b9ec-94a2-4899-b8b2-feb18162d20c None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:05.832843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7b9ec-94a2-4899-b8b2-feb18162d20c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:05.832940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7b9ec-94a2-4899-b8b2-feb18162d20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:05.859156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-c1e7b9ec-94a2-4899-b8b2-feb18162d20c None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:05.860365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-c1e7b9ec-94a2-4899-b8b2-feb18162d20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:05.956618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 11/23] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:05.991663 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbd3b3-ff93-4226-bc89-5f60de6984be None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:06.141795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbd3b3-ff93-4226-bc89-5f60de6984be None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:06.180073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29728a3e-e422-41cb-a5b0-a36e2269caf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:06.180752 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29728a3e-e422-41cb-a5b0-a36e2269caf1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.181033 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29728a3e-e422-41cb-a5b0-a36e2269caf1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.181306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29728a3e-e422-41cb-a5b0-a36e2269caf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.194696 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29728a3e-e422-41cb-a5b0-a36e2269caf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-29728a3e-e422-41cb-a5b0-a36e2269caf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:06.200192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:06.201324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:06.202397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 13/24] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:06.209582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7b9ec-94a2-4899-b8b2-feb18162d20c None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:06.219471 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5109e56-52fd-4ecf-8434-7e0c7c9b8f48 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.219789 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5109e56-52fd-4ecf-8434-7e0c7c9b8f48 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.220014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5109e56-52fd-4ecf-8434-7e0c7c9b8f48 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.221676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 14/25] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:06.231420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522320-a7f3-4ecc-9a02-85c2b79bc0a1 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.231744 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522320-a7f3-4ecc-9a02-85c2b79bc0a1 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.231955 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522320-a7f3-4ecc-9a02-85c2b79bc0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.245815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:06.262089 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-62522320-a7f3-4ecc-9a02-85c2b79bc0a1 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:06.263348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-62522320-a7f3-4ecc-9a02-85c2b79bc0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:06.289943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:06.384690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7b9ec-94a2-4899-b8b2-feb18162d20c None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:06.426444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 15/26] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:06.444489 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:06.445547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.446067 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.446622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.447600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:06.448145 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:06.449800 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:06.466668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62522320-a7f3-4ecc-9a02-85c2b79bc0a1 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:06.470506 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 12/27] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:06.476305 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:06.647377 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62522320-a7f3-4ecc-9a02-85c2b79bc0a1 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:06.703834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20337bb3-4b39-41c8-8fdc-29edc91323c4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:06.726812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5943b-e8f5-4853-99ce-1877ba87be59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:06.727512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5943b-e8f5-4853-99ce-1877ba87be59 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.727795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5943b-e8f5-4853-99ce-1877ba87be59 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.728070 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5943b-e8f5-4853-99ce-1877ba87be59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.734624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:06.748011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5943b-e8f5-4853-99ce-1877ba87be59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:06.749578 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5943b-e8f5-4853-99ce-1877ba87be59 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-9fd5943b-e8f5-4853-99ce-1877ba87be59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:55:06.772330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:06.773497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:06.774675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:55:06.774675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:55:06.774675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:55:06.774675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:06.779089 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 16/28] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:06.780703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106bb8b-135b-4657-879a-1032327002c1 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.780992 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106bb8b-135b-4657-879a-1032327002c1 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.781273 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106bb8b-135b-4657-879a-1032327002c1 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.782767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 17/29] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:06.784073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409cc01-746d-4a8e-87d6-dd579d46c0ef None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.784464 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409cc01-746d-4a8e-87d6-dd579d46c0ef None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.784748 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409cc01-746d-4a8e-87d6-dd579d46c0ef None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.786405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 18/30] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:06.787200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409cc01-746d-4a8e-87d6-dd579d46c0ef None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:06.851052 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:06.880116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0309e-ee00-407b-84c6-6c30e6bf7abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:06.881125 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0309e-ee00-407b-84c6-6c30e6bf7abd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.881577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0309e-ee00-407b-84c6-6c30e6bf7abd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.882019 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0309e-ee00-407b-84c6-6c30e6bf7abd None admin] PATH_INFO: `/v3/users/27091c013a8c447a81380abde9ae7c70` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.883790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0309e-ee00-407b-84c6-6c30e6bf7abd None admin] RBAC: Authorizing `identity:get_user(user_id=27091c013a8c447a81380abde9ae7c70)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:06.885514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0309e-ee00-407b-84c6-6c30e6bf7abd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:06.887387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 13/31] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:06 2022] GET /identity/v3/users/27091c013a8c447a81380abde9ae7c70 => generated 247 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:06.888973 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f0dc-183c-4551-85cd-d31fca6c48d0 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.889510 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f0dc-183c-4551-85cd-d31fca6c48d0 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.890091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f0dc-183c-4551-85cd-d31fca6c48d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.903995 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409cc01-746d-4a8e-87d6-dd579d46c0ef None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:06.909639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-3d26f0dc-183c-4551-85cd-d31fca6c48d0 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:06.911147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-3d26f0dc-183c-4551-85cd-d31fca6c48d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:06.939448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ab357-2dbb-4bbf-8a51-071d401fe714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:06.940229 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ab357-2dbb-4bbf-8a51-071d401fe714 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.940501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ab357-2dbb-4bbf-8a51-071d401fe714 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.940774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ab357-2dbb-4bbf-8a51-071d401fe714 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.941840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ab357-2dbb-4bbf-8a51-071d401fe714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:06.942190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ab357-2dbb-4bbf-8a51-071d401fe714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:06.943608 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ab357-2dbb-4bbf-8a51-071d401fe714 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:06.962614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 19/32] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:06.968589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4bbc3-1f43-418d-9f14-c5a6e00b3740 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:06.968885 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4bbc3-1f43-418d-9f14-c5a6e00b3740 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:06.969297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4bbc3-1f43-418d-9f14-c5a6e00b3740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:06.988009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 14/33] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:06.988399 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-04d4bbc3-1f43-418d-9f14-c5a6e00b3740 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:06.989434 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-04d4bbc3-1f43-418d-9f14-c5a6e00b3740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:06.989928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26f0dc-183c-4551-85cd-d31fca6c48d0 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:07.038098 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:07.066959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 20/34] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:07.074130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4bbc3-1f43-418d-9f14-c5a6e00b3740 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:07.104327 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26f0dc-183c-4551-85cd-d31fca6c48d0 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:07.112738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:07.145223 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756f8d2-9b51-4c07-af68-89da0b7e35dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:07.146422 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f8d2-9b51-4c07-af68-89da0b7e35dc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.146949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f8d2-9b51-4c07-af68-89da0b7e35dc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.147473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f8d2-9b51-4c07-af68-89da0b7e35dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.149091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756f8d2-9b51-4c07-af68-89da0b7e35dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:07.151025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756f8d2-9b51-4c07-af68-89da0b7e35dc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:07.153517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 15/35] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:07.154872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756f8d2-9b51-4c07-af68-89da0b7e35dc None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:07.272180 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756f8d2-9b51-4c07-af68-89da0b7e35dc None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:07.309412 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23724f-d6c2-4f3d-b0c5-5e4d36768159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:07.310519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23724f-d6c2-4f3d-b0c5-5e4d36768159 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.311044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23724f-d6c2-4f3d-b0c5-5e4d36768159 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.311539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23724f-d6c2-4f3d-b0c5-5e4d36768159 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.312506 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23724f-d6c2-4f3d-b0c5-5e4d36768159 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:07.314183 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23724f-d6c2-4f3d-b0c5-5e4d36768159 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:07.350172 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4bbc3-1f43-418d-9f14-c5a6e00b3740 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:07.396203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29268e51-6072-4223-99a8-647747267af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:07.397301 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29268e51-6072-4223-99a8-647747267af5 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.397819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29268e51-6072-4223-99a8-647747267af5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.398368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29268e51-6072-4223-99a8-647747267af5 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.399385 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29268e51-6072-4223-99a8-647747267af5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:07.401265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29268e51-6072-4223-99a8-647747267af5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:07.457799 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 16/36] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:55:07 2022] POST /identity/v3/roles => generated 219 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:07.458867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23724f-d6c2-4f3d-b0c5-5e4d36768159 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:07.503339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:07.581384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 21/37] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:55:07 2022] POST /identity/v3/projects => generated 301 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:07.583624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29268e51-6072-4223-99a8-647747267af5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:07.607904 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23724f-d6c2-4f3d-b0c5-5e4d36768159 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:07.628659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:07.719503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb62c8a-0365-41c0-8412-f58d105b93a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:07.720206 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb62c8a-0365-41c0-8412-f58d105b93a9 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.720495 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb62c8a-0365-41c0-8412-f58d105b93a9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.720863 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb62c8a-0365-41c0-8412-f58d105b93a9 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.721637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb62c8a-0365-41c0-8412-f58d105b93a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:07.726941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb62c8a-0365-41c0-8412-f58d105b93a9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:07.729537 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29268e51-6072-4223-99a8-647747267af5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:07.770395 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 17/38] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:55:07 2022] POST /identity/v3/roles => generated 225 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:07.772159 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7d7f8-d547-4562-bd9a-6206cbb209fe None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.772938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7d7f8-d547-4562-bd9a-6206cbb209fe None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.773243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7d7f8-d547-4562-bd9a-6206cbb209fe None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.775029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 18/39] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:07.779176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb5a43-a62f-4206-a61c-eae292fcf947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:07.779824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98519d05-41f4-474a-924e-56a8ccf68523 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.780164 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98519d05-41f4-474a-924e-56a8ccf68523 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.780286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5a43-a62f-4206-a61c-eae292fcf947 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.780687 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98519d05-41f4-474a-924e-56a8ccf68523 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.780815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5a43-a62f-4206-a61c-eae292fcf947 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.780902 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5a43-a62f-4206-a61c-eae292fcf947 None admin] PATH_INFO: `/v3/domains/default/users/27091c013a8c447a81380abde9ae7c70/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.784701 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb5a43-a62f-4206-a61c-eae292fcf947 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=27091c013a8c447a81380abde9ae7c70, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:07.784973 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 19/40] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:07.786281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e412-de74-436b-be9e-b67c854b0765 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.786577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e412-de74-436b-be9e-b67c854b0765 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.786862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e412-de74-436b-be9e-b67c854b0765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.788310 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb5a43-a62f-4206-a61c-eae292fcf947 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:07.812894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-e212e412-de74-436b-be9e-b67c854b0765 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:07.814041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 22/41] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jul 28 08:55:07 2022] PUT /identity/v3/domains/default/users/27091c013a8c447a81380abde9ae7c70/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:07.815471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-e212e412-de74-436b-be9e-b67c854b0765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:07.815989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee7f0-5cf6-4e81-97f8-bb0f60a89564 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:07.816668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee7f0-5cf6-4e81-97f8-bb0f60a89564 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:07.817002 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee7f0-5cf6-4e81-97f8-bb0f60a89564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:07.854312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-139ee7f0-5cf6-4e81-97f8-bb0f60a89564 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:07.858353 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-139ee7f0-5cf6-4e81-97f8-bb0f60a89564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:07.949106 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 20/42] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:07.972339 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 23/43] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:07.975313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212e412-de74-436b-be9e-b67c854b0765 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:07.997000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee7f0-5cf6-4e81-97f8-bb0f60a89564 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:08.014764 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:08.021820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:08.132543 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212e412-de74-436b-be9e-b67c854b0765 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:08.159097 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee7f0-5cf6-4e81-97f8-bb0f60a89564 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:08.175434 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d62ed1-6c77-4abf-83d6-07f6b8cb4913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:08.176120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62ed1-6c77-4abf-83d6-07f6b8cb4913 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:08.176380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62ed1-6c77-4abf-83d6-07f6b8cb4913 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:08.176662 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62ed1-6c77-4abf-83d6-07f6b8cb4913 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:08.178356 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d62ed1-6c77-4abf-83d6-07f6b8cb4913 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:08.179829 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d62ed1-6c77-4abf-83d6-07f6b8cb4913 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:08.199324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 21/44] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:55:07 2022] POST /identity/v3/roles => generated 223 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:08.203218 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baa980-239e-48af-a940-ae0670780461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:08.203956 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa980-239e-48af-a940-ae0670780461 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:08.204236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa980-239e-48af-a940-ae0670780461 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:08.204520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa980-239e-48af-a940-ae0670780461 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:08.205946 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baa980-239e-48af-a940-ae0670780461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:08.207576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baa980-239e-48af-a940-ae0670780461 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:08.209599 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 24/45] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:08.215845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d62ed1-6c77-4abf-83d6-07f6b8cb4913 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:08.320777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:08.344155 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d62ed1-6c77-4abf-83d6-07f6b8cb4913 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:08.385363 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e47a1-c02f-4460-b29e-5071d0a07624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:08.385719 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e47a1-c02f-4460-b29e-5071d0a07624 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:08.386008 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e47a1-c02f-4460-b29e-5071d0a07624 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:08.386320 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e47a1-c02f-4460-b29e-5071d0a07624 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:08.387031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e47a1-c02f-4460-b29e-5071d0a07624 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:08.388555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e47a1-c02f-4460-b29e-5071d0a07624 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:08.423716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 22/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:55:08 2022] POST /identity/v3/projects => generated 312 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:09.914945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb9ae9-7016-44c6-8e4c-1347b257b504 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:09.915574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb9ae9-7016-44c6-8e4c-1347b257b504 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:09.915574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb9ae9-7016-44c6-8e4c-1347b257b504 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:09.916878 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 25/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:09.922467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b36f80-7dc0-4a53-a9b7-2f003fb94f72 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:09.922915 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b36f80-7dc0-4a53-a9b7-2f003fb94f72 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:09.923066 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b36f80-7dc0-4a53-a9b7-2f003fb94f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:09.943141 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-f1b36f80-7dc0-4a53-a9b7-2f003fb94f72 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:09.944397 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-f1b36f80-7dc0-4a53-a9b7-2f003fb94f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:10.022810 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 23/48] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:10.046321 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb9ae9-7016-44c6-8e4c-1347b257b504 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:10.141349 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:10.158025 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb9ae9-7016-44c6-8e4c-1347b257b504 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:10.195727 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7f368-8e57-400c-94f3-c40dd2467087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:10.196298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7f368-8e57-400c-94f3-c40dd2467087 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.196590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7f368-8e57-400c-94f3-c40dd2467087 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.196868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7f368-8e57-400c-94f3-c40dd2467087 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.207531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7f368-8e57-400c-94f3-c40dd2467087 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-90d7f368-8e57-400c-94f3-c40dd2467087 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:10.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:10.213856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:10.214926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 26/49] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:10.220181 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b36f80-7dc0-4a53-a9b7-2f003fb94f72 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:10.288204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:10.327226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b36f80-7dc0-4a53-a9b7-2f003fb94f72 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:10.364067 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194110e-8133-4d82-926b-93b0ab7196f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:10.364715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194110e-8133-4d82-926b-93b0ab7196f7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.364980 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194110e-8133-4d82-926b-93b0ab7196f7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.365289 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194110e-8133-4d82-926b-93b0ab7196f7 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.366032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194110e-8133-4d82-926b-93b0ab7196f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:10.366378 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194110e-8133-4d82-926b-93b0ab7196f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:10.367892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194110e-8133-4d82-926b-93b0ab7196f7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:10.376725 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 24/50] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:10.382645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7f368-8e57-400c-94f3-c40dd2467087 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:10.466889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0e70c-9a3d-44a1-a506-d46827f7f113 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.467125 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0e70c-9a3d-44a1-a506-d46827f7f113 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.467316 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0e70c-9a3d-44a1-a506-d46827f7f113 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.468746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 25/51] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:10.473977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5375e55-4324-474e-9cdc-6946039d3b70 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.474337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5375e55-4324-474e-9cdc-6946039d3b70 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.474626 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5375e55-4324-474e-9cdc-6946039d3b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.490353 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7f368-8e57-400c-94f3-c40dd2467087 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:10.493201 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-e5375e55-4324-474e-9cdc-6946039d3b70 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:10.494529 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-e5375e55-4324-474e-9cdc-6946039d3b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:10.530645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a91c2d-2172-46f1-ac61-bf9e6d9345df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:10.531805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a91c2d-2172-46f1-ac61-bf9e6d9345df None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.532315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a91c2d-2172-46f1-ac61-bf9e6d9345df None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.532846 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a91c2d-2172-46f1-ac61-bf9e6d9345df None admin] PATH_INFO: `/v3/users/27091c013a8c447a81380abde9ae7c70` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.535082 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a91c2d-2172-46f1-ac61-bf9e6d9345df None admin] RBAC: Authorizing `identity:get_user(user_id=27091c013a8c447a81380abde9ae7c70)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:10.536880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a91c2d-2172-46f1-ac61-bf9e6d9345df None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:10.539261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 27/52] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:10 2022] GET /identity/v3/users/27091c013a8c447a81380abde9ae7c70 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:10.545104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a91c2d-2172-46f1-ac61-bf9e6d9345df None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:10.562777 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:10.576694 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 26/53] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:10.598847 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5375e55-4324-474e-9cdc-6946039d3b70 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:10.661498 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a91c2d-2172-46f1-ac61-bf9e6d9345df None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:10.666664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:10.699429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93398c33-4d7f-4188-a67d-35978fb9cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:10.700094 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93398c33-4d7f-4188-a67d-35978fb9cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.700377 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93398c33-4d7f-4188-a67d-35978fb9cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.700653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93398c33-4d7f-4188-a67d-35978fb9cbf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.701857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93398c33-4d7f-4188-a67d-35978fb9cbf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:10.703440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93398c33-4d7f-4188-a67d-35978fb9cbf7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:10.705147 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 28/54] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:10.711053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93398c33-4d7f-4188-a67d-35978fb9cbf7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:10.711326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5375e55-4324-474e-9cdc-6946039d3b70 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:10.747677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27772a8-568f-4eda-b591-0a1655a4ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:10.748257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27772a8-568f-4eda-b591-0a1655a4ff0d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.748554 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27772a8-568f-4eda-b591-0a1655a4ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.748829 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27772a8-568f-4eda-b591-0a1655a4ff0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.750060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27772a8-568f-4eda-b591-0a1655a4ff0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:10.751604 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27772a8-568f-4eda-b591-0a1655a4ff0d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:10.753496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 27/55] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:10.759302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27772a8-568f-4eda-b591-0a1655a4ff0d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:10.803530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:10.828966 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93398c33-4d7f-4188-a67d-35978fb9cbf7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:10.867956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27772a8-568f-4eda-b591-0a1655a4ff0d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:10.884128 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fd60-b780-4ff7-8dd5-276e4883511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:10.884773 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fd60-b780-4ff7-8dd5-276e4883511f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.885050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fd60-b780-4ff7-8dd5-276e4883511f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.885313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fd60-b780-4ff7-8dd5-276e4883511f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.886135 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3fd60-b780-4ff7-8dd5-276e4883511f None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.domain.id=default, user.id=27091c013a8c447a81380abde9ae7c70) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:10.886450 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3fd60-b780-4ff7-8dd5-276e4883511f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:10.887972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3fd60-b780-4ff7-8dd5-276e4883511f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:10.894616 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 29/56] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 08:55:10 2022] GET /identity/v3/role_assignments?user.id=27091c013a8c447a81380abde9ae7c70&scope.domain.id=default&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 530 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:10.896243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:10.906471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42796db-016c-41fe-8395-f78ef2050e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:10.907154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42796db-016c-41fe-8395-f78ef2050e13 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:10.907440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42796db-016c-41fe-8395-f78ef2050e13 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:10.907709 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42796db-016c-41fe-8395-f78ef2050e13 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:10.908590 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42796db-016c-41fe-8395-f78ef2050e13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:10.910054 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42796db-016c-41fe-8395-f78ef2050e13 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:10.953648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 28/57] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:55:10 2022] POST /identity/v3/projects => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:55:12.990426 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb968c8-990c-4c6f-8a52-d6951c7258e2 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:12.991052 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb968c8-990c-4c6f-8a52-d6951c7258e2 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:12.991052 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb968c8-990c-4c6f-8a52-d6951c7258e2 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:12.992383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 30/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:12.997979 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abdcfe-f2ec-4a96-b720-12c17492aed3 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:12.998548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abdcfe-f2ec-4a96-b720-12c17492aed3 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:12.998548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abdcfe-f2ec-4a96-b720-12c17492aed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:13.019543 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-08abdcfe-f2ec-4a96-b720-12c17492aed3 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:13.020780 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-08abdcfe-f2ec-4a96-b720-12c17492aed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:13.096137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 29/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:13.120583 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb968c8-990c-4c6f-8a52-d6951c7258e2 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:13.234683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb968c8-990c-4c6f-8a52-d6951c7258e2 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:13.271137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab5af5-7800-4d60-8b55-ec4a84555a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:13.271708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5af5-7800-4d60-8b55-ec4a84555a92 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:13.271996 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5af5-7800-4d60-8b55-ec4a84555a92 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:13.272276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5af5-7800-4d60-8b55-ec4a84555a92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:13.273696 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab5af5-7800-4d60-8b55-ec4a84555a92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:13.275263 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab5af5-7800-4d60-8b55-ec4a84555a92 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:13.276949 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 31/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:13.282886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abdcfe-f2ec-4a96-b720-12c17492aed3 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:13.390800 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abdcfe-f2ec-4a96-b720-12c17492aed3 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:13.401041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:13.430499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f506d-162b-471a-be4b-fb6e8669f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:13.431147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f506d-162b-471a-be4b-fb6e8669f0ab None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:13.431432 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f506d-162b-471a-be4b-fb6e8669f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:13.431713 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f506d-162b-471a-be4b-fb6e8669f0ab None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:13.432422 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f506d-162b-471a-be4b-fb6e8669f0ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:13.433930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f506d-162b-471a-be4b-fb6e8669f0ab None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:13.462602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 30/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:55:13 2022] POST /identity/v3/users => generated 275 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:16.635688 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b86d08-830b-4d9d-901f-2e86877201c4 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:16.636806 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b86d08-830b-4d9d-901f-2e86877201c4 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:16.637345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b86d08-830b-4d9d-901f-2e86877201c4 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:16.639012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 32/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:16.644663 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56147b83-4ba8-45a1-80f0-c6d26725bb8b None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:16.645592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56147b83-4ba8-45a1-80f0-c6d26725bb8b None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:16.646095 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56147b83-4ba8-45a1-80f0-c6d26725bb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:16.675249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-56147b83-4ba8-45a1-80f0-c6d26725bb8b None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:16.676909 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-56147b83-4ba8-45a1-80f0-c6d26725bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:16.764547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 31/63] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:16.795369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b86d08-830b-4d9d-901f-2e86877201c4 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:16.812679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:16.907635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b86d08-830b-4d9d-901f-2e86877201c4 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:16.915947 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072101a-9fb3-4b07-87b2-3c16fdcb8455 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:16.916243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072101a-9fb3-4b07-87b2-3c16fdcb8455 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:16.916732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072101a-9fb3-4b07-87b2-3c16fdcb8455 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:16.918376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 32/64] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:16.927636 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99d111-f1a5-42de-a001-d04d08733fd0 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:16.928188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99d111-f1a5-42de-a001-d04d08733fd0 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:16.928663 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99d111-f1a5-42de-a001-d04d08733fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:16.958147 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb439-1401-43aa-8342-094b5b98c615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:16.959069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb439-1401-43aa-8342-094b5b98c615 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:16.959583 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb439-1401-43aa-8342-094b5b98c615 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:16.960026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb439-1401-43aa-8342-094b5b98c615 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:16.967285 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-9f99d111-f1a5-42de-a001-d04d08733fd0 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:16.968590 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9f99d111-f1a5-42de-a001-d04d08733fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:16.972861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cb439-1401-43aa-8342-094b5b98c615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-9f9cb439-1401-43aa-8342-094b5b98c615 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:16.979290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:16.980652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:16.981925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:16.985068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 33/65] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:16.994154 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb439-1401-43aa-8342-094b5b98c615 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:17.086011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 33/66] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:17.115046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:17.119131 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99d111-f1a5-42de-a001-d04d08733fd0 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:17.149222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb439-1401-43aa-8342-094b5b98c615 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:17.189646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7b115-266e-4183-b395-e9d1680f0858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:17.190296 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7b115-266e-4183-b395-e9d1680f0858 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.190650 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7b115-266e-4183-b395-e9d1680f0858 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.190940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7b115-266e-4183-b395-e9d1680f0858 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.192098 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7b115-266e-4183-b395-e9d1680f0858 None admin] RBAC: Adding query filter params (name=member) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:17.192418 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7b115-266e-4183-b395-e9d1680f0858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:17.193929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7b115-266e-4183-b395-e9d1680f0858 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:17.206729 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 34/67] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:17.208848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3c96d-f4ab-4cb6-82d6-c8ca824ac796 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.209222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3c96d-f4ab-4cb6-82d6-c8ca824ac796 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.209577 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3c96d-f4ab-4cb6-82d6-c8ca824ac796 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.210507 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:17.211684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 35/68] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:17.217638 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3c96d-f4ab-4cb6-82d6-c8ca824ac796 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:17.299606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99d111-f1a5-42de-a001-d04d08733fd0 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:17.338164 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2696ea-965d-47cf-bc87-9c91151e8033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:17.338860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2696ea-965d-47cf-bc87-9c91151e8033 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.339142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2696ea-965d-47cf-bc87-9c91151e8033 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.339419 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2696ea-965d-47cf-bc87-9c91151e8033 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.350686 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3c96d-f4ab-4cb6-82d6-c8ca824ac796 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:17.354002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2696ea-965d-47cf-bc87-9c91151e8033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-fb2696ea-965d-47cf-bc87-9c91151e8033 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:17.360642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:17.362046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:17.363154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 34/69] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:17.364040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ea397-9cc9-4b7c-90c7-b9e253230503 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.364040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ea397-9cc9-4b7c-90c7-b9e253230503 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.364040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ea397-9cc9-4b7c-90c7-b9e253230503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.382388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-e80ea397-9cc9-4b7c-90c7-b9e253230503 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:17.383640 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-e80ea397-9cc9-4b7c-90c7-b9e253230503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:17.391344 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34b96e-0b42-45b0-adb7-c85a7918074d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:17.392002 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b96e-0b42-45b0-adb7-c85a7918074d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.392290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b96e-0b42-45b0-adb7-c85a7918074d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.392699 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b96e-0b42-45b0-adb7-c85a7918074d None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.395666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:17.407320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34b96e-0b42-45b0-adb7-c85a7918074d None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:17.408886 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34b96e-0b42-45b0-adb7-c85a7918074d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:17.410672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 36/70] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:17 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:17.411889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9fe94-5620-48d1-aa02-d9f1dbb2a847 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.412181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9fe94-5620-48d1-aa02-d9f1dbb2a847 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.412462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9fe94-5620-48d1-aa02-d9f1dbb2a847 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.413705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 37/71] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:17.414772 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aaf78-1fa8-4f6e-8e53-ac4a8261849c None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.415062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aaf78-1fa8-4f6e-8e53-ac4a8261849c None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.415343 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aaf78-1fa8-4f6e-8e53-ac4a8261849c None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.416558 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 38/72] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:17.417270 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796aaf78-1fa8-4f6e-8e53-ac4a8261849c None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:17.462321 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 35/73] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:17.467763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ea397-9cc9-4b7c-90c7-b9e253230503 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:17.485041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:17.524849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796aaf78-1fa8-4f6e-8e53-ac4a8261849c None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:17.561050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06134188-cc51-4497-8033-12e70a0a4a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:17.561687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06134188-cc51-4497-8033-12e70a0a4a2f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.561967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06134188-cc51-4497-8033-12e70a0a4a2f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.562280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06134188-cc51-4497-8033-12e70a0a4a2f None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.563006 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06134188-cc51-4497-8033-12e70a0a4a2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:17.563312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06134188-cc51-4497-8033-12e70a0a4a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:17.564879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06134188-cc51-4497-8033-12e70a0a4a2f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:17.569160 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ea397-9cc9-4b7c-90c7-b9e253230503 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:17.573796 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 39/74] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:17.574849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd43e-6744-4379-a0bc-442810508f31 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.575136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd43e-6744-4379-a0bc-442810508f31 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.575412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd43e-6744-4379-a0bc-442810508f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.594615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-f35bd43e-6744-4379-a0bc-442810508f31 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:17.595977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-f35bd43e-6744-4379-a0bc-442810508f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:17.606280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d0729-2d9c-4d23-9ee0-991e5fe9e1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:17.606933 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d0729-2d9c-4d23-9ee0-991e5fe9e1a7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.607219 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d0729-2d9c-4d23-9ee0-991e5fe9e1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.607494 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d0729-2d9c-4d23-9ee0-991e5fe9e1a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.647614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:17.665054 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 40/75] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:17.666376 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb20c-5741-4b63-bff5-177899b15114 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.666654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb20c-5741-4b63-bff5-177899b15114 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.666937 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb20c-5741-4b63-bff5-177899b15114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.679748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d0729-2d9c-4d23-9ee0-991e5fe9e1a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:17.681856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d0729-2d9c-4d23-9ee0-991e5fe9e1a7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:17.685020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-611cb20c-5741-4b63-bff5-177899b15114 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:17.686264 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-611cb20c-5741-4b63-bff5-177899b15114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:17.699035 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:17.755244 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 41/76] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:17.756020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cb20c-5741-4b63-bff5-177899b15114 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:17.793459 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 36/77] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:17.794368 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d0729-2d9c-4d23-9ee0-991e5fe9e1a7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:17.863330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cb20c-5741-4b63-bff5-177899b15114 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:17.881041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:17.899433 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d0729-2d9c-4d23-9ee0-991e5fe9e1a7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:17.899991 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab2f3-9f27-4da2-8d16-1d5a38ee47f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:17.900664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab2f3-9f27-4da2-8d16-1d5a38ee47f9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.900958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab2f3-9f27-4da2-8d16-1d5a38ee47f9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.901242 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab2f3-9f27-4da2-8d16-1d5a38ee47f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.912883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ab2f3-9f27-4da2-8d16-1d5a38ee47f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:17.914345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-127ab2f3-9f27-4da2-8d16-1d5a38ee47f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:17.917940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:17.919231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 42/78] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:17.920291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab2f3-9f27-4da2-8d16-1d5a38ee47f9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:17.940399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b6029-2cc9-4f6b-b343-634a2e966fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:17.941186 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b6029-2cc9-4f6b-b343-634a2e966fb4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:17.941839 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b6029-2cc9-4f6b-b343-634a2e966fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:17.942148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b6029-2cc9-4f6b-b343-634a2e966fb4 None admin] PATH_INFO: `/v3/users/27091c013a8c447a81380abde9ae7c70` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:17.943381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b6029-2cc9-4f6b-b343-634a2e966fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=27091c013a8c447a81380abde9ae7c70)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:17.944884 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b6029-2cc9-4f6b-b343-634a2e966fb4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:17.946590 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 37/79] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:17 2022] GET /identity/v3/users/27091c013a8c447a81380abde9ae7c70 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:17.947476 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b6029-2cc9-4f6b-b343-634a2e966fb4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.037187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab2f3-9f27-4da2-8d16-1d5a38ee47f9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.050496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b6029-2cc9-4f6b-b343-634a2e966fb4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.073155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39095fa-3fb1-485f-8303-d0f3c7069fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.073815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39095fa-3fb1-485f-8303-d0f3c7069fcd None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.074097 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39095fa-3fb1-485f-8303-d0f3c7069fcd None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.074410 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39095fa-3fb1-485f-8303-d0f3c7069fcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.075607 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39095fa-3fb1-485f-8303-d0f3c7069fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.077100 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39095fa-3fb1-485f-8303-d0f3c7069fcd None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.078768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 43/80] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:18.084046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39095fa-3fb1-485f-8303-d0f3c7069fcd None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.091516 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497e651-e74c-4cc2-a5a5-e003af44ebd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.092250 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497e651-e74c-4cc2-a5a5-e003af44ebd6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.092536 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497e651-e74c-4cc2-a5a5-e003af44ebd6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.092802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497e651-e74c-4cc2-a5a5-e003af44ebd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.104226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497e651-e74c-4cc2-a5a5-e003af44ebd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-7497e651-e74c-4cc2-a5a5-e003af44ebd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:18.109275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:18.110513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 38/81] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:18.112225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497e651-e74c-4cc2-a5a5-e003af44ebd6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.130945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:18.156570 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:18.188291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39095fa-3fb1-485f-8303-d0f3c7069fcd None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.214284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497e651-e74c-4cc2-a5a5-e003af44ebd6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.224903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0abf67-8cb4-4967-8cc7-40df9e79f41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.225579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0abf67-8cb4-4967-8cc7-40df9e79f41e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.225864 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0abf67-8cb4-4967-8cc7-40df9e79f41e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.226176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0abf67-8cb4-4967-8cc7-40df9e79f41e None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.227392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0abf67-8cb4-4967-8cc7-40df9e79f41e None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.229162 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0abf67-8cb4-4967-8cc7-40df9e79f41e None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.234160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 44/82] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:18.234160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0abf67-8cb4-4967-8cc7-40df9e79f41e None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.253420 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.254088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.254410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.254669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.255422 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:18.255721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.257153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.265831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 39/83] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:18.266696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.343550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0abf67-8cb4-4967-8cc7-40df9e79f41e None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.356794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:18.371837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d166013-ac9b-42ca-bc3c-b3146212f12f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.374992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:18.379626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef571e7f-4a79-4258-8c1a-bd83bb8aa3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.380275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef571e7f-4a79-4258-8c1a-bd83bb8aa3a9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.380563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef571e7f-4a79-4258-8c1a-bd83bb8aa3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.380844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef571e7f-4a79-4258-8c1a-bd83bb8aa3a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.405586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b99c4-d279-4f3d-8997-a96d6b2b37e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.406261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b99c4-d279-4f3d-8997-a96d6b2b37e7 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.406546 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b99c4-d279-4f3d-8997-a96d6b2b37e7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.406820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b99c4-d279-4f3d-8997-a96d6b2b37e7 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.407514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b99c4-d279-4f3d-8997-a96d6b2b37e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.409154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b99c4-d279-4f3d-8997-a96d6b2b37e7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.430888 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 40/84] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:55:18 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:18.431828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b99c4-d279-4f3d-8997-a96d6b2b37e7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.456380 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef571e7f-4a79-4258-8c1a-bd83bb8aa3a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.457946 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef571e7f-4a79-4258-8c1a-bd83bb8aa3a9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.543205 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b99c4-d279-4f3d-8997-a96d6b2b37e7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.581947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 45/85] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:18.583280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef571e7f-4a79-4258-8c1a-bd83bb8aa3a9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.595776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.596357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.596691 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.596997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.597738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:18.598878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.599593 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.604448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:18.610218 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 41/86] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:18.612090 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.641046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:18.741906 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b63f4-e780-4979-a47c-c69ce826bef7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.742683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef571e7f-4a79-4258-8c1a-bd83bb8aa3a9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.784802 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.785189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b963d68-1e72-4d89-99fc-2b3e309b547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.785857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b963d68-1e72-4d89-99fc-2b3e309b547d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.786170 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b963d68-1e72-4d89-99fc-2b3e309b547d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.786478 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b963d68-1e72-4d89-99fc-2b3e309b547d None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.786953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.787454 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.787719 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b963d68-1e72-4d89-99fc-2b3e309b547d None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.788161 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.789197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] RBAC: Adding query filter params (role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147, user.id=596bd6a71df14947b131621dfe20517f) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:18.789340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b963d68-1e72-4d89-99fc-2b3e309b547d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.789856 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.791166 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 42/87] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:18.791865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.792092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b963d68-1e72-4d89-99fc-2b3e309b547d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.803064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 46/88] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/role_assignments?user.id=596bd6a71df14947b131621dfe20517f&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:18.804296 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.912681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:18.915467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:18.926093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b963d68-1e72-4d89-99fc-2b3e309b547d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.942418 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8faf3d-5e87-4c13-a09c-7696d385b82d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:18.966896 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab03995-acfc-4f81-b8e8-383a18321631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.968035 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03995-acfc-4f81-b8e8-383a18321631 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.968575 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03995-acfc-4f81-b8e8-383a18321631 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.969283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03995-acfc-4f81-b8e8-383a18321631 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:18.970998 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab03995-acfc-4f81-b8e8-383a18321631 None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:18.972815 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab03995-acfc-4f81-b8e8-383a18321631 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:18.975387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 43/89] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:18.976550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab03995-acfc-4f81-b8e8-383a18321631 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:18.996643 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cee71-16c8-42cf-b72c-cf005cbf455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:18.997323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cee71-16c8-42cf-b72c-cf005cbf455c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:18.997610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cee71-16c8-42cf-b72c-cf005cbf455c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:18.997893 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cee71-16c8-42cf-b72c-cf005cbf455c None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:19.003169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cee71-16c8-42cf-b72c-cf005cbf455c None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:19.004799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cee71-16c8-42cf-b72c-cf005cbf455c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:19.006694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 47/90] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:19.007552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cee71-16c8-42cf-b72c-cf005cbf455c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:19.095001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab03995-acfc-4f81-b8e8-383a18321631 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:19.164247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfcec7-097a-4a8a-872a-91704d657fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:19.164947 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfcec7-097a-4a8a-872a-91704d657fb6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:19.165454 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfcec7-097a-4a8a-872a-91704d657fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:19.171587 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfcec7-097a-4a8a-872a-91704d657fb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:19.194774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cee71-16c8-42cf-b72c-cf005cbf455c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:19.233951 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:19.267387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:19.268084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:19.268375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:19.268657 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:19.269567 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147, user.id=27091c013a8c447a81380abde9ae7c70) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:19.269883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:19.271434 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:19.290252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 48/91] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:19 2022] GET /identity/v3/role_assignments?user.id=27091c013a8c447a81380abde9ae7c70&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:19.291273 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:19.318553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:19.322518 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfcec7-097a-4a8a-872a-91704d657fb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:19.324063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfcec7-097a-4a8a-872a-91704d657fb6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:19.496616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 44/92] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:19.501908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfcec7-097a-4a8a-872a-91704d657fb6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:19.544360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af0baa-a34e-4130-9392-31a626a824e7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:19.627291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b308bcc-46d7-4859-9baa-dbaa116e981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:19.627940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b308bcc-46d7-4859-9baa-dbaa116e981b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:19.628066 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:19.628244 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b308bcc-46d7-4859-9baa-dbaa116e981b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:19.628499 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b308bcc-46d7-4859-9baa-dbaa116e981b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:19.712715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfcec7-097a-4a8a-872a-91704d657fb6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:19.760435 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b308bcc-46d7-4859-9baa-dbaa116e981b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:19.761915 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b308bcc-46d7-4859-9baa-dbaa116e981b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:19.778467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:19.787315 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93337ad-1eac-47c2-bc9d-ea9eb57875e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:19.788005 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93337ad-1eac-47c2-bc9d-ea9eb57875e1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:19.788410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93337ad-1eac-47c2-bc9d-ea9eb57875e1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:19.788694 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93337ad-1eac-47c2-bc9d-ea9eb57875e1 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:19.790008 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93337ad-1eac-47c2-bc9d-ea9eb57875e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:19.795977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93337ad-1eac-47c2-bc9d-ea9eb57875e1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:19.798052 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 45/93] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:19 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:19.805477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93337ad-1eac-47c2-bc9d-ea9eb57875e1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:19.960368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 49/94] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:19.965821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b308bcc-46d7-4859-9baa-dbaa116e981b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:20.028417 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93337ad-1eac-47c2-bc9d-ea9eb57875e1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:20.090054 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:20.108511 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:20.109178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:20.109464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:20.109745 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:20.110711 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] RBAC: Adding query filter params (role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147, user.id=596bd6a71df14947b131621dfe20517f) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:20.111098 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:20.112646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:20.132677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 46/95] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:19 2022] GET /identity/v3/role_assignments?user.id=596bd6a71df14947b131621dfe20517f&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 259 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:20.185310 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b308bcc-46d7-4859-9baa-dbaa116e981b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:20.204985 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:20.236738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41ab3e-b73e-4d42-9a69-4004bdd0fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:20.237425 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ab3e-b73e-4d42-9a69-4004bdd0fdb6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:20.237705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ab3e-b73e-4d42-9a69-4004bdd0fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:20.237986 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ab3e-b73e-4d42-9a69-4004bdd0fdb6 None admin] PATH_INFO: `/v3/projects/2b220a1fd63f4bb3aacafbc1723926e4` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:20.242951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41ab3e-b73e-4d42-9a69-4004bdd0fdb6 None admin] RBAC: Authorizing `identity:get_project(project_id=2b220a1fd63f4bb3aacafbc1723926e4)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:20.244510 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41ab3e-b73e-4d42-9a69-4004bdd0fdb6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:20.246410 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 50/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:19 2022] GET /identity/v3/projects/2b220a1fd63f4bb3aacafbc1723926e4 => generated 312 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:20.255610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:20.393455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccafa8-0758-4545-a8d9-f6eb8e1154a9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:20.432831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcfa2e-d165-4ca8-bff5-fa5535f2f173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:20.433542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcfa2e-d165-4ca8-bff5-fa5535f2f173 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:20.433831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcfa2e-d165-4ca8-bff5-fa5535f2f173 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:20.434131 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcfa2e-d165-4ca8-bff5-fa5535f2f173 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:20.435009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcfa2e-d165-4ca8-bff5-fa5535f2f173 None admin] RBAC: Adding query filter params (role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=2b220a1fd63f4bb3aacafbc1723926e4, user.id=596bd6a71df14947b131621dfe20517f) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:20.435334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcfa2e-d165-4ca8-bff5-fa5535f2f173 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:20.436844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcfa2e-d165-4ca8-bff5-fa5535f2f173 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:20.444572 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 47/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:20 2022] GET /identity/v3/role_assignments?user.id=596bd6a71df14947b131621dfe20517f&scope.project.id=2b220a1fd63f4bb3aacafbc1723926e4&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:21.214202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c00f01-0895-4f69-b318-7c5c519b7448 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:21.214972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c00f01-0895-4f69-b318-7c5c519b7448 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:21.214972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c00f01-0895-4f69-b318-7c5c519b7448 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:21.216157 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 51/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:21.221603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70974b-3324-470b-9fe0-c79c6c336378 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:21.222059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70974b-3324-470b-9fe0-c79c6c336378 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:21.222219 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70974b-3324-470b-9fe0-c79c6c336378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:21.251214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-4f70974b-3324-470b-9fe0-c79c6c336378 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:21.252540 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-4f70974b-3324-470b-9fe0-c79c6c336378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:21.318114 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:21.340922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 48/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:21.380142 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c00f01-0895-4f69-b318-7c5c519b7448 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:21.510628 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c00f01-0895-4f69-b318-7c5c519b7448 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:21.548109 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10082a-ef78-44fb-99de-4d068037c516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:21.548857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10082a-ef78-44fb-99de-4d068037c516 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:21.549145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10082a-ef78-44fb-99de-4d068037c516 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:21.549428 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10082a-ef78-44fb-99de-4d068037c516 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:21.560680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10082a-ef78-44fb-99de-4d068037c516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-2f10082a-ef78-44fb-99de-4d068037c516 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:21.565786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:21.567383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:21.568442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 52/100] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:21.576121 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70974b-3324-470b-9fe0-c79c6c336378 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:21.647442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d6774-dbc4-43e0-9c05-78957b931a73 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:21.647829 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d6774-dbc4-43e0-9c05-78957b931a73 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:21.648002 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d6774-dbc4-43e0-9c05-78957b931a73 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:21.649525 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 53/101] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:21.654887 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acb1bc-e6be-4b4b-8aed-91fe7bf9ffa0 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:21.655178 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acb1bc-e6be-4b4b-8aed-91fe7bf9ffa0 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:21.655462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acb1bc-e6be-4b4b-8aed-91fe7bf9ffa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:21.698586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-f0acb1bc-e6be-4b4b-8aed-91fe7bf9ffa0 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:21.699849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-f0acb1bc-e6be-4b4b-8aed-91fe7bf9ffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:21.737096 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70974b-3324-470b-9fe0-c79c6c336378 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:21.769611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:21.813139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:21.813807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:21.814141 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:21.814432 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:21.815180 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] RBAC: Adding query filter params (name=reader) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:21.815494 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:21.817563 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:21.834494 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 49/102] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:21.838222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 54/103] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:21.843187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:21.887598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde80e48-7cfe-4b71-b7f5-c27c863a293a None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:21.887939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde80e48-7cfe-4b71-b7f5-c27c863a293a None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:21.888145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde80e48-7cfe-4b71-b7f5-c27c863a293a None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:21.889691 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 55/104] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:55:21.894608 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328c291-495d-4d2b-bc12-26c74f503f05 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:21.894902 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328c291-495d-4d2b-bc12-26c74f503f05 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:21.895186 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328c291-495d-4d2b-bc12-26c74f503f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:21.917086 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-b328c291-495d-4d2b-bc12-26c74f503f05 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:21.917438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:21.918791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b328c291-495d-4d2b-bc12-26c74f503f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:22.004354 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 56/105] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:22.005102 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328c291-495d-4d2b-bc12-26c74f503f05 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:22.088923 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a2ed7-01b2-4529-81ba-86f6a1dd0199 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:22.135359 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:22.138850 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7d178-e3c7-41ac-ae68-7d85ff82f232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:22.139542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7d178-e3c7-41ac-ae68-7d85ff82f232 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.139839 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7d178-e3c7-41ac-ae68-7d85ff82f232 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.140127 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7d178-e3c7-41ac-ae68-7d85ff82f232 None admin] PATH_INFO: `/v3/users/661eecaf09e449c19c1e1ce545060682` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.149070 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328c291-495d-4d2b-bc12-26c74f503f05 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:22.158074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7d178-e3c7-41ac-ae68-7d85ff82f232 None admin] RBAC: Authorizing `identity:get_user(user_id=661eecaf09e449c19c1e1ce545060682)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:22.161930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7d178-e3c7-41ac-ae68-7d85ff82f232 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:22.161930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 50/106] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:21 2022] GET /identity/v3/users/661eecaf09e449c19c1e1ce545060682 => generated 289 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:22.162573 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7d178-e3c7-41ac-ae68-7d85ff82f232 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:22.199521 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467d2b4-8d58-4313-824c-e2cffac78d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:22.200317 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d2b4-8d58-4313-824c-e2cffac78d51 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.200607 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d2b4-8d58-4313-824c-e2cffac78d51 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.200995 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d2b4-8d58-4313-824c-e2cffac78d51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.202220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467d2b4-8d58-4313-824c-e2cffac78d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:22.203762 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467d2b4-8d58-4313-824c-e2cffac78d51 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:22.205622 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 57/107] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:22.210359 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467d2b4-8d58-4313-824c-e2cffac78d51 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:22.322019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467d2b4-8d58-4313-824c-e2cffac78d51 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:22.359803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258e9c1-b932-419a-9f28-48c6a011572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:22.360374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e9c1-b932-419a-9f28-48c6a011572d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.360671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e9c1-b932-419a-9f28-48c6a011572d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.361090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e9c1-b932-419a-9f28-48c6a011572d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.377448 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7d178-e3c7-41ac-ae68-7d85ff82f232 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:22.380551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:22.414828 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:22.415062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd961-9a65-4e77-974d-f5e7e98c1c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:22.415798 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd961-9a65-4e77-974d-f5e7e98c1c7d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.416262 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd961-9a65-4e77-974d-f5e7e98c1c7d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.416635 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd961-9a65-4e77-974d-f5e7e98c1c7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.428723 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bd961-9a65-4e77-974d-f5e7e98c1c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-e15bd961-9a65-4e77-974d-f5e7e98c1c7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:22.434047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:22.435559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:22.437156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:22.438454 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 51/108] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:22.439436 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd961-9a65-4e77-974d-f5e7e98c1c7d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:22.449196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258e9c1-b932-419a-9f28-48c6a011572d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:22.450935 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258e9c1-b932-419a-9f28-48c6a011572d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:22.570759 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 58/109] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:22.572431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a962b3-3b0a-48cd-b5b4-e98aee7ebfd4 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.572646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd961-9a65-4e77-974d-f5e7e98c1c7d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:22.573111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a962b3-3b0a-48cd-b5b4-e98aee7ebfd4 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.573563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a962b3-3b0a-48cd-b5b4-e98aee7ebfd4 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.575327 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 59/110] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:22.576420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a962b3-3b0a-48cd-b5b4-e98aee7ebfd4 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:22.617437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b2d04-bf61-4daf-a05e-183181f75c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:22.617988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b2d04-bf61-4daf-a05e-183181f75c76 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.618308 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b2d04-bf61-4daf-a05e-183181f75c76 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.618595 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b2d04-bf61-4daf-a05e-183181f75c76 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.619325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b2d04-bf61-4daf-a05e-183181f75c76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:22.620826 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b2d04-bf61-4daf-a05e-183181f75c76 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:22.651108 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 52/111] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:55:22 2022] POST /identity/v3/projects => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:22.658433 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b2d04-bf61-4daf-a05e-183181f75c76 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:22.699937 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:22.702733 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:22.794495 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a962b3-3b0a-48cd-b5b4-e98aee7ebfd4 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:22.876716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bc39c-4874-47d4-91fe-1be1765a3078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:22.877377 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bc39c-4874-47d4-91fe-1be1765a3078 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.877723 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bc39c-4874-47d4-91fe-1be1765a3078 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.878053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bc39c-4874-47d4-91fe-1be1765a3078 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.880071 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bc39c-4874-47d4-91fe-1be1765a3078 None admin] RBAC: Adding query filter params (name=member) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:22.880400 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bc39c-4874-47d4-91fe-1be1765a3078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:22.881861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bc39c-4874-47d4-91fe-1be1765a3078 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:22.887863 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b2d04-bf61-4daf-a05e-183181f75c76 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:22.906627 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 60/112] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:22.907692 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9e2c-f86c-4c24-86f3-4f72fb9fb0b1 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.908060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9e2c-f86c-4c24-86f3-4f72fb9fb0b1 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.908346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9e2c-f86c-4c24-86f3-4f72fb9fb0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.950638 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-e05a9e2c-f86c-4c24-86f3-4f72fb9fb0b1 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:22.952707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-e05a9e2c-f86c-4c24-86f3-4f72fb9fb0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:22.963992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf91c9-6e49-4f18-a896-5715514577bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:22.964678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf91c9-6e49-4f18-a896-5715514577bd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:22.964964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf91c9-6e49-4f18-a896-5715514577bd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:22.965980 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf91c9-6e49-4f18-a896-5715514577bd None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:22.970278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf91c9-6e49-4f18-a896-5715514577bd None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:22.971763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf91c9-6e49-4f18-a896-5715514577bd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:22.973639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 53/113] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:22 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:22.974733 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf91c9-6e49-4f18-a896-5715514577bd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:23.042125 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 61/114] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:23.043287 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a9e2c-f86c-4c24-86f3-4f72fb9fb0b1 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:23.109391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:23.121669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:23.129941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf91c9-6e49-4f18-a896-5715514577bd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:23.184311 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a9e2c-f86c-4c24-86f3-4f72fb9fb0b1 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:23.197385 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701994dd-44b0-497c-984a-58e32cb15e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:23.198345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701994dd-44b0-497c-984a-58e32cb15e7c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.198835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701994dd-44b0-497c-984a-58e32cb15e7c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.199427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701994dd-44b0-497c-984a-58e32cb15e7c None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.201816 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701994dd-44b0-497c-984a-58e32cb15e7c None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:23.203389 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701994dd-44b0-497c-984a-58e32cb15e7c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:23.205192 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 54/115] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:22 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:23.205996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701994dd-44b0-497c-984a-58e32cb15e7c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:23.274881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:23.275541 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.275826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.276110 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.276929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] RBAC: Adding query filter params (role.id=61c2794c5a6d4f91b3a953f0ae6eab8b, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147, user.id=661eecaf09e449c19c1e1ce545060682) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:23.277243 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:23.282139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:23.285394 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 62/116] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:23 2022] GET /identity/v3/role_assignments?user.id=661eecaf09e449c19c1e1ce545060682&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=61c2794c5a6d4f91b3a953f0ae6eab8b => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:23.286017 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:23.388788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701994dd-44b0-497c-984a-58e32cb15e7c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:23.425298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e51ef8-5875-40e8-ab68-323ee7a03fe4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:23.451335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b962a-6fcc-4bd0-8747-5bc64986f38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:23.452400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b962a-6fcc-4bd0-8747-5bc64986f38b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.453607 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b962a-6fcc-4bd0-8747-5bc64986f38b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.454257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b962a-6fcc-4bd0-8747-5bc64986f38b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.464642 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:23.476899 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b962a-6fcc-4bd0-8747-5bc64986f38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-c49b962a-6fcc-4bd0-8747-5bc64986f38b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:23.483690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:23.484943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:23.486043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 55/117] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:23.487517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dee37-afb2-4708-aa03-b86e276264c2 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.487517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dee37-afb2-4708-aa03-b86e276264c2 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.487517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dee37-afb2-4708-aa03-b86e276264c2 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.488496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 56/118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:23.489581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288606-8838-412d-b1cf-8e422906506d None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.489868 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288606-8838-412d-b1cf-8e422906506d None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.490216 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288606-8838-412d-b1cf-8e422906506d None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.491730 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 57/119] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:23.496232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8288606-8838-412d-b1cf-8e422906506d None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:23.499902 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d60f4-9452-4c37-8e6e-dc6abe744e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:23.501118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d60f4-9452-4c37-8e6e-dc6abe744e05 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.501798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d60f4-9452-4c37-8e6e-dc6abe744e05 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.502362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d60f4-9452-4c37-8e6e-dc6abe744e05 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.503955 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d60f4-9452-4c37-8e6e-dc6abe744e05 None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:23.505813 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d60f4-9452-4c37-8e6e-dc6abe744e05 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:23.508347 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 63/120] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:23 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:23.515344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:23.517712 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9388870-bdd1-4d68-8453-a1653ba36967 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.518474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9388870-bdd1-4d68-8453-a1653ba36967 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.519081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9388870-bdd1-4d68-8453-a1653ba36967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.546695 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-a9388870-bdd1-4d68-8453-a1653ba36967 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:23.549215 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-a9388870-bdd1-4d68-8453-a1653ba36967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:23.636706 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8288606-8838-412d-b1cf-8e422906506d None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:23.645365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 64/121] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:23.647692 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d8285-067d-4e8a-804d-39ca769342cb None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.647978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d8285-067d-4e8a-804d-39ca769342cb None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.648261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d8285-067d-4e8a-804d-39ca769342cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.671994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-c54d8285-067d-4e8a-804d-39ca769342cb None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:23.673976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-c54d8285-067d-4e8a-804d-39ca769342cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:23.678024 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d13073-b684-496a-89df-835672352ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:23.679070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13073-b684-496a-89df-835672352ffc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.679586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13073-b684-496a-89df-835672352ffc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.680109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13073-b684-496a-89df-835672352ffc None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.681091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d13073-b684-496a-89df-835672352ffc None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:23.681626 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d13073-b684-496a-89df-835672352ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:23.683447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d13073-b684-496a-89df-835672352ffc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:23.694946 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 58/122] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:23.696261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d13073-b684-496a-89df-835672352ffc None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:23.765195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 65/123] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:23.765195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d8285-067d-4e8a-804d-39ca769342cb None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:23.810629 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:23.819976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:23.840642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d13073-b684-496a-89df-835672352ffc None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:23.880691 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3193a89-ae3e-4b59-b8e5-415275199450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:23.881977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3193a89-ae3e-4b59-b8e5-415275199450 None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.882577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3193a89-ae3e-4b59-b8e5-415275199450 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.883088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3193a89-ae3e-4b59-b8e5-415275199450 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/users/596bd6a71df14947b131621dfe20517f/roles/844a46aa3ad0499fb8bfa2b1bd023ddd` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.884369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d8285-067d-4e8a-804d-39ca769342cb None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:23.885792 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3193a89-ae3e-4b59-b8e5-415275199450 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49c1fbd83f844dcacce2b9aa7ab8147, user_id=596bd6a71df14947b131621dfe20517f, role_id=844a46aa3ad0499fb8bfa2b1bd023ddd)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:23.889492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3193a89-ae3e-4b59-b8e5-415275199450 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:23.922261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 59/124] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:23 2022] PUT /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/users/596bd6a71df14947b131621dfe20517f/roles/844a46aa3ad0499fb8bfa2b1bd023ddd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:23.922261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3193a89-ae3e-4b59-b8e5-415275199450 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:23.954295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32407f2b-4b84-4bb5-a8b4-b6c14789c1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:23.955624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32407f2b-4b84-4bb5-a8b4-b6c14789c1b3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:23.956144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32407f2b-4b84-4bb5-a8b4-b6c14789c1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:23.956651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32407f2b-4b84-4bb5-a8b4-b6c14789c1b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:23.986292 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32407f2b-4b84-4bb5-a8b4-b6c14789c1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-32407f2b-4b84-4bb5-a8b4-b6c14789c1b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:23.992095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:23.993280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:23.994389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:24.005076 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 66/125] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:24.007023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32407f2b-4b84-4bb5-a8b4-b6c14789c1b3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.046907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3193a89-ae3e-4b59-b8e5-415275199450 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.092992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3d914-80a5-4c0c-ad1b-8898727b5c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.093681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3d914-80a5-4c0c-ad1b-8898727b5c8c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.093967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3d914-80a5-4c0c-ad1b-8898727b5c8c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.094279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3d914-80a5-4c0c-ad1b-8898727b5c8c None admin] PATH_INFO: `/v3/users/27091c013a8c447a81380abde9ae7c70` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.102132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3d914-80a5-4c0c-ad1b-8898727b5c8c None admin] RBAC: Authorizing `identity:get_user(user_id=27091c013a8c447a81380abde9ae7c70)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:24.102132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3d914-80a5-4c0c-ad1b-8898727b5c8c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:24.102132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 60/126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:23 2022] GET /identity/v3/users/27091c013a8c447a81380abde9ae7c70 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:24.102132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3d914-80a5-4c0c-ad1b-8898727b5c8c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.105357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:24.226792 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3d914-80a5-4c0c-ad1b-8898727b5c8c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.235333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32407f2b-4b84-4bb5-a8b4-b6c14789c1b3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.251268 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:24.273061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.273889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.274410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.274755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.275463 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:24.275773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:24.277380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:24.287443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 61/127] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:24.288430 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.292359 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663eb36-9ed1-47c1-99e1-da9f9fbb4f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.293611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663eb36-9ed1-47c1-99e1-da9f9fbb4f74 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.294191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663eb36-9ed1-47c1-99e1-da9f9fbb4f74 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.294709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663eb36-9ed1-47c1-99e1-da9f9fbb4f74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.309313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663eb36-9ed1-47c1-99e1-da9f9fbb4f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-2663eb36-9ed1-47c1-99e1-da9f9fbb4f74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:24.315323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:24.316853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:24.317896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:24.320694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 67/128] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:24.321838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663eb36-9ed1-47c1-99e1-da9f9fbb4f74 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.358789 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:24.403827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bd1f6-30e6-43b9-9cc3-eee2e67b9e1a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.442557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663eb36-9ed1-47c1-99e1-da9f9fbb4f74 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.468344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a015cb8-865c-45ff-8308-2576a48877bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.469338 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a015cb8-865c-45ff-8308-2576a48877bd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.469815 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a015cb8-865c-45ff-8308-2576a48877bd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.470403 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a015cb8-865c-45ff-8308-2576a48877bd None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.475184 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a015cb8-865c-45ff-8308-2576a48877bd None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:24.476689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a015cb8-865c-45ff-8308-2576a48877bd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:24.478897 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 62/129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:24 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:24.479523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a015cb8-865c-45ff-8308-2576a48877bd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.484257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda36e-614d-4808-b8c5-303cba04d629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.485309 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda36e-614d-4808-b8c5-303cba04d629 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.485924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda36e-614d-4808-b8c5-303cba04d629 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.486458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda36e-614d-4808-b8c5-303cba04d629 None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.487945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeda36e-614d-4808-b8c5-303cba04d629 None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:24.489701 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeda36e-614d-4808-b8c5-303cba04d629 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:24.491832 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 68/130] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:24 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:24.492839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda36e-614d-4808-b8c5-303cba04d629 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.532144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:24.603064 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a015cb8-865c-45ff-8308-2576a48877bd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.604396 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda36e-614d-4808-b8c5-303cba04d629 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.639049 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:24.644055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3eaf9b-4749-4a81-bf00-ae208ce21128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.644895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eaf9b-4749-4a81-bf00-ae208ce21128 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.645390 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eaf9b-4749-4a81-bf00-ae208ce21128 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.645762 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eaf9b-4749-4a81-bf00-ae208ce21128 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/users/27091c013a8c447a81380abde9ae7c70/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.646207 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.647089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.647537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.647979 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.648355 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3eaf9b-4749-4a81-bf00-ae208ce21128 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49c1fbd83f844dcacce2b9aa7ab8147, user_id=27091c013a8c447a81380abde9ae7c70, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:24.649069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:24.649541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:24.650190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3eaf9b-4749-4a81-bf00-ae208ce21128 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:24.651390 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:24.663797 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 63/131] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:24.665346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.672222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 69/132] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:24 2022] PUT /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/users/27091c013a8c447a81380abde9ae7c70/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:24.673153 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3eaf9b-4749-4a81-bf00-ae208ce21128 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.786207 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47dfc5-6996-4971-9489-bcdafe52e866 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.789299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:24.796260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3eaf9b-4749-4a81-bf00-ae208ce21128 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:24.834093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b23ace-5828-4308-84bc-a8107bc7402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.835233 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b23ace-5828-4308-84bc-a8107bc7402d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.835763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b23ace-5828-4308-84bc-a8107bc7402d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.836301 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b23ace-5828-4308-84bc-a8107bc7402d None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.837806 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b23ace-5828-4308-84bc-a8107bc7402d None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:24.839421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7954-0f40-43f3-b92b-0b5ad334fd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:24.839997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b23ace-5828-4308-84bc-a8107bc7402d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:24.840118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7954-0f40-43f3-b92b-0b5ad334fd33 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:24.840381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7954-0f40-43f3-b92b-0b5ad334fd33 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:24.840664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7954-0f40-43f3-b92b-0b5ad334fd33 None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:24.841988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e7954-0f40-43f3-b92b-0b5ad334fd33 None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:24.842948 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 64/133] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:24 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:24.843659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e7954-0f40-43f3-b92b-0b5ad334fd33 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:24.845551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 70/134] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:24 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:24.851011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b23ace-5828-4308-84bc-a8107bc7402d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.851168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7954-0f40-43f3-b92b-0b5ad334fd33 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:24.934702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:24.984070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b23ace-5828-4308-84bc-a8107bc7402d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:25.007053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7954-0f40-43f3-b92b-0b5ad334fd33 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:25.024478 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d86b7-b540-48b1-8c29-c7dfee120454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:25.025627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d86b7-b540-48b1-8c29-c7dfee120454 None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:25.026182 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d86b7-b540-48b1-8c29-c7dfee120454 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:25.026687 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d86b7-b540-48b1-8c29-c7dfee120454 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/users/596bd6a71df14947b131621dfe20517f/roles/167df0c7f94644578aff0abc3a0c8e84` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:25.030034 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d86b7-b540-48b1-8c29-c7dfee120454 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49c1fbd83f844dcacce2b9aa7ab8147, user_id=596bd6a71df14947b131621dfe20517f, role_id=167df0c7f94644578aff0abc3a0c8e84)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:25.031926 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d86b7-b540-48b1-8c29-c7dfee120454 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:25.056307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 65/135] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:24 2022] PUT /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/users/596bd6a71df14947b131621dfe20517f/roles/167df0c7f94644578aff0abc3a0c8e84 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:25.087084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37c0c0-b704-40b1-9a9e-3a4e20d4d52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:25.088348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37c0c0-b704-40b1-9a9e-3a4e20d4d52f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:25.088915 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37c0c0-b704-40b1-9a9e-3a4e20d4d52f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:25.089531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37c0c0-b704-40b1-9a9e-3a4e20d4d52f None admin] PATH_INFO: `/v3/projects/2b220a1fd63f4bb3aacafbc1723926e4` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:25.094221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37c0c0-b704-40b1-9a9e-3a4e20d4d52f None admin] RBAC: Authorizing `identity:get_project(project_id=2b220a1fd63f4bb3aacafbc1723926e4)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:25.096505 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37c0c0-b704-40b1-9a9e-3a4e20d4d52f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:25.098885 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 71/136] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:24 2022] GET /identity/v3/projects/2b220a1fd63f4bb3aacafbc1723926e4 => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:25.105117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d86b7-b540-48b1-8c29-c7dfee120454 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:25.229748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d86b7-b540-48b1-8c29-c7dfee120454 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:25.255109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:25.267708 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d0994-74e0-40f1-850c-e52c2535c40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:25.269092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d0994-74e0-40f1-850c-e52c2535c40b None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:25.270299 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d0994-74e0-40f1-850c-e52c2535c40b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:25.270535 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d0994-74e0-40f1-850c-e52c2535c40b None admin] PATH_INFO: `/v3/projects/2b220a1fd63f4bb3aacafbc1723926e4/users/596bd6a71df14947b131621dfe20517f/roles/844a46aa3ad0499fb8bfa2b1bd023ddd` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:25.273090 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d0994-74e0-40f1-850c-e52c2535c40b None admin] RBAC: Authorizing `identity:create_grant(project_id=2b220a1fd63f4bb3aacafbc1723926e4, user_id=596bd6a71df14947b131621dfe20517f, role_id=844a46aa3ad0499fb8bfa2b1bd023ddd)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:25.274772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d0994-74e0-40f1-850c-e52c2535c40b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:25.299967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 66/137] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:25 2022] PUT /identity/v3/projects/2b220a1fd63f4bb3aacafbc1723926e4/users/596bd6a71df14947b131621dfe20517f/roles/844a46aa3ad0499fb8bfa2b1bd023ddd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:25.570186 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854f72-b26a-4475-b970-317ccc2d196c None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:25.571153 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854f72-b26a-4475-b970-317ccc2d196c None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:25.571593 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854f72-b26a-4475-b970-317ccc2d196c None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:25.573223 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 72/138] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:25.579227 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325456f-4922-4595-9584-e4101133f4ef None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:25.580204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325456f-4922-4595-9584-e4101133f4ef None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:25.580646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325456f-4922-4595-9584-e4101133f4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:25.607223 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-d325456f-4922-4595-9584-e4101133f4ef None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:25.608835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-d325456f-4922-4595-9584-e4101133f4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:25.691613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 67/139] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:25.719183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854f72-b26a-4475-b970-317ccc2d196c None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:25.755736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:25.842744 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854f72-b26a-4475-b970-317ccc2d196c None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:25.880092 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a82610-d591-4187-86e9-62800e3a1af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:25.880768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a82610-d591-4187-86e9-62800e3a1af5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:25.881055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a82610-d591-4187-86e9-62800e3a1af5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:25.881334 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a82610-d591-4187-86e9-62800e3a1af5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:25.882561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a82610-d591-4187-86e9-62800e3a1af5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:25.884086 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a82610-d591-4187-86e9-62800e3a1af5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:25.886049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 73/140] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:25.891810 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325456f-4922-4595-9584-e4101133f4ef None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:25.953384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39cbd1-3f83-4d2b-9efb-92f7721129c7 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:25.953734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39cbd1-3f83-4d2b-9efb-92f7721129c7 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:25.953945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39cbd1-3f83-4d2b-9efb-92f7721129c7 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:25.955618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 74/141] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:25.960737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1309f6c-65e0-48f1-8d8b-7596c192a1ee None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:25.961023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1309f6c-65e0-48f1-8d8b-7596c192a1ee None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:25.961307 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1309f6c-65e0-48f1-8d8b-7596c192a1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:25.980357 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-f1309f6c-65e0-48f1-8d8b-7596c192a1ee None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:25.981580 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-f1309f6c-65e0-48f1-8d8b-7596c192a1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:26.055272 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325456f-4922-4595-9584-e4101133f4ef None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:26.085341 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:26.108456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 75/142] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:26.126222 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd82335-86f0-4941-9e62-7d0349d191aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:26.126935 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd82335-86f0-4941-9e62-7d0349d191aa None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:26.127256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd82335-86f0-4941-9e62-7d0349d191aa None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:26.127492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd82335-86f0-4941-9e62-7d0349d191aa None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:26.128223 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd82335-86f0-4941-9e62-7d0349d191aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:26.129750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd82335-86f0-4941-9e62-7d0349d191aa None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:26.156437 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1309f6c-65e0-48f1-8d8b-7596c192a1ee None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:26.174333 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 68/143] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:55:25 2022] POST /identity/v3/users => generated 283 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:26.209350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:26.323923 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1309f6c-65e0-48f1-8d8b-7596c192a1ee None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:26.378820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a332a-f288-4606-b757-56157d97a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:26.379930 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a332a-f288-4606-b757-56157d97a724 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:26.380442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a332a-f288-4606-b757-56157d97a724 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:26.380939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a332a-f288-4606-b757-56157d97a724 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:26.398890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a332a-f288-4606-b757-56157d97a724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-c30a332a-f288-4606-b757-56157d97a724 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:26.404960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:26.406127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:26.407200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 76/144] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:26.412541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd82335-86f0-4941-9e62-7d0349d191aa None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:26.550119 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd82335-86f0-4941-9e62-7d0349d191aa None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:26.585272 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:26.586243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:26.586243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:26.586512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:26.587336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:26.587659 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:26.589088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:26.600477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 69/145] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:26.606345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a332a-f288-4606-b757-56157d97a724 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:26.727297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:26.739495 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a332a-f288-4606-b757-56157d97a724 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:26.790139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6130b42-01ab-4180-ad5a-43f514ed4ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:26.790820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130b42-01ab-4180-ad5a-43f514ed4ec9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:26.791149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130b42-01ab-4180-ad5a-43f514ed4ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:26.791424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130b42-01ab-4180-ad5a-43f514ed4ec9 None admin] PATH_INFO: `/v3/users/661eecaf09e449c19c1e1ce545060682` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:26.792737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6130b42-01ab-4180-ad5a-43f514ed4ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=661eecaf09e449c19c1e1ce545060682)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:26.794364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6130b42-01ab-4180-ad5a-43f514ed4ec9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:26.796064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 77/146] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:26 2022] GET /identity/v3/users/661eecaf09e449c19c1e1ce545060682 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:26.801949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:26.838421 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:26.928877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d49904-0a30-4e7a-9950-6d8b846d8b76 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:26.969012 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4532-58a6-4385-a093-38d7861b03de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:26.969700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4532-58a6-4385-a093-38d7861b03de None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:26.969971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4532-58a6-4385-a093-38d7861b03de None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:26.970323 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4532-58a6-4385-a093-38d7861b03de None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:26.971594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da4532-58a6-4385-a093-38d7861b03de None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:26.973175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da4532-58a6-4385-a093-38d7861b03de None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:26.975014 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 70/147] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:26 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:26.981246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6130b42-01ab-4180-ad5a-43f514ed4ec9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:27.127564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6130b42-01ab-4180-ad5a-43f514ed4ec9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:27.180620 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a721e29-4a6a-4577-837d-a5f41affba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:27.181320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721e29-4a6a-4577-837d-a5f41affba00 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:27.181600 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721e29-4a6a-4577-837d-a5f41affba00 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:27.181875 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721e29-4a6a-4577-837d-a5f41affba00 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/users/661eecaf09e449c19c1e1ce545060682/roles/61c2794c5a6d4f91b3a953f0ae6eab8b` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:27.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a721e29-4a6a-4577-837d-a5f41affba00 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49c1fbd83f844dcacce2b9aa7ab8147, user_id=661eecaf09e449c19c1e1ce545060682, role_id=61c2794c5a6d4f91b3a953f0ae6eab8b)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:27.189155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a721e29-4a6a-4577-837d-a5f41affba00 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:27.213088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 78/148] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:26 2022] PUT /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/users/661eecaf09e449c19c1e1ce545060682/roles/61c2794c5a6d4f91b3a953f0ae6eab8b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:27.294966 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732f8a-9603-4b61-b18e-5dc91f5502a8 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:27.295178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732f8a-9603-4b61-b18e-5dc91f5502a8 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:27.295440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732f8a-9603-4b61-b18e-5dc91f5502a8 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:27.297038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 71/149] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:27.303031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3410-63fb-4ecf-8225-ff772bc8eddd None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:27.303351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3410-63fb-4ecf-8225-ff772bc8eddd None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:27.303595 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3410-63fb-4ecf-8225-ff772bc8eddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:27.332270 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-eeeb3410-63fb-4ecf-8225-ff772bc8eddd None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:27.333498 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-eeeb3410-63fb-4ecf-8225-ff772bc8eddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:27.356791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:27.453051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 79/150] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:27.496975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a732f8a-9603-4b61-b18e-5dc91f5502a8 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:27.605924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:27.611247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a732f8a-9603-4b61-b18e-5dc91f5502a8 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:27.653177 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14db1-e92d-4a15-b424-7d0d7d6d1b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:27.653857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14db1-e92d-4a15-b424-7d0d7d6d1b5c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:27.654197 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14db1-e92d-4a15-b424-7d0d7d6d1b5c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:27.654480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14db1-e92d-4a15-b424-7d0d7d6d1b5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:27.666027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e14db1-e92d-4a15-b424-7d0d7d6d1b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-59e14db1-e92d-4a15-b424-7d0d7d6d1b5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:27.671264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:27.672550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:27.673776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 72/151] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:27.678972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3410-63fb-4ecf-8225-ff772bc8eddd None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:27.824224 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3410-63fb-4ecf-8225-ff772bc8eddd None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:27.866994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:27.867559 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:27.867844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:27.868118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:27.868866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] RBAC: Adding query filter params (name=member) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:27.869169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:27.870644 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:27.880175 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 80/152] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:27.886150 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14db1-e92d-4a15-b424-7d0d7d6d1b5c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:28.027123 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14db1-e92d-4a15-b424-7d0d7d6d1b5c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:28.081844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0945f23-13b2-4eef-b6c7-0b4cbc1fc501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:28.082552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0945f23-13b2-4eef-b6c7-0b4cbc1fc501 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:28.083996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0945f23-13b2-4eef-b6c7-0b4cbc1fc501 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:28.083996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0945f23-13b2-4eef-b6c7-0b4cbc1fc501 None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:28.086854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0945f23-13b2-4eef-b6c7-0b4cbc1fc501 None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:28.088427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0945f23-13b2-4eef-b6c7-0b4cbc1fc501 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:28.090235 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 73/153] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:27 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:28.096063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:28.109570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:28.239162 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dc02b-4e61-4506-97ee-9b4b277db4cd None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:28.286949 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31507ab-3e4f-409d-a7dd-4729fac2bda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:28.287622 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31507ab-3e4f-409d-a7dd-4729fac2bda7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:28.287910 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31507ab-3e4f-409d-a7dd-4729fac2bda7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:28.288198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31507ab-3e4f-409d-a7dd-4729fac2bda7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:28.367335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31507ab-3e4f-409d-a7dd-4729fac2bda7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:28.369083 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31507ab-3e4f-409d-a7dd-4729fac2bda7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:28.383724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:28.423703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3fb3c-7a22-4a1b-b1e4-39b678a418de None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:28.424399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3fb3c-7a22-4a1b-b1e4-39b678a418de None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:28.424908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3fb3c-7a22-4a1b-b1e4-39b678a418de None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:28.426560 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 74/154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:28.432172 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d363e-4ce8-455c-876f-09d81578db89 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:28.432777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d363e-4ce8-455c-876f-09d81578db89 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:28.433273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d363e-4ce8-455c-876f-09d81578db89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:28.475196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-8b7d363e-4ce8-455c-876f-09d81578db89 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:28.476841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-8b7d363e-4ce8-455c-876f-09d81578db89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:28.514726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 81/155] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:28.520623 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:28.527000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31507ab-3e4f-409d-a7dd-4729fac2bda7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:28.633189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 75/156] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:28.699703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d363e-4ce8-455c-876f-09d81578db89 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:28.719878 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31507ab-3e4f-409d-a7dd-4729fac2bda7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:28.743048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:28.805176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864384a3-8e1e-4d63-baf3-184f712b7af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:28.806282 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864384a3-8e1e-4d63-baf3-184f712b7af3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:28.806830 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864384a3-8e1e-4d63-baf3-184f712b7af3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:28.807423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864384a3-8e1e-4d63-baf3-184f712b7af3 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:28.808987 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864384a3-8e1e-4d63-baf3-184f712b7af3 None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:28.810775 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864384a3-8e1e-4d63-baf3-184f712b7af3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:28.812751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 82/157] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:28 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:28.819107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864384a3-8e1e-4d63-baf3-184f712b7af3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:28.862752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d363e-4ce8-455c-876f-09d81578db89 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:28.918290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99163f50-4e9b-4c09-95ea-75bfd89e9562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:28.918977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99163f50-4e9b-4c09-95ea-75bfd89e9562 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:28.919259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99163f50-4e9b-4c09-95ea-75bfd89e9562 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:28.919534 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99163f50-4e9b-4c09-95ea-75bfd89e9562 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:28.936405 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99163f50-4e9b-4c09-95ea-75bfd89e9562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-99163f50-4e9b-4c09-95ea-75bfd89e9562 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:28.942749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:28.943895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:28.944938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 76/158] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:28.950323 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99163f50-4e9b-4c09-95ea-75bfd89e9562 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:28.978428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:29.059744 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864384a3-8e1e-4d63-baf3-184f712b7af3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:29.117120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99163f50-4e9b-4c09-95ea-75bfd89e9562 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:29.121478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:29.122271 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.122568 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.122857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.123687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] RBAC: Adding query filter params (role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147, user.id=596bd6a71df14947b131621dfe20517f) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:29.124020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:29.125560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:29.142270 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 83/159] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:28 2022] GET /identity/v3/role_assignments?user.id=596bd6a71df14947b131621dfe20517f&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:29.179807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1267a0-7377-400f-8f4b-7ed71e2277fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:29.180476 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1267a0-7377-400f-8f4b-7ed71e2277fc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.180761 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1267a0-7377-400f-8f4b-7ed71e2277fc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.181045 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1267a0-7377-400f-8f4b-7ed71e2277fc None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.181797 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1267a0-7377-400f-8f4b-7ed71e2277fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:29.182100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1267a0-7377-400f-8f4b-7ed71e2277fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:29.183564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1267a0-7377-400f-8f4b-7ed71e2277fc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:29.193371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 77/160] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:29.199413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:29.269658 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:29.302080 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cfc08-262b-4c84-bb6c-b47bac7ea961 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.302377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cfc08-262b-4c84-bb6c-b47bac7ea961 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.302653 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cfc08-262b-4c84-bb6c-b47bac7ea961 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.304062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 78/161] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:29.316701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c2ce-4f53-4f2d-864a-c9c09f9182f4 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.316938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c2ce-4f53-4f2d-864a-c9c09f9182f4 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.317176 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c2ce-4f53-4f2d-864a-c9c09f9182f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.356045 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-c2e0c2ce-4f53-4f2d-864a-c9c09f9182f4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:29.357360 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-c2e0c2ce-4f53-4f2d-864a-c9c09f9182f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:29.361570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ad13b-02f4-4880-b095-405e76248336 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:29.412207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc20d2-9a58-4e8d-b78b-355f09e12ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:29.412909 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc20d2-9a58-4e8d-b78b-355f09e12ed2 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.413200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc20d2-9a58-4e8d-b78b-355f09e12ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.413487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc20d2-9a58-4e8d-b78b-355f09e12ed2 None admin] PATH_INFO: `/v3/users/27091c013a8c447a81380abde9ae7c70` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.414911 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc20d2-9a58-4e8d-b78b-355f09e12ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=27091c013a8c447a81380abde9ae7c70)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:29.416553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc20d2-9a58-4e8d-b78b-355f09e12ed2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:29.422269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 84/162] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:29 2022] GET /identity/v3/users/27091c013a8c447a81380abde9ae7c70 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:29.423931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264eb04f-256b-413a-8a70-51a653e7e657 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.424230 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264eb04f-256b-413a-8a70-51a653e7e657 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.424519 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264eb04f-256b-413a-8a70-51a653e7e657 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.426197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 85/163] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:29.428653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264eb04f-256b-413a-8a70-51a653e7e657 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:29.513933 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 79/164] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:29.519063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75110b-3e6a-4aca-8f08-45e7cc1633a9 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.519388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75110b-3e6a-4aca-8f08-45e7cc1633a9 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.519677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75110b-3e6a-4aca-8f08-45e7cc1633a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.555195 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ea75110b-3e6a-4aca-8f08-45e7cc1633a9 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:29.556415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ea75110b-3e6a-4aca-8f08-45e7cc1633a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:29.558792 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:29.596005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264eb04f-256b-413a-8a70-51a653e7e657 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:29.614479 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:29.641024 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 80/165] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:29.641905 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75110b-3e6a-4aca-8f08-45e7cc1633a9 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:29.662976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc97060-2959-467e-8139-7bea70904b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:29.663625 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc97060-2959-467e-8139-7bea70904b28 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.663906 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc97060-2959-467e-8139-7bea70904b28 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.664179 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc97060-2959-467e-8139-7bea70904b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.803448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc97060-2959-467e-8139-7bea70904b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:29.805054 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc97060-2959-467e-8139-7bea70904b28 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:29.881120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75110b-3e6a-4aca-8f08-45e7cc1633a9 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:29.936529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 86/166] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:29.937410 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc97060-2959-467e-8139-7bea70904b28 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:29.951507 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:29.974883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe742f-55a3-40d4-8f26-35368fe8ddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:29.975586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe742f-55a3-40d4-8f26-35368fe8ddc9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:29.975877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe742f-55a3-40d4-8f26-35368fe8ddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:29.976155 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe742f-55a3-40d4-8f26-35368fe8ddc9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:29.987815 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:29.994622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe742f-55a3-40d4-8f26-35368fe8ddc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-f8fe742f-55a3-40d4-8f26-35368fe8ddc9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:30.003115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:30.004324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:30.005414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:30.008563 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 81/167] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:30.009734 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe742f-55a3-40d4-8f26-35368fe8ddc9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:30.054223 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc97060-2959-467e-8139-7bea70904b28 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:30.092073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d5a06-d368-45f7-b65b-4207f5d7ec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:30.092752 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d5a06-d368-45f7-b65b-4207f5d7ec43 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:30.093018 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d5a06-d368-45f7-b65b-4207f5d7ec43 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:30.093296 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d5a06-d368-45f7-b65b-4207f5d7ec43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:30.107077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d5a06-d368-45f7-b65b-4207f5d7ec43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-7c6d5a06-d368-45f7-b65b-4207f5d7ec43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:30.112248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:30.113401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:30.114494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 87/168] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:30.116903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d5a06-d368-45f7-b65b-4207f5d7ec43 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:30.198029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe742f-55a3-40d4-8f26-35368fe8ddc9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:30.222507 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:30.231531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d5a06-d368-45f7-b65b-4207f5d7ec43 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:30.266951 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c4c10-7581-4db1-8425-6d875d64e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:30.267747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4c10-7581-4db1-8425-6d875d64e785 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:30.268041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4c10-7581-4db1-8425-6d875d64e785 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:30.268324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4c10-7581-4db1-8425-6d875d64e785 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:30.269149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:30.269804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:30.270089 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:30.270394 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:30.271133 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:30.271442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:30.273016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:30.274889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c4c10-7581-4db1-8425-6d875d64e785 None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:30.276399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c4c10-7581-4db1-8425-6d875d64e785 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:30.278380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 82/169] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:30 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:30.279092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c4c10-7581-4db1-8425-6d875d64e785 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:30.281656 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 88/170] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:30.287313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:30.402490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:30.464864 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b51458-8122-411d-950d-bfac503d428c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:30.505080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:30.509621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c4c10-7581-4db1-8425-6d875d64e785 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:30.516474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7420a27-744f-4298-a75c-ab854ed2acc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:30.517191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7420a27-744f-4298-a75c-ab854ed2acc1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:30.517477 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7420a27-744f-4298-a75c-ab854ed2acc1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:30.517758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7420a27-744f-4298-a75c-ab854ed2acc1 None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:30.523626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7420a27-744f-4298-a75c-ab854ed2acc1 None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:30.525158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7420a27-744f-4298-a75c-ab854ed2acc1 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:30.528497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 89/171] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:30 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:30.532964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7420a27-744f-4298-a75c-ab854ed2acc1 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:30.560035 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:30.560687 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:30.560970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:30.561246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:30.561988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:30.562332 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:30.563756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:30.574789 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 83/172] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:30.575662 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:30.694151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c43bd7-ba3c-482a-a7cb-3ccf6d560fff None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:30.728302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:30.740185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948e5dd-ae49-4b65-892d-1bd99b11c244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:30.741150 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948e5dd-ae49-4b65-892d-1bd99b11c244 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:30.741621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948e5dd-ae49-4b65-892d-1bd99b11c244 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:30.742138 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948e5dd-ae49-4b65-892d-1bd99b11c244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:30.815824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7420a27-744f-4298-a75c-ab854ed2acc1 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:30.826557 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948e5dd-ae49-4b65-892d-1bd99b11c244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:30.829076 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948e5dd-ae49-4b65-892d-1bd99b11c244 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:30.856932 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:30.857516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:30.857811 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:30.858093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:30.858933 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147, user.id=27091c013a8c447a81380abde9ae7c70) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:30.859251 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:30.860849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:30.867364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 90/173] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:30 2022] GET /identity/v3/role_assignments?user.id=27091c013a8c447a81380abde9ae7c70&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 608 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:30.870961 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:30.965843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 84/174] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:30.967698 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83b035-4313-4f6f-bf63-161835d84f28 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:30.968408 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83b035-4313-4f6f-bf63-161835d84f28 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:30.969052 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83b035-4313-4f6f-bf63-161835d84f28 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:30.970773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 85/175] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:30.972355 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83b035-4313-4f6f-bf63-161835d84f28 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:31.008715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:31.073224 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:31.132627 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717eeff-2523-408d-93c5-8d7f9c1fbd70 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:31.153678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83b035-4313-4f6f-bf63-161835d84f28 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:31.175412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd70320-f52d-4ede-b390-cc76470fa314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:31.177052 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70320-f52d-4ede-b390-cc76470fa314 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.177538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70320-f52d-4ede-b390-cc76470fa314 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.177990 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70320-f52d-4ede-b390-cc76470fa314 None admin] PATH_INFO: `/v3/users/596bd6a71df14947b131621dfe20517f` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.179478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd70320-f52d-4ede-b390-cc76470fa314 None admin] RBAC: Authorizing `identity:get_user(user_id=596bd6a71df14947b131621dfe20517f)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:31.182077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd70320-f52d-4ede-b390-cc76470fa314 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:31.184967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 91/176] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:30 2022] GET /identity/v3/users/596bd6a71df14947b131621dfe20517f => generated 275 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:31.186673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c55778-bf89-40e2-9613-1bd558a40ed8 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.187181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c55778-bf89-40e2-9613-1bd558a40ed8 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.188083 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c55778-bf89-40e2-9613-1bd558a40ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.198879 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff1dad-d3fe-40e6-adff-ad59e5939c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:31.199558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff1dad-d3fe-40e6-adff-ad59e5939c00 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.199846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff1dad-d3fe-40e6-adff-ad59e5939c00 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.200185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff1dad-d3fe-40e6-adff-ad59e5939c00 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.202484 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff1dad-d3fe-40e6-adff-ad59e5939c00 None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:31.204005 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff1dad-d3fe-40e6-adff-ad59e5939c00 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:31.205918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 86/177] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:30 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:31.207335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108974a3-a599-48ae-928c-c3c2a496af70 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.207640 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108974a3-a599-48ae-928c-c3c2a496af70 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.207911 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108974a3-a599-48ae-928c-c3c2a496af70 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.209173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 87/178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:31.209949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108974a3-a599-48ae-928c-c3c2a496af70 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:31.213601 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-82c55778-bf89-40e2-9613-1bd558a40ed8 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:31.216297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-82c55778-bf89-40e2-9613-1bd558a40ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:31.312376 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 92/179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:31.313989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd0573-7395-489b-afc1-a3208d325172 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.314318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd0573-7395-489b-afc1-a3208d325172 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.314588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd0573-7395-489b-afc1-a3208d325172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.334675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-0ccd0573-7395-489b-afc1-a3208d325172 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:31.335897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-0ccd0573-7395-489b-afc1-a3208d325172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:31.343669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:31.352291 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108974a3-a599-48ae-928c-c3c2a496af70 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:31.389063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c217db9-e9f1-4a3c-ba91-3199f49bd3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:31.389822 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c217db9-e9f1-4a3c-ba91-3199f49bd3ab None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.390159 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c217db9-e9f1-4a3c-ba91-3199f49bd3ab None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.390434 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c217db9-e9f1-4a3c-ba91-3199f49bd3ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.399496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:31.410861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 93/180] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:31.415717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd0573-7395-489b-afc1-a3208d325172 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:31.462940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c217db9-e9f1-4a3c-ba91-3199f49bd3ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:31.464431 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c217db9-e9f1-4a3c-ba91-3199f49bd3ab None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:31.523712 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd0573-7395-489b-afc1-a3208d325172 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:31.563397 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:31.568825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aaa83-7328-4e77-9c1b-5a8c3b9f09e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:31.569554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aaa83-7328-4e77-9c1b-5a8c3b9f09e7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.569837 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aaa83-7328-4e77-9c1b-5a8c3b9f09e7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.570118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aaa83-7328-4e77-9c1b-5a8c3b9f09e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.570888 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 88/181] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:31.571005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9aaa83-7328-4e77-9c1b-5a8c3b9f09e7 None admin] RBAC: Adding query filter params (role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147, user.id=596bd6a71df14947b131621dfe20517f) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:31.571322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9aaa83-7328-4e77-9c1b-5a8c3b9f09e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:31.572715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9aaa83-7328-4e77-9c1b-5a8c3b9f09e7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:31.575541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93374-5917-4170-b928-9b31ee7309e5 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.576042 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93374-5917-4170-b928-9b31ee7309e5 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.576530 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93374-5917-4170-b928-9b31ee7309e5 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.577915 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 89/182] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:31.578712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e93374-5917-4170-b928-9b31ee7309e5 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:31.580817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 94/183] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:31 2022] GET /identity/v3/role_assignments?user.id=596bd6a71df14947b131621dfe20517f&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:31.582446 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014141bc-d64c-4106-85dd-07c7be6b406c None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.582446 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014141bc-d64c-4106-85dd-07c7be6b406c None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.582768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014141bc-d64c-4106-85dd-07c7be6b406c None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.585203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 95/184] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:31.585203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014141bc-d64c-4106-85dd-07c7be6b406c None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:31.664635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:31.719340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014141bc-d64c-4106-85dd-07c7be6b406c None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:31.725119 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e93374-5917-4170-b928-9b31ee7309e5 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:31.771141 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9a102-f709-4dd1-b2a7-4bc466d23973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:31.771932 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a102-f709-4dd1-b2a7-4bc466d23973 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.772214 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a102-f709-4dd1-b2a7-4bc466d23973 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.772587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a102-f709-4dd1-b2a7-4bc466d23973 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.773693 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9a102-f709-4dd1-b2a7-4bc466d23973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:31.776429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9a102-f709-4dd1-b2a7-4bc466d23973 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:31.777160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 96/185] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:31.777950 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9a102-f709-4dd1-b2a7-4bc466d23973 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:31.781079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4f209-aa77-4e69-9421-b9116fbd45fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:31.781851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f209-aa77-4e69-9421-b9116fbd45fa None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.782302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f209-aa77-4e69-9421-b9116fbd45fa None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.782679 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f209-aa77-4e69-9421-b9116fbd45fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.796847 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4f209-aa77-4e69-9421-b9116fbd45fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-3cb4f209-aa77-4e69-9421-b9116fbd45fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:31.802509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:31.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:31.805136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 90/186] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:31.806269 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753acc5e-1cca-403e-9987-aa93f3ff974d None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.806269 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753acc5e-1cca-403e-9987-aa93f3ff974d None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.806269 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753acc5e-1cca-403e-9987-aa93f3ff974d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.826448 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-753acc5e-1cca-403e-9987-aa93f3ff974d None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:31.827987 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-753acc5e-1cca-403e-9987-aa93f3ff974d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:31.884999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:31.897835 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9a102-f709-4dd1-b2a7-4bc466d23973 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:31.902382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 91/187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:31.907279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd3f00-5356-442e-bc1c-1ec10077ac63 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.907645 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd3f00-5356-442e-bc1c-1ec10077ac63 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.907968 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd3f00-5356-442e-bc1c-1ec10077ac63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.926972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:31.928463 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-54fd3f00-5356-442e-bc1c-1ec10077ac63 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:31.930215 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-54fd3f00-5356-442e-bc1c-1ec10077ac63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:31.937834 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c59ead-1302-4623-81b8-7c5d14778a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:31.938535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c59ead-1302-4623-81b8-7c5d14778a73 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:31.939047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c59ead-1302-4623-81b8-7c5d14778a73 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:31.939329 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c59ead-1302-4623-81b8-7c5d14778a73 None admin] PATH_INFO: `/v3/projects/2b220a1fd63f4bb3aacafbc1723926e4` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:31.940610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c59ead-1302-4623-81b8-7c5d14778a73 None admin] RBAC: Authorizing `identity:get_project(project_id=2b220a1fd63f4bb3aacafbc1723926e4)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:31.942584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c59ead-1302-4623-81b8-7c5d14778a73 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:31.944426 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 97/188] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:31 2022] GET /identity/v3/projects/2b220a1fd63f4bb3aacafbc1723926e4 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:31.945240 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c59ead-1302-4623-81b8-7c5d14778a73 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.012655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 92/189] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:32.013561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd3f00-5356-442e-bc1c-1ec10077ac63 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.066964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c59ead-1302-4623-81b8-7c5d14778a73 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.110965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4658ffc-d10d-42d7-9504-10cecf6e0f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.111566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658ffc-d10d-42d7-9504-10cecf6e0f0b None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.111853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658ffc-d10d-42d7-9504-10cecf6e0f0b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.116136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658ffc-d10d-42d7-9504-10cecf6e0f0b None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.116934 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4658ffc-d10d-42d7-9504-10cecf6e0f0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:32.118476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4658ffc-d10d-42d7-9504-10cecf6e0f0b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:32.131767 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd3f00-5356-442e-bc1c-1ec10077ac63 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.151179 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 98/190] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:55:31 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:32.152049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4658ffc-d10d-42d7-9504-10cecf6e0f0b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.163891 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:32.180000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.180820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.181173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.181514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.182397 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:32.182778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:32.184337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:32.195041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 93/191] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:32.195041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.215680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:32.280641 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4658ffc-d10d-42d7-9504-10cecf6e0f0b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.319235 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986618a4-dc91-44db-96a8-9c1cb6808eca None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.329261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fe4ab-0139-4c28-9210-426a59aa9aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.330354 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fe4ab-0139-4c28-9210-426a59aa9aa5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.331177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fe4ab-0139-4c28-9210-426a59aa9aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.331177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fe4ab-0139-4c28-9210-426a59aa9aa5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.346023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fe4ab-0139-4c28-9210-426a59aa9aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-183fe4ab-0139-4c28-9210-426a59aa9aa5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:32.351152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:32.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 99/192] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:32.353705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fe4ab-0139-4c28-9210-426a59aa9aa5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.368945 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.370282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.370282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.370548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.371417 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] RBAC: Adding query filter params (role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=2b220a1fd63f4bb3aacafbc1723926e4, user.id=596bd6a71df14947b131621dfe20517f) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:32.371817 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:32.373392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:32.381643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 94/193] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/role_assignments?user.id=596bd6a71df14947b131621dfe20517f&scope.project.id=2b220a1fd63f4bb3aacafbc1723926e4&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:32.381643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.456336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:32.497099 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fe4ab-0139-4c28-9210-426a59aa9aa5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.509387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:32.541454 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd68900-f38f-49f7-a94c-354ac127f3fa None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.541688 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d65eb-3ac9-482b-999c-a768d18d924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.542303 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d65eb-3ac9-482b-999c-a768d18d924b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.542590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d65eb-3ac9-482b-999c-a768d18d924b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.542868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d65eb-3ac9-482b-999c-a768d18d924b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.557835 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d65eb-3ac9-482b-999c-a768d18d924b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-c34d65eb-3ac9-482b-999c-a768d18d924b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:32.564036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:32.565257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:32.566784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:32.569587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 100/194] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:32.571032 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d65eb-3ac9-482b-999c-a768d18d924b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.581855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2df67e-b307-4e27-ac37-567824ff4f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.582586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df67e-b307-4e27-ac37-567824ff4f3f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.582872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df67e-b307-4e27-ac37-567824ff4f3f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.583153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df67e-b307-4e27-ac37-567824ff4f3f None admin] PATH_INFO: `/v3/users/ebafb2e05b9f4da08179c97e1f806ef7` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.599980 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2df67e-b307-4e27-ac37-567824ff4f3f None admin] RBAC: Authorizing `identity:get_user(user_id=ebafb2e05b9f4da08179c97e1f806ef7)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:32.601500 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2df67e-b307-4e27-ac37-567824ff4f3f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:32.603346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 95/195] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/users/ebafb2e05b9f4da08179c97e1f806ef7 => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:32.604151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2df67e-b307-4e27-ac37-567824ff4f3f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.693293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d65eb-3ac9-482b-999c-a768d18d924b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.711638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:32.714661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2df67e-b307-4e27-ac37-567824ff4f3f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.735308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.736600 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.737118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.737706 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.738764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] RBAC: Adding query filter params (name=reader) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:32.739395 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:32.741381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:32.752464 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 101/196] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:32.752675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.753330 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.753441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.753689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.754440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:32.754755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:32.755326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.756187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:32.765776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 96/197] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:32.766701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.793237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:32.874950 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc48b0-27e8-41bc-b655-b5f6a1cbe764 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.877669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4ef49-d5dc-4444-b3e1-5ac5abc8d56c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:32.912782 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba029-6187-4b71-8295-79743039c507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.913352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d96701-9eed-415a-b96c-b2d11f460221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:32.913712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d96701-9eed-415a-b96c-b2d11f460221 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.913893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d96701-9eed-415a-b96c-b2d11f460221 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.914213 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d96701-9eed-415a-b96c-b2d11f460221 None admin] PATH_INFO: `/v3/users/661eecaf09e449c19c1e1ce545060682` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.915062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba029-6187-4b71-8295-79743039c507 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:32.915425 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d96701-9eed-415a-b96c-b2d11f460221 None admin] RBAC: Authorizing `identity:get_user(user_id=661eecaf09e449c19c1e1ce545060682)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:32.915807 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba029-6187-4b71-8295-79743039c507 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:32.916314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba029-6187-4b71-8295-79743039c507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:32.916919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d96701-9eed-415a-b96c-b2d11f460221 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:32.918845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 97/198] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/users/661eecaf09e449c19c1e1ce545060682 => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:32.919429 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d96701-9eed-415a-b96c-b2d11f460221 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:32.952732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:33.004347 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ba029-6187-4b71-8295-79743039c507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:33.006324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ba029-6187-4b71-8295-79743039c507 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:33.043701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d96701-9eed-415a-b96c-b2d11f460221 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:33.066849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:33.090914 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b49c33-00dd-48ec-ae35-9b2a6cd3105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:33.091736 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b49c33-00dd-48ec-ae35-9b2a6cd3105e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:33.093735 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b49c33-00dd-48ec-ae35-9b2a6cd3105e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:33.093942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b49c33-00dd-48ec-ae35-9b2a6cd3105e None admin] PATH_INFO: `/v3/users/ebafb2e05b9f4da08179c97e1f806ef7` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:33.095363 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b49c33-00dd-48ec-ae35-9b2a6cd3105e None admin] RBAC: Authorizing `identity:get_user(user_id=ebafb2e05b9f4da08179c97e1f806ef7)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:33.097851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b49c33-00dd-48ec-ae35-9b2a6cd3105e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:33.100842 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 98/199] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/users/ebafb2e05b9f4da08179c97e1f806ef7 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:33.102267 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b49c33-00dd-48ec-ae35-9b2a6cd3105e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:33.123580 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 102/200] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:33.124506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba029-6187-4b71-8295-79743039c507 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:33.220602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:33.224461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b49c33-00dd-48ec-ae35-9b2a6cd3105e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:33.261662 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b385e-4109-43fa-9f92-4a5862bb6052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:33.262142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b385e-4109-43fa-9f92-4a5862bb6052 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:33.262487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b385e-4109-43fa-9f92-4a5862bb6052 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:33.262769 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b385e-4109-43fa-9f92-4a5862bb6052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:33.265683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba029-6187-4b71-8295-79743039c507 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:33.306733 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:33.315197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a5178-a0d2-445c-aa05-e23b2b9e0bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:33.316784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5178-a0d2-445c-aa05-e23b2b9e0bdc None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:33.317305 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5178-a0d2-445c-aa05-e23b2b9e0bdc None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:33.317800 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5178-a0d2-445c-aa05-e23b2b9e0bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:33.342763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b385e-4109-43fa-9f92-4a5862bb6052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:33.344470 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b385e-4109-43fa-9f92-4a5862bb6052 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:33.408849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a5178-a0d2-445c-aa05-e23b2b9e0bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:33.411561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a5178-a0d2-445c-aa05-e23b2b9e0bdc None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:33.448751 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:33.454300 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 99/201] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:33.455314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b385e-4109-43fa-9f92-4a5862bb6052 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:33.543751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 103/202] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:33.544784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a5178-a0d2-445c-aa05-e23b2b9e0bdc None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:33.577246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:33.590359 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b385e-4109-43fa-9f92-4a5862bb6052 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:33.629147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bfe0-0500-4ce5-86f2-3470099f281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:33.629930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bfe0-0500-4ce5-86f2-3470099f281b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:33.630305 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bfe0-0500-4ce5-86f2-3470099f281b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:33.630684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bfe0-0500-4ce5-86f2-3470099f281b None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:33.632077 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6bfe0-0500-4ce5-86f2-3470099f281b None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:33.633768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6bfe0-0500-4ce5-86f2-3470099f281b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:33.635860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 100/203] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:33 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:33.636499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bfe0-0500-4ce5-86f2-3470099f281b None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:33.673443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a5178-a0d2-445c-aa05-e23b2b9e0bdc None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:33.711877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:33.721632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432aada-9ab7-4c3b-8abf-f008c04b7b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:33.722764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432aada-9ab7-4c3b-8abf-f008c04b7b39 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:33.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432aada-9ab7-4c3b-8abf-f008c04b7b39 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:33.723790 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432aada-9ab7-4c3b-8abf-f008c04b7b39 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:33.725834 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432aada-9ab7-4c3b-8abf-f008c04b7b39 None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:33.728055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432aada-9ab7-4c3b-8abf-f008c04b7b39 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:33.730753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 104/204] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:33 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:33.732139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432aada-9ab7-4c3b-8abf-f008c04b7b39 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:33.758511 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bfe0-0500-4ce5-86f2-3470099f281b None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:33.798136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b0d71-d66c-45ab-8c46-a92d9a8f1755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:33.799176 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b0d71-d66c-45ab-8c46-a92d9a8f1755 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:33.799464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b0d71-d66c-45ab-8c46-a92d9a8f1755 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:33.800006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b0d71-d66c-45ab-8c46-a92d9a8f1755 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:33.801478 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b0d71-d66c-45ab-8c46-a92d9a8f1755 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:33.803644 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b0d71-d66c-45ab-8c46-a92d9a8f1755 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:33.805921 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 101/205] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:33 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:33.807278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b0d71-d66c-45ab-8c46-a92d9a8f1755 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:33.843236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:33.847628 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432aada-9ab7-4c3b-8abf-f008c04b7b39 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:33.886516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:33.887401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:33.888079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:33.888369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:33.889495 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=96c6511840d04bcc8f165d1799252c4d, user.id=ebafb2e05b9f4da08179c97e1f806ef7) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:33.889817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:33.892084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:33.899731 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 105/206] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:33 2022] GET /identity/v3/role_assignments?user.id=ebafb2e05b9f4da08179c97e1f806ef7&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:33.902676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:33.971700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b0d71-d66c-45ab-8c46-a92d9a8f1755 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:34.087989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdd4d8-af8d-4ef3-8570-94c6ff5a8147 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:34.090249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c05f9-0b5a-4ffc-b703-9105de9e24f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:34.090919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c05f9-0b5a-4ffc-b703-9105de9e24f0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:34.091318 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c05f9-0b5a-4ffc-b703-9105de9e24f0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:34.091645 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c05f9-0b5a-4ffc-b703-9105de9e24f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:34.092602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c05f9-0b5a-4ffc-b703-9105de9e24f0 None admin] RBAC: Adding query filter params (role.id=61c2794c5a6d4f91b3a953f0ae6eab8b, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147, user.id=661eecaf09e449c19c1e1ce545060682) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:34.092917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c05f9-0b5a-4ffc-b703-9105de9e24f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:34.094651 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c05f9-0b5a-4ffc-b703-9105de9e24f0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:34.098625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:34.105352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 102/207] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:33 2022] GET /identity/v3/role_assignments?user.id=661eecaf09e449c19c1e1ce545060682&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=61c2794c5a6d4f91b3a953f0ae6eab8b => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:34.130498 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82336090-007d-482c-b605-f71c18d454dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:34.131170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82336090-007d-482c-b605-f71c18d454dd None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:34.131454 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82336090-007d-482c-b605-f71c18d454dd None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:34.131737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82336090-007d-482c-b605-f71c18d454dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:34.132579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82336090-007d-482c-b605-f71c18d454dd None admin] RBAC: Adding query filter params (role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=96c6511840d04bcc8f165d1799252c4d, user.id=ebafb2e05b9f4da08179c97e1f806ef7) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:34.132968 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82336090-007d-482c-b605-f71c18d454dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:34.134604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82336090-007d-482c-b605-f71c18d454dd None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:34.141598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 106/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:33 2022] GET /identity/v3/role_assignments?user.id=ebafb2e05b9f4da08179c97e1f806ef7&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:34.521807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab787f-add5-47ae-8434-7badde2558ee None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:34.523167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab787f-add5-47ae-8434-7badde2558ee None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:34.524001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab787f-add5-47ae-8434-7badde2558ee None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:34.527473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 103/209] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:34.533099 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f3b3-a496-44f7-aa32-b74ccba26f39 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:34.533374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f3b3-a496-44f7-aa32-b74ccba26f39 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:34.533653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f3b3-a496-44f7-aa32-b74ccba26f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:34.556372 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-be06f3b3-a496-44f7-aa32-b74ccba26f39 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:34.557614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-be06f3b3-a496-44f7-aa32-b74ccba26f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:34.587826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:34.656178 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 107/210] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:34.681447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab787f-add5-47ae-8434-7badde2558ee None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:34.732592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1784589-3829-45cd-b75f-3898bea5b233 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:34.733322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1784589-3829-45cd-b75f-3898bea5b233 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:34.733694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1784589-3829-45cd-b75f-3898bea5b233 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:34.735345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 108/211] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:34.741228 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af3a1-239b-4c19-ac21-a8040bf11838 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:34.741523 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af3a1-239b-4c19-ac21-a8040bf11838 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:34.741875 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af3a1-239b-4c19-ac21-a8040bf11838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:34.763313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-e98af3a1-239b-4c19-ac21-a8040bf11838 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:34.764545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-e98af3a1-239b-4c19-ac21-a8040bf11838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:34.850227 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab787f-add5-47ae-8434-7badde2558ee None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:34.851276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 109/212] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:34.877093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98af3a1-239b-4c19-ac21-a8040bf11838 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:34.889709 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53876aef-fab4-4d64-ba89-bdfac2563ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:34.890936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53876aef-fab4-4d64-ba89-bdfac2563ea1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:34.891228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53876aef-fab4-4d64-ba89-bdfac2563ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:34.891504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53876aef-fab4-4d64-ba89-bdfac2563ea1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:34.904232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53876aef-fab4-4d64-ba89-bdfac2563ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-53876aef-fab4-4d64-ba89-bdfac2563ea1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:34.913682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:34.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:34.916009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:34.923774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 104/213] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:34.925072 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53876aef-fab4-4d64-ba89-bdfac2563ea1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:34.988803 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:34.989415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:35.011864 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98af3a1-239b-4c19-ac21-a8040bf11838 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:35.042513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53876aef-fab4-4d64-ba89-bdfac2563ea1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:35.062132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0be1bd-62bf-44b9-b321-bdb70d957c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:35.062820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be1bd-62bf-44b9-b321-bdb70d957c5b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:35.063109 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be1bd-62bf-44b9-b321-bdb70d957c5b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:35.063391 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be1bd-62bf-44b9-b321-bdb70d957c5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:35.064614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0be1bd-62bf-44b9-b321-bdb70d957c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:35.066154 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0be1bd-62bf-44b9-b321-bdb70d957c5b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:35.074146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 110/214] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:35.075598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0be1bd-62bf-44b9-b321-bdb70d957c5b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:35.089245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:35.090551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:35.091071 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:35.091678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:35.092849 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:35.093397 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:35.095632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:35.105691 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 105/215] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:35.111799 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:35.190697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0be1bd-62bf-44b9-b321-bdb70d957c5b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:35.228143 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad888b81-2503-4b50-aee1-19d92bc6b4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:35.228787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad888b81-2503-4b50-aee1-19d92bc6b4be None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:35.229071 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad888b81-2503-4b50-aee1-19d92bc6b4be None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:35.229349 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad888b81-2503-4b50-aee1-19d92bc6b4be None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:35.230064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad888b81-2503-4b50-aee1-19d92bc6b4be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:35.231602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad888b81-2503-4b50-aee1-19d92bc6b4be None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:35.244097 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be608366-a473-43c7-ade0-da143da64f92 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:35.257925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 111/216] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:55:35 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:35.270677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:35.288159 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978491ec-a109-4b7d-b0b4-f924876bf51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:35.289363 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978491ec-a109-4b7d-b0b4-f924876bf51f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:35.289889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978491ec-a109-4b7d-b0b4-f924876bf51f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:35.290694 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978491ec-a109-4b7d-b0b4-f924876bf51f None admin] PATH_INFO: `/v3/users/b678411b7859457c9d999c00433509df` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:35.307588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978491ec-a109-4b7d-b0b4-f924876bf51f None admin] RBAC: Authorizing `identity:get_user(user_id=b678411b7859457c9d999c00433509df)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:35.309113 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978491ec-a109-4b7d-b0b4-f924876bf51f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:35.311244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 106/217] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:35 2022] GET /identity/v3/users/b678411b7859457c9d999c00433509df => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:35.317281 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad888b81-2503-4b50-aee1-19d92bc6b4be None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:35.337721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:35.452002 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad888b81-2503-4b50-aee1-19d92bc6b4be None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:35.488389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7c3c-8871-464a-874d-ee2dcca5c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:35.489046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7c3c-8871-464a-874d-ee2dcca5c775 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:35.489336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7c3c-8871-464a-874d-ee2dcca5c775 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:35.489620 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7c3c-8871-464a-874d-ee2dcca5c775 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:35.567776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed7c3c-8871-464a-874d-ee2dcca5c775 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:35.569342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed7c3c-8871-464a-874d-ee2dcca5c775 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:35.581980 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:35.681832 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 112/218] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:35.687339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978491ec-a109-4b7d-b0b4-f924876bf51f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:35.829174 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978491ec-a109-4b7d-b0b4-f924876bf51f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:35.875303 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9645b-8c29-4bb4-ae28-f46aa2547eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:35.876136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9645b-8c29-4bb4-ae28-f46aa2547eae None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:35.876381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9645b-8c29-4bb4-ae28-f46aa2547eae None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:35.876651 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9645b-8c29-4bb4-ae28-f46aa2547eae None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:35.882912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9645b-8c29-4bb4-ae28-f46aa2547eae None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:35.884523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9645b-8c29-4bb4-ae28-f46aa2547eae None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:35.886357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 107/219] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:35 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:35.892638 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7c3c-8871-464a-874d-ee2dcca5c775 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:36.040738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7c3c-8871-464a-874d-ee2dcca5c775 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:36.057898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:36.089823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c27bd8-c0c1-484b-89f5-2c6439191360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:36.090498 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c27bd8-c0c1-484b-89f5-2c6439191360 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.090784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c27bd8-c0c1-484b-89f5-2c6439191360 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.091069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c27bd8-c0c1-484b-89f5-2c6439191360 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.091891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c27bd8-c0c1-484b-89f5-2c6439191360 None admin] RBAC: Adding query filter params (role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=96c6511840d04bcc8f165d1799252c4d, user.id=b678411b7859457c9d999c00433509df) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:36.092214 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c27bd8-c0c1-484b-89f5-2c6439191360 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:36.093717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c27bd8-c0c1-484b-89f5-2c6439191360 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:36.102983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 113/220] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:35 2022] GET /identity/v3/role_assignments?user.id=b678411b7859457c9d999c00433509df&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:36.211489 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4583583-cf64-4d14-82df-88d3e085469d None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.211882 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4583583-cf64-4d14-82df-88d3e085469d None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.212073 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4583583-cf64-4d14-82df-88d3e085469d None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.213854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 108/221] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:36.219824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdfb7d-78e9-4d79-bd37-1c97dbc44c55 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.220574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdfb7d-78e9-4d79-bd37-1c97dbc44c55 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.220799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdfb7d-78e9-4d79-bd37-1c97dbc44c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.241021 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-d4bdfb7d-78e9-4d79-bd37-1c97dbc44c55 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:36.242282 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-d4bdfb7d-78e9-4d79-bd37-1c97dbc44c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:36.355496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 114/222] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:36.373457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ea2c-16f5-4a34-90a3-6058dbfe9b68 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.374162 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ea2c-16f5-4a34-90a3-6058dbfe9b68 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.374665 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ea2c-16f5-4a34-90a3-6058dbfe9b68 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.376438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 109/223] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:36.380642 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdfb7d-78e9-4d79-bd37-1c97dbc44c55 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:36.382387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b77ec-a7fa-4e26-ab7e-4a47643ef9b1 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.382921 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b77ec-a7fa-4e26-ab7e-4a47643ef9b1 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.383425 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b77ec-a7fa-4e26-ab7e-4a47643ef9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.403064 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-827b77ec-a7fa-4e26-ab7e-4a47643ef9b1 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:36.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-827b77ec-a7fa-4e26-ab7e-4a47643ef9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:36.463935 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:36.492903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 110/224] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:36.524273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b77ec-a7fa-4e26-ab7e-4a47643ef9b1 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:36.541846 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdfb7d-78e9-4d79-bd37-1c97dbc44c55 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:36.558388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:36.620819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52a4fb-6b51-4882-884f-a94eeb39d6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:36.622378 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52a4fb-6b51-4882-884f-a94eeb39d6e1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.622901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52a4fb-6b51-4882-884f-a94eeb39d6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.623450 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52a4fb-6b51-4882-884f-a94eeb39d6e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.635493 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52a4fb-6b51-4882-884f-a94eeb39d6e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-fa52a4fb-6b51-4882-884f-a94eeb39d6e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:36.641293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:36.642610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:36.643678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:36.646717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 115/225] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:36.666594 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52a4fb-6b51-4882-884f-a94eeb39d6e1 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:36.690247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b77ec-a7fa-4e26-ab7e-4a47643ef9b1 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:36.730514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ab34-aa9c-4364-b3b7-cd629adbf8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:36.731201 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ab34-aa9c-4364-b3b7-cd629adbf8b0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.731483 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ab34-aa9c-4364-b3b7-cd629adbf8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.731754 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ab34-aa9c-4364-b3b7-cd629adbf8b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.746611 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49ab34-aa9c-4364-b3b7-cd629adbf8b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-3f49ab34-aa9c-4364-b3b7-cd629adbf8b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:36.752856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:36.754061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:36.755161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 111/226] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:36.760749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ab34-aa9c-4364-b3b7-cd629adbf8b0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:36.814618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:36.877842 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ab34-aa9c-4364-b3b7-cd629adbf8b0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:36.921375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52a4fb-6b51-4882-884f-a94eeb39d6e1 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:36.951830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:36.952498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.952782 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.953058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.953806 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:36.954139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:36.955641 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:36.971414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 112/227] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:36.978033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:36.987866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:36.988510 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:36.988792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:36.989072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:36.989888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:36.990232 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:36.991707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:37.003168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 116/228] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:37.009435 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:37.044715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:37.135260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840c6ce-f174-4e6c-b4fa-28d9fa7791b6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:37.177990 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f743dc-decf-477f-8115-102a5692ff63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:37.178703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f743dc-decf-477f-8115-102a5692ff63 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:37.178989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f743dc-decf-477f-8115-102a5692ff63 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:37.179269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f743dc-decf-477f-8115-102a5692ff63 None admin] PATH_INFO: `/v3/users/ebafb2e05b9f4da08179c97e1f806ef7` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:37.180504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f743dc-decf-477f-8115-102a5692ff63 None admin] RBAC: Authorizing `identity:get_user(user_id=ebafb2e05b9f4da08179c97e1f806ef7)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:37.182786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f743dc-decf-477f-8115-102a5692ff63 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:37.184815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 117/229] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:37 2022] GET /identity/v3/users/ebafb2e05b9f4da08179c97e1f806ef7 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:37.190483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f743dc-decf-477f-8115-102a5692ff63 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:37.267980 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:37.276384 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8d1c1-ac42-4a5e-8af4-1e4049ab3601 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:37.295388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:37.303813 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f743dc-decf-477f-8115-102a5692ff63 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:37.342315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9925c1b-d8a9-4fb9-9cbe-7d3cfb5c91b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:37.343095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9925c1b-d8a9-4fb9-9cbe-7d3cfb5c91b6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:37.343387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9925c1b-d8a9-4fb9-9cbe-7d3cfb5c91b6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:37.343825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9925c1b-d8a9-4fb9-9cbe-7d3cfb5c91b6 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:37.345047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9925c1b-d8a9-4fb9-9cbe-7d3cfb5c91b6 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:37.346636 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9925c1b-d8a9-4fb9-9cbe-7d3cfb5c91b6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:37.348403 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 118/230] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:37 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:37.354336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9925c1b-d8a9-4fb9-9cbe-7d3cfb5c91b6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:37.371141 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223b39b-2092-4f31-a939-14f891267900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:37.371916 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223b39b-2092-4f31-a939-14f891267900 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:37.372209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223b39b-2092-4f31-a939-14f891267900 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:37.372491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223b39b-2092-4f31-a939-14f891267900 None admin] PATH_INFO: `/v3/users/ebafb2e05b9f4da08179c97e1f806ef7` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:37.373862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223b39b-2092-4f31-a939-14f891267900 None admin] RBAC: Authorizing `identity:get_user(user_id=ebafb2e05b9f4da08179c97e1f806ef7)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:37.375794 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223b39b-2092-4f31-a939-14f891267900 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:37.377974 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 113/231] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:36 2022] GET /identity/v3/users/ebafb2e05b9f4da08179c97e1f806ef7 => generated 283 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:37.394255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223b39b-2092-4f31-a939-14f891267900 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:37.469342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9925c1b-d8a9-4fb9-9cbe-7d3cfb5c91b6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:37.506961 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491632c9-f446-4ae5-942b-fb01f084fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:37.507666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491632c9-f446-4ae5-942b-fb01f084fdd6 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:37.507954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491632c9-f446-4ae5-942b-fb01f084fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:37.508236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491632c9-f446-4ae5-942b-fb01f084fdd6 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d/users/ebafb2e05b9f4da08179c97e1f806ef7/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:37.510332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491632c9-f446-4ae5-942b-fb01f084fdd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=96c6511840d04bcc8f165d1799252c4d, user_id=ebafb2e05b9f4da08179c97e1f806ef7, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:37.511841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491632c9-f446-4ae5-942b-fb01f084fdd6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:37.527105 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 119/232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:37 2022] PUT /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d/users/ebafb2e05b9f4da08179c97e1f806ef7/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:55:37.630375 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223b39b-2092-4f31-a939-14f891267900 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:37.685693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41e3a-c70f-442e-989b-c5beba9b15c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:37.686333 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41e3a-c70f-442e-989b-c5beba9b15c3 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:37.686626 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41e3a-c70f-442e-989b-c5beba9b15c3 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:37.686912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41e3a-c70f-442e-989b-c5beba9b15c3 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:37.688216 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41e3a-c70f-442e-989b-c5beba9b15c3 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:37.689812 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41e3a-c70f-442e-989b-c5beba9b15c3 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:37.693548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 114/233] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:37 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:37.698440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491632c9-f446-4ae5-942b-fb01f084fdd6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:37.728721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:37.817089 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491632c9-f446-4ae5-942b-fb01f084fdd6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:37.827572 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d5be-7f3a-4e44-8d5f-3e134a756286 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:37.827916 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d5be-7f3a-4e44-8d5f-3e134a756286 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:37.828130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d5be-7f3a-4e44-8d5f-3e134a756286 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:37.830030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 115/234] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:37.835852 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec4b9-47dc-4b3f-9a02-be8622bd7326 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:37.836160 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec4b9-47dc-4b3f-9a02-be8622bd7326 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:37.836449 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec4b9-47dc-4b3f-9a02-be8622bd7326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:37.858173 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d0965-0656-407a-a9c1-ae40781b97b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:37.858687 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-688ec4b9-47dc-4b3f-9a02-be8622bd7326 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:37.858904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d0965-0656-407a-a9c1-ae40781b97b2 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:37.859158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d0965-0656-407a-a9c1-ae40781b97b2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:37.859437 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d0965-0656-407a-a9c1-ae40781b97b2 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d/users/ebafb2e05b9f4da08179c97e1f806ef7/roles/167df0c7f94644578aff0abc3a0c8e84` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:37.860187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-688ec4b9-47dc-4b3f-9a02-be8622bd7326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:37.861772 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d0965-0656-407a-a9c1-ae40781b97b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=96c6511840d04bcc8f165d1799252c4d, user_id=ebafb2e05b9f4da08179c97e1f806ef7, role_id=167df0c7f94644578aff0abc3a0c8e84)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:37.863468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d0965-0656-407a-a9c1-ae40781b97b2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:37.890158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 120/235] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:37 2022] PUT /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d/users/ebafb2e05b9f4da08179c97e1f806ef7/roles/167df0c7f94644578aff0abc3a0c8e84 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:37.915248 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:37.984175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 116/236] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:38.017423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d0965-0656-407a-a9c1-ae40781b97b2 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:38.139059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:38.161790 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d0965-0656-407a-a9c1-ae40781b97b2 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:38.213890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccc1fe-1877-4f8b-9322-7a6304fb45c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:38.214629 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccc1fe-1877-4f8b-9322-7a6304fb45c1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:38.214915 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccc1fe-1877-4f8b-9322-7a6304fb45c1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:38.215189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccc1fe-1877-4f8b-9322-7a6304fb45c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:38.233878 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccc1fe-1877-4f8b-9322-7a6304fb45c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1fccc1fe-1877-4f8b-9322-7a6304fb45c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:38.239684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:38.241840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:38.245059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 121/237] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:38.251365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ec4b9-47dc-4b3f-9a02-be8622bd7326 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:38.417681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ec4b9-47dc-4b3f-9a02-be8622bd7326 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:38.468154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2194a82-cf55-4693-a347-6f9d77081cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:38.468821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194a82-cf55-4693-a347-6f9d77081cc8 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:38.469096 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194a82-cf55-4693-a347-6f9d77081cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:38.469369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194a82-cf55-4693-a347-6f9d77081cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:38.470213 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2194a82-cf55-4693-a347-6f9d77081cc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:38.470524 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2194a82-cf55-4693-a347-6f9d77081cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:38.471960 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2194a82-cf55-4693-a347-6f9d77081cc8 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:38.485865 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 117/238] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:38.491799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccc1fe-1877-4f8b-9322-7a6304fb45c1 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:38.586226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae689-1642-4908-89de-16bafa4f2e81 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:38.586686 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae689-1642-4908-89de-16bafa4f2e81 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:38.586967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae689-1642-4908-89de-16bafa4f2e81 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:38.589306 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 118/239] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:55:38.594270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f966-ab80-44c3-b5dd-ee0d9acf2f8a None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:38.594897 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f966-ab80-44c3-b5dd-ee0d9acf2f8a None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:38.595513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f966-ab80-44c3-b5dd-ee0d9acf2f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:38.615273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-4c91f966-ab80-44c3-b5dd-ee0d9acf2f8a None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:38.616866 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-4c91f966-ab80-44c3-b5dd-ee0d9acf2f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:38.635161 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccc1fe-1877-4f8b-9322-7a6304fb45c1 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:38.641413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:38.672065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad08f-7995-471b-b416-9acb3a553bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:38.672770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad08f-7995-471b-b416-9acb3a553bb8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:38.673187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad08f-7995-471b-b416-9acb3a553bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:38.673465 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad08f-7995-471b-b416-9acb3a553bb8 None admin] PATH_INFO: `/v3/users/36ca2ee835f3445093c205cb542bce37` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:38.695500 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:38.698832 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad08f-7995-471b-b416-9acb3a553bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=36ca2ee835f3445093c205cb542bce37)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:38.701304 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad08f-7995-471b-b416-9acb3a553bb8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:38.706281 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 122/240] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:38 2022] GET /identity/v3/users/36ca2ee835f3445093c205cb542bce37 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:38.710693 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad08f-7995-471b-b416-9acb3a553bb8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:38.715402 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 119/241] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:38.753070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91f966-ab80-44c3-b5dd-ee0d9acf2f8a None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:38.855156 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad08f-7995-471b-b416-9acb3a553bb8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:38.892938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91f966-ab80-44c3-b5dd-ee0d9acf2f8a None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:38.895713 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb2bfe-4173-4b62-affa-76807ca8cb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:38.896369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb2bfe-4173-4b62-affa-76807ca8cb46 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:38.896653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb2bfe-4173-4b62-affa-76807ca8cb46 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:38.896929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb2bfe-4173-4b62-affa-76807ca8cb46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:38.940971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06931dd9-a079-4bf1-b8f8-5ba7f11b0bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:38.942059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931dd9-a079-4bf1-b8f8-5ba7f11b0bf7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:38.942331 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931dd9-a079-4bf1-b8f8-5ba7f11b0bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:38.942601 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931dd9-a079-4bf1-b8f8-5ba7f11b0bf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:38.943828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06931dd9-a079-4bf1-b8f8-5ba7f11b0bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:38.945321 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06931dd9-a079-4bf1-b8f8-5ba7f11b0bf7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:38.950195 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 120/242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:38.951915 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54111f82-524b-4d78-abb6-f8039244e1a9 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:38.954556 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54111f82-524b-4d78-abb6-f8039244e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:38.954842 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54111f82-524b-4d78-abb6-f8039244e1a9 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:38.956354 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 121/243] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:38.956938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54111f82-524b-4d78-abb6-f8039244e1a9 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:38.991808 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:38.996300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:39.001905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb2bfe-4173-4b62-affa-76807ca8cb46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:39.004445 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb2bfe-4173-4b62-affa-76807ca8cb46 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:39.096319 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54111f82-524b-4d78-abb6-f8039244e1a9 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:39.137245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1dde54-74c9-4eb8-9370-c79aff9eaf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:39.139020 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dde54-74c9-4eb8-9370-c79aff9eaf0e None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:39.139467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dde54-74c9-4eb8-9370-c79aff9eaf0e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:39.139915 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dde54-74c9-4eb8-9370-c79aff9eaf0e None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:39.140781 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1dde54-74c9-4eb8-9370-c79aff9eaf0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:39.143087 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1dde54-74c9-4eb8-9370-c79aff9eaf0e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:39.158735 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 123/244] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:39.160475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441f95-055d-4886-8a1d-9dde972fe951 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:39.161014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441f95-055d-4886-8a1d-9dde972fe951 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:39.161423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441f95-055d-4886-8a1d-9dde972fe951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:39.172598 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 122/245] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:55:38 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:39.178266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1dde54-74c9-4eb8-9370-c79aff9eaf0e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:39.200127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-da441f95-055d-4886-8a1d-9dde972fe951 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:39.201384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-da441f95-055d-4886-8a1d-9dde972fe951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:39.354629 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:39.359735 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 124/246] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:39.380421 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1dde54-74c9-4eb8-9370-c79aff9eaf0e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:39.394489 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da441f95-055d-4886-8a1d-9dde972fe951 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:39.432925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:39.439480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5326d-ac17-4474-8f54-91926cae1c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:39.440169 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5326d-ac17-4474-8f54-91926cae1c31 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:39.440472 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5326d-ac17-4474-8f54-91926cae1c31 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:39.440824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5326d-ac17-4474-8f54-91926cae1c31 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:39.442927 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5326d-ac17-4474-8f54-91926cae1c31 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:39.444426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5326d-ac17-4474-8f54-91926cae1c31 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:39.446179 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 123/247] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:39 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:39.453025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5326d-ac17-4474-8f54-91926cae1c31 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:39.521713 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da441f95-055d-4886-8a1d-9dde972fe951 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:39.569284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e72d5-92c5-4248-a870-31ebb1d228ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:39.570238 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e72d5-92c5-4248-a870-31ebb1d228ee None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:39.570527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e72d5-92c5-4248-a870-31ebb1d228ee None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:39.570803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e72d5-92c5-4248-a870-31ebb1d228ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:39.580334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5326d-ac17-4474-8f54-91926cae1c31 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:39.586246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e72d5-92c5-4248-a870-31ebb1d228ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-ec9e72d5-92c5-4248-a870-31ebb1d228ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:39.591898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:39.593126 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:39.594184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 125/248] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:39.602379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e72d5-92c5-4248-a870-31ebb1d228ee None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:39.641501 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:39.643019 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:39.643966 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:39.644486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:39.645563 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] RBAC: Adding query filter params (role.id=61c2794c5a6d4f91b3a953f0ae6eab8b, scope.project.id=96c6511840d04bcc8f165d1799252c4d, user.id=36ca2ee835f3445093c205cb542bce37) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:39.646398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:39.648803 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:39.658614 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 124/249] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:39 2022] GET /identity/v3/role_assignments?user.id=36ca2ee835f3445093c205cb542bce37&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=61c2794c5a6d4f91b3a953f0ae6eab8b => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:39.734718 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:39.749123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e72d5-92c5-4248-a870-31ebb1d228ee None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:39.791819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:39.792854 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:39.793368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:39.793865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:39.794851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] RBAC: Adding query filter params (name=member) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:39.795381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:39.797026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:39.808013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 126/250] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:39.813633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:39.833206 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:39.938202 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23375b-8f07-469d-837f-3adc4308796f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:39.981837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b3ec4-63bd-465e-be21-8dd21df8ccbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:39.982552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b3ec4-63bd-465e-be21-8dd21df8ccbc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:39.982883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b3ec4-63bd-465e-be21-8dd21df8ccbc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:39.983167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b3ec4-63bd-465e-be21-8dd21df8ccbc None admin] PATH_INFO: `/v3/users/b678411b7859457c9d999c00433509df` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:39.984450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b3ec4-63bd-465e-be21-8dd21df8ccbc None admin] RBAC: Authorizing `identity:get_user(user_id=b678411b7859457c9d999c00433509df)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:39.986029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b3ec4-63bd-465e-be21-8dd21df8ccbc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:39.987752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 125/251] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:39 2022] GET /identity/v3/users/b678411b7859457c9d999c00433509df => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:39.993658 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:40.048777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff387bc-fe52-44ee-831c-5478367805b4 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.049450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff387bc-fe52-44ee-831c-5478367805b4 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.049952 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff387bc-fe52-44ee-831c-5478367805b4 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.051909 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 126/252] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:40.057870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbff92e-4e83-41a1-876d-95b8eb80713c None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.058549 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbff92e-4e83-41a1-876d-95b8eb80713c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.059079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbff92e-4e83-41a1-876d-95b8eb80713c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.103208 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-adbff92e-4e83-41a1-876d-95b8eb80713c None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:40.104812 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-adbff92e-4e83-41a1-876d-95b8eb80713c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:40.125889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf6473-8a79-4e26-8ab7-592d315c949a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:40.168239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81423ae-36d5-4834-9f11-b9e73f10159b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:40.169060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81423ae-36d5-4834-9f11-b9e73f10159b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.169352 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81423ae-36d5-4834-9f11-b9e73f10159b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.169636 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81423ae-36d5-4834-9f11-b9e73f10159b None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.170880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81423ae-36d5-4834-9f11-b9e73f10159b None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:40.172413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81423ae-36d5-4834-9f11-b9e73f10159b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:40.174559 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 127/253] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:39 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:40.180234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81423ae-36d5-4834-9f11-b9e73f10159b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:40.211122 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:40.259752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 127/254] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:40.298938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbff92e-4e83-41a1-876d-95b8eb80713c None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:40.302343 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81423ae-36d5-4834-9f11-b9e73f10159b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:40.319840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:40.341526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1cc00-0d07-4161-8454-3ea1307cb815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:40.342543 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1cc00-0d07-4161-8454-3ea1307cb815 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.343020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1cc00-0d07-4161-8454-3ea1307cb815 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.343480 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1cc00-0d07-4161-8454-3ea1307cb815 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d/users/b678411b7859457c9d999c00433509df/roles/844a46aa3ad0499fb8bfa2b1bd023ddd` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.345751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1cc00-0d07-4161-8454-3ea1307cb815 None admin] RBAC: Authorizing `identity:create_grant(project_id=96c6511840d04bcc8f165d1799252c4d, user_id=b678411b7859457c9d999c00433509df, role_id=844a46aa3ad0499fb8bfa2b1bd023ddd)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:40.347494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1cc00-0d07-4161-8454-3ea1307cb815 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:40.368475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 128/255] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:40 2022] PUT /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d/users/b678411b7859457c9d999c00433509df/roles/844a46aa3ad0499fb8bfa2b1bd023ddd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:40.370386 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de298b9-fa5b-41fd-9046-691f0111ef0e None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.370918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de298b9-fa5b-41fd-9046-691f0111ef0e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.371439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de298b9-fa5b-41fd-9046-691f0111ef0e None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.373105 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 129/256] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:40.382623 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ea77f-cbfb-4f1f-9aef-ff302c824899 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.384424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ea77f-cbfb-4f1f-9aef-ff302c824899 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.384712 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ea77f-cbfb-4f1f-9aef-ff302c824899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.417982 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-868ea77f-cbfb-4f1f-9aef-ff302c824899 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:40.419381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-868ea77f-cbfb-4f1f-9aef-ff302c824899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:40.538601 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbff92e-4e83-41a1-876d-95b8eb80713c None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:40.575776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 130/257] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:40.607008 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972e28f-4aec-48a8-9ec0-127367ca2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:40.607717 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e28f-4aec-48a8-9ec0-127367ca2243 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.607997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e28f-4aec-48a8-9ec0-127367ca2243 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.608269 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e28f-4aec-48a8-9ec0-127367ca2243 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.627072 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972e28f-4aec-48a8-9ec0-127367ca2243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:40.627594 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ea77f-cbfb-4f1f-9aef-ff302c824899 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-8972e28f-4aec-48a8-9ec0-127367ca2243 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:40.632268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:40.633490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:40.634579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 128/258] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:40.639712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972e28f-4aec-48a8-9ec0-127367ca2243 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:40.650728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:40.770962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:40.822648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ea77f-cbfb-4f1f-9aef-ff302c824899 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:40.838519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972e28f-4aec-48a8-9ec0-127367ca2243 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:40.916257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:40.916929 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.917212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.917489 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.918280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:40.918756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:40.920396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:40.937523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 129/259] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:40.942988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5402c6-67ac-487c-9aef-9cecd53d78a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:40.943690 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5402c6-67ac-487c-9aef-9cecd53d78a0 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:40.943959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:40.944108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5402c6-67ac-487c-9aef-9cecd53d78a0 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:40.944222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5402c6-67ac-487c-9aef-9cecd53d78a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:40.975117 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5402c6-67ac-487c-9aef-9cecd53d78a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1d5402c6-67ac-487c-9aef-9cecd53d78a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:40.980152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:40.981342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:40.982405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:40.986231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 131/260] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:40.988307 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5402c6-67ac-487c-9aef-9cecd53d78a0 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:41.135815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:41.176015 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5402c6-67ac-487c-9aef-9cecd53d78a0 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:41.235916 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:41.236410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e94431-06d0-4862-a3b0-aeca0526227e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:41.236868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:41.236868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:41.237123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:41.237820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:41.238148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:41.239618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:41.254300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 132/261] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:41.260123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:41.298459 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:41.305059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05589e22-da95-41c6-bc92-f551cb6f6293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:41.306368 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589e22-da95-41c6-bc92-f551cb6f6293 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:41.306891 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589e22-da95-41c6-bc92-f551cb6f6293 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:41.307410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589e22-da95-41c6-bc92-f551cb6f6293 None admin] PATH_INFO: `/v3/users/ebafb2e05b9f4da08179c97e1f806ef7` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:41.309079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05589e22-da95-41c6-bc92-f551cb6f6293 None admin] RBAC: Authorizing `identity:get_user(user_id=ebafb2e05b9f4da08179c97e1f806ef7)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:41.316569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05589e22-da95-41c6-bc92-f551cb6f6293 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:41.319760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 130/262] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:40 2022] GET /identity/v3/users/ebafb2e05b9f4da08179c97e1f806ef7 => generated 283 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:41.325169 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05589e22-da95-41c6-bc92-f551cb6f6293 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:41.425589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efe1d8-9987-45ef-b074-3e10cc34a94a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:41.457173 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:41.464679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98f518-f5aa-4125-a0b4-5c53275832db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:41.464679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f518-f5aa-4125-a0b4-5c53275832db None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:41.464679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f518-f5aa-4125-a0b4-5c53275832db None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:41.464679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f518-f5aa-4125-a0b4-5c53275832db None admin] PATH_INFO: `/v3/users/ebafb2e05b9f4da08179c97e1f806ef7` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:41.465879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98f518-f5aa-4125-a0b4-5c53275832db None admin] RBAC: Authorizing `identity:get_user(user_id=ebafb2e05b9f4da08179c97e1f806ef7)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:41.467960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98f518-f5aa-4125-a0b4-5c53275832db None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:41.470097 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 133/263] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:41 2022] GET /identity/v3/users/ebafb2e05b9f4da08179c97e1f806ef7 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:41.479551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98f518-f5aa-4125-a0b4-5c53275832db None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:41.548105 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05589e22-da95-41c6-bc92-f551cb6f6293 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:41.609610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98f518-f5aa-4125-a0b4-5c53275832db None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:41.620905 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9c9b3-3bc0-4f32-963d-185514554d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:41.622083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9c9b3-3bc0-4f32-963d-185514554d06 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:41.622613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9c9b3-3bc0-4f32-963d-185514554d06 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:41.623152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9c9b3-3bc0-4f32-963d-185514554d06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:41.646431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cd86c-db9f-4778-95dd-96cc04d27fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:41.647146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd86c-db9f-4778-95dd-96cc04d27fa6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:41.647429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd86c-db9f-4778-95dd-96cc04d27fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:41.647707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd86c-db9f-4778-95dd-96cc04d27fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:41.719132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:41.725199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cd86c-db9f-4778-95dd-96cc04d27fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:41.725199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cd86c-db9f-4778-95dd-96cc04d27fa6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:41.818472 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:41.824523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9c9b3-3bc0-4f32-963d-185514554d06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:41.826061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9c9b3-3bc0-4f32-963d-185514554d06 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:41.832437 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 134/264] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:55:41.840542 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cd86c-db9f-4778-95dd-96cc04d27fa6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:41.950180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:41.955092 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cd86c-db9f-4778-95dd-96cc04d27fa6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:41.994809 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e90e00-1e79-4639-9b57-7e827515f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:41.995391 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e90e00-1e79-4639-9b57-7e827515f87e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:41.995671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e90e00-1e79-4639-9b57-7e827515f87e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:41.995949 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e90e00-1e79-4639-9b57-7e827515f87e None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:41.997331 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e90e00-1e79-4639-9b57-7e827515f87e None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:41.998876 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e90e00-1e79-4639-9b57-7e827515f87e None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:42.000606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 135/265] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:41 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:42.007591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e90e00-1e79-4639-9b57-7e827515f87e None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:42.059029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 131/266] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 734 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:42.065092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9c9b3-3bc0-4f32-963d-185514554d06 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:42.168649 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e90e00-1e79-4639-9b57-7e827515f87e None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:42.203185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:42.210131 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9c9b3-3bc0-4f32-963d-185514554d06 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:42.213858 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:42.214929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:42.215394 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:42.215828 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:42.216802 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] RBAC: Adding query filter params (role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=96c6511840d04bcc8f165d1799252c4d, user.id=ebafb2e05b9f4da08179c97e1f806ef7) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:42.217308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:42.219078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:42.238217 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 136/267] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:42 2022] GET /identity/v3/role_assignments?user.id=ebafb2e05b9f4da08179c97e1f806ef7&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:42.252369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275a894-d051-4514-937e-c4f7ccfab511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:42.252988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275a894-d051-4514-937e-c4f7ccfab511 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:42.253267 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275a894-d051-4514-937e-c4f7ccfab511 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:42.253539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275a894-d051-4514-937e-c4f7ccfab511 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:42.254764 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275a894-d051-4514-937e-c4f7ccfab511 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:42.256265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275a894-d051-4514-937e-c4f7ccfab511 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:42.262356 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 132/268] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:42 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:42.264449 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:42.332378 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:42.474613 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e516c-d0c6-4adc-beb9-08d0aa565764 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:42.521804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d141be6-03ae-4950-8d77-8bae6cae3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:42.522871 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d141be6-03ae-4950-8d77-8bae6cae3ebf None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:42.523463 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d141be6-03ae-4950-8d77-8bae6cae3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:42.524065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d141be6-03ae-4950-8d77-8bae6cae3ebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:42.525168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d141be6-03ae-4950-8d77-8bae6cae3ebf None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=96c6511840d04bcc8f165d1799252c4d, user.id=ebafb2e05b9f4da08179c97e1f806ef7) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:42.525694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d141be6-03ae-4950-8d77-8bae6cae3ebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:42.527461 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d141be6-03ae-4950-8d77-8bae6cae3ebf None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:42.538298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 137/269] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:42 2022] GET /identity/v3/role_assignments?user.id=ebafb2e05b9f4da08179c97e1f806ef7&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:42.580696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fc49c-3e90-4bad-9f3c-e8a002067859 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:42.580956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fc49c-3e90-4bad-9f3c-e8a002067859 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:42.581224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fc49c-3e90-4bad-9f3c-e8a002067859 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:42.582928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 133/270] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:42.588248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915604b2-5da2-4fd0-8ccc-d6224ee41096 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:42.588669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915604b2-5da2-4fd0-8ccc-d6224ee41096 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:42.588959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915604b2-5da2-4fd0-8ccc-d6224ee41096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:42.618577 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-915604b2-5da2-4fd0-8ccc-d6224ee41096 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:42.619880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-915604b2-5da2-4fd0-8ccc-d6224ee41096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:42.724125 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 138/271] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:42.758884 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fc49c-3e90-4bad-9f3c-e8a002067859 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:42.889620 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fc49c-3e90-4bad-9f3c-e8a002067859 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:42.925765 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8625d-d887-4d87-98f1-31e1d88d0ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:42.926470 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8625d-d887-4d87-98f1-31e1d88d0ae9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:42.926747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8625d-d887-4d87-98f1-31e1d88d0ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:42.927018 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8625d-d887-4d87-98f1-31e1d88d0ae9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:42.940371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8625d-d887-4d87-98f1-31e1d88d0ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-53a8625d-d887-4d87-98f1-31e1d88d0ae9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:42.945964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:42.947124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:42.948758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 134/272] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:42.953202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915604b2-5da2-4fd0-8ccc-d6224ee41096 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:42.958218 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53401a-d8ee-4aa7-8b74-76f714bd3c9d None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:42.958767 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53401a-d8ee-4aa7-8b74-76f714bd3c9d None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:42.959333 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53401a-d8ee-4aa7-8b74-76f714bd3c9d None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:42.961243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 135/273] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:55:42.966297 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b28a-a5d0-4320-a1eb-d201a0a3f825 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:42.967033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b28a-a5d0-4320-a1eb-d201a0a3f825 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:42.967320 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b28a-a5d0-4320-a1eb-d201a0a3f825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:42.989979 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-cfc4b28a-a5d0-4320-a1eb-d201a0a3f825 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:42.991278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-cfc4b28a-a5d0-4320-a1eb-d201a0a3f825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:42.996125 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:43.017046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:43.086266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 136/274] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:43.087747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda10fb-5cf2-4456-a216-06405ca51719 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.088037 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda10fb-5cf2-4456-a216-06405ca51719 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.088312 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda10fb-5cf2-4456-a216-06405ca51719 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.093976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 137/275] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:43.095294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915604b2-5da2-4fd0-8ccc-d6224ee41096 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:43.095880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317448a-8a11-4226-b406-95274eefa2b1 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.096111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317448a-8a11-4226-b406-95274eefa2b1 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.096497 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317448a-8a11-4226-b406-95274eefa2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.117524 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-9317448a-8a11-4226-b406-95274eefa2b1 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:43.118878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9317448a-8a11-4226-b406-95274eefa2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:43.136791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:43.137430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.137717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.137992 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.138754 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] RBAC: Adding query filter params (name=member) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:43.139059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:43.140577 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:43.149352 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 139/276] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:43.150214 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:43.198589 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 138/277] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:43.199265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317448a-8a11-4226-b406-95274eefa2b1 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:43.258027 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:43.272903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cf04a-170b-4e2b-9210-71f2dd7d2410 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:43.295740 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:43.309388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317448a-8a11-4226-b406-95274eefa2b1 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:43.311242 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba2ef4-5257-4ad1-814b-c0a7aae36059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:43.312719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba2ef4-5257-4ad1-814b-c0a7aae36059 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.313009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba2ef4-5257-4ad1-814b-c0a7aae36059 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.313286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba2ef4-5257-4ad1-814b-c0a7aae36059 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.326077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba2ef4-5257-4ad1-814b-c0a7aae36059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-37ba2ef4-5257-4ad1-814b-c0a7aae36059 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:43.332879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:43.334084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 140/278] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:43.335609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba2ef4-5257-4ad1-814b-c0a7aae36059 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:43.351763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839399f3-eb8f-4fe3-bcfe-42a85f8406bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:43.352440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839399f3-eb8f-4fe3-bcfe-42a85f8406bc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.352722 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839399f3-eb8f-4fe3-bcfe-42a85f8406bc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.352997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839399f3-eb8f-4fe3-bcfe-42a85f8406bc None admin] PATH_INFO: `/v3/users/8004e4a5f40141c393461ad00fad5026` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.368368 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839399f3-eb8f-4fe3-bcfe-42a85f8406bc None admin] RBAC: Authorizing `identity:get_user(user_id=8004e4a5f40141c393461ad00fad5026)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:43.370162 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839399f3-eb8f-4fe3-bcfe-42a85f8406bc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:43.377480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 139/279] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:43 2022] GET /identity/v3/users/8004e4a5f40141c393461ad00fad5026 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:43.377775 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839399f3-eb8f-4fe3-bcfe-42a85f8406bc None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:43.442052 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba2ef4-5257-4ad1-814b-c0a7aae36059 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:43.478546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361c06-87f8-468e-8014-55989a138965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:43.479218 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361c06-87f8-468e-8014-55989a138965 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.479511 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361c06-87f8-468e-8014-55989a138965 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.479793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361c06-87f8-468e-8014-55989a138965 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.480977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb361c06-87f8-468e-8014-55989a138965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:43.482525 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb361c06-87f8-468e-8014-55989a138965 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:43.488911 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 141/280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:43.488911 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361c06-87f8-468e-8014-55989a138965 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:43.496600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839399f3-eb8f-4fe3-bcfe-42a85f8406bc None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:43.510703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:43.539401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:43.540163 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.540580 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.541018 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.541770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] RBAC: Adding query filter params (name=reader) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:43.542085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:43.543577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:43.552748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 140/281] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:43.553574 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:43.566828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:43.605905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361c06-87f8-468e-8014-55989a138965 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:43.640292 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:43.640974 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.641209 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.641491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.642400 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] RBAC: Adding query filter params (role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, user.id=8004e4a5f40141c393461ad00fad5026, scope.system=all) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:43.642715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:43.644245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:43.649641 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 142/282] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:55:43 2022] GET /identity/v3/role_assignments?user.id=8004e4a5f40141c393461ad00fad5026&scope.system=all&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:43.650515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:43.659467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9177a93-71be-4df8-9f4c-c90a8db28fef None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:43.703796 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15e319-f1e4-448a-9fcd-4d36aca266ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:43.704423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e319-f1e4-448a-9fcd-4d36aca266ee None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.705496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e319-f1e4-448a-9fcd-4d36aca266ee None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.705954 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e319-f1e4-448a-9fcd-4d36aca266ee None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.706588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15e319-f1e4-448a-9fcd-4d36aca266ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:43.708869 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15e319-f1e4-448a-9fcd-4d36aca266ee None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:43.738139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 141/283] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:55:43 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:43.738747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764cb0b-644f-4507-9fed-d01e401ab579 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.738948 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764cb0b-644f-4507-9fed-d01e401ab579 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.739369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764cb0b-644f-4507-9fed-d01e401ab579 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.741011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 142/284] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:43.750275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a039585-9f94-4fe8-bc8a-a63ee5c5cd77 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.750275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a039585-9f94-4fe8-bc8a-a63ee5c5cd77 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.750275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a039585-9f94-4fe8-bc8a-a63ee5c5cd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.774196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-1a039585-9f94-4fe8-bc8a-a63ee5c5cd77 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:43.776137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-1a039585-9f94-4fe8-bc8a-a63ee5c5cd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:43.778896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:43.808852 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf3f93-c4c0-4486-bdc6-c24a4b3647c7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:43.854044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30f3d5-b69e-4213-8422-709318ad6b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:43.854895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f3d5-b69e-4213-8422-709318ad6b1c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:43.855252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f3d5-b69e-4213-8422-709318ad6b1c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:43.855705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f3d5-b69e-4213-8422-709318ad6b1c None admin] PATH_INFO: `/v3/users/36ca2ee835f3445093c205cb542bce37` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:43.856976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30f3d5-b69e-4213-8422-709318ad6b1c None admin] RBAC: Authorizing `identity:get_user(user_id=36ca2ee835f3445093c205cb542bce37)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:43.858620 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30f3d5-b69e-4213-8422-709318ad6b1c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:43.860553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 143/285] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:43 2022] GET /identity/v3/users/36ca2ee835f3445093c205cb542bce37 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:43.866681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30f3d5-b69e-4213-8422-709318ad6b1c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:43.868253 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:43.886236 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 143/286] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:43.913834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a039585-9f94-4fe8-bc8a-a63ee5c5cd77 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:44.017031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30f3d5-b69e-4213-8422-709318ad6b1c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:44.054865 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a039585-9f94-4fe8-bc8a-a63ee5c5cd77 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:44.070749 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc86b5-e2e5-4bd4-b997-ddcc085a0a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:44.071895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc86b5-e2e5-4bd4-b997-ddcc085a0a9c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:44.072790 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc86b5-e2e5-4bd4-b997-ddcc085a0a9c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:44.073397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc86b5-e2e5-4bd4-b997-ddcc085a0a9c None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:44.075252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc86b5-e2e5-4bd4-b997-ddcc085a0a9c None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:44.077316 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc86b5-e2e5-4bd4-b997-ddcc085a0a9c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:44.080035 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 144/287] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:43 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:44.086606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc86b5-e2e5-4bd4-b997-ddcc085a0a9c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:44.108510 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92df73d-b90c-497f-a876-af154abfebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:44.109288 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92df73d-b90c-497f-a876-af154abfebfb None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:44.109673 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92df73d-b90c-497f-a876-af154abfebfb None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:44.110056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92df73d-b90c-497f-a876-af154abfebfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:44.115154 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:44.130773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92df73d-b90c-497f-a876-af154abfebfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-e92df73d-b90c-497f-a876-af154abfebfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:44.136167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:44.137689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:44.139189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 144/288] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:55:44.144732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92df73d-b90c-497f-a876-af154abfebfb None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:44.219107 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:44.246922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc86b5-e2e5-4bd4-b997-ddcc085a0a9c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:44.277036 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92df73d-b90c-497f-a876-af154abfebfb None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:44.294968 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca55be6-3184-4de0-8705-7eea42702a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:44.295954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55be6-3184-4de0-8705-7eea42702a39 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:44.296428 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55be6-3184-4de0-8705-7eea42702a39 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:44.296880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55be6-3184-4de0-8705-7eea42702a39 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d/users/36ca2ee835f3445093c205cb542bce37/roles/61c2794c5a6d4f91b3a953f0ae6eab8b` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:44.300525 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca55be6-3184-4de0-8705-7eea42702a39 None admin] RBAC: Authorizing `identity:create_grant(project_id=96c6511840d04bcc8f165d1799252c4d, user_id=36ca2ee835f3445093c205cb542bce37, role_id=61c2794c5a6d4f91b3a953f0ae6eab8b)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:44.303077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca55be6-3184-4de0-8705-7eea42702a39 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:44.324625 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 145/289] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:55:44 2022] PUT /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d/users/36ca2ee835f3445093c205cb542bce37/roles/61c2794c5a6d4f91b3a953f0ae6eab8b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:44.335946 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:44.337257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:44.337705 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:44.338270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:44.339173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:44.339855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:44.341240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:44.354732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 145/290] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:44.361169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca55be6-3184-4de0-8705-7eea42702a39 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:44.464907 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:44.498234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca55be6-3184-4de0-8705-7eea42702a39 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:44.537898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624734-4938-4ff9-a5c3-007187bb59fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:44.538508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624734-4938-4ff9-a5c3-007187bb59fd None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:44.538798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624734-4938-4ff9-a5c3-007187bb59fd None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:44.539081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624734-4938-4ff9-a5c3-007187bb59fd None admin] PATH_INFO: `/v3/users/b678411b7859457c9d999c00433509df` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:44.540345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b624734-4938-4ff9-a5c3-007187bb59fd None admin] RBAC: Authorizing `identity:get_user(user_id=b678411b7859457c9d999c00433509df)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:44.541960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b624734-4938-4ff9-a5c3-007187bb59fd None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:44.544063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 146/291] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:44 2022] GET /identity/v3/users/b678411b7859457c9d999c00433509df => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:44.552361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:44.694620 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6490c2-1e3b-409e-bdc2-0a830dca031f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:44.716873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:44.743443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2602c-7d29-4f35-b2ce-f7848aefc1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:44.744108 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2602c-7d29-4f35-b2ce-f7848aefc1b7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:44.744392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2602c-7d29-4f35-b2ce-f7848aefc1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:44.744663 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2602c-7d29-4f35-b2ce-f7848aefc1b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:44.834802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2602c-7d29-4f35-b2ce-f7848aefc1b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:44.836279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2602c-7d29-4f35-b2ce-f7848aefc1b7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:44.953896 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 146/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:44.959881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624734-4938-4ff9-a5c3-007187bb59fd None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:45.075193 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624734-4938-4ff9-a5c3-007187bb59fd None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:45.121788 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43364f-668c-4d6d-aa2f-bac80c9bacc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:45.122922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43364f-668c-4d6d-aa2f-bac80c9bacc4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:45.123444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43364f-668c-4d6d-aa2f-bac80c9bacc4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:45.124005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43364f-668c-4d6d-aa2f-bac80c9bacc4 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:45.131142 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43364f-668c-4d6d-aa2f-bac80c9bacc4 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:45.133095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43364f-668c-4d6d-aa2f-bac80c9bacc4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:45.135317 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 147/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:44 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:45.141825 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2602c-7d29-4f35-b2ce-f7848aefc1b7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:45.169748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:45.259265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2602c-7d29-4f35-b2ce-f7848aefc1b7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:45.298721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce938abc-b1d5-4f05-ae0d-659f74c8eaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:45.299382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938abc-b1d5-4f05-ae0d-659f74c8eaeb None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:45.299666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938abc-b1d5-4f05-ae0d-659f74c8eaeb None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:45.299941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938abc-b1d5-4f05-ae0d-659f74c8eaeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:45.300755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce938abc-b1d5-4f05-ae0d-659f74c8eaeb None admin] RBAC: Adding query filter params (role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=96c6511840d04bcc8f165d1799252c4d, user.id=b678411b7859457c9d999c00433509df) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:45.301069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce938abc-b1d5-4f05-ae0d-659f74c8eaeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:45.302580 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce938abc-b1d5-4f05-ae0d-659f74c8eaeb None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:45.310091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 147/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:45 2022] GET /identity/v3/role_assignments?user.id=b678411b7859457c9d999c00433509df&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:45.863473 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea2636-6ae0-4824-8fd8-cdd67b674423 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:45.863767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea2636-6ae0-4824-8fd8-cdd67b674423 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:45.864058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea2636-6ae0-4824-8fd8-cdd67b674423 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:45.865748 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 148/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:45.871799 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d6f0-bccf-4cda-9925-a9d9dda9c711 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:45.872339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d6f0-bccf-4cda-9925-a9d9dda9c711 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:45.872339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d6f0-bccf-4cda-9925-a9d9dda9c711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:45.891909 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-9ae8d6f0-bccf-4cda-9925-a9d9dda9c711 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:45.893143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9ae8d6f0-bccf-4cda-9925-a9d9dda9c711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:45.979320 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 148/296] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:46.001496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea2636-6ae0-4824-8fd8-cdd67b674423 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:46.018081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:46.116969 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea2636-6ae0-4824-8fd8-cdd67b674423 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:46.153890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa56d-ca76-486e-8e06-9e9ef8127e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:46.154601 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa56d-ca76-486e-8e06-9e9ef8127e0f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.154888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa56d-ca76-486e-8e06-9e9ef8127e0f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.155168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa56d-ca76-486e-8e06-9e9ef8127e0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.166933 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fa56d-ca76-486e-8e06-9e9ef8127e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-438fa56d-ca76-486e-8e06-9e9ef8127e0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:46.172681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:46.173921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:46.175044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 149/297] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:46.180575 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8d6f0-bccf-4cda-9925-a9d9dda9c711 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:46.222537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:46.311156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8d6f0-bccf-4cda-9925-a9d9dda9c711 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:46.366227 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e05b-1575-4015-b704-7af1f79097be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:46.367361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e05b-1575-4015-b704-7af1f79097be None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.367890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e05b-1575-4015-b704-7af1f79097be None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.368409 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e05b-1575-4015-b704-7af1f79097be None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.369491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1e05b-1575-4015-b704-7af1f79097be None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:46.370026 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1e05b-1575-4015-b704-7af1f79097be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:46.372018 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1e05b-1575-4015-b704-7af1f79097be None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:46.383547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 149/298] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:46.394139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa56d-ca76-486e-8e06-9e9ef8127e0f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:46.444009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad058c-6fbd-49aa-845f-d2d542c36caf None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.444356 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad058c-6fbd-49aa-845f-d2d542c36caf None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.444680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad058c-6fbd-49aa-845f-d2d542c36caf None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.449116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 150/299] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:55:46.452190 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6a7a9-2f94-46b2-9c45-2be909a8eddb None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.452478 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6a7a9-2f94-46b2-9c45-2be909a8eddb None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.452748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6a7a9-2f94-46b2-9c45-2be909a8eddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.471322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:46.476981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ffd6a7a9-2f94-46b2-9c45-2be909a8eddb None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:46.478280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ffd6a7a9-2f94-46b2-9c45-2be909a8eddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:46.507350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa56d-ca76-486e-8e06-9e9ef8127e0f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:46.546490 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9809af-fb62-44bf-b14c-e87737c9d50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:46.547103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9809af-fb62-44bf-b14c-e87737c9d50f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.547386 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9809af-fb62-44bf-b14c-e87737c9d50f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.547705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9809af-fb62-44bf-b14c-e87737c9d50f None admin] PATH_INFO: `/v3/users/8004e4a5f40141c393461ad00fad5026` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.549114 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9809af-fb62-44bf-b14c-e87737c9d50f None admin] RBAC: Authorizing `identity:get_user(user_id=8004e4a5f40141c393461ad00fad5026)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:46.550791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9809af-fb62-44bf-b14c-e87737c9d50f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:46.552994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 150/300] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:46 2022] GET /identity/v3/users/8004e4a5f40141c393461ad00fad5026 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:46.554386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 151/301] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:46.558419 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9809af-fb62-44bf-b14c-e87737c9d50f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:46.582031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6a7a9-2f94-46b2-9c45-2be909a8eddb None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:46.585117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:46.677455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9809af-fb62-44bf-b14c-e87737c9d50f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:46.699106 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6a7a9-2f94-46b2-9c45-2be909a8eddb None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:46.716576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a25b1-64d0-4673-bd50-fca9a4f9993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:46.717284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a25b1-64d0-4673-bd50-fca9a4f9993f None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.717962 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a25b1-64d0-4673-bd50-fca9a4f9993f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.718287 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a25b1-64d0-4673-bd50-fca9a4f9993f None admin] PATH_INFO: `/v3/system/users/8004e4a5f40141c393461ad00fad5026/roles/844a46aa3ad0499fb8bfa2b1bd023ddd` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.719993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a25b1-64d0-4673-bd50-fca9a4f9993f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8004e4a5f40141c393461ad00fad5026, role_id=844a46aa3ad0499fb8bfa2b1bd023ddd)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:46.721425 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a25b1-64d0-4673-bd50-fca9a4f9993f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:46.734599 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 151/302] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:55:46 2022] PUT /identity/v3/system/users/8004e4a5f40141c393461ad00fad5026/roles/844a46aa3ad0499fb8bfa2b1bd023ddd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:55:46.734599 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c86ea-f07e-41e8-aacf-01ec8bd13fc5 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.734945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c86ea-f07e-41e8-aacf-01ec8bd13fc5 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.735068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c86ea-f07e-41e8-aacf-01ec8bd13fc5 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.737240 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 152/303] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:46.743345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa80af0-8681-4c02-b92e-69bee1e2145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:46.744061 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0607359-eb6a-4b42-935e-b17a8b668b63 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.744207 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa80af0-8681-4c02-b92e-69bee1e2145e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.744380 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0607359-eb6a-4b42-935e-b17a8b668b63 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.744503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa80af0-8681-4c02-b92e-69bee1e2145e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.744662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0607359-eb6a-4b42-935e-b17a8b668b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.744849 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa80af0-8681-4c02-b92e-69bee1e2145e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.754536 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:46.759646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa80af0-8681-4c02-b92e-69bee1e2145e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-5fa80af0-8681-4c02-b92e-69bee1e2145e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:46.765464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:46.766795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:46.767844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 152/304] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:46.770476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-b0607359-eb6a-4b42-935e-b17a8b668b63 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:46.771822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b0607359-eb6a-4b42-935e-b17a8b668b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:46.773263 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa80af0-8681-4c02-b92e-69bee1e2145e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:46.855305 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:46.879493 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 153/305] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:46.901853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0607359-eb6a-4b42-935e-b17a8b668b63 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:46.906000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa80af0-8681-4c02-b92e-69bee1e2145e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:46.954418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ae25a-f64b-48e8-af59-6a231e646e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:46.955173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ae25a-f64b-48e8-af59-6a231e646e8e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.955458 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ae25a-f64b-48e8-af59-6a231e646e8e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.955755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ae25a-f64b-48e8-af59-6a231e646e8e None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.956493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ae25a-f64b-48e8-af59-6a231e646e8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:46.956802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ae25a-f64b-48e8-af59-6a231e646e8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:46.958275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ae25a-f64b-48e8-af59-6a231e646e8e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:46.969276 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 153/306] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:46.970772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7f102-303d-405a-bf40-a5b219bcbbdb None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:46.971569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7f102-303d-405a-bf40-a5b219bcbbdb None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:46.971569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7f102-303d-405a-bf40-a5b219bcbbdb None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:46.972795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 154/307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:46.976129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7f102-303d-405a-bf40-a5b219bcbbdb None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:47.027051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0607359-eb6a-4b42-935e-b17a8b668b63 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:47.070508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037455b-842d-4122-9713-1ed5d5cab7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:47.072129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037455b-842d-4122-9713-1ed5d5cab7c9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:47.072466 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037455b-842d-4122-9713-1ed5d5cab7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:47.072747 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037455b-842d-4122-9713-1ed5d5cab7c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:47.074444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037455b-842d-4122-9713-1ed5d5cab7c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:47.076443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037455b-842d-4122-9713-1ed5d5cab7c9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:47.082758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 154/308] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:47.082758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdbd2d-9f37-412b-9395-0c61b15dba06 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:47.082758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdbd2d-9f37-412b-9395-0c61b15dba06 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:47.082758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdbd2d-9f37-412b-9395-0c61b15dba06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:47.086476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:47.099808 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-1cfdbd2d-9f37-412b-9395-0c61b15dba06 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:47.101111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-1cfdbd2d-9f37-412b-9395-0c61b15dba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:47.107983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7f102-303d-405a-bf40-a5b219bcbbdb None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:47.134274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:47.146219 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f8422-34ac-41d8-bd40-47383b237f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:47.146940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f8422-34ac-41d8-bd40-47383b237f06 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:47.147226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f8422-34ac-41d8-bd40-47383b237f06 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:47.147510 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f8422-34ac-41d8-bd40-47383b237f06 None admin] PATH_INFO: `/v3/users/f37d4137775d41608ec9da37de2f3627` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:47.163129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f8422-34ac-41d8-bd40-47383b237f06 None admin] RBAC: Authorizing `identity:get_user(user_id=f37d4137775d41608ec9da37de2f3627)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:47.164625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f8422-34ac-41d8-bd40-47383b237f06 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:47.171332 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 155/309] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:46 2022] GET /identity/v3/users/f37d4137775d41608ec9da37de2f3627 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:47.172789 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f8422-34ac-41d8-bd40-47383b237f06 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:47.181613 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 155/310] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:47.182580 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdbd2d-9f37-412b-9395-0c61b15dba06 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:47.284174 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f8422-34ac-41d8-bd40-47383b237f06 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:47.296527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdbd2d-9f37-412b-9395-0c61b15dba06 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:47.321309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455c771-66fd-434d-bec9-9436bf9d7592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:47.322047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455c771-66fd-434d-bec9-9436bf9d7592 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:47.322367 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455c771-66fd-434d-bec9-9436bf9d7592 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:47.322644 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455c771-66fd-434d-bec9-9436bf9d7592 None admin] PATH_INFO: `/v3/groups` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:47.323350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455c771-66fd-434d-bec9-9436bf9d7592 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:47.324805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455c771-66fd-434d-bec9-9436bf9d7592 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:47.330664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:47.337796 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:47.338898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:47.339392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:47.339942 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:47.340819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] RBAC: Adding query filter params (role.id=61c2794c5a6d4f91b3a953f0ae6eab8b, user.id=f37d4137775d41608ec9da37de2f3627, scope.system=all) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:47.341154 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:47.342703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:47.343834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 156/311] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:55:47 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:55:47.344488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455c771-66fd-434d-bec9-9436bf9d7592 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:47.364001 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 156/312] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:55:47 2022] GET /identity/v3/role_assignments?user.id=f37d4137775d41608ec9da37de2f3627&scope.system=all&role.id=61c2794c5a6d4f91b3a953f0ae6eab8b => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:47.415891 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:47.495915 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455c771-66fd-434d-bec9-9436bf9d7592 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:47.546650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d805fc5-fbcc-4afb-8bac-2502c9385fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:47.547825 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805fc5-fbcc-4afb-8bac-2502c9385fc3 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:47.548344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805fc5-fbcc-4afb-8bac-2502c9385fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:47.548872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805fc5-fbcc-4afb-8bac-2502c9385fc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:47.562569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d805fc5-fbcc-4afb-8bac-2502c9385fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-5d805fc5-fbcc-4afb-8bac-2502c9385fc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:47.570083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:47.571377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:47.572857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 157/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:47.580711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:47.710021 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bea3f4-d8d5-4c68-ae1d-63ff19696343 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:47.751953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:47.752964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:47.753439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:47.753899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:47.754994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:47.755492 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:47.757176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:47.772108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 157/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:47.773922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d805fc5-fbcc-4afb-8bac-2502c9385fc3 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:47.887270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d805fc5-fbcc-4afb-8bac-2502c9385fc3 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:47.889033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:47.925729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c027c09-1166-47b3-a5c0-5018d5de266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:47.926450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027c09-1166-47b3-a5c0-5018d5de266c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:47.926698 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027c09-1166-47b3-a5c0-5018d5de266c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:47.926967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027c09-1166-47b3-a5c0-5018d5de266c None admin] PATH_INFO: `/v3/users/36ca2ee835f3445093c205cb542bce37` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:47.928208 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c027c09-1166-47b3-a5c0-5018d5de266c None admin] RBAC: Authorizing `identity:get_user(user_id=36ca2ee835f3445093c205cb542bce37)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:47.929863 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c027c09-1166-47b3-a5c0-5018d5de266c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:47.936423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 158/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:47 2022] GET /identity/v3/users/36ca2ee835f3445093c205cb542bce37 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:47.938017 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:48.007496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:48.054309 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb985dd-cfdf-43f3-ac89-787e45e00d31 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:48.091013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de034c47-bd3b-4b93-9917-d7851b4ab58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:48.091700 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de034c47-bd3b-4b93-9917-d7851b4ab58b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:48.092034 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de034c47-bd3b-4b93-9917-d7851b4ab58b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:48.092395 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de034c47-bd3b-4b93-9917-d7851b4ab58b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:48.165195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de034c47-bd3b-4b93-9917-d7851b4ab58b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:48.166686 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de034c47-bd3b-4b93-9917-d7851b4ab58b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:48.236166 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:48.289260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 158/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:48.290752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c027c09-1166-47b3-a5c0-5018d5de266c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:48.408902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c027c09-1166-47b3-a5c0-5018d5de266c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:48.447125 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8351e-57ab-4d04-b3e3-9caf5d588bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:48.447781 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8351e-57ab-4d04-b3e3-9caf5d588bb2 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:48.448064 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8351e-57ab-4d04-b3e3-9caf5d588bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:48.448402 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8351e-57ab-4d04-b3e3-9caf5d588bb2 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:48.449800 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8351e-57ab-4d04-b3e3-9caf5d588bb2 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:48.451349 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8351e-57ab-4d04-b3e3-9caf5d588bb2 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:48.458369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 159/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:48 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:48.459908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de034c47-bd3b-4b93-9917-d7851b4ab58b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:48.571976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de034c47-bd3b-4b93-9917-d7851b4ab58b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:48.609401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253092f2-dd95-45f6-90dc-3d83c4e54fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:48.610068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253092f2-dd95-45f6-90dc-3d83c4e54fb6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:48.610772 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253092f2-dd95-45f6-90dc-3d83c4e54fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:48.611049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253092f2-dd95-45f6-90dc-3d83c4e54fb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:48.611877 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253092f2-dd95-45f6-90dc-3d83c4e54fb6 None admin] RBAC: Adding query filter params (role.id=61c2794c5a6d4f91b3a953f0ae6eab8b, scope.project.id=96c6511840d04bcc8f165d1799252c4d, user.id=36ca2ee835f3445093c205cb542bce37) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:48.612192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253092f2-dd95-45f6-90dc-3d83c4e54fb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:48.613780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253092f2-dd95-45f6-90dc-3d83c4e54fb6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:48.626281 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 159/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:55:48 2022] GET /identity/v3/role_assignments?user.id=36ca2ee835f3445093c205cb542bce37&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=61c2794c5a6d4f91b3a953f0ae6eab8b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:48.925413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94a8dc-203c-4855-a212-60f4f5203424 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:48.925690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94a8dc-203c-4855-a212-60f4f5203424 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:48.925982 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94a8dc-203c-4855-a212-60f4f5203424 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:48.927638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 160/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:48.933060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06953d5-ad33-4405-84a8-948872569362 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:48.933477 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06953d5-ad33-4405-84a8-948872569362 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:48.933831 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06953d5-ad33-4405-84a8-948872569362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:48.953854 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-e06953d5-ad33-4405-84a8-948872569362 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:48.955122 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-e06953d5-ad33-4405-84a8-948872569362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:48.974980 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:49.039460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 160/320] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:49.062769 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94a8dc-203c-4855-a212-60f4f5203424 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:49.110910 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:49.176066 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94a8dc-203c-4855-a212-60f4f5203424 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:49.212357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c1fde-062d-47f2-986a-f7ac03e62920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:49.213039 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c1fde-062d-47f2-986a-f7ac03e62920 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.213322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c1fde-062d-47f2-986a-f7ac03e62920 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.213594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c1fde-062d-47f2-986a-f7ac03e62920 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.224967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c1fde-062d-47f2-986a-f7ac03e62920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-a47c1fde-062d-47f2-986a-f7ac03e62920 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:49.230006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:49.231187 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:49.232364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:49.234200 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 161/321] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:49.237719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06953d5-ad33-4405-84a8-948872569362 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:49.352543 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06953d5-ad33-4405-84a8-948872569362 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:49.394168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6a16b-08ad-471a-a981-80dd7402b66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:49.394837 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a16b-08ad-471a-a981-80dd7402b66c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.395122 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a16b-08ad-471a-a981-80dd7402b66c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.395401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a16b-08ad-471a-a981-80dd7402b66c None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.396155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6a16b-08ad-471a-a981-80dd7402b66c None admin] RBAC: Adding query filter params (name=member) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:49.396542 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6a16b-08ad-471a-a981-80dd7402b66c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:49.398083 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6a16b-08ad-471a-a981-80dd7402b66c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:49.413049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 161/322] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:49.414979 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c1fde-062d-47f2-986a-f7ac03e62920 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:49.533456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b6004-1470-440a-b1d4-42d01b89ca19 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.533667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b6004-1470-440a-b1d4-42d01b89ca19 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.533974 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b6004-1470-440a-b1d4-42d01b89ca19 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.534702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:49.535424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 162/323] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:55:49.541330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745aa11b-3ba1-41a1-9987-9de592b07ff0 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.541579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c1fde-062d-47f2-986a-f7ac03e62920 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:49.541706 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745aa11b-3ba1-41a1-9987-9de592b07ff0 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.541971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745aa11b-3ba1-41a1-9987-9de592b07ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.563879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-745aa11b-3ba1-41a1-9987-9de592b07ff0 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:49.565323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-745aa11b-3ba1-41a1-9987-9de592b07ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:49.580294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:49.587956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db20476-e0c4-4fa9-b5d7-38e599a0caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:49.589196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20476-e0c4-4fa9-b5d7-38e599a0caef None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.590036 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20476-e0c4-4fa9-b5d7-38e599a0caef None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.590521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20476-e0c4-4fa9-b5d7-38e599a0caef None admin] PATH_INFO: `/v3/users/8004e4a5f40141c393461ad00fad5026` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.592009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db20476-e0c4-4fa9-b5d7-38e599a0caef None admin] RBAC: Authorizing `identity:get_user(user_id=8004e4a5f40141c393461ad00fad5026)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:49.594474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db20476-e0c4-4fa9-b5d7-38e599a0caef None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:49.601683 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 162/324] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:49 2022] GET /identity/v3/users/8004e4a5f40141c393461ad00fad5026 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:49.603343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db20476-e0c4-4fa9-b5d7-38e599a0caef None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:49.646365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 163/325] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:49.648046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7481957-3d66-474e-81b5-af67d4b2c2c1 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.648348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7481957-3d66-474e-81b5-af67d4b2c2c1 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.648632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7481957-3d66-474e-81b5-af67d4b2c2c1 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.650185 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 164/326] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:49.655740 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fd79b-9ef3-4a8d-a30e-0fa0594e220a None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.656121 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fd79b-9ef3-4a8d-a30e-0fa0594e220a None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.656359 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fd79b-9ef3-4a8d-a30e-0fa0594e220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.674901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-074fd79b-9ef3-4a8d-a30e-0fa0594e220a None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:49.676131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-074fd79b-9ef3-4a8d-a30e-0fa0594e220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:49.721521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db20476-e0c4-4fa9-b5d7-38e599a0caef None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:49.748230 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 165/327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:49.749131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fd79b-9ef3-4a8d-a30e-0fa0594e220a None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:49.759224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:49.760264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.760536 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.760821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.761638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] RBAC: Adding query filter params (role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, user.id=8004e4a5f40141c393461ad00fad5026, scope.system=all) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:49.762000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:49.763553 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:49.777262 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 163/328] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:55:49 2022] GET /identity/v3/role_assignments?user.id=8004e4a5f40141c393461ad00fad5026&scope.system=all&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:49.778154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:49.806823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:49.837340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:49.870835 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fd79b-9ef3-4a8d-a30e-0fa0594e220a None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:49.894059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a74e7-5ce8-477a-93c1-e51592aea81f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:49.912177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d7a8e-299e-45e7-9303-099cff775f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:49.912973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7a8e-299e-45e7-9303-099cff775f83 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.913270 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7a8e-299e-45e7-9303-099cff775f83 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.913553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7a8e-299e-45e7-9303-099cff775f83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.914914 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d7a8e-299e-45e7-9303-099cff775f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:49.916467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d7a8e-299e-45e7-9303-099cff775f83 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:49.918544 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 166/329] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:55:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:49.924891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d7a8e-299e-45e7-9303-099cff775f83 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:49.934845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8ba02-d356-40c6-b73d-72e28f3aacee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:49.935675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8ba02-d356-40c6-b73d-72e28f3aacee None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:49.935965 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8ba02-d356-40c6-b73d-72e28f3aacee None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:49.936247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8ba02-d356-40c6-b73d-72e28f3aacee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:49.954401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8ba02-d356-40c6-b73d-72e28f3aacee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-e6e8ba02-d356-40c6-b73d-72e28f3aacee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:49.961100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:49.962334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:49.964022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 164/330] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:49.968555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8ba02-d356-40c6-b73d-72e28f3aacee None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:50.047590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d7a8e-299e-45e7-9303-099cff775f83 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:50.068394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:50.087498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8ba02-d356-40c6-b73d-72e28f3aacee None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:50.088571 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f884f34-24e6-4f78-bfeb-95a092b7b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:50.089239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f884f34-24e6-4f78-bfeb-95a092b7b800 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:50.089520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f884f34-24e6-4f78-bfeb-95a092b7b800 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:50.089922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f884f34-24e6-4f78-bfeb-95a092b7b800 None admin] PATH_INFO: `/v3/groups` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:50.090654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f884f34-24e6-4f78-bfeb-95a092b7b800 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:50.092385 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f884f34-24e6-4f78-bfeb-95a092b7b800 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:50.122618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 167/331] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 08:55:49 2022] POST /identity/v3/groups => generated 226 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:55:50.125603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:50.126371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:50.126666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:50.126945 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:50.127720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] RBAC: Adding query filter params (name=reader) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:50.128032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:50.129552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:50.139709 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 165/332] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:50.146387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f884f34-24e6-4f78-bfeb-95a092b7b800 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:50.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:50.270442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f884f34-24e6-4f78-bfeb-95a092b7b800 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:50.306968 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a10d0-407a-4923-a6e9-1e677df21532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:50.307662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a10d0-407a-4923-a6e9-1e677df21532 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:50.307940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a10d0-407a-4923-a6e9-1e677df21532 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:50.308238 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a10d0-407a-4923-a6e9-1e677df21532 None admin] PATH_INFO: `/v3/users/f37d4137775d41608ec9da37de2f3627` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:50.309462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a10d0-407a-4923-a6e9-1e677df21532 None admin] RBAC: Authorizing `identity:get_user(user_id=f37d4137775d41608ec9da37de2f3627)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:50.310994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a10d0-407a-4923-a6e9-1e677df21532 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:50.312652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 168/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:50 2022] GET /identity/v3/users/f37d4137775d41608ec9da37de2f3627 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:50.319119 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:50.430485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaeb2d-7203-492f-a514-c17d910f8558 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:50.470552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238cb46-64a8-449e-827c-be8937abae8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:50.471552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238cb46-64a8-449e-827c-be8937abae8f None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:50.471834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238cb46-64a8-449e-827c-be8937abae8f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:50.472109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238cb46-64a8-449e-827c-be8937abae8f None admin] PATH_INFO: `/v3/system/users/f37d4137775d41608ec9da37de2f3627/roles/61c2794c5a6d4f91b3a953f0ae6eab8b` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:50.473729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238cb46-64a8-449e-827c-be8937abae8f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f37d4137775d41608ec9da37de2f3627, role_id=61c2794c5a6d4f91b3a953f0ae6eab8b)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:50.475274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238cb46-64a8-449e-827c-be8937abae8f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:50.485472 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 166/334] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 08:55:50 2022] PUT /identity/v3/system/users/f37d4137775d41608ec9da37de2f3627/roles/61c2794c5a6d4f91b3a953f0ae6eab8b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:52.832897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8856e-cc7e-4496-afc2-ce130a7a75ea None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:52.834138 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8856e-cc7e-4496-afc2-ce130a7a75ea None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:52.834667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8856e-cc7e-4496-afc2-ce130a7a75ea None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:52.836675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 169/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:52.844291 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca28af-f7e6-4caa-b406-f414df720bb5 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:52.845447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca28af-f7e6-4caa-b406-f414df720bb5 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:52.845958 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca28af-f7e6-4caa-b406-f414df720bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:52.871892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ecca28af-f7e6-4caa-b406-f414df720bb5 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:52.873556 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ecca28af-f7e6-4caa-b406-f414df720bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:52.875972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0b20-4efd-49b2-8a17-f373c3babc18 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:52.876566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0b20-4efd-49b2-8a17-f373c3babc18 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:52.877058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0b20-4efd-49b2-8a17-f373c3babc18 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:52.877613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:52.878743 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 170/336] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:52.885328 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d968c-57c1-4ab3-851d-c992a4b69457 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:52.885998 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d968c-57c1-4ab3-851d-c992a4b69457 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:52.886529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d968c-57c1-4ab3-851d-c992a4b69457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:52.923261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-950d968c-57c1-4ab3-851d-c992a4b69457 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:52.924541 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-950d968c-57c1-4ab3-851d-c992a4b69457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:52.998197 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 167/337] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:53.000056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb339fa8-f50b-4d48-b7b7-3505936d584a None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.000523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb339fa8-f50b-4d48-b7b7-3505936d584a None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.001055 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb339fa8-f50b-4d48-b7b7-3505936d584a None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.002665 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 168/338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:53.017364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4c79-94a9-4bdf-9f1c-fc6bc24110dd None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.017969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4c79-94a9-4bdf-9f1c-fc6bc24110dd None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.018455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4c79-94a9-4bdf-9f1c-fc6bc24110dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.049978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:53.055734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 171/339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:53.056878 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d968c-57c1-4ab3-851d-c992a4b69457 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:53.063357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ab9e4c79-94a9-4bdf-9f1c-fc6bc24110dd None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:53.064603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ab9e4c79-94a9-4bdf-9f1c-fc6bc24110dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:53.213018 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 169/340] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:53.213723 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e4c79-94a9-4bdf-9f1c-fc6bc24110dd None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:53.261318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d968c-57c1-4ab3-851d-c992a4b69457 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:53.306639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2b340-7fa1-4497-bb4c-016136b701ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:53.307324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2b340-7fa1-4497-bb4c-016136b701ad None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.307669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2b340-7fa1-4497-bb4c-016136b701ad None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.307956 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2b340-7fa1-4497-bb4c-016136b701ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.323028 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2b340-7fa1-4497-bb4c-016136b701ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:53.327046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-7dd2b340-7fa1-4497-bb4c-016136b701ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:53.331189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:53.332705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 172/341] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:53.334069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2b340-7fa1-4497-bb4c-016136b701ad None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:53.374076 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e4c79-94a9-4bdf-9f1c-fc6bc24110dd None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:53.404517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:53.413201 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f453e-c235-49d0-acd2-0deec5b55cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:53.413985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f453e-c235-49d0-acd2-0deec5b55cff None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.414306 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f453e-c235-49d0-acd2-0deec5b55cff None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.414689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f453e-c235-49d0-acd2-0deec5b55cff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.426903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f453e-c235-49d0-acd2-0deec5b55cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-2d4f453e-c235-49d0-acd2-0deec5b55cff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:53.432240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:53.433427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:53.434539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 170/342] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:53.443011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac000419-9693-44bb-8c29-fc8e8858d71e None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.443302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac000419-9693-44bb-8c29-fc8e8858d71e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.443584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac000419-9693-44bb-8c29-fc8e8858d71e None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.449373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 171/343] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:53.449373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac000419-9693-44bb-8c29-fc8e8858d71e None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:53.478716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2b340-7fa1-4497-bb4c-016136b701ad None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:53.525871 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cc30-5ec6-4aca-b406-019ddc440922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:53.526570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cc30-5ec6-4aca-b406-019ddc440922 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.526882 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cc30-5ec6-4aca-b406-019ddc440922 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.527388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cc30-5ec6-4aca-b406-019ddc440922 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.544521 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320cc30-5ec6-4aca-b406-019ddc440922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-a320cc30-5ec6-4aca-b406-019ddc440922 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:53.550210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:53.551588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 173/344] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:53.552931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cc30-5ec6-4aca-b406-019ddc440922 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:53.589152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac000419-9693-44bb-8c29-fc8e8858d71e None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:53.633179 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:53.641584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c18ce3-09ae-4603-9ad3-5a7e27dd25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:53.642924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18ce3-09ae-4603-9ad3-5a7e27dd25b9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.643407 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18ce3-09ae-4603-9ad3-5a7e27dd25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.643904 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18ce3-09ae-4603-9ad3-5a7e27dd25b9 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.645174 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c18ce3-09ae-4603-9ad3-5a7e27dd25b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:53.645674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c18ce3-09ae-4603-9ad3-5a7e27dd25b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:53.647561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c18ce3-09ae-4603-9ad3-5a7e27dd25b9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:53.660395 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 172/345] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:53.662146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ff065-72c6-4997-87a6-f13332c408b7 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.663294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ff065-72c6-4997-87a6-f13332c408b7 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.663831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ff065-72c6-4997-87a6-f13332c408b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.689512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-170ff065-72c6-4997-87a6-f13332c408b7 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:53.691931 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-170ff065-72c6-4997-87a6-f13332c408b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:53.697576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cc30-5ec6-4aca-b406-019ddc440922 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:53.729857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:53.749367 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:53.750019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.750439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.750614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.751495 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:53.751804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:53.754659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:53.772636 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 174/346] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:53.772636 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:53.801105 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 173/347] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:53.801105 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ff065-72c6-4997-87a6-f13332c408b7 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:53.931615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1466-a64e-4e5c-ab66-3a740a630859 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:53.939302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ff065-72c6-4997-87a6-f13332c408b7 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:53.965922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:53.981775 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:53.982847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.983299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.983745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.984906 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] RBAC: Adding query filter params (name=member) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:53.985580 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:53.987291 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:53.987962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb79f7f-c125-41eb-bbcb-566125618404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:53.988641 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb79f7f-c125-41eb-bbcb-566125618404 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:53.988930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb79f7f-c125-41eb-bbcb-566125618404 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:53.989213 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb79f7f-c125-41eb-bbcb-566125618404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:53.999715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 175/348] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:54.001101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:54.073452 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:54.077676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb79f7f-c125-41eb-bbcb-566125618404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:54.079495 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb79f7f-c125-41eb-bbcb-566125618404 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:54.135886 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b6650-f76b-44d2-a5b5-81e8cf6bfb86 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:54.176159 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0b2e9-55d4-4807-a031-5855479bc158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:54.177212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0b2e9-55d4-4807-a031-5855479bc158 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:54.177703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0b2e9-55d4-4807-a031-5855479bc158 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:54.178223 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0b2e9-55d4-4807-a031-5855479bc158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:54.197129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 174/349] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:54.198506 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb79f7f-c125-41eb-bbcb-566125618404 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:54.226798 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:54.269124 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c0b2e9-55d4-4807-a031-5855479bc158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:54.270614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c0b2e9-55d4-4807-a031-5855479bc158 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:54.332257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:54.332480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb79f7f-c125-41eb-bbcb-566125618404 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:54.383747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a287b-3ca9-494f-8945-02a193f3ab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:54.384569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a287b-3ca9-494f-8945-02a193f3ab2e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:54.384926 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a287b-3ca9-494f-8945-02a193f3ab2e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:54.385418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a287b-3ca9-494f-8945-02a193f3ab2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:54.407584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a287b-3ca9-494f-8945-02a193f3ab2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-876a287b-3ca9-494f-8945-02a193f3ab2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:54.412883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:54.414016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:54.415129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 175/350] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:54.415991 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a287b-3ca9-494f-8945-02a193f3ab2e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:54.424786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 176/351] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:54.425975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191cba4-2e75-4f38-bf63-7dda216c09fc None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:54.427677 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191cba4-2e75-4f38-bf63-7dda216c09fc None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:54.427677 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191cba4-2e75-4f38-bf63-7dda216c09fc None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:54.428961 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 177/352] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:54.429656 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191cba4-2e75-4f38-bf63-7dda216c09fc None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:54.522005 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:54.553864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a287b-3ca9-494f-8945-02a193f3ab2e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:54.565435 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191cba4-2e75-4f38-bf63-7dda216c09fc None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:54.605415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b8178-6111-4d46-9dbf-ac3278d902f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:54.606070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8178-6111-4d46-9dbf-ac3278d902f4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:54.606914 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8178-6111-4d46-9dbf-ac3278d902f4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:54.607195 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8178-6111-4d46-9dbf-ac3278d902f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:54.614217 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c598-7e8a-4f80-98df-ae83e0902296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:54.615140 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c598-7e8a-4f80-98df-ae83e0902296 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:54.615612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c598-7e8a-4f80-98df-ae83e0902296 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:54.616068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c598-7e8a-4f80-98df-ae83e0902296 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:54.617917 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63c598-7e8a-4f80-98df-ae83e0902296 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:54.620101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63c598-7e8a-4f80-98df-ae83e0902296 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:54.622693 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 178/353] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:54 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:54.623895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c598-7e8a-4f80-98df-ae83e0902296 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:54.644955 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:54.694357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b8178-6111-4d46-9dbf-ac3278d902f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:54.696676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b8178-6111-4d46-9dbf-ac3278d902f4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:54.771310 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c598-7e8a-4f80-98df-ae83e0902296 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:54.819447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:54.820297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:54.821045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:54.821429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:54.821823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:54.822614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:54.822921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:54.824351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:54.841280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 179/354] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:54.841280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:54.846666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 176/355] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:54.847824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41363dd-d80e-4cc6-88f2-aab06b1d1dc1 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:54.848109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41363dd-d80e-4cc6-88f2-aab06b1d1dc1 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:54.848383 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41363dd-d80e-4cc6-88f2-aab06b1d1dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:54.873615 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-e41363dd-d80e-4cc6-88f2-aab06b1d1dc1 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:54.875182 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-e41363dd-d80e-4cc6-88f2-aab06b1d1dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:54.952240 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:54.963402 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 177/356] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:54.965206 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41363dd-d80e-4cc6-88f2-aab06b1d1dc1 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:54.991995 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638653b-37a6-4ae8-903a-09e7a90f83a1 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.037192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6201e-dd0d-49a6-9581-04991e47416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.037881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6201e-dd0d-49a6-9581-04991e47416c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.038202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6201e-dd0d-49a6-9581-04991e47416c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.038487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6201e-dd0d-49a6-9581-04991e47416c None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.040450 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6201e-dd0d-49a6-9581-04991e47416c None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.042358 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6201e-dd0d-49a6-9581-04991e47416c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.044164 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 180/357] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:54 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.049713 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6201e-dd0d-49a6-9581-04991e47416c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.093561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41363dd-d80e-4cc6-88f2-aab06b1d1dc1 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.108491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:55.132398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543652da-7747-43e9-8c5e-61b893bfa616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.133076 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543652da-7747-43e9-8c5e-61b893bfa616 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.133362 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543652da-7747-43e9-8c5e-61b893bfa616 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.133642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543652da-7747-43e9-8c5e-61b893bfa616 None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.140134 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543652da-7747-43e9-8c5e-61b893bfa616 None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.142304 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543652da-7747-43e9-8c5e-61b893bfa616 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.143461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 178/358] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:55:54 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.145196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543652da-7747-43e9-8c5e-61b893bfa616 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.180044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6201e-dd0d-49a6-9581-04991e47416c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.220458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6515b07-e497-4d65-af63-df1106fdf523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.221138 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6515b07-e497-4d65-af63-df1106fdf523 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.221418 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6515b07-e497-4d65-af63-df1106fdf523 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.221689 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6515b07-e497-4d65-af63-df1106fdf523 None admin] PATH_INFO: `/v3/users/f37d4137775d41608ec9da37de2f3627` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.222966 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6515b07-e497-4d65-af63-df1106fdf523 None admin] RBAC: Authorizing `identity:get_user(user_id=f37d4137775d41608ec9da37de2f3627)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.224434 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6515b07-e497-4d65-af63-df1106fdf523 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.227020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 181/359] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/users/f37d4137775d41608ec9da37de2f3627 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.227388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4088d-5854-4240-8715-fc2e3785691f None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.227673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4088d-5854-4240-8715-fc2e3785691f None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.227953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4088d-5854-4240-8715-fc2e3785691f None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.229334 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 182/360] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:55.229909 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4088d-5854-4240-8715-fc2e3785691f None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.249715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:55.262827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543652da-7747-43e9-8c5e-61b893bfa616 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.309371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a25a2-faaa-4eab-b70b-93b20a7300bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.310362 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a25a2-faaa-4eab-b70b-93b20a7300bd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.310646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a25a2-faaa-4eab-b70b-93b20a7300bd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.311563 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a25a2-faaa-4eab-b70b-93b20a7300bd None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.312514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a25a2-faaa-4eab-b70b-93b20a7300bd None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.314090 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a25a2-faaa-4eab-b70b-93b20a7300bd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.318819 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 179/361] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.318997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a25a2-faaa-4eab-b70b-93b20a7300bd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.364374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4088d-5854-4240-8715-fc2e3785691f None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.374356 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:55.409600 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d330bd1-0f84-4347-9977-f0966995fef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.410316 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330bd1-0f84-4347-9977-f0966995fef8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.411181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330bd1-0f84-4347-9977-f0966995fef8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.411628 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330bd1-0f84-4347-9977-f0966995fef8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.425706 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d330bd1-0f84-4347-9977-f0966995fef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-7d330bd1-0f84-4347-9977-f0966995fef8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:55.431406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:55.432576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:55.433630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:55.434482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 183/362] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:55.435268 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d330bd1-0f84-4347-9977-f0966995fef8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.449662 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a25a2-faaa-4eab-b70b-93b20a7300bd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.496527 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ac6dd-79aa-4ea2-a896-39a8c3c6fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.497219 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac6dd-79aa-4ea2-a896-39a8c3c6fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.497504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac6dd-79aa-4ea2-a896-39a8c3c6fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.497785 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac6dd-79aa-4ea2-a896-39a8c3c6fbc9 None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.499128 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ac6dd-79aa-4ea2-a896-39a8c3c6fbc9 None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.500678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ac6dd-79aa-4ea2-a896-39a8c3c6fbc9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.502465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 180/363] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.503322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ac6dd-79aa-4ea2-a896-39a8c3c6fbc9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.531810 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:55.570128 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d330bd1-0f84-4347-9977-f0966995fef8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.614885 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360f524-bfd3-462f-be20-a3e4b79c3f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.615562 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360f524-bfd3-462f-be20-a3e4b79c3f77 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.615848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360f524-bfd3-462f-be20-a3e4b79c3f77 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.616331 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360f524-bfd3-462f-be20-a3e4b79c3f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.617048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360f524-bfd3-462f-be20-a3e4b79c3f77 None admin] RBAC: Adding query filter params (group.id=28be246b5d0c408da6b57009c6b857a5, role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=96c6511840d04bcc8f165d1799252c4d) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:55.617364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360f524-bfd3-462f-be20-a3e4b79c3f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.618908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360f524-bfd3-462f-be20-a3e4b79c3f77 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.623022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ac6dd-79aa-4ea2-a896-39a8c3c6fbc9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.625288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 184/364] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/role_assignments?group.id=28be246b5d0c408da6b57009c6b857a5&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.626736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b764b-6dc4-4129-9d26-55985b18d517 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.627028 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b764b-6dc4-4129-9d26-55985b18d517 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.627312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b764b-6dc4-4129-9d26-55985b18d517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.644358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:55.652870 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-1e2b764b-6dc4-4129-9d26-55985b18d517 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:55.657879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-1e2b764b-6dc4-4129-9d26-55985b18d517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:55.677351 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.678053 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.678392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.678677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.679526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] RBAC: Adding query filter params (role.id=61c2794c5a6d4f91b3a953f0ae6eab8b, user.id=f37d4137775d41608ec9da37de2f3627, scope.system=all) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:55.679851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.681377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.703303 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 181/365] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/role_assignments?user.id=f37d4137775d41608ec9da37de2f3627&scope.system=all&role.id=61c2794c5a6d4f91b3a953f0ae6eab8b => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.703303 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.756364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 185/366] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:55.757752 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b764b-6dc4-4129-9d26-55985b18d517 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.838432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:55.872856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df293346-c5d6-4515-9af4-d55f88ce75e7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.884620 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b764b-6dc4-4129-9d26-55985b18d517 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:55.924615 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef39b25-30e8-48ae-b443-d9aa16c7798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.925917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39b25-30e8-48ae-b443-d9aa16c7798d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.926826 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39b25-30e8-48ae-b443-d9aa16c7798d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.927475 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39b25-30e8-48ae-b443-d9aa16c7798d None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.928880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef39b25-30e8-48ae-b443-d9aa16c7798d None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.930644 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef39b25-30e8-48ae-b443-d9aa16c7798d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.931475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:55.932123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:55.932407 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:55.932685 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:55.933279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 182/367] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.933410 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:55.933724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:55.934619 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef39b25-30e8-48ae-b443-d9aa16c7798d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.938218 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:55.946622 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 186/368] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:55.950727 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:55.988748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:56.068349 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef39b25-30e8-48ae-b443-d9aa16c7798d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:56.112075 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:56.112728 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:56.112939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac354e3-63cd-4c04-83b5-3cb4ac478823 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:56.113395 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:56.113395 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:56.114079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] RBAC: Adding query filter params (group.id=28be246b5d0c408da6b57009c6b857a5, role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:56.114500 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:56.116005 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:56.126355 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 183/369] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/role_assignments?group.id=28be246b5d0c408da6b57009c6b857a5&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:56.127786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:56.148964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:56.171518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62a21f-eb27-4eaa-8f38-7fe13c689c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:56.172205 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62a21f-eb27-4eaa-8f38-7fe13c689c5d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:56.172485 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62a21f-eb27-4eaa-8f38-7fe13c689c5d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:56.172839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62a21f-eb27-4eaa-8f38-7fe13c689c5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:56.187673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62a21f-eb27-4eaa-8f38-7fe13c689c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-2a62a21f-eb27-4eaa-8f38-7fe13c689c5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:56.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:56.195458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 187/370] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:55:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:56.196939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62a21f-eb27-4eaa-8f38-7fe13c689c5d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:56.274936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc556cbc-fc58-4116-ab84-1e8b2cd0c315 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:56.276976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:56.319310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:56.320182 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:56.320476 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:56.320760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:56.322092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] RBAC: Adding query filter params (group.id=28be246b5d0c408da6b57009c6b857a5, role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:56.322464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:56.324384 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:56.333280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 184/371] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:55:56 2022] GET /identity/v3/role_assignments?group.id=28be246b5d0c408da6b57009c6b857a5&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:56.333633 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62a21f-eb27-4eaa-8f38-7fe13c689c5d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:56.334223 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:56.394745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd42f6-8eaa-4d88-8b7f-d9823a8e8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:56.395325 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd42f6-8eaa-4d88-8b7f-d9823a8e8a52 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:56.395670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd42f6-8eaa-4d88-8b7f-d9823a8e8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:56.395952 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd42f6-8eaa-4d88-8b7f-d9823a8e8a52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:56.474261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdcc63-e8c9-4ccc-ac8a-11a20da489d0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:56.484925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:56.503275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd42f6-8eaa-4d88-8b7f-d9823a8e8a52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:56.505200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd42f6-8eaa-4d88-8b7f-d9823a8e8a52 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:56.537520 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:56.539180 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:56.539180 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:56.539180 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:56.540127 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:56.540127 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:56.542026 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:56.551471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 185/372] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:56.557673 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:56.595388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:56.680074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a09d48-580e-4f56-a81d-66e155be1279 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:56.687364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 188/373] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:55:56.695323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd42f6-8eaa-4d88-8b7f-d9823a8e8a52 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:56.724396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b05e23-2e10-4c35-9aa1-10062e934746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:56.725050 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b05e23-2e10-4c35-9aa1-10062e934746 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:56.725337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b05e23-2e10-4c35-9aa1-10062e934746 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:56.725613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b05e23-2e10-4c35-9aa1-10062e934746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:56.804134 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b05e23-2e10-4c35-9aa1-10062e934746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:56.805638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b05e23-2e10-4c35-9aa1-10062e934746 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:56.838633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:56.882172 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:56.918647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd42f6-8eaa-4d88-8b7f-d9823a8e8a52 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:56.925214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 186/374] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:55:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:55:56.932507 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b05e23-2e10-4c35-9aa1-10062e934746 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:56.978520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcdec4-3610-462b-ab77-5225874e1ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:56.979560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcdec4-3610-462b-ab77-5225874e1ecc None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:56.979939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcdec4-3610-462b-ab77-5225874e1ecc None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:56.980312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcdec4-3610-462b-ab77-5225874e1ecc None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:56.981687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcdec4-3610-462b-ab77-5225874e1ecc None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:56.984016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcdec4-3610-462b-ab77-5225874e1ecc None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:56.986739 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 189/375] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:56 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:56.995683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcdec4-3610-462b-ab77-5225874e1ecc None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:57.126199 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b05e23-2e10-4c35-9aa1-10062e934746 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:57.130572 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcdec4-3610-462b-ab77-5225874e1ecc None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:57.170578 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:57.187607 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039e5e-a5ff-46af-9383-3e2307788394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:57.188065 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039e5e-a5ff-46af-9383-3e2307788394 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:57.188485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039e5e-a5ff-46af-9383-3e2307788394 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:57.188766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039e5e-a5ff-46af-9383-3e2307788394 None admin] PATH_INFO: `/v3/projects/9904f995f8d54e70b75420b12eb6d746` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:57.198130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db039e5e-a5ff-46af-9383-3e2307788394 None admin] RBAC: Authorizing `identity:get_project(project_id=9904f995f8d54e70b75420b12eb6d746)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:57.198130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db039e5e-a5ff-46af-9383-3e2307788394 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:57.198130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 187/376] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:56 2022] GET /identity/v3/projects/9904f995f8d54e70b75420b12eb6d746 => generated 344 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:57.200461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039e5e-a5ff-46af-9383-3e2307788394 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:57.202100 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b74597-a443-47f2-809f-e7b1ec8b3ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:57.203037 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74597-a443-47f2-809f-e7b1ec8b3ab0 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:57.203610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74597-a443-47f2-809f-e7b1ec8b3ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:57.204167 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74597-a443-47f2-809f-e7b1ec8b3ab0 None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:57.210711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b74597-a443-47f2-809f-e7b1ec8b3ab0 None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:57.212638 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b74597-a443-47f2-809f-e7b1ec8b3ab0 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:57.215000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 190/377] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:55:56 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:57.222718 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b74597-a443-47f2-809f-e7b1ec8b3ab0 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:57.263948 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:57.388313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039e5e-a5ff-46af-9383-3e2307788394 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:57.412636 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b74597-a443-47f2-809f-e7b1ec8b3ab0 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:57.453162 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb130-1e07-42c8-b3fe-834618fd6d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:57.454273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb130-1e07-42c8-b3fe-834618fd6d0f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:57.454762 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb130-1e07-42c8-b3fe-834618fd6d0f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:57.455246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb130-1e07-42c8-b3fe-834618fd6d0f None admin] PATH_INFO: `/v3/groups/f61881c55911434f98606cf3554253d8` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:57.466277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb130-1e07-42c8-b3fe-834618fd6d0f None admin] RBAC: Authorizing `identity:get_group(group_id=f61881c55911434f98606cf3554253d8)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:57.468101 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb130-1e07-42c8-b3fe-834618fd6d0f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:57.470159 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 188/378] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:55:57 2022] GET /identity/v3/groups/f61881c55911434f98606cf3554253d8 => generated 229 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:57.477941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb130-1e07-42c8-b3fe-834618fd6d0f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:57.483872 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3772f6-2b27-47bb-9c58-5faa2f17718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:57.484797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3772f6-2b27-47bb-9c58-5faa2f17718b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:57.486265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3772f6-2b27-47bb-9c58-5faa2f17718b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:57.486755 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3772f6-2b27-47bb-9c58-5faa2f17718b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:57.491967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3772f6-2b27-47bb-9c58-5faa2f17718b None admin] RBAC: Adding query filter params (group.id=28be246b5d0c408da6b57009c6b857a5, role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=96c6511840d04bcc8f165d1799252c4d) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:57.492535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3772f6-2b27-47bb-9c58-5faa2f17718b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:57.494278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3772f6-2b27-47bb-9c58-5faa2f17718b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:57.507795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 191/379] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:55:57 2022] GET /identity/v3/role_assignments?group.id=28be246b5d0c408da6b57009c6b857a5&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 260 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:57.569487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:57.633081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb130-1e07-42c8-b3fe-834618fd6d0f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:57.677660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d1e21-f604-4886-ab21-3a0c2a0a92f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:57.678718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d1e21-f604-4886-ab21-3a0c2a0a92f4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:57.679191 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d1e21-f604-4886-ab21-3a0c2a0a92f4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:57.679672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d1e21-f604-4886-ab21-3a0c2a0a92f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:57.680655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d1e21-f604-4886-ab21-3a0c2a0a92f4 None admin] RBAC: Adding query filter params (group.id=f61881c55911434f98606cf3554253d8, role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=9904f995f8d54e70b75420b12eb6d746) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:57.681170 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d1e21-f604-4886-ab21-3a0c2a0a92f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:57.682961 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d1e21-f604-4886-ab21-3a0c2a0a92f4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:57.694147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 189/380] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:55:57 2022] GET /identity/v3/role_assignments?group.id=f61881c55911434f98606cf3554253d8&scope.project.id=9904f995f8d54e70b75420b12eb6d746&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:57.925293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000538f-8ae1-4121-9d9b-0b4bd509eb60 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:57.925936 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000538f-8ae1-4121-9d9b-0b4bd509eb60 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:57.926444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000538f-8ae1-4121-9d9b-0b4bd509eb60 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:57.928288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 192/381] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:57.934327 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d35e3-fa73-4685-bd3f-68b9d5e88d0e None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:57.934908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d35e3-fa73-4685-bd3f-68b9d5e88d0e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:57.935402 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d35e3-fa73-4685-bd3f-68b9d5e88d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:57.955953 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-5f0d35e3-fa73-4685-bd3f-68b9d5e88d0e None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:57.957631 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-5f0d35e3-fa73-4685-bd3f-68b9d5e88d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:58.049778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 190/382] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:55:58.086699 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000538f-8ae1-4121-9d9b-0b4bd509eb60 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:58.211207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:58.238875 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000538f-8ae1-4121-9d9b-0b4bd509eb60 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:58.290910 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf144e-80da-4029-bea1-79eb93f6e12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:58.291591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf144e-80da-4029-bea1-79eb93f6e12f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:58.291880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf144e-80da-4029-bea1-79eb93f6e12f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:58.292160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf144e-80da-4029-bea1-79eb93f6e12f None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:58.309290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf144e-80da-4029-bea1-79eb93f6e12f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-7fbf144e-80da-4029-bea1-79eb93f6e12f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:58.314474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:58.315645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:58.317501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 193/383] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:58.322192 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d35e3-fa73-4685-bd3f-68b9d5e88d0e None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:58.416925 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:58.476024 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d35e3-fa73-4685-bd3f-68b9d5e88d0e None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:58.527499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:58.528154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:58.528434 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:58.528705 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:58.529610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:58.529918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:58.531357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:58.542852 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 191/384] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:58.548765 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf144e-80da-4029-bea1-79eb93f6e12f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:58.696412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf144e-80da-4029-bea1-79eb93f6e12f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:58.742172 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c85701-bd7d-44f1-a589-e74d1e5de0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:58.742776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85701-bd7d-44f1-a589-e74d1e5de0a9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:58.743060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85701-bd7d-44f1-a589-e74d1e5de0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:58.743337 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85701-bd7d-44f1-a589-e74d1e5de0a9 None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:58.744658 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c85701-bd7d-44f1-a589-e74d1e5de0a9 None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:58.746369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c85701-bd7d-44f1-a589-e74d1e5de0a9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:58.748373 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 194/385] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:55:58 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:58.754147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:58.824816 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149cc19-4503-4280-b617-b0d7efeaa6fd None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:58.825697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149cc19-4503-4280-b617-b0d7efeaa6fd None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:58.826572 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149cc19-4503-4280-b617-b0d7efeaa6fd None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:58.829636 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 195/386] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:58.836121 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074fa4c-4a28-42ef-a9f9-e7c57b806f40 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:58.836776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074fa4c-4a28-42ef-a9f9-e7c57b806f40 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:58.837293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074fa4c-4a28-42ef-a9f9-e7c57b806f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:58.882167 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-b074fa4c-4a28-42ef-a9f9-e7c57b806f40 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:58.883925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b074fa4c-4a28-42ef-a9f9-e7c57b806f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:58.898671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:58.905294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f783031-e715-42a6-a912-43d8a2f10540 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:58.923684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:58.944963 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392db70-9073-400d-a1ab-0207e14ac1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:58.945642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392db70-9073-400d-a1ab-0207e14ac1d8 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:58.945924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392db70-9073-400d-a1ab-0207e14ac1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:58.946234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392db70-9073-400d-a1ab-0207e14ac1d8 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:58.947526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392db70-9073-400d-a1ab-0207e14ac1d8 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:58.949104 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392db70-9073-400d-a1ab-0207e14ac1d8 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:58.951415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 192/387] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:58 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:58.952622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7343-d1b4-4939-917a-b189897fa65e None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:58.952906 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7343-d1b4-4939-917a-b189897fa65e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:58.953182 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7343-d1b4-4939-917a-b189897fa65e None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:58.954836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 193/388] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:55:58.960284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25487608-6d1e-483e-ad20-b47096359bf8 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:58.960693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25487608-6d1e-483e-ad20-b47096359bf8 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:58.961069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25487608-6d1e-483e-ad20-b47096359bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:58.980154 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 196/389] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:58.981026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074fa4c-4a28-42ef-a9f9-e7c57b806f40 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:58.982282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-25487608-6d1e-483e-ad20-b47096359bf8 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:55:58.983725 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-25487608-6d1e-483e-ad20-b47096359bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:55:59.080303 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 194/390] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:55:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:55:59.081433 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25487608-6d1e-483e-ad20-b47096359bf8 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:59.113882 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074fa4c-4a28-42ef-a9f9-e7c57b806f40 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:59.157982 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6a0b0-aa7e-4dda-b0f0-19b67f9b80d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:59.158723 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6a0b0-aa7e-4dda-b0f0-19b67f9b80d5 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:59.159004 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6a0b0-aa7e-4dda-b0f0-19b67f9b80d5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:59.159279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6a0b0-aa7e-4dda-b0f0-19b67f9b80d5 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d/groups/28be246b5d0c408da6b57009c6b857a5/roles/844a46aa3ad0499fb8bfa2b1bd023ddd` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:59.161269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6a0b0-aa7e-4dda-b0f0-19b67f9b80d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=96c6511840d04bcc8f165d1799252c4d, group_id=28be246b5d0c408da6b57009c6b857a5, role_id=844a46aa3ad0499fb8bfa2b1bd023ddd)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:59.162801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6a0b0-aa7e-4dda-b0f0-19b67f9b80d5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:59.181525 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 197/391] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:55:58 2022] PUT /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d/groups/28be246b5d0c408da6b57009c6b857a5/roles/844a46aa3ad0499fb8bfa2b1bd023ddd => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:55:59.182400 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6a0b0-aa7e-4dda-b0f0-19b67f9b80d5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:59.196242 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:59.212908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25487608-6d1e-483e-ad20-b47096359bf8 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:59.237451 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:59.264456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a2011-ddc2-44e7-aa3f-47372571ec85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:59.265884 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a2011-ddc2-44e7-aa3f-47372571ec85 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:59.266560 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a2011-ddc2-44e7-aa3f-47372571ec85 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:59.267079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a2011-ddc2-44e7-aa3f-47372571ec85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:59.284465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a2011-ddc2-44e7-aa3f-47372571ec85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-624a2011-ddc2-44e7-aa3f-47372571ec85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:59.291912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:59.293142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:55:59.294257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:55:59.298616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 195/392] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:55:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:59.311293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a2011-ddc2-44e7-aa3f-47372571ec85 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:59.380297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6a0b0-aa7e-4dda-b0f0-19b67f9b80d5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:59.425193 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fab66-148b-4078-9299-415667bc19ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:59.425953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fab66-148b-4078-9299-415667bc19ec None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:59.426273 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fab66-148b-4078-9299-415667bc19ec None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:59.426554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fab66-148b-4078-9299-415667bc19ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:59.432162 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a2011-ddc2-44e7-aa3f-47372571ec85 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:59.442235 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fab66-148b-4078-9299-415667bc19ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-2a5fab66-148b-4078-9299-415667bc19ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:55:59.449269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:55:59.452545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:55:59.453604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 198/393] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:55:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:55:59.456644 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fab66-148b-4078-9299-415667bc19ec None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:59.494041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:59.496180 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:59.496666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:59.497388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:59.498362 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:59.499130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:59.502846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:59.517840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 196/394] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:55:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:59.525124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:59.552272 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:59.614506 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:59.635292 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fab66-148b-4078-9299-415667bc19ec None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:59.715479 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:59.716336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:59.716730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:59.717112 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:59.717957 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:55:59.718390 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:59.720008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:59.737781 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186245e-2371-423b-9adb-768ff4a7528c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:59.739839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 199/395] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:55:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:59.741824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:59.861387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfe561-8b71-4dea-846f-eb557e1d63fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:59.861995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfe561-8b71-4dea-846f-eb557e1d63fd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:59.862310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfe561-8b71-4dea-846f-eb557e1d63fd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:59.862587 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfe561-8b71-4dea-846f-eb557e1d63fd None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:59.863942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bfe561-8b71-4dea-846f-eb557e1d63fd None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:59.865534 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bfe561-8b71-4dea-846f-eb557e1d63fd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:59.867543 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 197/396] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:55:59 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:55:59.868020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda9ca-25a4-4780-be47-20b488c7e8be None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:55:59.873107 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfe561-8b71-4dea-846f-eb557e1d63fd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:55:59.906634 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:55:59.923420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952c3c9-52b1-48ad-b7fe-bb274c8dc6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:55:59.924289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c3c9-52b1-48ad-b7fe-bb274c8dc6c4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:55:59.924671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c3c9-52b1-48ad-b7fe-bb274c8dc6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:55:59.925042 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c3c9-52b1-48ad-b7fe-bb274c8dc6c4 None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:55:59.926677 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952c3c9-52b1-48ad-b7fe-bb274c8dc6c4 None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:55:59.929765 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952c3c9-52b1-48ad-b7fe-bb274c8dc6c4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:55:59.934967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 200/397] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:55:59 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:55:59.941001 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952c3c9-52b1-48ad-b7fe-bb274c8dc6c4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:00.053238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfe561-8b71-4dea-846f-eb557e1d63fd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:00.056741 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:00.095313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713cb07-57ec-414f-8275-bb3c72288daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:00.097398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713cb07-57ec-414f-8275-bb3c72288daa None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.097807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713cb07-57ec-414f-8275-bb3c72288daa None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.098202 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713cb07-57ec-414f-8275-bb3c72288daa None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.099617 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713cb07-57ec-414f-8275-bb3c72288daa None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:00.101198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713cb07-57ec-414f-8275-bb3c72288daa None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:00.103377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 198/398] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:59 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:00.104903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7a20c-d5cd-4ae3-aad1-788554a4edef None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.105283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7a20c-d5cd-4ae3-aad1-788554a4edef None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.105839 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7a20c-d5cd-4ae3-aad1-788554a4edef None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.107724 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 199/399] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:00.108702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7a20c-d5cd-4ae3-aad1-788554a4edef None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:00.119874 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952c3c9-52b1-48ad-b7fe-bb274c8dc6c4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:00.156244 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d45eb0-8c35-42cb-bed1-3d98177d3a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:00.157188 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d45eb0-8c35-42cb-bed1-3d98177d3a7b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.157533 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d45eb0-8c35-42cb-bed1-3d98177d3a7b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.157874 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d45eb0-8c35-42cb-bed1-3d98177d3a7b None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.159189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d45eb0-8c35-42cb-bed1-3d98177d3a7b None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:00.161013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d45eb0-8c35-42cb-bed1-3d98177d3a7b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:00.163388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 201/400] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:55:59 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:00.164979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da11fc-2c58-4530-9860-db1c7d8d902e None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.165436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da11fc-2c58-4530-9860-db1c7d8d902e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.165888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da11fc-2c58-4530-9860-db1c7d8d902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.189180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-b3da11fc-2c58-4530-9860-db1c7d8d902e None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:00.190836 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b3da11fc-2c58-4530-9860-db1c7d8d902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:00.240777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7a20c-d5cd-4ae3-aad1-788554a4edef None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:00.273662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:00.278099 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53d3d3-6b40-42dd-bc5a-666909eb9bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:00.278656 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d3d3-6b40-42dd-bc5a-666909eb9bd6 None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.279022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d3d3-6b40-42dd-bc5a-666909eb9bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.279618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d3d3-6b40-42dd-bc5a-666909eb9bd6 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/groups/28be246b5d0c408da6b57009c6b857a5/roles/844a46aa3ad0499fb8bfa2b1bd023ddd` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.282931 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d3d3-6b40-42dd-bc5a-666909eb9bd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49c1fbd83f844dcacce2b9aa7ab8147, group_id=28be246b5d0c408da6b57009c6b857a5, role_id=844a46aa3ad0499fb8bfa2b1bd023ddd)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:00.284446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d3d3-6b40-42dd-bc5a-666909eb9bd6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:00.289125 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 202/401] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:00.290479 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da11fc-2c58-4530-9860-db1c7d8d902e None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:00.318507 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 200/402] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:56:00 2022] PUT /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/groups/28be246b5d0c408da6b57009c6b857a5/roles/844a46aa3ad0499fb8bfa2b1bd023ddd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:00.322474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53d3d3-6b40-42dd-bc5a-666909eb9bd6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:00.361931 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:00.424873 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da11fc-2c58-4530-9860-db1c7d8d902e None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:00.446218 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53d3d3-6b40-42dd-bc5a-666909eb9bd6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:00.471516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17863ae1-1a2f-4b8f-997f-9b228b13f8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:00.472232 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17863ae1-1a2f-4b8f-997f-9b228b13f8e4 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.472817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17863ae1-1a2f-4b8f-997f-9b228b13f8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.473103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17863ae1-1a2f-4b8f-997f-9b228b13f8e4 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/groups/28be246b5d0c408da6b57009c6b857a5/roles/167df0c7f94644578aff0abc3a0c8e84` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.476224 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17863ae1-1a2f-4b8f-997f-9b228b13f8e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49c1fbd83f844dcacce2b9aa7ab8147, group_id=28be246b5d0c408da6b57009c6b857a5, role_id=167df0c7f94644578aff0abc3a0c8e84)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:00.478058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17863ae1-1a2f-4b8f-997f-9b228b13f8e4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:00.490996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa43fbe-e1f3-46da-8897-a1e1958c5237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:00.492217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43fbe-e1f3-46da-8897-a1e1958c5237 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.492825 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43fbe-e1f3-46da-8897-a1e1958c5237 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43fbe-e1f3-46da-8897-a1e1958c5237 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.497236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 203/403] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:56:00 2022] PUT /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147/groups/28be246b5d0c408da6b57009c6b857a5/roles/167df0c7f94644578aff0abc3a0c8e84 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:00.499177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a0e81-c89c-46f4-8b60-2bc8f7d541c9 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.499470 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a0e81-c89c-46f4-8b60-2bc8f7d541c9 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.499803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a0e81-c89c-46f4-8b60-2bc8f7d541c9 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.501804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 204/404] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:00.507640 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cfa6-f82e-450d-9d03-d96a62c23894 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.508378 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cfa6-f82e-450d-9d03-d96a62c23894 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.508886 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cfa6-f82e-450d-9d03-d96a62c23894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.522810 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa43fbe-e1f3-46da-8897-a1e1958c5237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-eaa43fbe-e1f3-46da-8897-a1e1958c5237 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:00.532579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:00.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:56:00.534940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:00.536223 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 201/405] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:00.541792 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa43fbe-e1f3-46da-8897-a1e1958c5237 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:00.549549 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-7d08cfa6-f82e-450d-9d03-d96a62c23894 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:00.551225 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-7d08cfa6-f82e-450d-9d03-d96a62c23894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:00.614027 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:00.642166 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 205/406] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:56:00.677532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08cfa6-f82e-450d-9d03-d96a62c23894 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:00.690183 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:00.707989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa43fbe-e1f3-46da-8897-a1e1958c5237 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:00.751637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:00.752854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.753161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.753444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.754515 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:00.754830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:00.756950 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:00.768310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 202/407] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:00.774296 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:00.828583 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08cfa6-f82e-450d-9d03-d96a62c23894 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:00.896925 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64629a5-c743-49ac-a4cc-71570e1e11f7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:00.904772 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0b109-48ef-43fe-be6d-e62839533ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:00.905758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0b109-48ef-43fe-be6d-e62839533ac9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.906297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0b109-48ef-43fe-be6d-e62839533ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.906866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0b109-48ef-43fe-be6d-e62839533ac9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.941796 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0b109-48ef-43fe-be6d-e62839533ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:00.947038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd2991-5bc7-4b3e-86b7-111bbf49b239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:00.948127 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd2991-5bc7-4b3e-86b7-111bbf49b239 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:00.948652 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd2991-5bc7-4b3e-86b7-111bbf49b239 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:00.949151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd2991-5bc7-4b3e-86b7-111bbf49b239 None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:00.951162 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd2991-5bc7-4b3e-86b7-111bbf49b239 None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-d5f0b109-48ef-43fe-be6d-e62839533ac9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:00.954036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:00.955236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:00.956336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:00.958618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd2991-5bc7-4b3e-86b7-111bbf49b239 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:00.960648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 206/408] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:00.962496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 203/409] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:56:00 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:00.966551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0b109-48ef-43fe-be6d-e62839533ac9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:00.968905 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd2991-5bc7-4b3e-86b7-111bbf49b239 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:01.014360 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:01.056431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:01.163197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0b109-48ef-43fe-be6d-e62839533ac9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:01.166839 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd2991-5bc7-4b3e-86b7-111bbf49b239 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:01.229268 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:01.229502 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fdff5-38ca-43c2-a458-893f71e57a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:01.229865 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fdff5-38ca-43c2-a458-893f71e57a37 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:01.230177 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fdff5-38ca-43c2-a458-893f71e57a37 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:01.230462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fdff5-38ca-43c2-a458-893f71e57a37 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:01.231139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:01.231678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fdff5-38ca-43c2-a458-893f71e57a37 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:01.231954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:01.232475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:01.233160 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fdff5-38ca-43c2-a458-893f71e57a37 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:01.233698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:01.234266 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:01.235348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 204/410] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:56:00 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:01.236187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:01.245871 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 207/411] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:01.247400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fdff5-38ca-43c2-a458-893f71e57a37 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:01.251745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:01.362440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595fa7c-9d80-4036-864e-0ce48057b908 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:01.370541 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:01.399649 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24d4dc-b161-4857-9bcc-d5c7ce290655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:01.400319 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d4dc-b161-4857-9bcc-d5c7ce290655 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:01.400604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d4dc-b161-4857-9bcc-d5c7ce290655 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:01.400888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d4dc-b161-4857-9bcc-d5c7ce290655 None admin] PATH_INFO: `/v3/groups/f61881c55911434f98606cf3554253d8` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:01.402090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24d4dc-b161-4857-9bcc-d5c7ce290655 None admin] RBAC: Authorizing `identity:get_group(group_id=f61881c55911434f98606cf3554253d8)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:01.404146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24d4dc-b161-4857-9bcc-d5c7ce290655 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:01.406146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 208/412] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:56:01 2022] GET /identity/v3/groups/f61881c55911434f98606cf3554253d8 => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:01.411766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24d4dc-b161-4857-9bcc-d5c7ce290655 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:01.528310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:01.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24d4dc-b161-4857-9bcc-d5c7ce290655 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:01.570840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fdff5-38ca-43c2-a458-893f71e57a37 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:01.594159 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890812c-f1b7-41b5-8f54-64b9ea09aee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:01.594866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890812c-f1b7-41b5-8f54-64b9ea09aee3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:01.595159 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890812c-f1b7-41b5-8f54-64b9ea09aee3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:01.595852 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890812c-f1b7-41b5-8f54-64b9ea09aee3 None admin] PATH_INFO: `/v3/projects/9904f995f8d54e70b75420b12eb6d746` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:01.597193 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890812c-f1b7-41b5-8f54-64b9ea09aee3 None admin] RBAC: Authorizing `identity:get_project(project_id=9904f995f8d54e70b75420b12eb6d746)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:01.598989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890812c-f1b7-41b5-8f54-64b9ea09aee3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:01.601057 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 209/413] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:56:01 2022] GET /identity/v3/projects/9904f995f8d54e70b75420b12eb6d746 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:01.606945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890812c-f1b7-41b5-8f54-64b9ea09aee3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:01.645896 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc909e9-add3-45c0-843a-b7e241394e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:01.646704 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc909e9-add3-45c0-843a-b7e241394e5c None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:01.646992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc909e9-add3-45c0-843a-b7e241394e5c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:01.647272 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc909e9-add3-45c0-843a-b7e241394e5c None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d/groups/28be246b5d0c408da6b57009c6b857a5/roles/167df0c7f94644578aff0abc3a0c8e84` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:01.649576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc909e9-add3-45c0-843a-b7e241394e5c None admin] RBAC: Authorizing `identity:create_grant(project_id=96c6511840d04bcc8f165d1799252c4d, group_id=28be246b5d0c408da6b57009c6b857a5, role_id=167df0c7f94644578aff0abc3a0c8e84)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:01.651214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc909e9-add3-45c0-843a-b7e241394e5c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:01.668694 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 205/414] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:56:01 2022] PUT /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d/groups/28be246b5d0c408da6b57009c6b857a5/roles/167df0c7f94644578aff0abc3a0c8e84 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:01.669701 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:01.671843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa4f37-ec69-4510-b9a7-e316c31d992f None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:01.672912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa4f37-ec69-4510-b9a7-e316c31d992f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:01.672912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa4f37-ec69-4510-b9a7-e316c31d992f None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:01.679325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 206/415] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:01.682409 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae066f-8446-4ee9-8377-bd75a84f9cbd None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:01.682768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae066f-8446-4ee9-8377-bd75a84f9cbd None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:01.683050 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae066f-8446-4ee9-8377-bd75a84f9cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:01.703245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-5fae066f-8446-4ee9-8377-bd75a84f9cbd None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:01.704626 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-5fae066f-8446-4ee9-8377-bd75a84f9cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:01.753396 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890812c-f1b7-41b5-8f54-64b9ea09aee3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:01.798179 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb9cf0-4704-47fd-8854-524ebb9c0238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:01.799392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb9cf0-4704-47fd-8854-524ebb9c0238 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:01.799902 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb9cf0-4704-47fd-8854-524ebb9c0238 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:01.800420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb9cf0-4704-47fd-8854-524ebb9c0238 None admin] PATH_INFO: `/v3/projects/9904f995f8d54e70b75420b12eb6d746/groups/f61881c55911434f98606cf3554253d8/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:01.803081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb9cf0-4704-47fd-8854-524ebb9c0238 None admin] RBAC: Authorizing `identity:create_grant(project_id=9904f995f8d54e70b75420b12eb6d746, group_id=f61881c55911434f98606cf3554253d8, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:01.804913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb9cf0-4704-47fd-8854-524ebb9c0238 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:01.828259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 210/416] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 08:56:01 2022] PUT /identity/v3/projects/9904f995f8d54e70b75420b12eb6d746/groups/f61881c55911434f98606cf3554253d8/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:01.856020 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 207/417] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:56:01.880914 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb9cf0-4704-47fd-8854-524ebb9c0238 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:02.008425 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:02.012555 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb9cf0-4704-47fd-8854-524ebb9c0238 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:02.051432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bd075-a90a-4774-a58c-e7b4563fc952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:02.052031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bd075-a90a-4774-a58c-e7b4563fc952 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:02.052323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bd075-a90a-4774-a58c-e7b4563fc952 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:02.052598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bd075-a90a-4774-a58c-e7b4563fc952 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:02.064006 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bd075-a90a-4774-a58c-e7b4563fc952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-ed9bd075-a90a-4774-a58c-e7b4563fc952 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:02.069977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:02.071165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:02.076792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 211/418] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:02.077666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae066f-8446-4ee9-8377-bd75a84f9cbd None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:02.166925 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:02.182335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae066f-8446-4ee9-8377-bd75a84f9cbd None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:02.218994 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:02.219957 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:02.220438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:02.220935 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:02.222327 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] RBAC: Adding query filter params (name=member) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:02.222872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:02.225296 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:02.236544 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 208/419] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:56:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:02.242326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bd075-a90a-4774-a58c-e7b4563fc952 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:02.349395 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bd075-a90a-4774-a58c-e7b4563fc952 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:02.384883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3eeac-879e-450a-af03-40bb71de09cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:02.385536 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3eeac-879e-450a-af03-40bb71de09cb None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:02.385828 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3eeac-879e-450a-af03-40bb71de09cb None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:02.386098 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3eeac-879e-450a-af03-40bb71de09cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:02.418596 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:02.464524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3eeac-879e-450a-af03-40bb71de09cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:02.466489 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3eeac-879e-450a-af03-40bb71de09cb None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:02.578722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 212/420] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:02.591347 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:02.764249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8cbe4-2578-46a3-8f46-afe1a9ebc733 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:02.793486 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c672a-f365-4b3e-a50b-03cc724ce58f None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:02.793882 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c672a-f365-4b3e-a50b-03cc724ce58f None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:02.794079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c672a-f365-4b3e-a50b-03cc724ce58f None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:02.797825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 213/421] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:02.803708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920e42e-33d1-4c02-a131-f3355ddc0c04 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:02.804084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920e42e-33d1-4c02-a131-f3355ddc0c04 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:02.804269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920e42e-33d1-4c02-a131-f3355ddc0c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:02.817418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a8d84-67bc-40a3-b743-dda6bf7c8973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:02.818622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a8d84-67bc-40a3-b743-dda6bf7c8973 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:02.819175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a8d84-67bc-40a3-b743-dda6bf7c8973 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:02.819676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a8d84-67bc-40a3-b743-dda6bf7c8973 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:02.821169 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a8d84-67bc-40a3-b743-dda6bf7c8973 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:02.822959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a8d84-67bc-40a3-b743-dda6bf7c8973 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:02.825313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 209/422] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:56:02 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:02.833908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a8d84-67bc-40a3-b743-dda6bf7c8973 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:02.846549 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:02.847297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-4920e42e-33d1-4c02-a131-f3355ddc0c04 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:02.848590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-4920e42e-33d1-4c02-a131-f3355ddc0c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:02.944225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a8d84-67bc-40a3-b743-dda6bf7c8973 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:02.983921 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e549b3-30e0-4e32-b47e-735ee0c35b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:02.984603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e549b3-30e0-4e32-b47e-735ee0c35b32 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:02.984954 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e549b3-30e0-4e32-b47e-735ee0c35b32 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:02.985236 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e549b3-30e0-4e32-b47e-735ee0c35b32 None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:02.986627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e549b3-30e0-4e32-b47e-735ee0c35b32 None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:02.988122 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e549b3-30e0-4e32-b47e-735ee0c35b32 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:02.991579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 210/423] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:56:02 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:02.991820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:02.999228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e549b3-30e0-4e32-b47e-735ee0c35b32 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:03.019605 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 214/424] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:03.062448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920e42e-33d1-4c02-a131-f3355ddc0c04 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:03.114541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:03.134135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e549b3-30e0-4e32-b47e-735ee0c35b32 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:03.172280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c008296-eb08-4e59-998a-116da5af48ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:03.172931 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c008296-eb08-4e59-998a-116da5af48ad None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:03.173217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c008296-eb08-4e59-998a-116da5af48ad None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:03.173495 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c008296-eb08-4e59-998a-116da5af48ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:03.174388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c008296-eb08-4e59-998a-116da5af48ad None admin] RBAC: Adding query filter params (group.id=28be246b5d0c408da6b57009c6b857a5, role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=96c6511840d04bcc8f165d1799252c4d) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:03.174675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c008296-eb08-4e59-998a-116da5af48ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:03.176245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c008296-eb08-4e59-998a-116da5af48ad None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:03.187358 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920e42e-33d1-4c02-a131-f3355ddc0c04 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:03.190292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 211/425] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:56:02 2022] GET /identity/v3/role_assignments?group.id=28be246b5d0c408da6b57009c6b857a5&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:03.192337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5b50d-af62-48c5-a31e-0dd9cc4c438c None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:03.192628 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5b50d-af62-48c5-a31e-0dd9cc4c438c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:03.192908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5b50d-af62-48c5-a31e-0dd9cc4c438c None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:03.194630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 212/426] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:03.200139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b07754-c2f0-4f2b-8a75-c9e2a8a887df None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:03.200482 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b07754-c2f0-4f2b-8a75-c9e2a8a887df None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:03.200773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b07754-c2f0-4f2b-8a75-c9e2a8a887df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:03.238474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25bc39-b055-4fe7-930a-9ff0d00b2832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:03.239618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25bc39-b055-4fe7-930a-9ff0d00b2832 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:03.240254 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25bc39-b055-4fe7-930a-9ff0d00b2832 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:03.240755 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25bc39-b055-4fe7-930a-9ff0d00b2832 None admin] PATH_INFO: `/v3/roles/member` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:03.242676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-26b07754-c2f0-4f2b-8a75-c9e2a8a887df None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:03.244447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-26b07754-c2f0-4f2b-8a75-c9e2a8a887df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:03.265030 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25bc39-b055-4fe7-930a-9ff0d00b2832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-ec25bc39-b055-4fe7-930a-9ff0d00b2832 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:03.271351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:03.272564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:56:03.273632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:03.280266 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 215/427] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:03.287248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25bc39-b055-4fe7-930a-9ff0d00b2832 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:03.386836 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:03.402626 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 213/428] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:56:03.426199 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b07754-c2f0-4f2b-8a75-c9e2a8a887df None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:03.442491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25bc39-b055-4fe7-930a-9ff0d00b2832 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:03.485726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:03.486774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:03.487263 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:03.487724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:03.488811 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] RBAC: Adding query filter params (name=member) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:03.490159 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:03.491836 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:03.503794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 216/429] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:56:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:03.509104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:03.549279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:03.585111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b07754-c2f0-4f2b-8a75-c9e2a8a887df None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:03.638684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dd240-001c-4ea8-8355-37d1a746a1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:03.640332 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dd240-001c-4ea8-8355-37d1a746a1bb None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:03.640913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dd240-001c-4ea8-8355-37d1a746a1bb None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:03.641586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dd240-001c-4ea8-8355-37d1a746a1bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:03.647236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27814b4d-a453-44d8-b4cd-976b28e8a75f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:03.655511 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dd240-001c-4ea8-8355-37d1a746a1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:03.661181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-a42dd240-001c-4ea8-8355-37d1a746a1bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:03.662752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:03.663883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:56:03.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:03.668030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 214/430] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:03.673692 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dd240-001c-4ea8-8355-37d1a746a1bb None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:03.684292 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a0aa-8ef0-4232-93f0-8fbb77d6dd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:03.684787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a0aa-8ef0-4232-93f0-8fbb77d6dd27 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:03.685075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a0aa-8ef0-4232-93f0-8fbb77d6dd27 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:03.685354 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a0aa-8ef0-4232-93f0-8fbb77d6dd27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:03.761355 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622a0aa-8ef0-4232-93f0-8fbb77d6dd27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:03.763013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622a0aa-8ef0-4232-93f0-8fbb77d6dd27 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:03.786133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dd240-001c-4ea8-8355-37d1a746a1bb None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:03.819861 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:03.826245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:03.827346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:03.828283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:03.829000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:03.829986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:03.830552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:03.832353 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:03.842351 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 215/431] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:03.849411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:03.881457 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 217/432] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:03.886672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a0aa-8ef0-4232-93f0-8fbb77d6dd27 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:03.906079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:03.993618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0df79-a37c-4d9e-93b6-3b3b9ead84c4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.018296 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a0aa-8ef0-4232-93f0-8fbb77d6dd27 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.032754 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ed524-21db-4df9-a78d-76f8fefcdc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.033410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed524-21db-4df9-a78d-76f8fefcdc95 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.033725 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed524-21db-4df9-a78d-76f8fefcdc95 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.034001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed524-21db-4df9-a78d-76f8fefcdc95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.059437 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6fd9c-7c67-4b4f-8e8e-ce14b77dc8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.060488 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6fd9c-7c67-4b4f-8e8e-ce14b77dc8c1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.060999 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6fd9c-7c67-4b4f-8e8e-ce14b77dc8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.061459 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6fd9c-7c67-4b4f-8e8e-ce14b77dc8c1 None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.063024 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6fd9c-7c67-4b4f-8e8e-ce14b77dc8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:04.064848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6fd9c-7c67-4b4f-8e8e-ce14b77dc8c1 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:04.067646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 218/433] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:56:03 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:04.073136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6fd9c-7c67-4b4f-8e8e-ce14b77dc8c1 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.107514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:04.117613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ed524-21db-4df9-a78d-76f8fefcdc95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:04.119158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ed524-21db-4df9-a78d-76f8fefcdc95 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:04.167862 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:04.191619 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6fd9c-7c67-4b4f-8e8e-ce14b77dc8c1 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.227767 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 216/434] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:04.232726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672add8-b789-424b-b657-a85d5426781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.232726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672add8-b789-424b-b657-a85d5426781a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.232726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672add8-b789-424b-b657-a85d5426781a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.232726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672add8-b789-424b-b657-a85d5426781a None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.233101 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98927c7-586a-4ca9-9492-d99b905cd6a3 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.233201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672add8-b789-424b-b657-a85d5426781a None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:04.233285 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98927c7-586a-4ca9-9492-d99b905cd6a3 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.233285 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98927c7-586a-4ca9-9492-d99b905cd6a3 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.233451 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672add8-b789-424b-b657-a85d5426781a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:04.237621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 217/435] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:04.237621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98927c7-586a-4ca9-9492-d99b905cd6a3 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.239822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 219/436] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:04.241065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3b15e-6ae1-4c65-933b-6ab59fcbe6f7 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.241354 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3b15e-6ae1-4c65-933b-6ab59fcbe6f7 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.241633 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3b15e-6ae1-4c65-933b-6ab59fcbe6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.262374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-5ee3b15e-6ae1-4c65-933b-6ab59fcbe6f7 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:04.263122 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-5ee3b15e-6ae1-4c65-933b-6ab59fcbe6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:04.338139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 220/437] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:04.338938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3b15e-6ae1-4c65-933b-6ab59fcbe6f7 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.354452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:04.365634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98927c7-586a-4ca9-9492-d99b905cd6a3 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.402198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec127a-3f89-447b-9ab6-5ccb77d9782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.402874 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec127a-3f89-447b-9ab6-5ccb77d9782a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.403245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec127a-3f89-447b-9ab6-5ccb77d9782a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.403428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec127a-3f89-447b-9ab6-5ccb77d9782a None admin] PATH_INFO: `/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.404611 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec127a-3f89-447b-9ab6-5ccb77d9782a None admin] RBAC: Authorizing `identity:get_project(project_id=c49c1fbd83f844dcacce2b9aa7ab8147)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:04.406306 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec127a-3f89-447b-9ab6-5ccb77d9782a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:04.408217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 218/438] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/projects/c49c1fbd83f844dcacce2b9aa7ab8147 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:04.413331 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec127a-3f89-447b-9ab6-5ccb77d9782a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.424382 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:04.451519 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3b15e-6ae1-4c65-933b-6ab59fcbe6f7 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.487185 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.487831 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.488117 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.488403 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.489215 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] RBAC: Adding query filter params (group.id=28be246b5d0c408da6b57009c6b857a5, role.id=844a46aa3ad0499fb8bfa2b1bd023ddd, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:04.489627 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:04.491078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:04.497385 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 221/439] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/role_assignments?group.id=28be246b5d0c408da6b57009c6b857a5&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=844a46aa3ad0499fb8bfa2b1bd023ddd => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:04.498039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.527486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec127a-3f89-447b-9ab6-5ccb77d9782a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.576081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffe8d4-83f5-40d2-a6b5-5bbbb9e24b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.577271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe8d4-83f5-40d2-a6b5-5bbbb9e24b7b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.577559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe8d4-83f5-40d2-a6b5-5bbbb9e24b7b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.578140 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe8d4-83f5-40d2-a6b5-5bbbb9e24b7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.591872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffe8d4-83f5-40d2-a6b5-5bbbb9e24b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-7bffe8d4-83f5-40d2-a6b5-5bbbb9e24b7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:04.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:04.598581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:04.599625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 219/440] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:04.600487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c212ac8-0d34-4d4d-b0f5-7ac82f1bf633 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.600487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c212ac8-0d34-4d4d-b0f5-7ac82f1bf633 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.600487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c212ac8-0d34-4d4d-b0f5-7ac82f1bf633 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.606205 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 220/441] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:04.607708 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c212ac8-0d34-4d4d-b0f5-7ac82f1bf633 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.628964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a7c2c-0c95-4ada-b58f-d6ed362a8a7c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.637548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:04.668353 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d90f4-7817-49e8-94fd-94c5eb1cd6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.669115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d90f4-7817-49e8-94fd-94c5eb1cd6dc None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.669403 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d90f4-7817-49e8-94fd-94c5eb1cd6dc None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.669683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d90f4-7817-49e8-94fd-94c5eb1cd6dc None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.670929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d90f4-7817-49e8-94fd-94c5eb1cd6dc None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:04.672562 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d90f4-7817-49e8-94fd-94c5eb1cd6dc None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:04.681483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 222/442] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:04.681483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098007d-38bc-4d51-808a-c7c0c096984b None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.681483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098007d-38bc-4d51-808a-c7c0c096984b None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.681483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098007d-38bc-4d51-808a-c7c0c096984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.703328 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-f098007d-38bc-4d51-808a-c7c0c096984b None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:04.704626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-f098007d-38bc-4d51-808a-c7c0c096984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:04.708195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:04.733926 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c212ac8-0d34-4d4d-b0f5-7ac82f1bf633 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.772385 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.773045 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.773329 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.773606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.774382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:04.774701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:04.776142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:04.785977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 221/443] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:04.789381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.789574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 223/444] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:04.791942 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098007d-38bc-4d51-808a-c7c0c096984b None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.889685 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:04.906178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279bcd7-b4b2-4a35-bc03-96789fa7650a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.916059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098007d-38bc-4d51-808a-c7c0c096984b None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:04.940320 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.940957 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.941597 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.941872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.942711 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] RBAC: Adding query filter params (group.id=28be246b5d0c408da6b57009c6b857a5, role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:04.943048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:04.944521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:04.951478 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 222/445] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/role_assignments?group.id=28be246b5d0c408da6b57009c6b857a5&scope.project.id=c49c1fbd83f844dcacce2b9aa7ab8147&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:04.952203 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:04.968726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1af259-76ef-4bb8-b4eb-e4b077aa86aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:04.969579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1af259-76ef-4bb8-b4eb-e4b077aa86aa None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:04.969855 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1af259-76ef-4bb8-b4eb-e4b077aa86aa None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:04.970169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1af259-76ef-4bb8-b4eb-e4b077aa86aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:04.985080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:05.066728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1af259-76ef-4bb8-b4eb-e4b077aa86aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:05.069413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1af259-76ef-4bb8-b4eb-e4b077aa86aa None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:05.094029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954ca7b-c0fb-43eb-9700-c52dbce1ee17 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:05.136154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bdc69-5032-4322-8f5a-5dcc8c97527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:05.136827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bdc69-5032-4322-8f5a-5dcc8c97527e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:05.137109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bdc69-5032-4322-8f5a-5dcc8c97527e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:05.137385 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bdc69-5032-4322-8f5a-5dcc8c97527e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:05.149903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bdc69-5032-4322-8f5a-5dcc8c97527e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-496bdc69-5032-4322-8f5a-5dcc8c97527e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:05.155504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:05.156855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:05.157913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 223/446] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:05.162941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bdc69-5032-4322-8f5a-5dcc8c97527e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:05.176194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:05.201253 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 224/447] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:05.207635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1af259-76ef-4bb8-b4eb-e4b077aa86aa None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:05.264440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:05.297069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bdc69-5032-4322-8f5a-5dcc8c97527e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:05.336895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:05.337548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:05.337828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:05.338133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:05.338872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:05.339178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:05.340614 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:05.340805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1af259-76ef-4bb8-b4eb-e4b077aa86aa None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:05.353779 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 224/448] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:05.359601 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:05.390010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c5603-3c29-456c-854c-88d5908de723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:05.390758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c5603-3c29-456c-854c-88d5908de723 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:05.391224 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c5603-3c29-456c-854c-88d5908de723 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:05.391390 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c5603-3c29-456c-854c-88d5908de723 None admin] PATH_INFO: `/v3/projects/96c6511840d04bcc8f165d1799252c4d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:05.392640 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c5603-3c29-456c-854c-88d5908de723 None admin] RBAC: Authorizing `identity:get_project(project_id=96c6511840d04bcc8f165d1799252c4d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:05.394246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c5603-3c29-456c-854c-88d5908de723 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:05.396251 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 225/449] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:56:05 2022] GET /identity/v3/projects/96c6511840d04bcc8f165d1799252c4d => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:05.401840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c5603-3c29-456c-854c-88d5908de723 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:05.452350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:05.489893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df2d7-8e67-466c-92c4-3c73d26586bb None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:05.521895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c5603-3c29-456c-854c-88d5908de723 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:05.528025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa47a-ecb7-4766-ba1b-fa623a3de0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:05.528706 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa47a-ecb7-4766-ba1b-fa623a3de0ed None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:05.528993 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa47a-ecb7-4766-ba1b-fa623a3de0ed None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:05.529266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa47a-ecb7-4766-ba1b-fa623a3de0ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:05.531961 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:05.557462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1540134-eaeb-42ce-a86a-b4e81d6a8998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:05.558150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1540134-eaeb-42ce-a86a-b4e81d6a8998 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:05.558439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1540134-eaeb-42ce-a86a-b4e81d6a8998 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:05.558693 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1540134-eaeb-42ce-a86a-b4e81d6a8998 None admin] PATH_INFO: `/v3/groups/28be246b5d0c408da6b57009c6b857a5` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:05.559882 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1540134-eaeb-42ce-a86a-b4e81d6a8998 None admin] RBAC: Authorizing `identity:get_group(group_id=28be246b5d0c408da6b57009c6b857a5)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:05.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1540134-eaeb-42ce-a86a-b4e81d6a8998 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:05.563564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 226/450] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:56:05 2022] GET /identity/v3/groups/28be246b5d0c408da6b57009c6b857a5 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:05.569491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1540134-eaeb-42ce-a86a-b4e81d6a8998 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:05.600435 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fa47a-ecb7-4766-ba1b-fa623a3de0ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:05.601949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fa47a-ecb7-4766-ba1b-fa623a3de0ed None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:05.670204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:05.671170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1540134-eaeb-42ce-a86a-b4e81d6a8998 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:05.705169 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 225/451] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:05.708640 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:05.709263 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:05.709543 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:05.709834 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:05.710654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] RBAC: Adding query filter params (group.id=28be246b5d0c408da6b57009c6b857a5, role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=96c6511840d04bcc8f165d1799252c4d) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:05.710964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:05.711104 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa47a-ecb7-4766-ba1b-fa623a3de0ed None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:05.712448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:05.718564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 227/452] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:56:05 2022] GET /identity/v3/role_assignments?group.id=28be246b5d0c408da6b57009c6b857a5&scope.project.id=96c6511840d04bcc8f165d1799252c4d&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:05.823482 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa47a-ecb7-4766-ba1b-fa623a3de0ed None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:05.862791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e360c-7296-4283-99a9-c175c136e374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:05.863501 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e360c-7296-4283-99a9-c175c136e374 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:05.863780 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e360c-7296-4283-99a9-c175c136e374 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:05.864052 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e360c-7296-4283-99a9-c175c136e374 None admin] PATH_INFO: `/v3/projects/9904f995f8d54e70b75420b12eb6d746` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:05.865480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e360c-7296-4283-99a9-c175c136e374 None admin] RBAC: Authorizing `identity:get_project(project_id=9904f995f8d54e70b75420b12eb6d746)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:05.867047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e360c-7296-4283-99a9-c175c136e374 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:05.868862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 226/453] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:56:05 2022] GET /identity/v3/projects/9904f995f8d54e70b75420b12eb6d746 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:05.875302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:05.929341 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:06.002241 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502edbc5-901b-4e11-b737-bf82f75003d6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:06.044864 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740576d1-2d4f-4cb3-b8b5-4c24952fcf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:06.045553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740576d1-2d4f-4cb3-b8b5-4c24952fcf83 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:06.045840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740576d1-2d4f-4cb3-b8b5-4c24952fcf83 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:06.046143 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740576d1-2d4f-4cb3-b8b5-4c24952fcf83 None admin] PATH_INFO: `/v3/groups/f61881c55911434f98606cf3554253d8` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:06.047580 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740576d1-2d4f-4cb3-b8b5-4c24952fcf83 None admin] RBAC: Authorizing `identity:get_group(group_id=f61881c55911434f98606cf3554253d8)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:06.049124 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740576d1-2d4f-4cb3-b8b5-4c24952fcf83 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:06.050929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 228/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 08:56:05 2022] GET /identity/v3/groups/f61881c55911434f98606cf3554253d8 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:06.057200 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e360c-7296-4283-99a9-c175c136e374 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:06.105218 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:06.171520 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e360c-7296-4283-99a9-c175c136e374 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:06.216734 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d184ec-ab3f-474d-b000-dcec7ddf4331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:06.217275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d184ec-ab3f-474d-b000-dcec7ddf4331 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:06.217555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d184ec-ab3f-474d-b000-dcec7ddf4331 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:06.217821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d184ec-ab3f-474d-b000-dcec7ddf4331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:06.218658 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d184ec-ab3f-474d-b000-dcec7ddf4331 None admin] RBAC: Adding query filter params (group.id=f61881c55911434f98606cf3554253d8, role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=9904f995f8d54e70b75420b12eb6d746) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:06.218943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d184ec-ab3f-474d-b000-dcec7ddf4331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:06.220485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d184ec-ab3f-474d-b000-dcec7ddf4331 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:06.226403 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 227/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 08:56:06 2022] GET /identity/v3/role_assignments?group.id=f61881c55911434f98606cf3554253d8&scope.project.id=9904f995f8d54e70b75420b12eb6d746&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:08.752472 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bc8a7-c9b6-48ab-9c98-018afb03ea11 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:08.753075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bc8a7-c9b6-48ab-9c98-018afb03ea11 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:08.753075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bc8a7-c9b6-48ab-9c98-018afb03ea11 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:08.754777 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 229/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:08.760466 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70b0a-b7ae-47c6-bc98-14f79089f9e6 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:08.760964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70b0a-b7ae-47c6-bc98-14f79089f9e6 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:08.761083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70b0a-b7ae-47c6-bc98-14f79089f9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:08.787193 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-10b70b0a-b7ae-47c6-bc98-14f79089f9e6 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:08.788453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-10b70b0a-b7ae-47c6-bc98-14f79089f9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:08.876881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf83dd-b407-4f2d-99a6-5fc1e0fb8a2e None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:08.877263 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf83dd-b407-4f2d-99a6-5fc1e0fb8a2e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:08.877464 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf83dd-b407-4f2d-99a6-5fc1e0fb8a2e None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:08.879165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 230/457] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:08.884125 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290a330-74c2-40b1-afcc-75fd25cb0fb4 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:08.884416 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290a330-74c2-40b1-afcc-75fd25cb0fb4 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:08.884696 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290a330-74c2-40b1-afcc-75fd25cb0fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:08.895596 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 228/458] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:08.915279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-4290a330-74c2-40b1-afcc-75fd25cb0fb4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:08.916508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-4290a330-74c2-40b1-afcc-75fd25cb0fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:08.918346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b70b0a-b7ae-47c6-bc98-14f79089f9e6 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:08.969279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:09.018259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 231/459] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:56:09.020012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163fd9e-fea2-4d42-8f91-ee510fa8a4b0 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.020557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163fd9e-fea2-4d42-8f91-ee510fa8a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.021048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163fd9e-fea2-4d42-8f91-ee510fa8a4b0 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.023793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 232/460] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:09.039953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ea56b-3abc-4e42-bc09-c4c74b384102 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.043052 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ea56b-3abc-4e42-bc09-c4c74b384102 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.043555 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ea56b-3abc-4e42-bc09-c4c74b384102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.067247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-e90ea56b-3abc-4e42-bc09-c4c74b384102 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:09.068863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-e90ea56b-3abc-4e42-bc09-c4c74b384102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:09.075977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b70b0a-b7ae-47c6-bc98-14f79089f9e6 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:09.114138 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:09.122296 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:09.123190 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.123720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.124233 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.138459 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:09.142307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:09.160837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:09.162015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:09.163154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:09.163154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:09.163154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:09.163154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:09.170150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 233/462] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:09.170150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ea56b-3abc-4e42-bc09-c4c74b384102 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:09.171098 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 229/462] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:09.175593 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:09.306702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ea56b-3abc-4e42-bc09-c4c74b384102 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:09.312615 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:09.335302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8462726-8eb5-41c0-9e6a-7b5e783b182f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:09.346203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:09.346483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.346857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.347204 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.358955 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:09.360675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:09.377362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:09.378612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:09.379722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:09.379722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:09.379722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:09.379722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:09.382194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 234/463] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:09.384400 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:09.385304 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:09.387048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.387404 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.387774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.388572 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:09.388951 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:09.391977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:09.414152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 230/464] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:09.414152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:09.420448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:09.513815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b0485-50e9-4882-9978-d9e4b2082113 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:09.531031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc8b7f-6ba8-46bc-9835-0dd5d717f542 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:09.550335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bfccf-a9ae-43aa-a29a-c26760f98935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:09.550994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bfccf-a9ae-43aa-a29a-c26760f98935 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.551282 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bfccf-a9ae-43aa-a29a-c26760f98935 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.551563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bfccf-a9ae-43aa-a29a-c26760f98935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.563311 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bfccf-a9ae-43aa-a29a-c26760f98935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:09.564758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bfccf-a9ae-43aa-a29a-c26760f98935 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:09.569011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:09.569753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.570118 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.570411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.571218 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:09.571527 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:09.573739 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-e08bfccf-a9ae-43aa-a29a-c26760f98935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:09.583563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:09.584847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:09.585959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:09.585959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:09.585959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:09.585959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:09.585959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 235/465] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:09.586324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ded1a-31e0-4b05-a564-be834ea2767e None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.586494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ded1a-31e0-4b05-a564-be834ea2767e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.586970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ded1a-31e0-4b05-a564-be834ea2767e None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.588260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 236/466] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:09.593117 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ded1a-31e0-4b05-a564-be834ea2767e None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:09.593678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 231/467] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:09.598897 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:09.602846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:09.675283 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:09.710501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ded1a-31e0-4b05-a564-be834ea2767e None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:09.711572 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c25fd-1cd0-4a95-9f91-9808b0a06dc1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:09.745877 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3060e-ad0a-4d17-8de1-fbf6d4352d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:09.746556 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3060e-ad0a-4d17-8de1-fbf6d4352d51 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.746848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3060e-ad0a-4d17-8de1-fbf6d4352d51 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.747133 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3060e-ad0a-4d17-8de1-fbf6d4352d51 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.747839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3060e-ad0a-4d17-8de1-fbf6d4352d51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:09.749327 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e2748-6802-455a-b38f-34b9df42c73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:09.749493 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3060e-ad0a-4d17-8de1-fbf6d4352d51 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:09.749980 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e2748-6802-455a-b38f-34b9df42c73c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.750324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e2748-6802-455a-b38f-34b9df42c73c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.750608 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e2748-6802-455a-b38f-34b9df42c73c None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.751318 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e2748-6802-455a-b38f-34b9df42c73c None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:09.751628 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e2748-6802-455a-b38f-34b9df42c73c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:09.753129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e2748-6802-455a-b38f-34b9df42c73c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:09.767333 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 232/468] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:09.772443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180938a1-089e-45ab-975b-947ab5ca724b None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.772443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180938a1-089e-45ab-975b-947ab5ca724b None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.772443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180938a1-089e-45ab-975b-947ab5ca724b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.776865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 237/469] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:56:09 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:09.778210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3060e-ad0a-4d17-8de1-fbf6d4352d51 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:09.806793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-180938a1-089e-45ab-975b-947ab5ca724b None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:09.808255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-180938a1-089e-45ab-975b-947ab5ca724b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:09.900584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:09.912368 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 233/470] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:09.913261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180938a1-089e-45ab-975b-947ab5ca724b None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:09.925221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3060e-ad0a-4d17-8de1-fbf6d4352d51 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:09.972119 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:09.983007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480cdf5-c0a4-478f-898a-2a1b4ba4da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:09.983930 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480cdf5-c0a4-478f-898a-2a1b4ba4da94 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:09.984396 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480cdf5-c0a4-478f-898a-2a1b4ba4da94 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:09.984937 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480cdf5-c0a4-478f-898a-2a1b4ba4da94 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:09.986574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480cdf5-c0a4-478f-898a-2a1b4ba4da94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:09.987510 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480cdf5-c0a4-478f-898a-2a1b4ba4da94 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:10.015371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 238/471] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:56:09 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:10.016770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480cdf5-c0a4-478f-898a-2a1b4ba4da94 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:10.059091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180938a1-089e-45ab-975b-947ab5ca724b None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:10.116099 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e6f65-686a-4868-a544-95e6d6fb19ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:10.116835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6f65-686a-4868-a544-95e6d6fb19ce None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.117123 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6f65-686a-4868-a544-95e6d6fb19ce None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.117400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6f65-686a-4868-a544-95e6d6fb19ce None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.118135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e6f65-686a-4868-a544-95e6d6fb19ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:10.119669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e6f65-686a-4868-a544-95e6d6fb19ce None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:10.153266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 234/472] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:56:09 2022] POST /identity/v3/users => generated 246 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:10.162074 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480cdf5-c0a4-478f-898a-2a1b4ba4da94 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:10.164827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f095e-f26e-4707-a345-c98f5fa74fe9 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.165121 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f095e-f26e-4707-a345-c98f5fa74fe9 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.165405 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f095e-f26e-4707-a345-c98f5fa74fe9 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.168495 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 235/473] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:10.174389 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53088540-fbab-4234-8d97-112a499b93bc None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.175093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53088540-fbab-4234-8d97-112a499b93bc None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.175581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53088540-fbab-4234-8d97-112a499b93bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.216998 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-53088540-fbab-4234-8d97-112a499b93bc None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:10.218390 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-53088540-fbab-4234-8d97-112a499b93bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:10.224896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:10.225581 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.225867 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.226186 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.247322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:10.249866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:10.269350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:10.270557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:10.271698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:10.271698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:10.271698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:10.271698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:10.271698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 239/474] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:10.278687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:10.294613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:10.326157 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:10.328140 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 236/475] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:10.352476 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53088540-fbab-4234-8d97-112a499b93bc None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:10.423149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f02c9-ad89-44bd-a32a-b0e505c5f860 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:10.475563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:10.476213 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.476500 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.476782 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.477502 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:10.477809 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:10.479420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:10.497913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53088540-fbab-4234-8d97-112a499b93bc None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:10.499491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 240/476] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:10.501181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:10.541512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:10.542239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.542528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.542807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.543547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:10.544977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:10.555528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.sql.core [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:56:10.555528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:56:10.555528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [parameters: {'id': 'bb6681a2a2eb4b35b215d774318846c6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:56:10.555528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=347146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application result.read() Jul 28 08:56:10.576707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:56:10.577920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application yield resource Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise self.value Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application session.commit() Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:56:10.579373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:56:10.580503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:56:10.581637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application result.read() Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:56:10.584399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [parameters: {'id': 'bb6681a2a2eb4b35b215d774318846c6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:10.586770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:56:10.595551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:10.597846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 237/477] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:56:10 2022] POST /identity/v3/roles => generated 144 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:10.597846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:10.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:10.675678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:10.693400 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753496c-9f25-4691-9199-85d5a3fefd89 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:10.714138 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59ab09-9d23-419e-853c-0d221156c4bd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:10.750750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d067b8-e26a-48a6-9350-ea62cfcff4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:10.751477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d067b8-e26a-48a6-9350-ea62cfcff4d4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.751755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d067b8-e26a-48a6-9350-ea62cfcff4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.752033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d067b8-e26a-48a6-9350-ea62cfcff4d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.764319 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d067b8-e26a-48a6-9350-ea62cfcff4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-82d067b8-e26a-48a6-9350-ea62cfcff4d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:10.770124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:10.771627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:10.772660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 238/478] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:10.778563 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d067b8-e26a-48a6-9350-ea62cfcff4d4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:10.792050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34997d-dc53-49d2-ac97-743beeba6747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:10.792795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34997d-dc53-49d2-ac97-743beeba6747 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.793094 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34997d-dc53-49d2-ac97-743beeba6747 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.793380 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34997d-dc53-49d2-ac97-743beeba6747 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.794084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34997d-dc53-49d2-ac97-743beeba6747 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:10.795615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34997d-dc53-49d2-ac97-743beeba6747 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:10.847630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 241/479] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:56:10 2022] POST /identity/v3/users => generated 249 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:56:10.893706 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d067b8-e26a-48a6-9350-ea62cfcff4d4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:10.896812 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:10.933846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0becc-d910-488f-adc3-05bf7b9875ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:10.934442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0becc-d910-488f-adc3-05bf7b9875ae None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:10.934726 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0becc-d910-488f-adc3-05bf7b9875ae None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:10.935004 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0becc-d910-488f-adc3-05bf7b9875ae None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:10.935744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0becc-d910-488f-adc3-05bf7b9875ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:10.936066 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0becc-d910-488f-adc3-05bf7b9875ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:10.937547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0becc-d910-488f-adc3-05bf7b9875ae None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:10.948387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 239/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:12.689371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05140e16-89d2-461c-ba77-16b30fb02f93 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:12.690028 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05140e16-89d2-461c-ba77-16b30fb02f93 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:12.690028 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05140e16-89d2-461c-ba77-16b30fb02f93 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:12.692195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 242/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:12.697747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe0f1c-fb71-4f86-9afb-3e163d48ce8e None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:12.698322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe0f1c-fb71-4f86-9afb-3e163d48ce8e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:12.698322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe0f1c-fb71-4f86-9afb-3e163d48ce8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:12.720012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de7c6-789e-4ae4-8265-772890bd3297 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:12.720279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de7c6-789e-4ae4-8265-772890bd3297 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:12.720574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de7c6-789e-4ae4-8265-772890bd3297 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:12.722375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 243/482] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:56:12.727282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-0cfe0f1c-fb71-4f86-9afb-3e163d48ce8e None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:12.728572 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-0cfe0f1c-fb71-4f86-9afb-3e163d48ce8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:12.733506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802e087-2016-4868-a9d8-a727ba24193c None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:12.733864 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802e087-2016-4868-a9d8-a727ba24193c None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:12.734085 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802e087-2016-4868-a9d8-a727ba24193c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:12.771427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-a802e087-2016-4868-a9d8-a727ba24193c None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:12.772379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-a802e087-2016-4868-a9d8-a727ba24193c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:12.810590 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 240/483] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:12.812313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695434be-910c-4c99-a6b0-34fb80864af6 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:12.812871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695434be-910c-4c99-a6b0-34fb80864af6 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:12.813445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695434be-910c-4c99-a6b0-34fb80864af6 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:12.815350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 241/484] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:12.827670 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515919a-5a58-4b96-80e2-9659a7bca08f None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:12.828479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515919a-5a58-4b96-80e2-9659a7bca08f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:12.828976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515919a-5a58-4b96-80e2-9659a7bca08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:12.850220 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-c515919a-5a58-4b96-80e2-9659a7bca08f None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:12.851680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-c515919a-5a58-4b96-80e2-9659a7bca08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:12.852618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 244/485] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:12.853548 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802e087-2016-4868-a9d8-a727ba24193c None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:12.943389 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 242/486] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:56:12.944184 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515919a-5a58-4b96-80e2-9659a7bca08f None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:12.949806 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:12.974518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802e087-2016-4868-a9d8-a727ba24193c None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:12.983685 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:13.019116 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cf339-11eb-4bdd-921c-0f7d7096db61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.019785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cf339-11eb-4bdd-921c-0f7d7096db61 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.020069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cf339-11eb-4bdd-921c-0f7d7096db61 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.020347 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cf339-11eb-4bdd-921c-0f7d7096db61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.031637 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cf339-11eb-4bdd-921c-0f7d7096db61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-573cf339-11eb-4bdd-921c-0f7d7096db61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:13.039148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:13.040324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 245/487] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:13.041375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cf339-11eb-4bdd-921c-0f7d7096db61 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.084182 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515919a-5a58-4b96-80e2-9659a7bca08f None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.131334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cab8ee-edea-49b8-9c52-51141f76e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.132741 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab8ee-edea-49b8-9c52-51141f76e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.133179 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab8ee-edea-49b8-9c52-51141f76e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.134234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab8ee-edea-49b8-9c52-51141f76e3b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.148586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cab8ee-edea-49b8-9c52-51141f76e3b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:13.152745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cf339-11eb-4bdd-921c-0f7d7096db61 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d4cab8ee-edea-49b8-9c52-51141f76e3b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:13.154824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:13.157450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 243/488] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:13.159790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cab8ee-edea-49b8-9c52-51141f76e3b6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.192256 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b786b9-2fe2-40fc-86dc-c4cf8f6bbb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.192928 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b786b9-2fe2-40fc-86dc-c4cf8f6bbb1b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.193210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b786b9-2fe2-40fc-86dc-c4cf8f6bbb1b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.193548 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b786b9-2fe2-40fc-86dc-c4cf8f6bbb1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.204967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b786b9-2fe2-40fc-86dc-c4cf8f6bbb1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-94b786b9-2fe2-40fc-86dc-c4cf8f6bbb1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:13.210080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:13.212589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 246/489] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:13.214779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b786b9-2fe2-40fc-86dc-c4cf8f6bbb1b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.223211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:13.267688 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:13.293067 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cab8ee-edea-49b8-9c52-51141f76e3b6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.321658 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b786b9-2fe2-40fc-86dc-c4cf8f6bbb1b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.336851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.337615 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.337907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.338221 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.338953 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:13.339267 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:13.340912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:13.355290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 244/490] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:13.356373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.359696 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.360353 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.360640 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.360916 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.361667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:13.361974 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:13.363456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:13.373141 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 247/491] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:13.374087 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.440563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:13.483275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05403d23-1892-4197-aaf7-66d2517dc7b7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.487034 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db308c7-6924-4357-a426-893815f795f6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.522791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.523890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.524180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.524726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.528867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.529614 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.529996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.530392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.531225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:13.531697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:13.533299 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:13.537808 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:13.539894 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:13.548060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 245/492] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:13.549232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:13.557732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:13.559136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:13.560481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:13.560481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:13.560481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:13.560481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:13.560481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 248/493] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:13.560481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.566897 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:13.688936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9e4a7-a32f-461a-ba5d-39a5ba57f6dc None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.695490 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a4974-5cd6-42d7-bb6e-d685f26dca2f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.715835 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:13.733437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.734037 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.734324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.734654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.736146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec41c4-74f8-4f62-a01c-bb4faf95356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.736808 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec41c4-74f8-4f62-a01c-bb4faf95356c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.737090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec41c4-74f8-4f62-a01c-bb4faf95356c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.737365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec41c4-74f8-4f62-a01c-bb4faf95356c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.746701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:13.748230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:13.749159 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec41c4-74f8-4f62-a01c-bb4faf95356c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:13.750967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec41c4-74f8-4f62-a01c-bb4faf95356c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:13.764284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:13.765721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:13.767006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:13.767006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:13.767006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:13.767006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:13.767006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 246/494] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:13.767006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1fec41c4-74f8-4f62-a01c-bb4faf95356c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:13.767481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:13.768538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:13.769824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:13.769824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:13.769824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:13.769824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:13.769824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 249/495] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:13.769824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ed3e-760a-47e9-a11b-2cf2391d6d90 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.769824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ed3e-760a-47e9-a11b-2cf2391d6d90 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.769824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ed3e-760a-47e9-a11b-2cf2391d6d90 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.775563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 250/496] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:13.775563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dcd6d-0e35-4f4f-a8d2-373f8e491d2f None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.775563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dcd6d-0e35-4f4f-a8d2-373f8e491d2f None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.775563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dcd6d-0e35-4f4f-a8d2-373f8e491d2f None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.775563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 251/497] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:13.775563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dcd6d-0e35-4f4f-a8d2-373f8e491d2f None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.846906 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:13.885796 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dcd6d-0e35-4f4f-a8d2-373f8e491d2f None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.886461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d8a3-bc02-4d03-8e8e-bc79912ff2f1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:13.920664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.920931 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cf783-179a-4528-80bd-94d52fd02dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:13.921294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.921423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cf783-179a-4528-80bd-94d52fd02dee None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.921593 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.921709 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cf783-179a-4528-80bd-94d52fd02dee None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.921868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.921985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cf783-179a-4528-80bd-94d52fd02dee None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.922601 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:13.922728 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cf783-179a-4528-80bd-94d52fd02dee None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:13.922886 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:13.922990 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cf783-179a-4528-80bd-94d52fd02dee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:13.924310 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:13.924439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cf783-179a-4528-80bd-94d52fd02dee None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:13.936794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 252/498] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:13.937178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 247/499] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:13.938381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:13.938604 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a60369-44ff-4b3b-86a4-30a09595b3a6 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:13.938879 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a60369-44ff-4b3b-86a4-30a09595b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:13.939155 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a60369-44ff-4b3b-86a4-30a09595b3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:13.957315 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-a6a60369-44ff-4b3b-86a4-30a09595b3a6 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:13.958580 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-a6a60369-44ff-4b3b-86a4-30a09595b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:13.970692 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.029126 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 248/500] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:14.032766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e00c8a-3928-4c5a-91ce-b06823f8fc48 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.032766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e00c8a-3928-4c5a-91ce-b06823f8fc48 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.032766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e00c8a-3928-4c5a-91ce-b06823f8fc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.051202 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-a6e00c8a-3928-4c5a-91ce-b06823f8fc48 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:14.052428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-a6e00c8a-3928-4c5a-91ce-b06823f8fc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:14.053433 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90708a07-8cbc-4747-8ea1-b2d9cfcf1a22 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.092347 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.093023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.093306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.093577 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.094346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:14.095066 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.096484 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.102006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.109210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 253/501] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:14.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.123648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 249/502] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:14.124510 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e00c8a-3928-4c5a-91ce-b06823f8fc48 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.205786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.221259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7d08f-0901-4c42-adb6-05a50bc67ecc None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.237901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e00c8a-3928-4c5a-91ce-b06823f8fc48 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.255668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.256330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.256617 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.256898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] PATH_INFO: `/v3/users/nova` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.264975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.267058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.274649 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.275320 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.275610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.275890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:14.280199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:14.281467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 254/503] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:14.282736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.284151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.285449 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:14.299232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:14.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 250/504] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:14.301582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.349507 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.389475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dd833-b018-4cde-92b5-764ac5a5e786 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.409720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bca67-ce7b-47ef-b4a9-bdbd671850d9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.423293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03357f21-6d63-417b-9561-70b7714e839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.423962 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03357f21-6d63-417b-9561-70b7714e839c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.424238 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03357f21-6d63-417b-9561-70b7714e839c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.424514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03357f21-6d63-417b-9561-70b7714e839c None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.436845 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03357f21-6d63-417b-9561-70b7714e839c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-03357f21-6d63-417b-9561-70b7714e839c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:14.442899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:14.444063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 255/505] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:14.445095 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03357f21-6d63-417b-9561-70b7714e839c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.452398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.453174 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.453467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.453744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] PATH_INFO: `/v3/users/glance` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.460013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.461970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.463685 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:14.475938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:14.477139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 251/506] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:14.478456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.556609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03357f21-6d63-417b-9561-70b7714e839c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.583522 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3f878-11d9-4d5a-9de2-0071cd749857 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.585106 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.591686 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.592339 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.592627 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.592908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.605292 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.606883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.622156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.622815 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:14.623041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:14.624194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:14.625560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:14.625560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:14.625560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:14.625560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:14.625866 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.625866 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.625866 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:14.626195 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.627652 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.629276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 256/507] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:14.630226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.669128 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 252/508] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:14.670326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.701363 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.744253 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1d262-b5b2-4bd8-95ab-e1ac06073a54 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.774303 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eba1b-bb64-45ea-b61b-e18d2de25394 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.778890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.779530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.779812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.780124 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.780941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:14.781252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.782865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.809587 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.810152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.810479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.810730 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.811467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:14.811774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.813206 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.821715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 257/509] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:14.822123 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 253/510] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:14.822611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.823036 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.848328 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.922630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b11202-c356-417b-8f2c-00febeaad5a4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.924063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d6a14-4ffb-44e0-8708-4b3bd063ada7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:14.941492 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:14.959147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.959787 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.960071 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.960346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.960971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:14.961098 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:14.961344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.961795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:14.962172 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:14.962520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:14.962790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.963466 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:14.963850 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:14.965409 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:14.974933 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 254/511] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:14.975766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:14.984364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 258/512] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:14.985598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:15.064292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:15.086126 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0e8c-2f9a-4c48-9af7-30cf9ba2a9f8 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:15.093595 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880951a-95b2-4419-8c8d-fcfb8b1facb4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:15.123710 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c969b18-7302-4df5-9a97-8967ca9c8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:15.124358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c969b18-7302-4df5-9a97-8967ca9c8037 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:15.124645 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c969b18-7302-4df5-9a97-8967ca9c8037 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:15.124919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c969b18-7302-4df5-9a97-8967ca9c8037 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:15.129977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e7c10-b387-4b48-bb25-510891a7ea95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:15.130641 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e7c10-b387-4b48-bb25-510891a7ea95 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:15.130924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e7c10-b387-4b48-bb25-510891a7ea95 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:15.131206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e7c10-b387-4b48-bb25-510891a7ea95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:15.179860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:15.197814 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c969b18-7302-4df5-9a97-8967ca9c8037 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:15.199389 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c969b18-7302-4df5-9a97-8967ca9c8037 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:15.204592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e7c10-b387-4b48-bb25-510891a7ea95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:15.206202 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e7c10-b387-4b48-bb25-510891a7ea95 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:15.283031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:15.303532 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 255/513] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:15.304420 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c969b18-7302-4df5-9a97-8967ca9c8037 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:15.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 259/514] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:15.309991 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e7c10-b387-4b48-bb25-510891a7ea95 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:15.386825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:15.405778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c969b18-7302-4df5-9a97-8967ca9c8037 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:15.411478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e7c10-b387-4b48-bb25-510891a7ea95 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:15.441884 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:15.442587 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:15.442858 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:15.443129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:15.447497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbe88f-7cd0-40b0-a1e4-53fa7bdce0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:15.448120 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbe88f-7cd0-40b0-a1e4-53fa7bdce0fe None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:15.448397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbe88f-7cd0-40b0-a1e4-53fa7bdce0fe None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:15.448671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbe88f-7cd0-40b0-a1e4-53fa7bdce0fe None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:15.449349 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbe88f-7cd0-40b0-a1e4-53fa7bdce0fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:15.450873 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbe88f-7cd0-40b0-a1e4-53fa7bdce0fe None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:15.453729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:15.455268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:15.471111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:15.472396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:15.473538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:15.473538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:15.473538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:15.473538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:15.473538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 256/516] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:15.473888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 260/516] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:56:15 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:15.473988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:15.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbe88f-7cd0-40b0-a1e4-53fa7bdce0fe None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:15.519666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:15.607826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbe88f-7cd0-40b0-a1e4-53fa7bdce0fe None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:15.624820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29132b-c543-48b9-bef6-7b73acf3c8d7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:15.643427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:15.652574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:15.653261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:15.653571 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:15.653981 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:15.668261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:15.669804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:15.683906 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23c0c-00b2-44b2-bcc3-ae78058eff45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:15.684693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23c0c-00b2-44b2-bcc3-ae78058eff45 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:15.685032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23c0c-00b2-44b2-bcc3-ae78058eff45 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:15.685388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23c0c-00b2-44b2-bcc3-ae78058eff45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:15.689839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:15.691422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:15.692530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:15.692530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:15.692530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:15.692530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:15.692530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 261/517] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:15.692865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:15.763325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d23c0c-00b2-44b2-bcc3-ae78058eff45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:15.764810 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d23c0c-00b2-44b2-bcc3-ae78058eff45 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:15.805521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:15.836017 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afdaef-0111-487b-a541-3b4f1c9649e0 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:15.872646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:15.873368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:15.873659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:15.874169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:15.881021 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 257/518] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:15.881716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23c0c-00b2-44b2-bcc3-ae78058eff45 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:15.891031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:15.892960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:15.908912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:15.910165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:15.911293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:15.911293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:15.911293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:15.911293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:15.911293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 262/519] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:15.911293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:15.950660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:15.996564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23c0c-00b2-44b2-bcc3-ae78058eff45 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.027506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e73d52-3026-4e7d-aa0e-046cbf8149e2 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.038574 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.039832 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.040589 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.041068 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.042179 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:16.043061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.045132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.047338 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:16.063421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.064052 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.064332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.064604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.065327 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:16.065630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.066569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 258/520] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:16.069815 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.069996 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.085024 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 263/521] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:16.085024 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.190611 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbace0f-6bcc-4ebd-a976-c71aeff02622 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.193867 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:16.200708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5e17e-5857-4f1a-a250-43bb592565bb None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.227584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.228326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.228659 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.228979 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.240818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.242372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.243220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.243865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.244422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.244704 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.245859 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:16.246189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.247623 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:16.258198 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:16.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:16.260416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:16.260416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:16.260416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:16.260416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:16.262564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 259/522] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:16.267949 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 264/523] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:16.267949 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.268192 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.321987 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:16.377528 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e1a7-a4e5-42e4-beff-6032bea64fc3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f00bd-f7a3-4b3c-bc25-8cb9c74c4324 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.413265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.414297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.414828 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.415103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.419001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.419674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.419958 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.420250 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.422394 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.423907 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.429835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.431853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:16.436432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:16.437515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 265/524] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:16.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:16.449350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:16.450531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:16.451788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 260/525] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:16.452529 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.465401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:16.557290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d50d3-73f4-4dc9-9fa0-7f817461287e None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.560987 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fc865-0ba9-4063-80e5-ecde10b1fc06 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.569539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:16.600450 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.600866 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233efb46-2291-4486-822b-c8214886f42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.601089 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.601375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.601484 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233efb46-2291-4486-822b-c8214886f42f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.601661 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.601785 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233efb46-2291-4486-822b-c8214886f42f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.601985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233efb46-2291-4486-822b-c8214886f42f None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.602391 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:16.602699 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.604322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.609732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233efb46-2291-4486-822b-c8214886f42f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.611255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233efb46-2291-4486-822b-c8214886f42f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.621833 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 266/526] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:16.621833 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-233efb46-2291-4486-822b-c8214886f42f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:16.629129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:16.630382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:16.631493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 261/527] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:16.632091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233efb46-2291-4486-822b-c8214886f42f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.710252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:16.734796 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48d2bf-0ca6-4d42-8ec5-0250c825b88c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.746865 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233efb46-2291-4486-822b-c8214886f42f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.772074 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.772739 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.773021 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.773297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.774091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:16.774430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.775987 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.787094 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.788048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.789249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.789747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.790832 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:16.791314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.793031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.797404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 267/528] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:16.802055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.816622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 262/529] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:16.817532 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:16.846653 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:16.929138 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf30a3-c573-4fdc-87d7-c09fb2d963b6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.933043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528b78c-2539-4e86-9d54-c9680f0afdf0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:16.969591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.970199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.970530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.970761 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.972485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:16.973280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:16.973564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:16.973852 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:16.974360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:16.974669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:16.974985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.976519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.979797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:16.982003 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:16.998612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:16.999803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:17.001007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:17.001007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:17.001007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:17.001007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:17.001007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:17.001007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:17.001007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:17.001007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:17.002473 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 268/530] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:17.003696 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.005117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 263/531] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:17.006444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.113548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:17.128635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15c18f-a039-476e-bf98-b4f5b09c55a4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:17.131825 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faa9c3-e4f2-4388-851a-04f09bd17c54 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:17.168748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175e163-0609-466c-a8d2-d78c9e6c5446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:17.169147 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:17.169517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175e163-0609-466c-a8d2-d78c9e6c5446 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.169697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.169820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175e163-0609-466c-a8d2-d78c9e6c5446 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.169978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.170082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175e163-0609-466c-a8d2-d78c9e6c5446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.170294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.171155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=4fe070acb9a84d90baaa3b08a4fd3191) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:17.171470 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:17.173403 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:17.180870 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 269/532] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/role_assignments?user.id=4fe070acb9a84d90baaa3b08a4fd3191&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:17.182603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.251011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175e163-0609-466c-a8d2-d78c9e6c5446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:17.252473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175e163-0609-466c-a8d2-d78c9e6c5446 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:17.269803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:17.329332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92cfad-736d-4390-ae54-a306bb2832d3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:17.348224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:17.368183 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 264/533] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:17.369042 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175e163-0609-466c-a8d2-d78c9e6c5446 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.381547 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:17.382356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.382710 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.383048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.383884 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:17.384197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:17.385692 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:17.399502 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 270/534] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:17.403650 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.499325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175e163-0609-466c-a8d2-d78c9e6c5446 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:17.533609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f759490-3a51-4e64-bc36-496eec2c9fd2 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:17.549845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:17.551234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.552036 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.552555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.554041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=021350d0e5bc4eb59b53166712073877) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:17.554613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:17.556622 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:17.557548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:17.565379 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 265/535] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/role_assignments?user.id=021350d0e5bc4eb59b53166712073877&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:17.567284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.573657 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaba474-dbb3-4289-aaf7-3e6565bc1c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:17.574371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaba474-dbb3-4289-aaf7-3e6565bc1c79 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.574651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaba474-dbb3-4289-aaf7-3e6565bc1c79 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.574926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaba474-dbb3-4289-aaf7-3e6565bc1c79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.587103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaba474-dbb3-4289-aaf7-3e6565bc1c79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:17.588699 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaba474-dbb3-4289-aaf7-3e6565bc1c79 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-bbaba474-dbb3-4289-aaf7-3e6565bc1c79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:17.612233 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:17.613415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:17.615013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:17.615013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:17.615013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:17.615013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:17.622195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 271/536] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:17.624259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2c4d3-a6ae-4b6e-a423-fd83b048f987 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.624832 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2c4d3-a6ae-4b6e-a423-fd83b048f987 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.625407 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2c4d3-a6ae-4b6e-a423-fd83b048f987 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.627536 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 272/537] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:17.628770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2c4d3-a6ae-4b6e-a423-fd83b048f987 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.633003 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:17.701217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aee161-b22f-4012-bd56-a5d5dbc63aef None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:17.741796 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81e454-6b10-4db4-b1bd-e46e39d72522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:17.742498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e454-6b10-4db4-b1bd-e46e39d72522 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.742766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e454-6b10-4db4-b1bd-e46e39d72522 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.743048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e454-6b10-4db4-b1bd-e46e39d72522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.744513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81e454-6b10-4db4-b1bd-e46e39d72522 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=b27b8079c98346efbc94b558e152da86) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:17.744833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81e454-6b10-4db4-b1bd-e46e39d72522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:17.746573 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81e454-6b10-4db4-b1bd-e46e39d72522 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:17.756381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 266/538] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/role_assignments?user.id=b27b8079c98346efbc94b558e152da86&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:17.756830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a5bbf-ecf4-4398-8f48-2899532807d3 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.757118 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a5bbf-ecf4-4398-8f48-2899532807d3 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.757508 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a5bbf-ecf4-4398-8f48-2899532807d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.764928 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2c4d3-a6ae-4b6e-a423-fd83b048f987 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:17.781526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-220a5bbf-ecf4-4398-8f48-2899532807d3 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:17.782834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-220a5bbf-ecf4-4398-8f48-2899532807d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:17.814057 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:17.814915 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:17.815208 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:17.815491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:17.816732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:17.817050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:17.819190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:17.836820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 273/539] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:17.842616 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.870279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 267/540] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:17.881969 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:17.887393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a5bbf-ecf4-4398-8f48-2899532807d3 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:17.942589 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:17.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe440-f906-4465-8aea-576941c915dd None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:18.012324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:18.013724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:18.014021 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:18.014346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:18.015583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a5bbf-ecf4-4398-8f48-2899532807d3 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:18.023644 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:18.025201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:18.042891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:18.044016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:18.045129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 274/541] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:18.052186 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:18.069367 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4359-5311-4b6b-bad5-f6f9f2c48fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:18.070035 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4359-5311-4b6b-bad5-f6f9f2c48fe1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:18.070355 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4359-5311-4b6b-bad5-f6f9f2c48fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:18.070633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4359-5311-4b6b-bad5-f6f9f2c48fe1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:18.091270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b4359-5311-4b6b-bad5-f6f9f2c48fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-e33b4359-5311-4b6b-bad5-f6f9f2c48fe1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:18.096499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:18.097696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:18.098864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:18.101875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 268/542] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:18.104609 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4359-5311-4b6b-bad5-f6f9f2c48fe1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:18.213138 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4359-5311-4b6b-bad5-f6f9f2c48fe1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:18.215064 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:18.252571 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:18.252571 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:18.252571 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:18.252571 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:18.253221 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:18.253487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:18.254979 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:18.269900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 269/543] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:18.270790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:18.339328 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:18.360481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43840a52-a4d9-4a48-bf3f-580cad82f908 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:18.381909 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ba1c6-8a59-4c67-81ef-d9195c389e2e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:18.423464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:18.424297 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:18.424588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:18.424919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:18.436614 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:18.438294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:18.446855 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:18.447517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:18.447805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:18.448085 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:18.448866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:18.449184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:18.450791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:18.455621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:18.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:18.457772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:18.457772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:18.457772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:18.457772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:18.461697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 270/544] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:18.463013 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:18.472651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 275/545] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:18.478669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:18.502575 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:18.592542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a13c4d-3cd8-478b-a9df-21efce5f7f1f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:18.627118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b42f0-4b50-408d-95e5-65e1c77d13d8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:18.655596 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:18.656306 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:18.656608 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:18.656986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:18.657876 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:18.658193 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:18.659685 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:18.685487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 271/546] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:18.687075 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:18.690868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:18.691893 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:18.692183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:18.692467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:18.693294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=2f8958e3b5dd46bb9e5c835214654ca8) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:18.693609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:18.699407 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:18.714621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 276/547] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:18 2022] GET /identity/v3/role_assignments?user.id=2f8958e3b5dd46bb9e5c835214654ca8&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:18.786616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:18.815527 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af5738-b48e-4dc0-8cde-2f96f9a8f3cf None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:18.860280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:18.860901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:18.861192 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:18.861471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] PATH_INFO: `/v3/users/swift` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:18.870808 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:18.872493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:18.888151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:18.890836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:18.893358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:18.894851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 272/548] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:18.902171 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:18.926036 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:19.040779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec17553-d3be-4dde-8789-65ee7253d2d5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:19.094189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:19.094880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:19.095157 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:19.095439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:19.096257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:19.096626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:19.098131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:19.122418 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 277/549] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:56:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:19.128403 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:19.250971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa4298-20f9-4e77-baf8-01895dd66906 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:19.287019 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99883a-16db-4b15-bd5c-94e7ed02246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:19.287673 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99883a-16db-4b15-bd5c-94e7ed02246a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:19.287966 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99883a-16db-4b15-bd5c-94e7ed02246a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:19.288246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99883a-16db-4b15-bd5c-94e7ed02246a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:19.292055 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:19.361459 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99883a-16db-4b15-bd5c-94e7ed02246a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:19.363112 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99883a-16db-4b15-bd5c-94e7ed02246a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:19.510186 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 273/550] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:19.516313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:19.632759 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:19.665845 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ba848-83fd-4b76-b175-eefe4d8ca4c7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:19.726866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:19.727548 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:19.727834 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:19.728111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:19.741967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:19.743529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:19.752644 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b7c95-9508-4236-9e3f-b95a37759dea None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:19.753366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b7c95-9508-4236-9e3f-b95a37759dea None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:19.753863 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b7c95-9508-4236-9e3f-b95a37759dea None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:19.755816 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 274/551] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:19.763606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f2b27-4286-4874-8bc5-8acf31ead670 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:19.764199 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f2b27-4286-4874-8bc5-8acf31ead670 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:19.764672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:19.765801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:19.766953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:19.766953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:19.766953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:19.766953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:19.766953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 278/552] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:19.767840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f2b27-4286-4874-8bc5-8acf31ead670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:19.775220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:19.787110 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-757f2b27-4286-4874-8bc5-8acf31ead670 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:19.788358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-757f2b27-4286-4874-8bc5-8acf31ead670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:19.823120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:19.889455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 275/553] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:19.925398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f2b27-4286-4874-8bc5-8acf31ead670 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:19.944852 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ca867-024b-4143-bb11-be3bab105dd8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:19.979905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:19.992227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd526ad-5e5f-46ab-8b9f-ac2faffa864f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:19.992891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd526ad-5e5f-46ab-8b9f-ac2faffa864f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:19.993180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd526ad-5e5f-46ab-8b9f-ac2faffa864f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:19.993466 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd526ad-5e5f-46ab-8b9f-ac2faffa864f None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:19.994490 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd526ad-5e5f-46ab-8b9f-ac2faffa864f None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:19.994812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd526ad-5e5f-46ab-8b9f-ac2faffa864f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:19.996686 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd526ad-5e5f-46ab-8b9f-ac2faffa864f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:20.018419 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 279/554] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:20.020920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f72dd4-3c85-47af-9b4e-8ae732be2f23 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.021211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f72dd4-3c85-47af-9b4e-8ae732be2f23 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.021495 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f72dd4-3c85-47af-9b4e-8ae732be2f23 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.024153 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 280/555] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:20.024153 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f72dd4-3c85-47af-9b4e-8ae732be2f23 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.068416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f2b27-4286-4874-8bc5-8acf31ead670 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.106293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d38f7e-c848-4128-9a56-cb2011886397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.107402 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d38f7e-c848-4128-9a56-cb2011886397 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.107911 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d38f7e-c848-4128-9a56-cb2011886397 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.108407 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d38f7e-c848-4128-9a56-cb2011886397 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.120565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d38f7e-c848-4128-9a56-cb2011886397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-78d38f7e-c848-4128-9a56-cb2011886397 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:20.126023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:20.127292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:20.128357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:20.134489 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 276/556] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:20.135415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f72dd4-3c85-47af-9b4e-8ae732be2f23 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.136723 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2e0a-6eb1-4a82-8e41-b120e14a20ab None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.137241 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2e0a-6eb1-4a82-8e41-b120e14a20ab None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.137845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2e0a-6eb1-4a82-8e41-b120e14a20ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.161154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-4a4d2e0a-6eb1-4a82-8e41-b120e14a20ab None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:20.162907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-4a4d2e0a-6eb1-4a82-8e41-b120e14a20ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:20.175695 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.176352 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.176632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.176909 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.184857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:20.186436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:20.189416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:20.200512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:20.201797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:20.203728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:20.203728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:20.203728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:20.203728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:20.203728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:20.203728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:20.203728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:20.203728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:20.207177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 281/557] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:20.208972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.253060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 277/558] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:20.254574 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d2e0a-6eb1-4a82-8e41-b120e14a20ab None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.270226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:20.332329 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f80b7-28fa-4b7a-9f42-d21f7b204dab None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.370301 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.371037 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.371344 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.371622 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.372356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:20.372659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:20.374090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:20.388147 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 282/559] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:20.388147 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.393521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d2e0a-6eb1-4a82-8e41-b120e14a20ab None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.431443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.432496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.433004 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.433497 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.434633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:20.435182 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:20.436892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:20.451022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 278/560] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:20.452344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.471981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:20.504531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536c47b-90b7-43d2-9050-77370f1f320a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.515380 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:20.542407 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b93f-869c-4002-800d-e19df02c556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.542981 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b93f-869c-4002-800d-e19df02c556f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.543257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b93f-869c-4002-800d-e19df02c556f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.543533 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b93f-869c-4002-800d-e19df02c556f None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.554604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b93f-869c-4002-800d-e19df02c556f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-6ca8b93f-869c-4002-800d-e19df02c556f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:20.561014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:20.562221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:20.563365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:20.564175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6bebf-8819-42d7-afc6-2412c789d12a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.564276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 283/561] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:20.564276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b93f-869c-4002-800d-e19df02c556f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.605355 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee9edb-ed8c-470a-9681-671369291ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.606047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee9edb-ed8c-470a-9681-671369291ef4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.606428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee9edb-ed8c-470a-9681-671369291ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.606740 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee9edb-ed8c-470a-9681-671369291ef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.618901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee9edb-ed8c-470a-9681-671369291ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:20.620403 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee9edb-ed8c-470a-9681-671369291ef4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-9eee9edb-ed8c-470a-9681-671369291ef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:20.635699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:20.636942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:20.638075 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:20.638075 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:20.638075 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:20.638075 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:20.638075 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 279/562] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:20.639729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e72c-5b77-4bd5-9444-30412dfd0a87 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.639729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e72c-5b77-4bd5-9444-30412dfd0a87 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.639729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e72c-5b77-4bd5-9444-30412dfd0a87 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.644318 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 280/563] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:20.644318 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81e72c-5b77-4bd5-9444-30412dfd0a87 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.683845 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b93f-869c-4002-800d-e19df02c556f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.718259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:20.720804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.721500 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.721783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.722053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.722891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=f3d1c0e873e94d85b590846747152e24) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:20.723203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:20.724708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:20.731058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 284/564] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/role_assignments?user.id=f3d1c0e873e94d85b590846747152e24&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:20.735362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.765170 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81e72c-5b77-4bd5-9444-30412dfd0a87 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.796457 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:20.814801 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af80e2-c018-4fb4-a1e8-0f6154eed5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.816490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af80e2-c018-4fb4-a1e8-0f6154eed5b0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.816887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af80e2-c018-4fb4-a1e8-0f6154eed5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.817272 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af80e2-c018-4fb4-a1e8-0f6154eed5b0 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.818335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af80e2-c018-4fb4-a1e8-0f6154eed5b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:20.819636 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af80e2-c018-4fb4-a1e8-0f6154eed5b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:20.821112 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af80e2-c018-4fb4-a1e8-0f6154eed5b0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:20.836194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 281/565] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:20.836194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdea3aa-500f-42c1-880d-0326a7469148 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.836194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdea3aa-500f-42c1-880d-0326a7469148 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.836194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdea3aa-500f-42c1-880d-0326a7469148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.855623 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-5fdea3aa-500f-42c1-880d-0326a7469148 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:20.856389 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-5fdea3aa-500f-42c1-880d-0326a7469148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:20.895594 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ee3d8-e8b4-4cc8-9321-11bb01b51703 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:20.934421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:20.935607 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:20.936186 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:20.936736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:20.937711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:20.938307 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:20.940015 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:20.942152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 282/566] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:20.943199 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdea3aa-500f-42c1-880d-0326a7469148 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:20.955554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 285/567] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:20.960602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.036336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:21.090325 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:21.094044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdea3aa-500f-42c1-880d-0326a7469148 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.099891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcde991-4707-4d4a-b7c8-3bfe41ae3b70 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.135790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.136907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.137485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.138081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.143130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.143803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.144091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.144373 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.152083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:21.152563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:21.154090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:21.154434 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:21.167192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:21.168427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 286/568] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:21.169667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:21.177063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:21.178265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:21.179438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:21.179438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:21.179438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:21.179438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:21.182475 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 283/569] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:21.183375 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.301605 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73375223-9ebe-4295-87c5-eb0653626c2f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.301876 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f31a-844f-4ca7-96fa-ea950d616400 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.317656 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:21.340845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f8a98-83a3-4ef2-9928-e2a71dec8240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.341064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a5ed5-a556-4fba-b7ad-14ff7ac52980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.341397 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f8a98-83a3-4ef2-9928-e2a71dec8240 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.341543 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a5ed5-a556-4fba-b7ad-14ff7ac52980 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.341716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f8a98-83a3-4ef2-9928-e2a71dec8240 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.341836 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a5ed5-a556-4fba-b7ad-14ff7ac52980 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.341997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f8a98-83a3-4ef2-9928-e2a71dec8240 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.342212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a5ed5-a556-4fba-b7ad-14ff7ac52980 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.342796 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f8a98-83a3-4ef2-9928-e2a71dec8240 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:21.343109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f8a98-83a3-4ef2-9928-e2a71dec8240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:21.344635 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f8a98-83a3-4ef2-9928-e2a71dec8240 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:21.354031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a5ed5-a556-4fba-b7ad-14ff7ac52980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-a70a5ed5-a556-4fba-b7ad-14ff7ac52980 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:21.359673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:21.360841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:21.361904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 287/570] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:21.362825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3fb95-ef6b-4608-b203-584f95a57766 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.362825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3fb95-ef6b-4608-b203-584f95a57766 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.362825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3fb95-ef6b-4608-b203-584f95a57766 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.368524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 288/571] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:21.368524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3fb95-ef6b-4608-b203-584f95a57766 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.371261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 284/572] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:21.376406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fcc0f-319c-4ec0-a491-560ae44517d0 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.376406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fcc0f-319c-4ec0-a491-560ae44517d0 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.376406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fcc0f-319c-4ec0-a491-560ae44517d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.384646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:21.392155 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-a18fcc0f-319c-4ec0-a491-560ae44517d0 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:21.393396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-a18fcc0f-319c-4ec0-a491-560ae44517d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:21.466156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 285/573] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:21.466817 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fcc0f-319c-4ec0-a491-560ae44517d0 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.505115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3fb95-ef6b-4608-b203-584f95a57766 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.541231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.542006 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.542317 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.542598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.543298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:21.543612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:21.545020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:21.557256 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 289/574] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:21.557256 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.576777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:21.579903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fcc0f-319c-4ec0-a491-560ae44517d0 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.617859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.618448 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.618762 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.619007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.619745 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:21.620052 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:21.621461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:21.629786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 286/575] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:21.630647 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.643624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:21.676220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c162e33-24d8-4ddc-9014-ae09063e56a7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.714708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.715489 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.715771 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.716050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.727684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:21.729312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:21.738526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8b92-99e4-4cde-80b6-951855d49d6f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:21.745766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:21.746970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:21.748508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:21.748508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:21.748508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:21.748508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:21.750259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 290/576] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:21.751791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.779150 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e91c-8ba0-47db-9543-e6e321461acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.780428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e91c-8ba0-47db-9543-e6e321461acd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e91c-8ba0-47db-9543-e6e321461acd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.781091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e91c-8ba0-47db-9543-e6e321461acd None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.800500 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0e91c-8ba0-47db-9543-e6e321461acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-b5b0e91c-8ba0-47db-9543-e6e321461acd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:21.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:21.809756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:21.810886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 287/577] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:21.812413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e91c-8ba0-47db-9543-e6e321461acd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.837655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:21.879130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11720929-ce65-4edf-9e75-74cab0bffbf8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.909046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:21.922796 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.923823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.923823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.924051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.933255 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:21.933819 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e91c-8ba0-47db-9543-e6e321461acd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:21.934857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:21.949197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:21.950603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:56:21.952055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:21.954296 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 291/578] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:21.956341 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:21.977245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:21.977934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:21.978247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:21.978524 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:21.991715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:21.993906 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:22.010386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:22.011680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:22.012840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:22.012840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:22.012840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:22.012840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:22.012840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 288/579] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:22.012840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.089026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d71640-853f-49d6-be1c-3d84b90ec280 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.111080 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:22.128522 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.129229 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.129452 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.129738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.131056 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:22.131367 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.133027 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.145588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1cc99-4ff2-4317-83d2-1869812d07f1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.146311 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 292/580] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:22.147189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.191570 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.192719 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.192719 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.192961 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.193498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:22.193811 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.195273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.205090 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 289/581] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:22.205749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.211776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:22.272167 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4f2a4-0e2b-46d8-bae1-d97fbb86167d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.315616 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.316907 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.317381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.317851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.318716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:22.319096 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.320630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.331523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb17a0c-8d67-4b49-93ba-2c88763fb88b None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.346131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 293/582] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:22.347111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.370753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.371410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.371700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.371982 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.372701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:22.373021 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.374467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.377161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:22.395878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 290/583] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:22.395878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.476034 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404408c-3701-45b3-90f1-4b4b27096843 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.477972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:22.519432 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc6944-399a-4087-b4ed-f179b0bc792d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.523988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.524672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.524960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.525239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.533897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.535482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:22.548734 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:22.549908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 294/584] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:22.551115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.568611 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.570024 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.570433 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.570627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.583556 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.585217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:22.601763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:22.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:22.604271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:22.604271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:22.604271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:22.604271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:22.604271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 291/585] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:22.604271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.661804 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:22.691860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113085-ec84-41e2-a302-789762bb4ea3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.727260 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7cf13-bc24-4110-8bd4-f4fde19bd99a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.733022 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.733689 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.733983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.734276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.746024 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.747558 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:22.764165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:22.765504 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:22.767012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:22.767012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:22.767012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:22.767012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:22.767012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 295/586] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:22.769153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.770051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.770429 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.770802 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.770928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] PATH_INFO: `/v3/users/glance` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.781898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.783976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.789358 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:22.799975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:22.801366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 292/587] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:22.802630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.888977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af099ca1-ce75-4ce3-b80d-45c96c9701dc None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.917120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd9012-8c4a-40f7-b638-8df292ddcc37 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:22.927255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:22.928527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.929188 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.929497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.929752 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.930584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:22.930885 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.932446 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.946260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 296/588] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:22.947138 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:22.965606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930100b4-ceb3-4799-a449-6cff86f8473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:22.966386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930100b4-ceb3-4799-a449-6cff86f8473a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.966675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930100b4-ceb3-4799-a449-6cff86f8473a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.966956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930100b4-ceb3-4799-a449-6cff86f8473a None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.967690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930100b4-ceb3-4799-a449-6cff86f8473a None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:22.967998 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930100b4-ceb3-4799-a449-6cff86f8473a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:22.969635 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930100b4-ceb3-4799-a449-6cff86f8473a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:22.982079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 293/589] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:22.983420 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba867e9-0b3f-4c37-b5e6-484886cfc1af None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:22.983689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba867e9-0b3f-4c37-b5e6-484886cfc1af None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:22.983971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba867e9-0b3f-4c37-b5e6-484886cfc1af None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:22.985439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 294/590] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:22.985979 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba867e9-0b3f-4c37-b5e6-484886cfc1af None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.044917 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:23.073464 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2565f1-bc28-4a0f-9517-465a2778e931 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.104428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba867e9-0b3f-4c37-b5e6-484886cfc1af None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.114286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.114925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.115210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.115478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.116176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:23.116480 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:23.117899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:23.130860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 297/591] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:23.131684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.140919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.141568 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.141848 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.142143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.142959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:23.143270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:23.144744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:23.163802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 295/592] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:56:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:23.164779 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.204116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:23.256002 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9037a8-b36a-498f-9eb1-76fec46d1ea5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.291761 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce22bb-fbe3-4048-9b66-c48819d65c48 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.292113 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c17960-80d0-4339-a756-c26d7be078fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.292811 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c17960-80d0-4339-a756-c26d7be078fe None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.293101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c17960-80d0-4339-a756-c26d7be078fe None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.293382 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c17960-80d0-4339-a756-c26d7be078fe None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/4fe070acb9a84d90baaa3b08a4fd3191/roles/e5f9b7b560ca414994499e84782fa22d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.301103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:23.315655 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c17960-80d0-4339-a756-c26d7be078fe None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=4fe070acb9a84d90baaa3b08a4fd3191, role_id=e5f9b7b560ca414994499e84782fa22d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:23.317514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c17960-80d0-4339-a756-c26d7be078fe None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:23.332119 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 298/593] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:56:23 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/4fe070acb9a84d90baaa3b08a4fd3191/roles/e5f9b7b560ca414994499e84782fa22d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:23.333377 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4412799-1064-4744-b686-5211ef3f8463 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.333671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4412799-1064-4744-b686-5211ef3f8463 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.333947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4412799-1064-4744-b686-5211ef3f8463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.334214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.334915 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.335218 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.335494 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.353366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:23.355088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:23.355871 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-c4412799-1064-4744-b686-5211ef3f8463 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:23.357154 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-c4412799-1064-4744-b686-5211ef3f8463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:23.369746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:23.371158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 296/594] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:23.372693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.459859 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 299/595] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:23.461313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4412799-1064-4744-b686-5211ef3f8463 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.513422 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:23.531697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ea665-5092-4886-8c3b-665b3efe5e0e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.575890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.575890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.576329 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.576523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.584643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:23.586618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:23.598022 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4412799-1064-4744-b686-5211ef3f8463 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:23.601078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:23.602350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 297/596] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:23.603677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.620272 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:23.651843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.652832 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.653319 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.653591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.665939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:23.667511 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:23.685108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:23.686413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:23.687606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:23.687606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:23.687606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:23.687606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:23.691520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 300/597] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:23.692606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.735246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b755-3055-44e3-8f0b-8029935f33af None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.773268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.773908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.774258 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.774535 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.775315 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:23.775628 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:23.777121 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:23.801360 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 298/598] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:56:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:23.801360 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.807812 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:23.813150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00babf-fc01-4acc-a8f1-4950a45bad19 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.857351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1823e4-8d7e-4066-a011-b570c4557c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.858019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1823e4-8d7e-4066-a011-b570c4557c1b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.858337 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1823e4-8d7e-4066-a011-b570c4557c1b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.858613 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1823e4-8d7e-4066-a011-b570c4557c1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.870455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1823e4-8d7e-4066-a011-b570c4557c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-9a1823e4-8d7e-4066-a011-b570c4557c1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:23.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:23.878973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 301/599] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:23.880669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1823e4-8d7e-4066-a011-b570c4557c1b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:23.918517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:23.940800 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac7503-91f3-4d6c-a000-aac728336ad2 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:23.987262 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:23.987976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:23.988077 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:23.988443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:23.989221 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:23.989538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:23.991051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.004343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 299/600] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:24.005236 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.022461 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1823e4-8d7e-4066-a011-b570c4557c1b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.077647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.077647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.079005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.079005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.079226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:24.082115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.082115 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.090566 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:24.099965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 302/601] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:24.099965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.135099 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3095-be53-46ab-baff-6f7844170142 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.171387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.172648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.173173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.173673 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.186411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.188457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.202846 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:24.205386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:24.206503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:24.213681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:24.213681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:24.213681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:24.213681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:24.213681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 300/602] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:24.213681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.238480 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326c30e-d7fd-4849-82e6-8b42e3bbf505 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.290485 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.291981 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.292265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.292567 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.293313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:24.293619 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.295058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.304393 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 303/603] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:24.305033 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.344709 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe7437-66b9-4659-8b2f-c5afd0edd3a8 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.364603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:24.390083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54ee31-20d6-4e50-aa3a-33e5b24d46b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.391091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54ee31-20d6-4e50-aa3a-33e5b24d46b8 None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.391390 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54ee31-20d6-4e50-aa3a-33e5b24d46b8 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.391667 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54ee31-20d6-4e50-aa3a-33e5b24d46b8 None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/021350d0e5bc4eb59b53166712073877/roles/e5f9b7b560ca414994499e84782fa22d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.408414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54ee31-20d6-4e50-aa3a-33e5b24d46b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=021350d0e5bc4eb59b53166712073877, role_id=e5f9b7b560ca414994499e84782fa22d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.409925 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54ee31-20d6-4e50-aa3a-33e5b24d46b8 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.425136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6733-d9d9-4e2d-aa86-052f7d98ec23 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.431674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 301/604] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:56:24 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/021350d0e5bc4eb59b53166712073877/roles/e5f9b7b560ca414994499e84782fa22d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:24.431674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54ee31-20d6-4e50-aa3a-33e5b24d46b8 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.476588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.477269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.477610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.477899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.486670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.488302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:24.502338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:24.503531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 304/605] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:24.504670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.523685 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:24.597544 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54ee31-20d6-4e50-aa3a-33e5b24d46b8 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.643448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d720ec6-8d14-44cc-93f3-528eb69a5b7f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.652656 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.653327 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.653611 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.654035 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.655093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:24.655093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.656405 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.672059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 302/606] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:24.673227 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.696393 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.697070 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.697816 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.698150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.713056 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.715563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.720101 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:24.741774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:24.743198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:24.744526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:24.744526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:24.744526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:24.744526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:24.744526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 305/607] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:24.744526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.809754 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ab62-5664-4f13-9f73-41fb3b41c452 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.814322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:24.852441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.853480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.853976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.854471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.855600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:24.856107 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.857909 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.861690 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8a5b4-c4d5-493f-b873-fe403b26c1a3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:24.873625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 303/608] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:24.874834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.925818 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:24.926628 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:24.926920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:24.927199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:24.936714 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:24.938663 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:24.955929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:24.957822 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 306/609] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:24.958993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:24.991138 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:25.012898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79cdcc-8b93-43f8-99fc-5e095c68c063 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:25.057788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:25.058813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.059282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.059861 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.060786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:25.061105 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:25.062644 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:25.081755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 304/610] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:25.086571 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:25.112844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79399c-3456-4a86-99f4-42aac01def21 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:25.141597 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:25.166275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858f17-3129-48eb-b938-b77077709fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:25.168300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858f17-3129-48eb-b938-b77077709fae None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.168835 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858f17-3129-48eb-b938-b77077709fae None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.169336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858f17-3129-48eb-b938-b77077709fae None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/b27b8079c98346efbc94b558e152da86/roles/e5f9b7b560ca414994499e84782fa22d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.190597 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858f17-3129-48eb-b938-b77077709fae None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=b27b8079c98346efbc94b558e152da86, role_id=e5f9b7b560ca414994499e84782fa22d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:25.193626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858f17-3129-48eb-b938-b77077709fae None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:25.201863 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85988c-9345-417e-a43a-e143da5b06a0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:25.222221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 307/611] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:56:24 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/b27b8079c98346efbc94b558e152da86/roles/e5f9b7b560ca414994499e84782fa22d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:25.223698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858f17-3129-48eb-b938-b77077709fae None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:25.255674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:25.256462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.256670 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.256951 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.257805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:25.258137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:25.259770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:25.284074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 305/612] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:25.289746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:25.304982 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:25.354388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858f17-3129-48eb-b938-b77077709fae None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:25.396368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:25.397468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.398009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.398578 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.407745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:25.409908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:25.424212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:25.425496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:25.426736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:25.429799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 308/613] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:25.439259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:25.480592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:25.512184 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2037-8224-4769-8cca-2a7ba6123afd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:25.553314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15011ff-e3df-497f-b53b-992815c1d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:25.554406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15011ff-e3df-497f-b53b-992815c1d3f6 None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.554692 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15011ff-e3df-497f-b53b-992815c1d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.554974 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15011ff-e3df-497f-b53b-992815c1d3f6 None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/2f8958e3b5dd46bb9e5c835214654ca8/roles/e5f9b7b560ca414994499e84782fa22d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.572340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eeab5-3cc3-4c9a-aee4-9906a34c1c69 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:25.576204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15011ff-e3df-497f-b53b-992815c1d3f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=2f8958e3b5dd46bb9e5c835214654ca8, role_id=e5f9b7b560ca414994499e84782fa22d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:25.577725 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15011ff-e3df-497f-b53b-992815c1d3f6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:25.595966 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 306/614] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:56:25 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/2f8958e3b5dd46bb9e5c835214654ca8/roles/e5f9b7b560ca414994499e84782fa22d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:56:25.597399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bccf2-f750-40b6-9444-3dc6ca441ddc None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.598346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bccf2-f750-40b6-9444-3dc6ca441ddc None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.598657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bccf2-f750-40b6-9444-3dc6ca441ddc None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.603665 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 307/615] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:25.613464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e733426-384a-4af3-860e-1baf42803d34 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.613924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e733426-384a-4af3-860e-1baf42803d34 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.614425 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e733426-384a-4af3-860e-1baf42803d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.623890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:25.625117 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.625628 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.626157 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.628584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:25.629152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:25.630898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:25.652099 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-8e733426-384a-4af3-860e-1baf42803d34 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:25.653530 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-8e733426-384a-4af3-860e-1baf42803d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:25.654650 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 309/616] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:56:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:25.661989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:25.745939 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:25.783224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 308/617] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:56:25.811964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e733426-384a-4af3-860e-1baf42803d34 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:25.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:25.857631 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e225849-650b-470f-9c5b-946c7a82bef2 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:25.898634 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:25.899300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:25.899587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:25.899863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:25.911425 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:25.913247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:25.929646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:25.930913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:25.932053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:25.932053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:25.932053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:25.932053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:25.932329 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e733426-384a-4af3-860e-1baf42803d34 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:25.938436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 310/618] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:25.940303 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:26.001876 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276b94b-8a8c-4153-9bef-60f585651e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.005167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b94b-8a8c-4153-9bef-60f585651e98 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.005473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b94b-8a8c-4153-9bef-60f585651e98 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.005791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b94b-8a8c-4153-9bef-60f585651e98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.031322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276b94b-8a8c-4153-9bef-60f585651e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-9276b94b-8a8c-4153-9bef-60f585651e98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:26.041231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:26.043669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:26.046002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 309/619] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:26.050429 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276b94b-8a8c-4153-9bef-60f585651e98 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:26.143051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe3255-fdf9-40d1-91c7-1b162128b07a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:26.146151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:26.198038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.199219 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.199740 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.200239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.201271 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:26.201805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:26.203609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:26.217395 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:26.221080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 311/620] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:26.226428 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:26.246194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276b94b-8a8c-4153-9bef-60f585651e98 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:26.315523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.317004 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.317546 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.318048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.319041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:26.319576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:26.321394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:26.331742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 310/621] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:26.337629 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:26.448906 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b69b33-38e6-486f-9534-ef2b0fb0b2bc None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:26.460221 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:26.477770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbd406-d2cd-4522-a9fc-0aded89ba2e9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:26.492988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.494230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.494766 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.495276 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.507750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:26.509603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:26.525654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:26.526862 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:26.527997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:26.527997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:26.527997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:26.527997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:26.530671 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 311/622] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:26.536626 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:26.541759 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.541970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.542205 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.542479 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.559706 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:26.561250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:26.583038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:26.584227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:26.585398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 312/623] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:26.590555 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:26.652679 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643e99-876c-4dc8-8379-f41cc68b1ab0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:26.694857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:26.712071 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.713111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.713624 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.714183 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.715266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:26.715854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:26.717502 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:26.746447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 312/624] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:26.749474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8bb78-9a17-4b5b-85a7-4459302c5bf9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:26.752886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:26.788492 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf16bd7-a822-46b3-a6bd-c78748b52a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.789131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16bd7-a822-46b3-a6bd-c78748b52a75 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.789418 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16bd7-a822-46b3-a6bd-c78748b52a75 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.789746 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16bd7-a822-46b3-a6bd-c78748b52a75 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.790545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf16bd7-a822-46b3-a6bd-c78748b52a75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:26.790857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf16bd7-a822-46b3-a6bd-c78748b52a75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:26.791738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:26.793601 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf16bd7-a822-46b3-a6bd-c78748b52a75 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:26.815556 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 313/625] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:26.817810 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec869fd-1c70-4118-8992-ba15dbbc6d4d None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.817810 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec869fd-1c70-4118-8992-ba15dbbc6d4d None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.818253 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec869fd-1c70-4118-8992-ba15dbbc6d4d None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.819833 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 314/626] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:26.820928 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec869fd-1c70-4118-8992-ba15dbbc6d4d None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:26.903668 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ccd5e-8ec9-40c7-b2a3-5266f4b3ddf1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:26.939761 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec869fd-1c70-4118-8992-ba15dbbc6d4d None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:26.946930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd2246-3a64-449e-8d10-e5fc32e440d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.947924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd2246-3a64-449e-8d10-e5fc32e440d6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.948438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd2246-3a64-449e-8d10-e5fc32e440d6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.949048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd2246-3a64-449e-8d10-e5fc32e440d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.957583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd2246-3a64-449e-8d10-e5fc32e440d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:26.959418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd2246-3a64-449e-8d10-e5fc32e440d6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:26.964553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d0cd2246-3a64-449e-8d10-e5fc32e440d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:26.975585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:26.976901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:26.978214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:26.984244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 313/627] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:56:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:26.984244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a68684-0202-4bc0-b862-feedeb22c44a None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.984244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a68684-0202-4bc0-b862-feedeb22c44a None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.984244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a68684-0202-4bc0-b862-feedeb22c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:26.997043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee8c2c-8145-4bbc-a9e9-8773c3f8d058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:26.998327 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8c2c-8145-4bbc-a9e9-8773c3f8d058 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:26.998843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8c2c-8145-4bbc-a9e9-8773c3f8d058 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:26.999346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8c2c-8145-4bbc-a9e9-8773c3f8d058 None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/f3d1c0e873e94d85b590846747152e24/roles/e5f9b7b560ca414994499e84782fa22d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.007893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-d1a68684-0202-4bc0-b862-feedeb22c44a None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:27.009132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-d1a68684-0202-4bc0-b862-feedeb22c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:27.018466 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee8c2c-8145-4bbc-a9e9-8773c3f8d058 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=f3d1c0e873e94d85b590846747152e24, role_id=e5f9b7b560ca414994499e84782fa22d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:27.020292 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee8c2c-8145-4bbc-a9e9-8773c3f8d058 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:27.050282 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 315/628] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:56:26 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/f3d1c0e873e94d85b590846747152e24/roles/e5f9b7b560ca414994499e84782fa22d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:56:27.051675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee8c2c-8145-4bbc-a9e9-8773c3f8d058 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:27.163648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 314/629] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:27.195322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a68684-0202-4bc0-b862-feedeb22c44a None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:27.210854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:27.281190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee8c2c-8145-4bbc-a9e9-8773c3f8d058 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:27.321335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:27.322028 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.322736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.322942 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.324351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:27.324880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:27.326950 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:27.347988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 316/630] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:56:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:27.352038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a68684-0202-4bc0-b862-feedeb22c44a None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:27.353748 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:27.385109 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:27.398017 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031138cb-e219-4e95-9bef-0e860aba99cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:27.399239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031138cb-e219-4e95-9bef-0e860aba99cc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.399750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031138cb-e219-4e95-9bef-0e860aba99cc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.400221 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031138cb-e219-4e95-9bef-0e860aba99cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.413041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031138cb-e219-4e95-9bef-0e860aba99cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-031138cb-e219-4e95-9bef-0e860aba99cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:27.418479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:27.419655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:27.420823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:27.423916 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 315/631] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:27.429298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031138cb-e219-4e95-9bef-0e860aba99cc None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:27.468086 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f241a3-6639-4ecc-b106-bed135c5713b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:27.508271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:27.514914 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6513-d176-48f1-a144-4302f2baac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:27.515803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6513-d176-48f1-a144-4302f2baac80 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.516153 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6513-d176-48f1-a144-4302f2baac80 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.516269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6513-d176-48f1-a144-4302f2baac80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.547219 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031138cb-e219-4e95-9bef-0e860aba99cc None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:27.606180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e6513-d176-48f1-a144-4302f2baac80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:27.607720 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e6513-d176-48f1-a144-4302f2baac80 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:27.638839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:27.650871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:27.651542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.651828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.652103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.652854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:27.653158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:27.654606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:27.673744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 316/632] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:27.678940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:27.722982 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 317/633] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:56:27.728828 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6513-d176-48f1-a144-4302f2baac80 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:27.844185 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6513-d176-48f1-a144-4302f2baac80 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:27.882485 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:27.887127 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69141bc5-4a2f-4907-be08-954118406565 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:27.894144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f26f9a-e252-4659-8410-730109ffedda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:27.894144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f26f9a-e252-4659-8410-730109ffedda None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.894144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f26f9a-e252-4659-8410-730109ffedda None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.894144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f26f9a-e252-4659-8410-730109ffedda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.911356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f26f9a-e252-4659-8410-730109ffedda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:27.913014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f26f9a-e252-4659-8410-730109ffedda None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:27.924911 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:27.931794 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0025ae3-95fd-4f0e-bdd0-e4c1d2ea6922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:27.932841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025ae3-95fd-4f0e-bdd0-e4c1d2ea6922 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.933339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025ae3-95fd-4f0e-bdd0-e4c1d2ea6922 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.933843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025ae3-95fd-4f0e-bdd0-e4c1d2ea6922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-78f26f9a-e252-4659-8410-730109ffedda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:27.935853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:27.936924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:27.938550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:27.938550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:27.938550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:27.938550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:27.938550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 318/634] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:27.939370 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91da09-b6a2-44c4-8b3f-17c1bccc85de None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.939662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91da09-b6a2-44c4-8b3f-17c1bccc85de None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.939936 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91da09-b6a2-44c4-8b3f-17c1bccc85de None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.945637 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 319/635] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:27.945637 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784dfa4-198f-4486-977a-0b02e07e5e2e None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.945637 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784dfa4-198f-4486-977a-0b02e07e5e2e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.945637 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784dfa4-198f-4486-977a-0b02e07e5e2e None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.945637 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 320/636] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:27.945637 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784dfa4-198f-4486-977a-0b02e07e5e2e None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:27.956872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0025ae3-95fd-4f0e-bdd0-e4c1d2ea6922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:27.958409 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0025ae3-95fd-4f0e-bdd0-e4c1d2ea6922 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d0025ae3-95fd-4f0e-bdd0-e4c1d2ea6922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:27.973986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:27.975152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:27.976283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:27.976283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:27.976283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:27.976283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:27.976283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 317/637] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:27.976640 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32ad8-babe-422c-9ea2-d21dd7fdb61f None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:27.976732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32ad8-babe-422c-9ea2-d21dd7fdb61f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:27.977405 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32ad8-babe-422c-9ea2-d21dd7fdb61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:27.996479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-83e32ad8-babe-422c-9ea2-d21dd7fdb61f None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:27.997819 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-83e32ad8-babe-422c-9ea2-d21dd7fdb61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:28.065489 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784dfa4-198f-4486-977a-0b02e07e5e2e None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:28.091155 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 318/638] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:28.103795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c558-82ce-4c4e-b042-c17665b5bc58 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.104029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c558-82ce-4c4e-b042-c17665b5bc58 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.104255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c558-82ce-4c4e-b042-c17665b5bc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.116595 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:28.117430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.117851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.118280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.119113 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:28.119421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:28.120910 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:28.132549 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ee76c558-82ce-4c4e-b042-c17665b5bc58 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:28.133773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ee76c558-82ce-4c4e-b042-c17665b5bc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:28.135422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 321/639] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:28.140874 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.193065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:28.211984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 319/640] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:28.212892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76c558-82ce-4c4e-b042-c17665b5bc58 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.262654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ebd0c-27e7-467b-8fc9-f4119d1ec319 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:28.274362 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:28.302267 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:28.303124 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.303461 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.303806 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.304522 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:28.304830 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:28.306288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:28.320629 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 322/641] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:28.321341 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.343320 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76c558-82ce-4c4e-b042-c17665b5bc58 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:28.384584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625004f-5ad1-486e-a6b5-b920f67152ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:28.385526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625004f-5ad1-486e-a6b5-b920f67152ca None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.385830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625004f-5ad1-486e-a6b5-b920f67152ca None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.386513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625004f-5ad1-486e-a6b5-b920f67152ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.400279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625004f-5ad1-486e-a6b5-b920f67152ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-b625004f-5ad1-486e-a6b5-b920f67152ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:28.405376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:28.406682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 320/642] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:28.413643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625004f-5ad1-486e-a6b5-b920f67152ca None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.439632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:28.443542 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5444b4-7295-4ea4-a2e5-1176fff65ba0 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:28.487058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:28.487735 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.488018 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.488302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.497380 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:28.499326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:28.512977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:28.514222 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:28.515881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:28.515881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:28.515881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:28.515881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:28.515881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:28.515881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:28.515881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:28.515881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:28.518619 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 323/643] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:28.520337 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.551749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625004f-5ad1-486e-a6b5-b920f67152ca None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:28.555581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:28.601377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536e18f-bdee-43d9-949f-58a9ab7b76cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:28.602059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536e18f-bdee-43d9-949f-58a9ab7b76cc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.602366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536e18f-bdee-43d9-949f-58a9ab7b76cc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.602652 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536e18f-bdee-43d9-949f-58a9ab7b76cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.617147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536e18f-bdee-43d9-949f-58a9ab7b76cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-9536e18f-bdee-43d9-949f-58a9ab7b76cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:28.623996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:28.625224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 321/644] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:28.630934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536e18f-bdee-43d9-949f-58a9ab7b76cc None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.631995 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b0cce-df32-4cd4-99d2-a38e593f7eed None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:28.682176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:28.683111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.683679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.684237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.696437 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:28.697991 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:28.711721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:28.713041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 324/645] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:28.714259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.750208 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:28.781372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536e18f-bdee-43d9-949f-58a9ab7b76cc None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:28.825210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:28.826243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.826740 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.826890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.828055 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:28.828055 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:28.829282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:28.843851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 322/646] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:28.843851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.849950 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff59959-d6ff-4296-a7e2-5f556ec888f6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:28.881443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:28.888481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:28.889123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:28.889405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:28.889678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:28.890506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:28.890795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:28.892295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:28.917266 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 325/647] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:28.918268 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:28.969229 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de1341-add9-48ec-adb7-ae089873c501 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.019674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:29.020489 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.020783 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.021056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.021802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:29.022381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:29.023557 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:29.027324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:29.034854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 323/648] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:29.035172 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:29.065232 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347b7fa-ea03-4991-b725-fc1feaacda01 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.120718 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:29.121353 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.121654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.121919 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.122762 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:29.123064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:29.124566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:29.144528 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 326/649] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:56:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:29.145411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:29.177556 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:29.198892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254ed87-92da-439f-86b7-0898ad327ba0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.239535 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:29.240113 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.240396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.240670 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.253194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:29.254761 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:29.270992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:29.272661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:29.274370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:29.274370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:29.274370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:29.274370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:29.274370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 324/650] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:29.274370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:29.283476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc245c1-3e15-48a7-ae87-1508d10dcb69 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.320400 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:29.321037 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.321321 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.321598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.322435 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=4fe070acb9a84d90baaa3b08a4fd3191) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:29.322753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:29.324247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:29.331041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 327/651] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:29 2022] GET /identity/v3/role_assignments?user.id=4fe070acb9a84d90baaa3b08a4fd3191&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:29.331727 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:29.336432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:29.435381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb53b2-eae7-4c90-9a98-95f93fbfa4b9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.472971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31776790-8680-4f6b-ae93-83b1bb8c9568 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.475606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:29.476400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.476744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.477145 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.488137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:29.491058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:29.492673 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:29.511023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:29.512230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:29.513867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:29.513867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:29.513867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:29.513867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:29.513867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 325/652] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:29.513867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:29.514332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd9769-54cb-4805-8ab8-04584d9cb808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:29.514710 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9769-54cb-4805-8ab8-04584d9cb808 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.515664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9769-54cb-4805-8ab8-04584d9cb808 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.515664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9769-54cb-4805-8ab8-04584d9cb808 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.610800 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd9769-54cb-4805-8ab8-04584d9cb808 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:29.612374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd9769-54cb-4805-8ab8-04584d9cb808 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:29.614386 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:29.635521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810be7c-0a81-4890-9ebe-51f5ddeae97a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.671992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76237c92-af48-4f0c-b057-b44571ebc701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:29.672643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76237c92-af48-4f0c-b057-b44571ebc701 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.672929 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76237c92-af48-4f0c-b057-b44571ebc701 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.673207 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76237c92-af48-4f0c-b057-b44571ebc701 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.673928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76237c92-af48-4f0c-b057-b44571ebc701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:29.674271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76237c92-af48-4f0c-b057-b44571ebc701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:29.675711 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76237c92-af48-4f0c-b057-b44571ebc701 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:29.688944 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 326/653] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:29.690365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc241a-454b-4653-b561-7ad7cf95e116 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.690668 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc241a-454b-4653-b561-7ad7cf95e116 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.690875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc241a-454b-4653-b561-7ad7cf95e116 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.692732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 327/654] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:29.693626 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc241a-454b-4653-b561-7ad7cf95e116 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:29.728275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 328/655] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:29.729679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66d22-db0a-4320-a438-44296a3bd530 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.729963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66d22-db0a-4320-a438-44296a3bd530 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.730261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66d22-db0a-4320-a438-44296a3bd530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.750625 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-79a66d22-db0a-4320-a438-44296a3bd530 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:29.751885 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-79a66d22-db0a-4320-a438-44296a3bd530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:29.760050 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:29.813612 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc241a-454b-4653-b561-7ad7cf95e116 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.846290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 329/656] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:29.847439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66d22-db0a-4320-a438-44296a3bd530 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:29.852670 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:29.853278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:29.853561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:29.853831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:29.855095 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:29.855095 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:29.856362 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:29.869311 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 328/657] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:29.870334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:29.894965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:29.970590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66d22-db0a-4320-a438-44296a3bd530 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.982801 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127705f-4f4e-4670-b61d-711103cfe9b2 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:29.997358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:30.007653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.008216 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.008497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.008775 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] PATH_INFO: `/v3/users/glance` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.016756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.018507 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.022583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.023321 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.023664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.024003 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:30.033546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:30.034954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 330/658] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:30.036381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.045014 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.046568 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:30.064033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:30.065238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:30.066802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:30.066802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:30.066802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:30.066802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:30.066802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 329/659] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:30.068171 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.147732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:30.159191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a204902-a8f0-4807-8229-a9b86505da83 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.186622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d0c1-7503-45d5-88c1-86176644a265 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.198757 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190daa3-8cbf-4f37-9e5d-806b7bc5e6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.199705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190daa3-8cbf-4f37-9e5d-806b7bc5e6b3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.199986 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190daa3-8cbf-4f37-9e5d-806b7bc5e6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.200267 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190daa3-8cbf-4f37-9e5d-806b7bc5e6b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.211428 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190daa3-8cbf-4f37-9e5d-806b7bc5e6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1190daa3-8cbf-4f37-9e5d-806b7bc5e6b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:30.216889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:30.218312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 331/660] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:30.219561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190daa3-8cbf-4f37-9e5d-806b7bc5e6b3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.228488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.229514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.230001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.230570 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.239413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.240989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:30.253894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:30.255343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:30.256697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 330/661] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:30.260988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.289104 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:30.332013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190daa3-8cbf-4f37-9e5d-806b7bc5e6b3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.369058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.369843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.370239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.370584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.371456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:30.371835 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.372653 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692aed1-4a76-432d-a44d-e04acf5978e9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.373455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.393618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 332/662] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:30.394593 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.408635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:30.417034 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.417684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.417970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.418282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.418996 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:30.419335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.420732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.434678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 331/663] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:30.435552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.504384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d77ae-c191-498a-a52f-9d9f515c17f4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.536889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:30.546217 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.546992 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.547279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.547568 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.548301 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:30.548611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.550049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.556493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d138d-a9af-4446-92a0-3ede226923ab None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.560087 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 333/664] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:30.561045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.595624 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.596340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.596569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.596846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.597694 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:30.598012 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.600457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.625146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 332/665] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:30.625146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.642010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:30.676082 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9248e-4958-4d51-9d58-b6ed3d9f3310 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.714196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173593a-9ac4-48c0-b575-7c08dfce16bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.715073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173593a-9ac4-48c0-b575-7c08dfce16bc None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.715218 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173593a-9ac4-48c0-b575-7c08dfce16bc None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.715379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173593a-9ac4-48c0-b575-7c08dfce16bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.736603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e422787-5abc-4121-8036-33ef9ba73349 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.774442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.775119 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.775407 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.775687 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.782884 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.784365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.788155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173593a-9ac4-48c0-b575-7c08dfce16bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.789823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173593a-9ac4-48c0-b575-7c08dfce16bc None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:30.796153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:30.797576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 333/666] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:30.798889 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.805143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:30.878968 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:30.906616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35215a03-151e-4453-b35a-d0750f8f94b7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:30.924529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 334/667] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:30.924529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173593a-9ac4-48c0-b575-7c08dfce16bc None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:30.948454 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:30.949291 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:30.949593 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:30.949867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:30.963081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:30.964584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:30.980843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:30.982212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:30.984290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:30.984290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:30.984290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:30.984290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:30.984290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 334/668] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:30.984290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.044040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173593a-9ac4-48c0-b575-7c08dfce16bc None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.048591 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:31.084191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e630-55db-4f74-81ee-4ee4e0322cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.084830 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e630-55db-4f74-81ee-4ee4e0322cbe None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.085113 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e630-55db-4f74-81ee-4ee4e0322cbe None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.085388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e630-55db-4f74-81ee-4ee4e0322cbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.100196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f07d59-19a8-4a1a-8901-804dd6d5806f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.132332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:31.136714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.137360 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.137821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.138138 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.139315 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:31.139655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.141213 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.154040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 335/669] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:31.156700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.168437 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760e630-55db-4f74-81ee-4ee4e0322cbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.169960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760e630-55db-4f74-81ee-4ee4e0322cbe None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.279685 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2426-518f-4ca6-9033-6077acc7efe1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.291104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 335/670] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:31.291970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e630-55db-4f74-81ee-4ee4e0322cbe None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.296702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:31.320116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.320789 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.321073 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.321351 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.332256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.333783 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:31.349669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:31.355738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:31.356880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:31.356880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:31.356880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:31.356880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:31.356880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 336/671] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:31.356880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.377094 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:31.410302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e630-55db-4f74-81ee-4ee4e0322cbe None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.450835 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.451469 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.451779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.452027 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.452726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:31.453132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.454591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.462942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda643b1-ffca-4514-822a-c7c51eeed23a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.466590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 336/672] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:31.469944 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.496930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.497572 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.497860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.498168 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.498987 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=021350d0e5bc4eb59b53166712073877) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:31.499306 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.500813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.506722 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 337/673] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/role_assignments?user.id=021350d0e5bc4eb59b53166712073877&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:31.507518 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.544894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:31.576423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae75e3-d0a3-4c93-a8be-e999a1c05b0b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.610212 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:31.616436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70b76c-f1e4-4297-94e8-79a44233904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.617104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70b76c-f1e4-4297-94e8-79a44233904d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.617384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70b76c-f1e4-4297-94e8-79a44233904d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.617654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70b76c-f1e4-4297-94e8-79a44233904d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.629949 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70b76c-f1e4-4297-94e8-79a44233904d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.631520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70b76c-f1e4-4297-94e8-79a44233904d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.631657 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42856d84-b976-4706-a7ef-7c4fa1cfcc9c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1f70b76c-f1e4-4297-94e8-79a44233904d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:31.649866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:31.651137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:31.652294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:31.652294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:31.652294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:31.652294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:31.652294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 337/674] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:31.652669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9c307-1fb9-4d1e-9ab7-e3154ebc767d None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.652775 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9c307-1fb9-4d1e-9ab7-e3154ebc767d None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.653462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9c307-1fb9-4d1e-9ab7-e3154ebc767d None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.658183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 338/675] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:31.658183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9c307-1fb9-4d1e-9ab7-e3154ebc767d None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.685267 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.686270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.686554 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.686826 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.687703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:31.688004 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.689429 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.704489 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 338/676] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:31.710185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.781191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9c307-1fb9-4d1e-9ab7-e3154ebc767d None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.812224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:31.818711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125d729-1501-4316-bb19-5e7519182d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.819371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125d729-1501-4316-bb19-5e7519182d03 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.819658 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125d729-1501-4316-bb19-5e7519182d03 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.819938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125d729-1501-4316-bb19-5e7519182d03 None admin] PATH_INFO: `/v3/users/swift` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.824827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82aa75-6371-49b3-ba72-5f049a5691f0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:31.828249 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125d729-1501-4316-bb19-5e7519182d03 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.829719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125d729-1501-4316-bb19-5e7519182d03 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1125d729-1501-4316-bb19-5e7519182d03 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:31.842206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:31.843621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:31.844840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:31.848301 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 339/677] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:31.849513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e8843-8e44-4520-86ee-9e1573ba1627 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.849803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e8843-8e44-4520-86ee-9e1573ba1627 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.850089 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e8843-8e44-4520-86ee-9e1573ba1627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.868836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:31.869163 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-970e8843-8e44-4520-86ee-9e1573ba1627 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:31.869807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:31.870097 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:31.870449 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-970e8843-8e44-4520-86ee-9e1573ba1627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:31.870592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:31.871148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:31.871456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:31.872879 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:31.889278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 339/678] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:31.889278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.920588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:31.952497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 340/679] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:31.953497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e8843-8e44-4520-86ee-9e1573ba1627 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:31.999096 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ecd9-69ef-42ba-affb-492cd7c25bc0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.038291 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.038998 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.039253 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.039532 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.047177 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.048609 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:32.062552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:32.063834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 340/680] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:32.064962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.081140 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:32.092733 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e8843-8e44-4520-86ee-9e1573ba1627 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.132331 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.132965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.133248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.133522 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.134367 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:32.134707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.136182 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.154392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 341/681] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:56:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:32.159472 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.183793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874d91e-42f6-4f6b-b32f-63b18db3c12f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.204997 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:32.226370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.227072 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.227345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.227636 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.236294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.238736 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:32.255843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:32.257109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:32.258485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:32.258485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:32.258485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:32.258485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:32.258485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:32.258485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:32.258485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:32.258485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:32.260508 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 341/682] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:32.261626 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.291772 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1916c5-a1df-43a3-9dfd-2d8773508025 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.332874 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893effea-0644-4124-85d6-2619e3abc73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.333878 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893effea-0644-4124-85d6-2619e3abc73b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.334190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893effea-0644-4124-85d6-2619e3abc73b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.334468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893effea-0644-4124-85d6-2619e3abc73b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.342809 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:32.347103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893effea-0644-4124-85d6-2619e3abc73b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-893effea-0644-4124-85d6-2619e3abc73b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:32.352181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:32.358552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 342/683] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:32.359918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893effea-0644-4124-85d6-2619e3abc73b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.387989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30630df-d1ec-403d-bfb0-5c3501942590 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.426755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.427556 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.427940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.428228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.429003 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:32.429315 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.430842 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.445300 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 342/684] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:32.447473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.470441 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:32.482782 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893effea-0644-4124-85d6-2619e3abc73b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.523632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db079a1-778b-47b2-b786-347c3fadfe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.524714 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db079a1-778b-47b2-b786-347c3fadfe0b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.525006 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db079a1-778b-47b2-b786-347c3fadfe0b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.525284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db079a1-778b-47b2-b786-347c3fadfe0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.561780 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b4cbf-726b-4664-93d3-ab8bf1071fa7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.600487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:32.602995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.603899 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.604180 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.604453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.605227 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:32.605531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.607209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.611223 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db079a1-778b-47b2-b786-347c3fadfe0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.612761 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db079a1-778b-47b2-b786-347c3fadfe0b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.623048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 343/685] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:32.623930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.732008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:32.748939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 343/686] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:32.752198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db079a1-778b-47b2-b786-347c3fadfe0b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.757537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1318e-5495-42ed-8ef9-660036c98964 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.810385 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.811119 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.812846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.812846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.812846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:32.812846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.814060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.828977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 344/687] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:32.828977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.885251 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:32.888494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db079a1-778b-47b2-b786-347c3fadfe0b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.929329 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.930830 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.931290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.931904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.933184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=b27b8079c98346efbc94b558e152da86) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:32.933914 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.935499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657e0a-fbee-4a60-a5f3-b0abc3de50bd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:32.935800 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.943865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 344/688] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/role_assignments?user.id=b27b8079c98346efbc94b558e152da86&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:32.944830 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:32.984164 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:32.985453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:32.985877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:32.986324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:32.987611 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=2f8958e3b5dd46bb9e5c835214654ca8) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:32.988063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:32.990781 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:32.999728 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 345/689] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/role_assignments?user.id=2f8958e3b5dd46bb9e5c835214654ca8&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:33.000893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:33.025420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:33.096028 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ef56-1cd4-4e2a-8403-7c80772001b9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:33.167003 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:33.167790 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.168079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.168356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.181808 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa528d-e204-44ac-8877-013c99d078d2 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:33.200849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:33.202559 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:33.215809 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:33.225524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:33.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:33.227911 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:33.227911 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:33.227911 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:33.227911 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:33.227911 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 345/690] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:33.238558 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:33.249386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:33.250584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.251556 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.252088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.268071 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:33.270238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:33.291834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:33.293027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:33.294168 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:33.294168 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:33.294168 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:33.294168 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:33.296794 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 346/691] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:33.303950 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:33.379062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdee2ff-24c3-4af0-b3d0-4568c0ef3c4d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:33.389680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:33.419999 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:33.420250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.420496 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.420677 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.421467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:33.421914 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:33.423587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:33.427343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b043a-e3ea-47bf-beb0-56c44cdbb25b None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:33.442045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 346/692] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:33.445436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:33.479172 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:33.479894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.480234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.480901 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.481963 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:33.482308 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:33.484112 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:33.501526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 347/693] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:33.507377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:33.561437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:33.610550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76cfc3-3f9d-4a00-8e7f-41f6d2b34474 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:33.651148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd2edc-db2f-4a7a-8eae-ca63688d99e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:33.651848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd2edc-db2f-4a7a-8eae-ca63688d99e3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.652112 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd2edc-db2f-4a7a-8eae-ca63688d99e3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.652346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd2edc-db2f-4a7a-8eae-ca63688d99e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.662382 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd2edc-db2f-4a7a-8eae-ca63688d99e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:33.663946 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd2edc-db2f-4a7a-8eae-ca63688d99e3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:33.664165 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a945f46-e96d-4454-9d69-038fc5e14ea1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-fdbd2edc-db2f-4a7a-8eae-ca63688d99e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:33.678584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:33.679724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:33.680861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 347/694] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:33.681942 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7cf8d-0ae1-42f7-bfd8-0bf3995c5069 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.686146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7cf8d-0ae1-42f7-bfd8-0bf3995c5069 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.686146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7cf8d-0ae1-42f7-bfd8-0bf3995c5069 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.686553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 348/695] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:33.687005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7cf8d-0ae1-42f7-bfd8-0bf3995c5069 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:33.711964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d38fa-477c-468f-ac99-b00cefcd2e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:33.712807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d38fa-477c-468f-ac99-b00cefcd2e8a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.713128 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d38fa-477c-468f-ac99-b00cefcd2e8a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.713801 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d38fa-477c-468f-ac99-b00cefcd2e8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.722248 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d38fa-477c-468f-ac99-b00cefcd2e8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:33.724024 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:33.725089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d38fa-477c-468f-ac99-b00cefcd2e8a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-b68d38fa-477c-468f-ac99-b00cefcd2e8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:33.739894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:33.742406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 348/696] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:56:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe960-8126-4986-bc45-eee935013863 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.744934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe960-8126-4986-bc45-eee935013863 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.747172 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe960-8126-4986-bc45-eee935013863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.764541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-4dcbe960-8126-4986-bc45-eee935013863 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:33.765834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-4dcbe960-8126-4986-bc45-eee935013863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:33.813778 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7cf8d-0ae1-42f7-bfd8-0bf3995c5069 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:33.842900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 349/697] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:33.843846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbe960-8126-4986-bc45-eee935013863 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:33.863786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:33.874978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:33.875646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.875928 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.876207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.876971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:33.877280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:33.878795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:33.897923 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 349/698] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:33.898887 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:33.953765 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbe960-8126-4986-bc45-eee935013863 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:33.990193 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:33.990577 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:33.991305 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:33.991627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:33.992004 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:33.992913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:33.993216 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:33.994802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.004411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3958e4-af6a-4bad-8f65-f68eb28722e5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:34.015124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 350/699] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:56:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:34.016068 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:34.044556 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:34.044867 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:34.045149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:34.045423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:34.046065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:34.047562 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:34.058565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:34.061047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:34.063414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:34.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 350/700] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:56:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:34.065680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:34.100266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:34.127076 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e782e81-55b8-44ab-a03d-276a31bae2ed None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:34.163477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:34.164147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:34.164485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:34.164864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:34.165729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=f3d1c0e873e94d85b590846747152e24) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:34.166165 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:34.167748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.174905 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 351/701] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:34 2022] GET /identity/v3/role_assignments?user.id=f3d1c0e873e94d85b590846747152e24&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:34.175714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:34.185005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e195d0-eccd-4ecf-9dd6-8b9a41d0a45e None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:34.231256 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f422a-266a-4f96-825b-456272c7f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:34.231798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f422a-266a-4f96-825b-456272c7f002 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:34.232077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f422a-266a-4f96-825b-456272c7f002 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:34.232346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f422a-266a-4f96-825b-456272c7f002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:34.255989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:34.293497 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5b23-332b-4b9a-a12a-7b8dc953ef0c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:34.307485 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f422a-266a-4f96-825b-456272c7f002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:34.308979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f422a-266a-4f96-825b-456272c7f002 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.335445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:34.336517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:34.337045 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:34.337676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:34.338820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:34.339377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:34.341181 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.346732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 352/702] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:56:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:34.360845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:34.366637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:34.429198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 351/703] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:34.430729 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f422a-266a-4f96-825b-456272c7f002 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:34.487943 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:34.504000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101279a9-1dba-418a-bf96-dadc1bb073f7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:34.541701 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f422a-266a-4f96-825b-456272c7f002 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:34.543157 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:34.544162 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:34.544632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:34.545096 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:34.545959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:34.546473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:34.548119 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.555036 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 353/704] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:56:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:34.589176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:34.589847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:34.590151 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:34.590430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:34.603029 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:34.604604 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:34.622387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:34.623786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:34.625225 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:34.625225 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:34.625225 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:34.625225 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:34.628434 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 352/705] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:34.629984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:34.707453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:34.745695 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947100a-c288-4096-864f-33e0a6d03de3 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:34.784491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:34.785142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:34.785423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:34.785720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:34.786447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:34.786748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:34.788165 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.803350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 354/706] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:34.808032 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:34.922408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7352f-fde1-4549-84e4-f50e6bba4336 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:34.926615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:34.963067 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:34.963731 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:34.964010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:34.964284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:34.972388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:34.973920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:34.987570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:34.988760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:34.989900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:34.989900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:34.989900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:34.989900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:34.989900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:34.989900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:34.989900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:34.989900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:34.993199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 353/707] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:34.995912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:35.191456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f292df4-0370-4310-a4f1-bc9271c82544 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:35.223954 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:35.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8150fc6-8b87-4ec3-8b2c-cf6ac25709ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:35.233911 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150fc6-8b87-4ec3-8b2c-cf6ac25709ed None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:35.234698 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150fc6-8b87-4ec3-8b2c-cf6ac25709ed None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:35.235032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150fc6-8b87-4ec3-8b2c-cf6ac25709ed None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:35.236209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8150fc6-8b87-4ec3-8b2c-cf6ac25709ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:35.237002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8150fc6-8b87-4ec3-8b2c-cf6ac25709ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:35.238622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8150fc6-8b87-4ec3-8b2c-cf6ac25709ed None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:35.252250 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 355/708] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:35.258451 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:35.422883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948fdf5-0b81-42a9-a439-8661469ab43c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:35.475859 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0f8f5-0299-44f6-bfb1-ed51efb4e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:35.476526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0f8f5-0299-44f6-bfb1-ed51efb4e885 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:35.476814 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0f8f5-0299-44f6-bfb1-ed51efb4e885 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:35.477092 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0f8f5-0299-44f6-bfb1-ed51efb4e885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:35.477917 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0f8f5-0299-44f6-bfb1-ed51efb4e885 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=4fe070acb9a84d90baaa3b08a4fd3191) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:35.478263 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0f8f5-0299-44f6-bfb1-ed51efb4e885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:35.479799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0f8f5-0299-44f6-bfb1-ed51efb4e885 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:35.489465 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 354/709] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:35 2022] GET /identity/v3/role_assignments?user.id=4fe070acb9a84d90baaa3b08a4fd3191&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:35.574347 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf9781-d502-4d38-825d-b9dd359a94a8 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:35.574926 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf9781-d502-4d38-825d-b9dd359a94a8 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:35.574926 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf9781-d502-4d38-825d-b9dd359a94a8 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:35.576587 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 356/710] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:35.583127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a9e98-c41c-4cc2-847b-e967ad5b3233 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:35.583508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a9e98-c41c-4cc2-847b-e967ad5b3233 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:35.583701 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a9e98-c41c-4cc2-847b-e967ad5b3233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:35.612205 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-096a9e98-c41c-4cc2-847b-e967ad5b3233 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:35.613452 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-096a9e98-c41c-4cc2-847b-e967ad5b3233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:35.649872 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:35.692869 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33ebe5-847d-48bc-8268-62137cce3a15 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:35.693350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33ebe5-847d-48bc-8268-62137cce3a15 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:35.693791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33ebe5-847d-48bc-8268-62137cce3a15 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:35.695814 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 357/711] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:35.703230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe2f3d-fa98-455b-a48c-869af0e05a8a None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:35.703538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe2f3d-fa98-455b-a48c-869af0e05a8a None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:35.703813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe2f3d-fa98-455b-a48c-869af0e05a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:35.708776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 355/712] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:35.739278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-6dfe2f3d-fa98-455b-a48c-869af0e05a8a None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:35.740567 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-6dfe2f3d-fa98-455b-a48c-869af0e05a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:35.761106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a9e98-c41c-4cc2-847b-e967ad5b3233 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:35.872182 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 358/713] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:35.911042 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe2f3d-fa98-455b-a48c-869af0e05a8a None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:35.914327 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a9e98-c41c-4cc2-847b-e967ad5b3233 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:35.986629 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:35.988815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:35.990923 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:35.991459 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] PATH_INFO: `/v3/services/network` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:35.993152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:35.996226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:36.022318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:36.024101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:36.025611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:36.029199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 356/714] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:35 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:36.042181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:36.096703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:36.172313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe2f3d-fa98-455b-a48c-869af0e05a8a None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:36.202583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:36.224928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:36.225624 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:36.226214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:36.226501 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:36.239935 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:36.241692 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:36.254439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c36351-cc88-4bff-a9d5-1d7502b8d35c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:36.260496 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:36.261936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:36.263137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:36.263137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:36.263137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:36.263137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:36.263137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 359/715] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:36.273053 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:36.365206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:36.366239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:36.366653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:36.367109 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:36.368339 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] RBAC: Adding query filter params (name=network) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:36.371816 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:36.373707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:36.392744 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 357/716] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:56:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:36.400987 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:36.514608 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:36.517668 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567c4eb-43f9-45a8-9b86-e8853d34e4cb None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:36.536722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9b973-0d0d-4f34-b30c-56555c3d0f5c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:36.585180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:36.585831 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:36.586161 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:36.586456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:36.587166 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] RBAC: Adding query filter params (type=network) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:36.587470 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:36.588996 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:36.596703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 358/717] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:56:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:36.618266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36df24-35bf-45c1-a768-611e14763542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:36.619354 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36df24-35bf-45c1-a768-611e14763542 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:36.619354 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36df24-35bf-45c1-a768-611e14763542 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:36.619874 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36df24-35bf-45c1-a768-611e14763542 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:36.620444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36df24-35bf-45c1-a768-611e14763542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:36.620754 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36df24-35bf-45c1-a768-611e14763542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:36.622304 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36df24-35bf-45c1-a768-611e14763542 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:36.639884 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 360/718] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:36.647076 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:36.842583 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12ea7f-0f2a-44be-8915-3e3221d23bb7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:36.886374 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a86f09-19ed-49cb-9f26-3437cafd59f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:36.887229 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86f09-19ed-49cb-9f26-3437cafd59f9 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:36.887531 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86f09-19ed-49cb-9f26-3437cafd59f9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:36.887861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86f09-19ed-49cb-9f26-3437cafd59f9 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:36.888732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86f09-19ed-49cb-9f26-3437cafd59f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:36.890491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86f09-19ed-49cb-9f26-3437cafd59f9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:36.922661 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 359/719] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:56:36 2022] POST /identity/v3/users => generated 254 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:37.270279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559a927-732a-491c-95c9-4771975f7952 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:37.270628 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559a927-732a-491c-95c9-4771975f7952 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:37.270804 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559a927-732a-491c-95c9-4771975f7952 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:37.272761 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 361/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:37.281050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ea359-15ca-43d7-a3b3-9282ac47cec0 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:37.281394 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ea359-15ca-43d7-a3b3-9282ac47cec0 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:37.281687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ea359-15ca-43d7-a3b3-9282ac47cec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:37.303803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:37.317793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-9c8ea359-15ca-43d7-a3b3-9282ac47cec0 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:37.319646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-9c8ea359-15ca-43d7-a3b3-9282ac47cec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:37.451823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 360/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:37.504382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559a927-732a-491c-95c9-4771975f7952 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:37.536293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8455ee-1e87-456b-a212-605a3af1ac51 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:37.537096 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8455ee-1e87-456b-a212-605a3af1ac51 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:37.537606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8455ee-1e87-456b-a212-605a3af1ac51 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:37.540083 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 361/722] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:37.546260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090e3f9-694e-4e71-8b1b-6d5b2c06449b None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:37.547253 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090e3f9-694e-4e71-8b1b-6d5b2c06449b None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:37.547774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090e3f9-694e-4e71-8b1b-6d5b2c06449b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:37.548486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:37.583838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-9090e3f9-694e-4e71-8b1b-6d5b2c06449b None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:37.587990 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-9090e3f9-694e-4e71-8b1b-6d5b2c06449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:37.683304 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559a927-732a-491c-95c9-4771975f7952 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:37.755423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 362/723] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:37.765855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce12ef-6711-446e-8f6b-062f9e254de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:37.766594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce12ef-6711-446e-8f6b-062f9e254de9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:37.766885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce12ef-6711-446e-8f6b-062f9e254de9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:37.767230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce12ef-6711-446e-8f6b-062f9e254de9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:37.787486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce12ef-6711-446e-8f6b-062f9e254de9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-efce12ef-6711-446e-8f6b-062f9e254de9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:37.793175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:37.794669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:37.795919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 362/724] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:37.803435 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090e3f9-694e-4e71-8b1b-6d5b2c06449b None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:37.803705 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce12ef-6711-446e-8f6b-062f9e254de9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:37.966494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:37.973755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:37.987222 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce12ef-6711-446e-8f6b-062f9e254de9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:38.043748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:38.046340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:38.046340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:38.046340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:38.046340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:38.046340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:38.049001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:38.061383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090e3f9-694e-4e71-8b1b-6d5b2c06449b None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:38.067759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 363/725] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:38.079462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:38.165146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c75f4-96ff-43fa-8f24-4dec1f207b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:38.167315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c75f4-96ff-43fa-8f24-4dec1f207b52 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:38.168392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c75f4-96ff-43fa-8f24-4dec1f207b52 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:38.170498 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c75f4-96ff-43fa-8f24-4dec1f207b52 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:38.172108 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c75f4-96ff-43fa-8f24-4dec1f207b52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:38.175479 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c75f4-96ff-43fa-8f24-4dec1f207b52 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:38.227326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 363/726] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:56:37 2022] POST /identity/v3/services => generated 251 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:38.296371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f53384-cfe9-44b4-8b1f-52cb2f9dfb41 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:38.371789 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469e923-9620-4857-9c27-8c1afd57748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:38.372504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e923-9620-4857-9c27-8c1afd57748a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:38.372799 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e923-9620-4857-9c27-8c1afd57748a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:38.373186 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e923-9620-4857-9c27-8c1afd57748a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:38.386410 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469e923-9620-4857-9c27-8c1afd57748a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:38.388041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469e923-9620-4857-9c27-8c1afd57748a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:38.404820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-a469e923-9620-4857-9c27-8c1afd57748a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:38.423032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:38.425259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:38.427540 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:38.427540 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:38.427540 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:38.427540 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:38.427540 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 364/727] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:38.439356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c75f4-96ff-43fa-8f24-4dec1f207b52 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:38.531840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7bdaf-899e-41f2-b8e1-92f082ef725c None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:38.532233 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7bdaf-899e-41f2-b8e1-92f082ef725c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:38.532442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7bdaf-899e-41f2-b8e1-92f082ef725c None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:38.533922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 365/728] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:56:38.550848 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a80d4-ce75-438f-933e-4f52d7c46c2f None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:38.551066 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a80d4-ce75-438f-933e-4f52d7c46c2f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:38.551302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a80d4-ce75-438f-933e-4f52d7c46c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:38.579393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-6b1a80d4-ce75-438f-933e-4f52d7c46c2f None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:38.580696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-6b1a80d4-ce75-438f-933e-4f52d7c46c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:38.600603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c75f4-96ff-43fa-8f24-4dec1f207b52 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:38.632434 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:38.653392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:38.654054 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:38.654429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:38.654705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:38.655483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:38.655846 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:38.657348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:38.675027 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 364/729] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:38.676429 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 366/730] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:56:38.683032 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:38.723702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a80d4-ce75-438f-933e-4f52d7c46c2f None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:38.811567 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c7a9a-4b98-4216-a647-610d5ae6d9cb None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:38.860368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:38.861165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:38.861551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:38.861859 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] PATH_INFO: `/v3/users/swift` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:38.872599 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:38.874444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:38.897460 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:38.898760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:38.899964 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:38.900766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 365/731] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:38.906732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:38.941564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a80d4-ce75-438f-933e-4f52d7c46c2f None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:38.976147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:38.985731 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003c720-bff8-4071-a3e3-cf8c02a3b910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:38.986993 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003c720-bff8-4071-a3e3-cf8c02a3b910 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:38.987470 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003c720-bff8-4071-a3e3-cf8c02a3b910 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:38.987930 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003c720-bff8-4071-a3e3-cf8c02a3b910 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:39.004087 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003c720-bff8-4071-a3e3-cf8c02a3b910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-4003c720-bff8-4071-a3e3-cf8c02a3b910 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:39.010423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:39.011937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:39.013204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:39.016860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 367/732] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:39.021539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:39.024277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003c720-bff8-4071-a3e3-cf8c02a3b910 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:39.127304 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94aea0-c023-499a-a9e2-926672ac03af None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:39.179919 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:39.180656 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:39.180940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:39.181211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:39.182153 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:39.182465 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:39.184046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:39.211853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 366/733] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:56:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:39.217923 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:39.341270 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad46a-5723-4541-a860-565bca405a4f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:39.345457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003c720-bff8-4071-a3e3-cf8c02a3b910 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:39.346554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:39.420096 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:39.420650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:39.420939 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:39.421242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:39.422077 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:39.422406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:39.423881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:39.428967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aff9c0-9e10-41ce-95e1-40c4abaf56d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:39.429686 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aff9c0-9e10-41ce-95e1-40c4abaf56d6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:39.429990 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aff9c0-9e10-41ce-95e1-40c4abaf56d6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:39.430309 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aff9c0-9e10-41ce-95e1-40c4abaf56d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:39.448108 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 368/734] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:39.456753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:39.585342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aff9c0-9e10-41ce-95e1-40c4abaf56d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:39.586405 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:39.587532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aff9c0-9e10-41ce-95e1-40c4abaf56d6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:39.657935 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8f02b-6c6d-4a87-879c-b0f091897aa0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:39.700406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df033a-aa3f-4ede-a9f3-17f82ec0d071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:39.701492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df033a-aa3f-4ede-a9f3-17f82ec0d071 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:39.701970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df033a-aa3f-4ede-a9f3-17f82ec0d071 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:39.702486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df033a-aa3f-4ede-a9f3-17f82ec0d071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:39.717123 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df033a-aa3f-4ede-a9f3-17f82ec0d071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:39.719201 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df033a-aa3f-4ede-a9f3-17f82ec0d071 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-89df033a-aa3f-4ede-a9f3-17f82ec0d071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:39.738030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:39.739900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:39.741340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:39.741340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:39.741340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:39.741340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:39.748621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 369/735] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:39.749696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96ffb-0b00-42cd-9fbb-aa5c10ebd5d6 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:39.750139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96ffb-0b00-42cd-9fbb-aa5c10ebd5d6 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:39.750628 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96ffb-0b00-42cd-9fbb-aa5c10ebd5d6 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:39.753758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 370/736] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:39.755440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc96ffb-0b00-42cd-9fbb-aa5c10ebd5d6 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:39.780033 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 367/737] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:39 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:56:39.782579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9032a85-52b9-4eaa-b7d9-11f54e9cdaf3 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:39.783176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9032a85-52b9-4eaa-b7d9-11f54e9cdaf3 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:39.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9032a85-52b9-4eaa-b7d9-11f54e9cdaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:39.795684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:39.816473 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-a9032a85-52b9-4eaa-b7d9-11f54e9cdaf3 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:39.817745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-a9032a85-52b9-4eaa-b7d9-11f54e9cdaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:39.897864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc96ffb-0b00-42cd-9fbb-aa5c10ebd5d6 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:39.901350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:39.941475 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:39.942312 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:39.942720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:39.943070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:39.943900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:39.944271 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:39.945850 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:39.963246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 368/738] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:56:39.965353 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9032a85-52b9-4eaa-b7d9-11f54e9cdaf3 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:39.965753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 371/739] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:39.966908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:40.114289 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be6363-fd05-4cd6-bb09-7f8798cb8309 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:40.134586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9032a85-52b9-4eaa-b7d9-11f54e9cdaf3 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:40.160184 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90222-1292-400c-92c9-a4f498fb8ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:40.161171 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90222-1292-400c-92c9-a4f498fb8ea9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.161676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90222-1292-400c-92c9-a4f498fb8ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.162161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90222-1292-400c-92c9-a4f498fb8ea9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.171665 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab90222-1292-400c-92c9-a4f498fb8ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:40.173597 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab90222-1292-400c-92c9-a4f498fb8ea9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:40.176697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:40.177367 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.177649 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.177983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-5ab90222-1292-400c-92c9-a4f498fb8ea9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:40.188784 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:40.196109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 372/740] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:56:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84903b5-498b-453a-850a-3484c2087d4f None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.197411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84903b5-498b-453a-850a-3484c2087d4f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.198376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84903b5-498b-453a-850a-3484c2087d4f None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.198376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 373/741] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:40.198376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84903b5-498b-453a-850a-3484c2087d4f None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:40.207760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:40.210556 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:40.224811 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:40.241516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:40.243008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:40.244187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:40.244187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:40.244187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:40.244187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:40.244187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 369/742] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:40.245329 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:40.327619 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84903b5-498b-453a-850a-3484c2087d4f None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:40.372625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56836ea-4fd8-449c-8a4e-408570024eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:40.373411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56836ea-4fd8-449c-8a4e-408570024eda None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.373791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56836ea-4fd8-449c-8a4e-408570024eda None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.374203 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56836ea-4fd8-449c-8a4e-408570024eda None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.375038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56836ea-4fd8-449c-8a4e-408570024eda None admin] RBAC: Authorizing `identity:create_service()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:40.379739 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56836ea-4fd8-449c-8a4e-408570024eda None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:40.388476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de793d-1cbc-4af6-8318-c8a04d5dee87 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:40.414577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 374/743] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:56:40 2022] POST /identity/v3/services => generated 240 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:40.416680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa8bdc-1372-48ba-aab8-b361d19c5f1d None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.417119 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa8bdc-1372-48ba-aab8-b361d19c5f1d None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.417542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa8bdc-1372-48ba-aab8-b361d19c5f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.445937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-34aa8bdc-1372-48ba-aab8-b361d19c5f1d None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:40.447510 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-34aa8bdc-1372-48ba-aab8-b361d19c5f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:40.462332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:40.463096 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.463527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.463883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.465087 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:40.465540 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:40.467466 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:40.492226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 370/744] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:56:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:40.498022 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:40.568762 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 375/745] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:40.568762 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa8bdc-1372-48ba-aab8-b361d19c5f1d None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:40.571245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:40.587286 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:40.664326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ef4a5-b797-4827-bef7-2ec35283fc54 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:40.707192 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa8bdc-1372-48ba-aab8-b361d19c5f1d None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:40.710185 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:40.710860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.711148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.711443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.712153 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:40.712466 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:40.713967 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:40.728552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 371/746] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:40.735241 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:40.783653 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:40.785833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.786733 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.787585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.809521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:40.812972 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:40.843702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:40.845714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:40.847818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:40.847818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:40.847818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:40.847818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:40.858457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 376/747] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:40.860468 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:40.871206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:40.876841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cb181-287d-4046-a309-6217981ee35d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:40.921170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ca801-42eb-496c-bb2c-98fdc186e1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:40.921880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca801-42eb-496c-bb2c-98fdc186e1bc None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:40.922204 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca801-42eb-496c-bb2c-98fdc186e1bc None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:40.922540 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca801-42eb-496c-bb2c-98fdc186e1bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:40.937251 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ca801-42eb-496c-bb2c-98fdc186e1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-fe5ca801-42eb-496c-bb2c-98fdc186e1bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:40.948579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:40.951738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:40.954084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:40.957369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 372/748] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:40.958957 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ca801-42eb-496c-bb2c-98fdc186e1bc None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:41.014283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:41.084120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dabe3-4af7-47ae-807c-3ca003382328 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:41.108034 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ca801-42eb-496c-bb2c-98fdc186e1bc None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:41.141883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:41.143151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.143632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.144107 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.157193 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:41.159867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:41.162452 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd655a57-1cfd-47df-9017-51dff95f2673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:41.163293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd655a57-1cfd-47df-9017-51dff95f2673 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.163658 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd655a57-1cfd-47df-9017-51dff95f2673 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.163937 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd655a57-1cfd-47df-9017-51dff95f2673 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.164673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd655a57-1cfd-47df-9017-51dff95f2673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:41.164983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd655a57-1cfd-47df-9017-51dff95f2673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:41.166516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd655a57-1cfd-47df-9017-51dff95f2673 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:41.182598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 373/749] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:41.183873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:41.185903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:41.188444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:41.188444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:41.188444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:41.188444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:41.188444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:41.188444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:41.188444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:41.188444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:41.189860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed8ee-7e52-4cf6-9723-6269bc2a8ae0 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.189860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed8ee-7e52-4cf6-9723-6269bc2a8ae0 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.189860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed8ee-7e52-4cf6-9723-6269bc2a8ae0 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.189860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 374/750] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:41.189860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eed8ee-7e52-4cf6-9723-6269bc2a8ae0 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:41.197764 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 377/751] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:41.197764 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:41.237898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:41.338820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe93590-531c-46ed-8717-60e64805275a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:41.356099 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eed8ee-7e52-4cf6-9723-6269bc2a8ae0 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:41.374441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:41.386413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:41.387087 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.387372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.387650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.395508 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:41.396299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97d0c4-b0c7-4312-b87d-a4ba6a5298ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:41.396948 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97d0c4-b0c7-4312-b87d-a4ba6a5298ff None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.397075 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:41.397258 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97d0c4-b0c7-4312-b87d-a4ba6a5298ff None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.397510 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97d0c4-b0c7-4312-b87d-a4ba6a5298ff None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.398276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97d0c4-b0c7-4312-b87d-a4ba6a5298ff None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:41.398585 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97d0c4-b0c7-4312-b87d-a4ba6a5298ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:41.399983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97d0c4-b0c7-4312-b87d-a4ba6a5298ff None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:41.408935 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 375/752] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:41.410245 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:41.411309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:41.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:41.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:41.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:41.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:41.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:41.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:41.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:41.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:41.412970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d23e0-7882-4f9e-b99c-3acf66075b69 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.412970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d23e0-7882-4f9e-b99c-3acf66075b69 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.412970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d23e0-7882-4f9e-b99c-3acf66075b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.414917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 378/753] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:41.415983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:41.430446 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-b60d23e0-7882-4f9e-b99c-3acf66075b69 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:41.431684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b60d23e0-7882-4f9e-b99c-3acf66075b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:41.506177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 376/754] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:41.506177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d23e0-7882-4f9e-b99c-3acf66075b69 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:41.522532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:41.542586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad553b1-9f00-4ea7-bbe1-435a5c3da357 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:41.585422 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:41.586069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.586387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.586656 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.587557 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:41.587886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:41.589432 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:41.621916 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d23e0-7882-4f9e-b99c-3acf66075b69 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:41.666908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:41.667642 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.667900 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.668209 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.681218 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:41.682819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:41.700825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:41.702065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:41.703512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:41.703512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:41.703512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:41.703512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:41.706556 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 377/755] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:41.707909 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:41.735603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 379/756] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:41.737440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:41.792378 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:41.810283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:41.845258 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376641c7-12ba-42f7-abe6-67c91932f9d3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:41.877731 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8d900-6382-4ba0-b93c-117aee03bc3b None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:41.896386 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:41.897342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.897756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.898195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.899257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:41.899706 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:41.901655 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:41.918254 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 378/757] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:41.919744 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:41.927662 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:41.929145 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:41.929407 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:41.929776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:41.930577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:41.932114 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:41.945442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:41.951962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:41.954081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 380/758] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:56:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:41.956131 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:42.062467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331e6ff-0cc5-441e-9ff9-3abe978105c3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:42.094737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:42.110256 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:42.110844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.111132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.111346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.112180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:42.112589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:42.114965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:42.116130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c3d3-e979-4952-9d1d-179eeea59922 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:42.129554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 379/759] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:42.130447 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:42.131285 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:42.171928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:42.172897 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.173348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.173917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.175045 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=f3d1c0e873e94d85b590846747152e24) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:42.175527 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:42.177322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:42.183776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 381/760] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:41 2022] GET /identity/v3/role_assignments?user.id=f3d1c0e873e94d85b590846747152e24&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:42.184866 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:42.276890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f96e1e-99d5-497c-8a7a-93332d836692 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:42.316564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b6378-e3a9-45ce-a5d8-f0db78f4eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:42.317305 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b6378-e3a9-45ce-a5d8-f0db78f4eff6 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.317587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b6378-e3a9-45ce-a5d8-f0db78f4eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.317863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b6378-e3a9-45ce-a5d8-f0db78f4eff6 None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/4fe070acb9a84d90baaa3b08a4fd3191/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.320143 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b6378-e3a9-45ce-a5d8-f0db78f4eff6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=4fe070acb9a84d90baaa3b08a4fd3191, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:42.321624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b6378-e3a9-45ce-a5d8-f0db78f4eff6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:42.342800 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed49c-a347-412a-9b3f-4b8a2b3be1d1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:42.346592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 380/761] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:56:42 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/4fe070acb9a84d90baaa3b08a4fd3191/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:42.349639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b6378-e3a9-45ce-a5d8-f0db78f4eff6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:42.439039 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:42.439720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.440006 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.440320 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.440616 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:42.441776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:42.442089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:42.443580 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:42.451558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 382/762] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:56:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:42.459435 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:42.506222 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:42.514479 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b6378-e3a9-45ce-a5d8-f0db78f4eff6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:42.556761 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:42.557543 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.557844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.558136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.568001 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:42.569696 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:42.577969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df523d12-bff0-4fc1-a024-f869885d03af None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:42.583275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:42.586424 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 381/763] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:56:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:42.594698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:42.615442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:42.615874 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.616125 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.616399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.617119 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:42.617430 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:42.618911 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:42.633276 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 383/764] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:56:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:42.640170 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:42.771155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fb4c4-48b7-4c09-91da-9c7f4cd92604 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:42.775148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:42.786223 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bda43-19d7-4910-a68d-22678075da5f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:42.789230 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:42.825958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ed465-853a-4245-b40d-5dd5234641ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:42.827958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ed465-853a-4245-b40d-5dd5234641ae None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.828463 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ed465-853a-4245-b40d-5dd5234641ae None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.828971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ed465-853a-4245-b40d-5dd5234641ae None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.830430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ed465-853a-4245-b40d-5dd5234641ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:42.830975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ed465-853a-4245-b40d-5dd5234641ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:42.833063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ed465-853a-4245-b40d-5dd5234641ae None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:42.851648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8c9c2-41c3-493c-93fb-db033925ff51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:42.852818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c9c2-41c3-493c-93fb-db033925ff51 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.853441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c9c2-41c3-493c-93fb-db033925ff51 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.854031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c9c2-41c3-493c-93fb-db033925ff51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.855531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8c9c2-41c3-493c-93fb-db033925ff51 None admin] RBAC: Adding query filter params (interface=public, service_id=a3821c24ff214e81a552721006399e34, region_id=RegionOne) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:42.856197 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8c9c2-41c3-493c-93fb-db033925ff51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:42.859042 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 382/765] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:56:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:42.864124 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8c9c2-41c3-493c-93fb-db033925ff51 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:42.866026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bb957-372e-46a3-8a9e-18aabc72a437 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.866572 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bb957-372e-46a3-8a9e-18aabc72a437 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.867061 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bb957-372e-46a3-8a9e-18aabc72a437 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.869011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 383/766] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:42.870072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bb957-372e-46a3-8a9e-18aabc72a437 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:42.884416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 384/767] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:56:42 2022] GET /identity/v3/endpoints?service_id=a3821c24ff214e81a552721006399e34&interface=public®ion_id=RegionOne => generated 194 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:42.889224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7449-eb95-4b3b-bd48-7416ad18baf3 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:42.889224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7449-eb95-4b3b-bd48-7416ad18baf3 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:42.889224 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7449-eb95-4b3b-bd48-7416ad18baf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:42.927923 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-c0fd7449-eb95-4b3b-bd48-7416ad18baf3 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:42.930821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-c0fd7449-eb95-4b3b-bd48-7416ad18baf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:43.000558 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bb957-372e-46a3-8a9e-18aabc72a437 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:43.071273 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42d60a-0dcd-4510-8ce7-93036aa6562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:43.071273 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42d60a-0dcd-4510-8ce7-93036aa6562f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:43.078208 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 385/768] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:43.078208 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd7449-eb95-4b3b-bd48-7416ad18baf3 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:43.078512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42d60a-0dcd-4510-8ce7-93036aa6562f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:43.078512 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42d60a-0dcd-4510-8ce7-93036aa6562f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:43.132309 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:43.188980 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42d60a-0dcd-4510-8ce7-93036aa6562f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:43.191667 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42d60a-0dcd-4510-8ce7-93036aa6562f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:43.261459 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:43.270692 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd7449-eb95-4b3b-bd48-7416ad18baf3 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:43.322499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202b868-b21b-4bf0-baff-16b61b6ce7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:43.323114 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202b868-b21b-4bf0-baff-16b61b6ce7eb None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:43.323504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202b868-b21b-4bf0-baff-16b61b6ce7eb None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:43.323788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202b868-b21b-4bf0-baff-16b61b6ce7eb None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:43.324474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202b868-b21b-4bf0-baff-16b61b6ce7eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:43.326010 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202b868-b21b-4bf0-baff-16b61b6ce7eb None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:43.332813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 386/769] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:56:43 2022] GET /identity/v3/services => generated 772 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:43.333832 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202b868-b21b-4bf0-baff-16b61b6ce7eb None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:43.486415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202b868-b21b-4bf0-baff-16b61b6ce7eb None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:43.538222 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:43.539243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:43.539566 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:43.539855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] PATH_INFO: `/v3/services/network` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:43.540517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:43.542424 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:43.542884 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 384/770] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:56:43.550058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42d60a-0dcd-4510-8ce7-93036aa6562f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:43.571313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:43.575956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:43.584074 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 387/771] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:43 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:43.595118 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:43.655118 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:43.710498 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:43.787255 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42d60a-0dcd-4510-8ce7-93036aa6562f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:43.798476 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b75f4b-48d0-415e-9275-acb2651fc914 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:43.853672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:43.854685 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:43.855341 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:43.855798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:43.871877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:43.872924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:43.873382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:43.873820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:43.875701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] RBAC: Adding query filter params (name=network) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:43.876560 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:43.877635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:43.878984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:43.880323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:43.888958 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 388/772] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:56:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:43.902466 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:43.929051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:43.930338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:43.931517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:43.931517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:43.931517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:43.931517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:43.935726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 385/773] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:43.943864 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:44.164786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:44.188265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cffa2-356f-4366-aa82-6cc2f53efeef None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:44.200464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11085f0-b8d5-4ff5-bfcf-85c501731d2f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:44.220043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:44.259920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:44.260598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:44.260894 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:44.261185 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:44.261916 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:44.262261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:44.263747 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:44.282089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:44.282664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:44.282955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:44.283242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:44.283967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] RBAC: Adding query filter params (type=network) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:44.284270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:44.285720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:44.295681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 386/774] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:44.302879 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 389/775] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:56:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:44.304024 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:44.312880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:44.652240 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636bfbf-861f-46d9-a9b2-0cade6a745ec None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:44.661072 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7de1d-9406-45b0-a297-17a379e61f64 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:44.742143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:44.742143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:44.742143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:44.742143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:44.742858 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] RBAC: Adding query filter params (interface=public, service_id=0577ba86a4ce4d04b5d053f3ec03414a, region_id=RegionOne) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:44.742858 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:44.743419 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:44.748583 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:44.749798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:44.750325 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:44.750815 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:44.761995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 390/776] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:56:44 2022] GET /identity/v3/endpoints?service_id=0577ba86a4ce4d04b5d053f3ec03414a&interface=public®ion_id=RegionOne => generated 194 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:44.763169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:44.766296 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:44.771374 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:44.785817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:44.788040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:44.790823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 387/777] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 484 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:44.794521 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:44.833778 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:44.847859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:44.918162 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f926b-48c5-4305-9f54-24d8f27885db None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:44.959415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:44.960084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:44.960429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:44.960732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:44.961488 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:44.961866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:44.963336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:44.980486 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 388/778] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:44.987031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:45.105946 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:45.116205 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3469e3-1f84-4ddb-bff3-0ff277dd6765 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:45.142579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c03f29-ed7a-4b65-8a5e-41f6fa24c500 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:45.158956 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242593b5-f61f-4e2a-8e21-b9eb68a38f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:45.159735 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242593b5-f61f-4e2a-8e21-b9eb68a38f0c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.160029 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242593b5-f61f-4e2a-8e21-b9eb68a38f0c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.160312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242593b5-f61f-4e2a-8e21-b9eb68a38f0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.161227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242593b5-f61f-4e2a-8e21-b9eb68a38f0c None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=ac2817fe7c654a29abe66f6e51b1dec7) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:45.161547 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242593b5-f61f-4e2a-8e21-b9eb68a38f0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:45.163244 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242593b5-f61f-4e2a-8e21-b9eb68a38f0c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:45.170943 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 389/779] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:44 2022] GET /identity/v3/role_assignments?user.id=ac2817fe7c654a29abe66f6e51b1dec7&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:45.232428 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a420d-5c76-4e30-a243-0a0d8818332a None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.232770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a420d-5c76-4e30-a243-0a0d8818332a None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.233110 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a420d-5c76-4e30-a243-0a0d8818332a None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.234871 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 390/780] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:56:45.243200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e7d19-cc17-4e28-8bb6-b5e8b36a6bb4 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.243545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e7d19-cc17-4e28-8bb6-b5e8b36a6bb4 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.243883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e7d19-cc17-4e28-8bb6-b5e8b36a6bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.267824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da288bbf-34ae-43c9-91d0-9d8ea99ef832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:45.268778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da288bbf-34ae-43c9-91d0-9d8ea99ef832 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.269059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da288bbf-34ae-43c9-91d0-9d8ea99ef832 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.269375 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da288bbf-34ae-43c9-91d0-9d8ea99ef832 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.270203 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da288bbf-34ae-43c9-91d0-9d8ea99ef832 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:45.272078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da288bbf-34ae-43c9-91d0-9d8ea99ef832 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:45.286594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 391/781] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:56:44 2022] GET /identity/v3/services => generated 772 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:45.308206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-ca8e7d19-cc17-4e28-8bb6-b5e8b36a6bb4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:45.309779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-ca8e7d19-cc17-4e28-8bb6-b5e8b36a6bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:45.438846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84fe76-1641-4251-896d-98b6ee71ecdb None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.439126 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84fe76-1641-4251-896d-98b6ee71ecdb None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.439549 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84fe76-1641-4251-896d-98b6ee71ecdb None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.446079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 392/783] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:45.446484 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 391/783] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:45.451995 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd20be-6bf7-404e-a698-6e95d620e63a None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.452294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd20be-6bf7-404e-a698-6e95d620e63a None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.452617 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd20be-6bf7-404e-a698-6e95d620e63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.469614 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84fe76-1641-4251-896d-98b6ee71ecdb None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:45.477821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-74fd20be-6bf7-404e-a698-6e95d620e63a None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:45.480650 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-74fd20be-6bf7-404e-a698-6e95d620e63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:45.593991 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 392/784] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:45.622894 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd20be-6bf7-404e-a698-6e95d620e63a None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:45.645755 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:45.651969 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:45.673033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84fe76-1641-4251-896d-98b6ee71ecdb None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:45.713619 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde5679-aa06-4a34-806d-7e4cd824a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:45.714759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5679-aa06-4a34-806d-7e4cd824a447 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.715298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5679-aa06-4a34-806d-7e4cd824a447 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.715881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5679-aa06-4a34-806d-7e4cd824a447 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.728982 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde5679-aa06-4a34-806d-7e4cd824a447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-ccde5679-aa06-4a34-806d-7e4cd824a447 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:45.735047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:45.736281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:45.737519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:45.740946 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 393/785] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:45.747430 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde5679-aa06-4a34-806d-7e4cd824a447 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:45.752841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd20be-6bf7-404e-a698-6e95d620e63a None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:45.806130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220293-a250-4770-91c4-a538daa1f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:45.807068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220293-a250-4770-91c4-a538daa1f6ce None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.807208 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220293-a250-4770-91c4-a538daa1f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.807490 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220293-a250-4770-91c4-a538daa1f6ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.824475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29220293-a250-4770-91c4-a538daa1f6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-29220293-a250-4770-91c4-a538daa1f6ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:45.837031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:45.838420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:45.843680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 393/786] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:45.856433 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220293-a250-4770-91c4-a538daa1f6ce None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:45.881536 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde5679-aa06-4a34-806d-7e4cd824a447 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:45.924520 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:45.925731 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:45.926366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:45.926959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:45.928164 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:45.928777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:45.930823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:45.942639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 394/787] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:45.953062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:45.966592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:46.061636 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:46.112287 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220293-a250-4770-91c4-a538daa1f6ce None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:46.190143 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:46.190778 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:46.191426 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:46.191706 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:46.192492 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:46.193079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:46.196659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:46.203942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913599d-648a-4e2e-bf3b-e1240e804522 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:46.215009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 394/788] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:46.223138 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:46.286656 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:46.287342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:46.287631 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:46.289386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:46.315892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:46.318073 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:46.349934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:46.351294 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:46.353831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:46.353831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:46.353831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:46.353831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:46.353831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 395/789] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:46.359258 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:46.461980 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289627a7-86ce-410b-8a96-c31728591178 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:46.490581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:46.550598 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969fcd7-e9d3-4028-8c9d-6264206634cf None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:46.579017 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:46.579711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:46.579997 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:46.580275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:46.592120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:46.593117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:46.593409 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:46.593726 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:46.594548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:46.594864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:46.596518 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:46.600515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:46.611665 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:46.613676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:46.615371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 396/790] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:46.622165 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:46.645611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:46.646895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:46.648087 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:46.648087 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:46.648087 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:46.648087 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:46.656266 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 395/791] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:46.662336 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:46.767147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a45d0b-588b-474c-b9d2-601e1e6e06b7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:46.786818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:46.817451 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1bc31-c8f9-480c-99f1-02be66bafeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:46.818839 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1bc31-c8f9-480c-99f1-02be66bafeec None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:46.819471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1bc31-c8f9-480c-99f1-02be66bafeec None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:46.819875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1bc31-c8f9-480c-99f1-02be66bafeec None admin] PATH_INFO: `/v3/users/swift` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:46.828699 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43aaf-30d2-4294-9f26-8c0200caac1b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:46.831364 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1bc31-c8f9-480c-99f1-02be66bafeec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:46.833349 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1bc31-c8f9-480c-99f1-02be66bafeec None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-3ed1bc31-c8f9-480c-99f1-02be66bafeec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:46.849898 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:46.852437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:46.854989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:46.856970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 397/792] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:46.858598 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d8a93-51dd-432b-a890-ec444c4da31f None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:46.859054 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d8a93-51dd-432b-a890-ec444c4da31f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:46.859396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d8a93-51dd-432b-a890-ec444c4da31f None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:46.861402 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 398/793] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:46.868922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d8a93-51dd-432b-a890-ec444c4da31f None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:46.903045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27936c6-3cf0-4b65-9c36-5b0cf9a82a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:46.903888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27936c6-3cf0-4b65-9c36-5b0cf9a82a04 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:46.904231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27936c6-3cf0-4b65-9c36-5b0cf9a82a04 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:46.904520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27936c6-3cf0-4b65-9c36-5b0cf9a82a04 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:46.905412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27936c6-3cf0-4b65-9c36-5b0cf9a82a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:46.905794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27936c6-3cf0-4b65-9c36-5b0cf9a82a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:46.908302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27936c6-3cf0-4b65-9c36-5b0cf9a82a04 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:46.937625 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 396/794] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:46.942563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc701ec-565c-431e-a19b-9bece3e41029 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:46.942942 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc701ec-565c-431e-a19b-9bece3e41029 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:46.943162 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc701ec-565c-431e-a19b-9bece3e41029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:46.969065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-abc701ec-565c-431e-a19b-9bece3e41029 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:46.970552 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-abc701ec-565c-431e-a19b-9bece3e41029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:47.041749 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:47.063415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d8a93-51dd-432b-a890-ec444c4da31f None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:47.074196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 397/795] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:47.074196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc701ec-565c-431e-a19b-9bece3e41029 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:47.132580 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:47.133456 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:47.133910 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:47.134317 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:47.135388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:47.135809 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:47.137541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:47.163661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 399/796] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:56:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:47.167176 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:47.218458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc701ec-565c-431e-a19b-9bece3e41029 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:47.223358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:47.273254 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:47.274582 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:47.275031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:47.275535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:47.289708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:47.292607 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:47.318330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:47.319756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:56:47.321158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:47.327011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 398/797] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:56:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:47.328801 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6b40-4d35-490f-9c21-2f3012a44b9d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:47.328953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:47.393938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:47.398710 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:47.400573 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:47.401149 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:47.401650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:47.402980 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:47.406041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:47.426790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:47.428387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:56:47.429714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:47.434583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 400/798] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 08:56:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:47.436395 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:47.495329 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3b96b-eb37-4569-8a9e-d0e9d95544e3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:47.581707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:47.582760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:47.583138 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:47.583567 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:47.604481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:47.609195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:47.625564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:47.639881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:47.641360 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:47.642729 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:47.642729 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:47.642729 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:47.642729 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:47.643012 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57ba0-ef97-4058-bc66-180c3f55b003 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:47.644999 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 399/799] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:47.646675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:47.712988 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:47.714048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:47.714573 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:47.715472 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:47.716984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:47.717543 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:47.719471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:47.750974 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 401/800] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:56:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:47.752086 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:47.795140 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:47.804960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d859e2-9dd8-4c13-985f-d89620dba144 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:47.848184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:47.849924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:47.849924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:47.849924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:47.849924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:47.850794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:47.852280 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:47.862170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 400/801] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:56:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:47.862170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:47.948357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dfe36-bfe5-4072-bdf9-61cae98e09d1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:47.992616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6ba84-8c34-4a2c-b21f-d50d941578e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:47.993498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6ba84-8c34-4a2c-b21f-d50d941578e0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:47.993774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6ba84-8c34-4a2c-b21f-d50d941578e0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:47.994054 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6ba84-8c34-4a2c-b21f-d50d941578e0 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:47.994841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6ba84-8c34-4a2c-b21f-d50d941578e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:47.995149 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6ba84-8c34-4a2c-b21f-d50d941578e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:47.996590 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6ba84-8c34-4a2c-b21f-d50d941578e0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:48.003935 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a6bb-9560-4ceb-90fc-6c23955c91f1 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:48.011190 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 402/802] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:48.012703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25067-6697-47df-9bd4-91aa9ccb5e4a None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.012981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25067-6697-47df-9bd4-91aa9ccb5e4a None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.013265 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25067-6697-47df-9bd4-91aa9ccb5e4a None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.016705 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 403/803] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:48.016705 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25067-6697-47df-9bd4-91aa9ccb5e4a None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:48.039552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:48.061641 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde3cbb-3be5-4682-b25d-4b836dadee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:48.062369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde3cbb-3be5-4682-b25d-4b836dadee08 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.062646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde3cbb-3be5-4682-b25d-4b836dadee08 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.062919 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde3cbb-3be5-4682-b25d-4b836dadee08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.104102 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:48.157133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25067-6697-47df-9bd4-91aa9ccb5e4a None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:48.161820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde3cbb-3be5-4682-b25d-4b836dadee08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:48.163421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde3cbb-3be5-4682-b25d-4b836dadee08 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:48.203258 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de8544-5a5c-4cd5-8a79-f5c85df51e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:48.204635 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de8544-5a5c-4cd5-8a79-f5c85df51e41 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.205123 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de8544-5a5c-4cd5-8a79-f5c85df51e41 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.205596 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de8544-5a5c-4cd5-8a79-f5c85df51e41 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.206870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de8544-5a5c-4cd5-8a79-f5c85df51e41 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:48.207400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de8544-5a5c-4cd5-8a79-f5c85df51e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:48.209848 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de8544-5a5c-4cd5-8a79-f5c85df51e41 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:48.218525 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 404/804] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 08:56:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:48.220702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28424dc5-73e2-4dee-ab76-f4a351f507bf None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.221324 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28424dc5-73e2-4dee-ab76-f4a351f507bf None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.221773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28424dc5-73e2-4dee-ab76-f4a351f507bf None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.224155 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 405/805] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:48.225661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28424dc5-73e2-4dee-ab76-f4a351f507bf None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:48.322521 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 401/806] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:56:48.324150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f787-1445-4d50-8fc9-4c07186c25c4 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.324562 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f787-1445-4d50-8fc9-4c07186c25c4 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.324862 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f787-1445-4d50-8fc9-4c07186c25c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.326837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:48.345877 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-a5d9f787-1445-4d50-8fc9-4c07186c25c4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:48.347338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-a5d9f787-1445-4d50-8fc9-4c07186c25c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:48.359120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28424dc5-73e2-4dee-ab76-f4a351f507bf None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:48.407344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:48.408194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.408479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.408785 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.420166 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:48.422309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:48.430979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:48.443716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:48.445041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:48.446217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 406/807] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:48.446841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:48.452526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 402/808] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:48.455440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363a676-c2ec-434b-9893-fbf56d4d69b6 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.455968 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363a676-c2ec-434b-9893-fbf56d4d69b6 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.456501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363a676-c2ec-434b-9893-fbf56d4d69b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.482723 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-5363a676-c2ec-434b-9893-fbf56d4d69b6 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:48.484158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-5363a676-c2ec-434b-9893-fbf56d4d69b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:48.569299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 403/809] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:48.570226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363a676-c2ec-434b-9893-fbf56d4d69b6 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:48.592143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365fe0a-56ac-4d40-823b-b7dba5199d23 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:48.630844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc0f54-cb57-4656-a19a-419b4b3c409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:48.631525 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0f54-cb57-4656-a19a-419b4b3c409c None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.631853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0f54-cb57-4656-a19a-419b4b3c409c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.632102 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0f54-cb57-4656-a19a-419b4b3c409c None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.632867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc0f54-cb57-4656-a19a-419b4b3c409c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:48.634436 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc0f54-cb57-4656-a19a-419b4b3c409c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:48.645952 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:48.666933 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 407/810] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:56:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:48.668920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc0f54-cb57-4656-a19a-419b4b3c409c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:48.706500 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363a676-c2ec-434b-9893-fbf56d4d69b6 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:48.742652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:48.779682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:48.780524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.780936 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.781451 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.798672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:48.800478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:48.831289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:48.832678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:48.834069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:48.834069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:48.834069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:48.834069 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:48.839905 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc0f54-cb57-4656-a19a-419b4b3c409c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:48.841882 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 404/811] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:48.843808 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:48.886059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:48.886707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:48.886984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:48.887262 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:48.888183 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:48.888490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:48.890055 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:48.904542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 408/812] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:48.906300 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:48.975718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:49.036439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1c5f3-cddc-4929-a234-e8b35fbacfc8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:49.060079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a67dc-ef65-4b56-9c80-cabaa65b7437 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:49.094881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f229e0-a64c-43fc-9eb2-c8828bd20d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:49.095575 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f229e0-a64c-43fc-9eb2-c8828bd20d9e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:49.095852 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f229e0-a64c-43fc-9eb2-c8828bd20d9e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:49.096156 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f229e0-a64c-43fc-9eb2-c8828bd20d9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:49.109726 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:49.110394 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f229e0-a64c-43fc-9eb2-c8828bd20d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:49.110761 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:49.111048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:49.111328 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] PATH_INFO: `/v3/services/network` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:49.111974 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:49.113399 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-20f229e0-a64c-43fc-9eb2-c8828bd20d9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:49.116443 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:49.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 405/813] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:49.120073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f229e0-a64c-43fc-9eb2-c8828bd20d9e None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:49.129835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:49.138632 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:49.140588 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 409/814] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:48 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:49.142318 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:49.170085 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:49.260889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f229e0-a64c-43fc-9eb2-c8828bd20d9e None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:49.275184 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aafb0-a5f9-4268-8a1e-78f63b7c8e2c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:49.299247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:49.319217 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:49.319867 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:49.320227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:49.320453 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:49.321120 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:49.321423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:49.322045 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80994a-50de-418e-8b6c-b70a487f03af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:49.322798 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80994a-50de-418e-8b6c-b70a487f03af None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:49.322913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:49.323134 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80994a-50de-418e-8b6c-b70a487f03af None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:49.323393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80994a-50de-418e-8b6c-b70a487f03af None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/f3d1c0e873e94d85b590846747152e24/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:49.325460 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80994a-50de-418e-8b6c-b70a487f03af None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=f3d1c0e873e94d85b590846747152e24, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:49.326989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80994a-50de-418e-8b6c-b70a487f03af None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:49.344472 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 406/815] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:49.345900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 410/816] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:56:49 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/f3d1c0e873e94d85b590846747152e24/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:49.346088 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:49.354278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80994a-50de-418e-8b6c-b70a487f03af None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:49.516404 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80994a-50de-418e-8b6c-b70a487f03af None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:49.560887 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:49.586021 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:49.586956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:49.587366 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:49.587829 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:49.588998 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] RBAC: Adding query filter params (name=network) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:49.589412 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:49.591803 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:49.594925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28f809-fe8f-432c-b578-f75f2cb2063e None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:49.608929 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 411/817] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:56:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:49.611548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:49.645147 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:49.645830 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:49.646098 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:49.646395 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:49.647135 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:49.647445 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:49.648887 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:49.659790 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 407/818] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:49.660797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:49.722917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:49.803902 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78823026-907f-4d30-bc05-42bb331ba41d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:49.812639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea497a57-8237-4f37-aeda-8483ea9dca5a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:49.857805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:49.858708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:49.859101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:49.859523 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:49.860359 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] RBAC: Adding query filter params (type=network) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:49.860798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:49.862768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:49.864871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:49.866222 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:49.866730 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:49.867292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:49.874274 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 408/819] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:56:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:49.875609 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:49.876976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:49.879149 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:49.894011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:49.895462 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:49.896696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:49.896696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:49.896696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:49.896696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:49.896696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:49.896696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:49.896696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:49.896696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:49.898167 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:49.899957 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 412/820] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:49.905669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:50.012846 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768089d-e75a-4498-8008-37b0f81b5fd1 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:50.053792 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfa164-bb52-425e-b0f9-e9e487ae1d7f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:50.055872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:50.077266 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:50.078796 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:50.079334 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:50.079842 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:50.092858 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:50.095063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:50.100415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f65ede-61c5-4fff-86a7-023cdac8d0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:50.101133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f65ede-61c5-4fff-86a7-023cdac8d0ef None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:50.101426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f65ede-61c5-4fff-86a7-023cdac8d0ef None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:50.101726 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f65ede-61c5-4fff-86a7-023cdac8d0ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:50.102622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f65ede-61c5-4fff-86a7-023cdac8d0ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:50.105263 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f65ede-61c5-4fff-86a7-023cdac8d0ef None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:50.116554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:50.117958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:50.119320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:50.119320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:50.119320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:50.119320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:50.122756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 409/821] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:50.124341 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:50.150765 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 413/822] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:56:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:50.152714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f65ede-61c5-4fff-86a7-023cdac8d0ef None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:50.222415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:50.281920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf620e8-d4a9-4987-af65-b2fea6f05bb0 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:50.343582 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:50.344910 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:50.345665 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:50.346230 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:50.347356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:50.347948 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:50.349752 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:50.369774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 410/823] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:50.377205 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:50.471164 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f65ede-61c5-4fff-86a7-023cdac8d0ef None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:50.514080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942658af-a899-4755-83e7-5999bbef0a51 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:50.540658 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:50.547994 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:50.548752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:50.549060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:50.549339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:50.550260 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:50.550511 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:50.552175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:50.564805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:50.566540 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:50.567094 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:50.567863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:50.569150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=4fe070acb9a84d90baaa3b08a4fd3191) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:50.569726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:50.572225 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:50.577622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 414/824] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:50.579066 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:50.586279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 411/825] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:50 2022] GET /identity/v3/role_assignments?user.id=4fe070acb9a84d90baaa3b08a4fd3191&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:50.614535 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:50.735837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4f7bf-7c00-415e-9696-005031661cd5 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:50.787521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:50.788205 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:50.788550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:50.788831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:50.798007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:50.799960 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:50.814853 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:50.816976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:56:50.818256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:50.822148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 415/826] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:56:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:50.823988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:50.966196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06c35-7204-40d0-a658-8362ebf3a79f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:51.012594 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee80e3-c414-46d7-be50-7708dc61a081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:51.013680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee80e3-c414-46d7-be50-7708dc61a081 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.014226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee80e3-c414-46d7-be50-7708dc61a081 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.014737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee80e3-c414-46d7-be50-7708dc61a081 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.016172 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee80e3-c414-46d7-be50-7708dc61a081 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:51.016491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee80e3-c414-46d7-be50-7708dc61a081 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:51.018358 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee80e3-c414-46d7-be50-7708dc61a081 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:51.041739 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 412/827] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:56:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:51.047817 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:51.069738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9f805-52b1-45e3-9cf6-347a5139ae65 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.069738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9f805-52b1-45e3-9cf6-347a5139ae65 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.070083 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9f805-52b1-45e3-9cf6-347a5139ae65 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.071694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 413/828] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:56:51.077610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd716f8-224f-49b0-b2fe-767c89d3428c None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.078269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd716f8-224f-49b0-b2fe-767c89d3428c None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.078269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd716f8-224f-49b0-b2fe-767c89d3428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.098430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-2dd716f8-224f-49b0-b2fe-767c89d3428c None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:51.099670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-2dd716f8-224f-49b0-b2fe-767c89d3428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:51.152286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:51.153990 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:51.189069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff85618-908e-4fd2-8f38-ac56d5043248 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:51.202040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 414/829] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:56:51.229132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd716f8-224f-49b0-b2fe-767c89d3428c None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:51.241519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:51.242695 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.243643 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.244173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.259054 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:51.261419 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:51.280818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:51.282081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:51.283261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:51.283261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:51.283261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:51.283261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:51.285820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 416/830] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:51.292929 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:51.356297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd716f8-224f-49b0-b2fe-767c89d3428c None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:51.398235 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:51.398998 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.399283 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.399559 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.400199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:51.401854 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:51.412666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:51.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:51.419618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 415/831] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:51.422201 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5429cc-9e48-4c63-aea3-0a3d5edab92d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:51.423380 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:51.462178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:51.471147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:51.472214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.473237 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.473758 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.474847 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:51.475428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:51.475942 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:51.478925 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:51.494903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 417/832] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:51.506143 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:51.558548 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282fb8c-e273-4d62-ab0f-0d7c69ad47ec None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:51.601179 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:51.601832 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.602131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.602456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.603299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:51.603594 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:51.605196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:51.615598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 416/833] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:51.617853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:51.627289 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7765c-71a4-4d25-8e4a-d70f2d6712d5 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:51.665975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317096a-8264-417e-8536-a7e030a0a685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:51.667036 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317096a-8264-417e-8536-a7e030a0a685 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.667506 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317096a-8264-417e-8536-a7e030a0a685 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.668039 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317096a-8264-417e-8536-a7e030a0a685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.676471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317096a-8264-417e-8536-a7e030a0a685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:51.678285 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317096a-8264-417e-8536-a7e030a0a685 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-6317096a-8264-417e-8536-a7e030a0a685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:51.691870 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:51.693273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:51.694638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:51.694638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:51.694638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:51.694638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:51.694638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:51.694638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:51.694638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:51.694638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:51.696824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 418/834] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:51.698404 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d95-7e4f-4647-8fc4-0c93515974a5 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.698857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d95-7e4f-4647-8fc4-0c93515974a5 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.699386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d95-7e4f-4647-8fc4-0c93515974a5 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.701318 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 419/835] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:51.706818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39332d95-7e4f-4647-8fc4-0c93515974a5 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:51.736205 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:51.740415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040673d-dc46-4a9a-847a-f9350a434819 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:51.771962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:51.791821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea757b9-541a-4a6f-9553-b94ec558ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:51.793073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea757b9-541a-4a6f-9553-b94ec558ad54 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.793551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea757b9-541a-4a6f-9553-b94ec558ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.794132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea757b9-541a-4a6f-9553-b94ec558ad54 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.795170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea757b9-541a-4a6f-9553-b94ec558ad54 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:51.795606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea757b9-541a-4a6f-9553-b94ec558ad54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:51.797340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea757b9-541a-4a6f-9553-b94ec558ad54 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:51.805068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 417/836] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:51.806781 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea811ae6-4c3e-4760-8a50-7fdda4559c01 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.807256 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea811ae6-4c3e-4760-8a50-7fdda4559c01 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.807719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea811ae6-4c3e-4760-8a50-7fdda4559c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.833472 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-ea811ae6-4c3e-4760-8a50-7fdda4559c01 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:51.834813 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-ea811ae6-4c3e-4760-8a50-7fdda4559c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:51.860215 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39332d95-7e4f-4647-8fc4-0c93515974a5 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:51.910028 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:51.911135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.911672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.912532 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.914251 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:51.914832 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:51.916778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:51.932070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 420/837] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:51.941474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:51.951687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 418/838] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:51.981826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea811ae6-4c3e-4760-8a50-7fdda4559c01 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:52.108404 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08de6f-f71d-4a80-8d16-4ba7c055ca3d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:52.121299 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:52.147068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:52.153135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7b285-1b62-4169-888b-27ada4b04e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:52.153959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7b285-1b62-4169-888b-27ada4b04e7b None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:52.154322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7b285-1b62-4169-888b-27ada4b04e7b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:52.154668 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7b285-1b62-4169-888b-27ada4b04e7b None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/ac2817fe7c654a29abe66f6e51b1dec7/roles/e5f9b7b560ca414994499e84782fa22d` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:52.156436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea811ae6-4c3e-4760-8a50-7fdda4559c01 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:52.172450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7b285-1b62-4169-888b-27ada4b04e7b None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=ac2817fe7c654a29abe66f6e51b1dec7, role_id=e5f9b7b560ca414994499e84782fa22d)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:52.174157 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7b285-1b62-4169-888b-27ada4b04e7b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:52.189000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 421/839] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:56:51 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/ac2817fe7c654a29abe66f6e51b1dec7/roles/e5f9b7b560ca414994499e84782fa22d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:56:52.205719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f1dd1-4b3f-4d74-9f83-b751b03dce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:52.206451 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f1dd1-4b3f-4d74-9f83-b751b03dce5c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:52.206737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f1dd1-4b3f-4d74-9f83-b751b03dce5c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:52.207064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f1dd1-4b3f-4d74-9f83-b751b03dce5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:52.222588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f1dd1-4b3f-4d74-9f83-b751b03dce5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-2b0f1dd1-4b3f-4d74-9f83-b751b03dce5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:52.229078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:52.231716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:56:52.234532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:52.236384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 419/840] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:52.242150 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7b285-1b62-4169-888b-27ada4b04e7b None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:52.399929 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7b285-1b62-4169-888b-27ada4b04e7b None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:52.459293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:52.459957 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:52.460247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:52.460528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:52.461286 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:52.461634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:52.463206 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:52.475323 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 422/841] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:52.480905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f1dd1-4b3f-4d74-9f83-b751b03dce5c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:52.598191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f1dd1-4b3f-4d74-9f83-b751b03dce5c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:52.636511 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:52.637196 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:52.637572 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:52.637886 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:52.650517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:52.651966 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:52.669131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:52.672821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:52.676417 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:52.676417 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:52.676417 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:52.676417 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:52.676417 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 420/842] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:52.677518 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:52.705227 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:52.789911 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bc23c-0964-4fd5-8290-84874e8b2b8c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:52.829018 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885a998-5b11-41d1-a86a-4a5fdde6fc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:52.829672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885a998-5b11-41d1-a86a-4a5fdde6fc3c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:52.829952 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885a998-5b11-41d1-a86a-4a5fdde6fc3c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:52.830262 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885a998-5b11-41d1-a86a-4a5fdde6fc3c None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:52.831044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885a998-5b11-41d1-a86a-4a5fdde6fc3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:52.831346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885a998-5b11-41d1-a86a-4a5fdde6fc3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:52.832774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885a998-5b11-41d1-a86a-4a5fdde6fc3c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:52.846676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 423/843] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:52.852600 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:52.863489 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:52.974805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5c8d8-e078-4243-be30-cdb8cb825fad None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:53.014824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:53.015935 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:53.016440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:53.016947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] PATH_INFO: `/v3/users/swift` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:53.026506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:53.028429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:53.032656 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29583c2b-3b98-42e5-8086-0723e769c263 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:53.032977 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29583c2b-3b98-42e5-8086-0723e769c263 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:53.033292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29583c2b-3b98-42e5-8086-0723e769c263 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:53.038137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 424/844] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:53.044056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4b530-3f65-4e25-8697-0dfe4f3badf2 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:53.044344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4b530-3f65-4e25-8697-0dfe4f3badf2 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:53.044622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4b530-3f65-4e25-8697-0dfe4f3badf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:53.046412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:53.047670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:53.048863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 421/845] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:56:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:53.053803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:53.071764 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-dbb4b530-3f65-4e25-8697-0dfe4f3badf2 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:53.073879 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-dbb4b530-3f65-4e25-8697-0dfe4f3badf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:53.168201 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 425/846] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:56:53.178520 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:53.195289 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4b530-3f65-4e25-8697-0dfe4f3badf2 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:53.213326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:53.241591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1679d-7aea-4ea8-823d-906cda6bb01a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:53.292863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:53.293937 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:53.294478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:53.295155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:53.296282 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:53.296859 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:53.299200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:53.328676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 422/847] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 08:56:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:53.333847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:53.363665 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4b530-3f65-4e25-8697-0dfe4f3badf2 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:53.424696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:53.426251 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:53.426833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:53.427804 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:53.429030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:53.430958 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:53.444307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:53.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:56:53.447298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:53.451872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 426/848] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:56:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:53.463705 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:53.513697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58689e-286e-4275-99bf-294263db8337 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:53.570912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:53.582458 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:53.590588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe482d-ab3c-401f-97f0-baeffb80d78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:53.592492 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe482d-ab3c-401f-97f0-baeffb80d78b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:53.593669 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe482d-ab3c-401f-97f0-baeffb80d78b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:53.595016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe482d-ab3c-401f-97f0-baeffb80d78b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:53.612437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d20750-ae18-4834-88e4-9a37f27e396a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:53.659025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:53.660650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:53.660955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:53.661355 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:53.662092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:53.662491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:53.664356 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:53.676371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 427/849] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:56:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:53.677374 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:53.724408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe482d-ab3c-401f-97f0-baeffb80d78b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:53.726425 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe482d-ab3c-401f-97f0-baeffb80d78b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:53.823888 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d2d79-58a5-411c-b665-5fcdffae3e17 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:53.853151 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 423/850] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:53 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:56:53.862160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe482d-ab3c-401f-97f0-baeffb80d78b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:53.889305 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:53.901129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:53.904384 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:53.905115 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:53.905885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:53.907226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:53.908058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:53.911963 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:53.931217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 428/851] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:56:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:53.995104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe482d-ab3c-401f-97f0-baeffb80d78b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:54.043387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33f2d6-eda1-4c02-894b-25e8b478535a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:54.044149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33f2d6-eda1-4c02-894b-25e8b478535a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.044459 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33f2d6-eda1-4c02-894b-25e8b478535a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.044736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33f2d6-eda1-4c02-894b-25e8b478535a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.057857 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33f2d6-eda1-4c02-894b-25e8b478535a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:54.059679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33f2d6-eda1-4c02-894b-25e8b478535a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1d33f2d6-eda1-4c02-894b-25e8b478535a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:54.079411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:54.080565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:54.081682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:54.081682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:54.081682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:54.081682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:54.085998 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 424/852] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:54.087750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:54.102537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:54.175274 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21144734-24c6-45a6-8036-472b3ce050ed None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.175785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21144734-24c6-45a6-8036-472b3ce050ed None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.175972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21144734-24c6-45a6-8036-472b3ce050ed None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.178157 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 425/853] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:56:54.190603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54859e9-a9e4-4570-ae04-747f5d8c53b5 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.192176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54859e9-a9e4-4570-ae04-747f5d8c53b5 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.192830 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54859e9-a9e4-4570-ae04-747f5d8c53b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.217850 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-c54859e9-a9e4-4570-ae04-747f5d8c53b5 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:54.219344 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-c54859e9-a9e4-4570-ae04-747f5d8c53b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:54.256530 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf444b1-dbf1-409c-8e83-5cb490535902 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:54.305955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:54.307173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.307682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.308184 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.309137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:54.309680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:54.311693 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:54.313508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 426/854] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:56:54.330208 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 429/855] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:54.340679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54859e9-a9e4-4570-ae04-747f5d8c53b5 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:54.348128 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:54.350402 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:54.417768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:54.457078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54859e9-a9e4-4570-ae04-747f5d8c53b5 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:54.482438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b556a-1c72-44ea-99c6-a68eb922c143 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:54.497335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:54.498364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.498855 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.499371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.507557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:54.509408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:54.523048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:54.524566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:54.526075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:54.526075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:54.526075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:54.526075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:54.526075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:54.526075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:54.526075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:54.526075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:54.527587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 427/856] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:54.529378 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a7de3-837d-405b-a5ff-05cd10e83ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:54.530030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7de3-837d-405b-a5ff-05cd10e83ef3 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.530371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7de3-837d-405b-a5ff-05cd10e83ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.530682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7de3-837d-405b-a5ff-05cd10e83ef3 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.531443 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a7de3-837d-405b-a5ff-05cd10e83ef3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:54.532902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a7de3-837d-405b-a5ff-05cd10e83ef3 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:54.534308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:54.558231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 430/857] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:56:54 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:56:54.619470 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:54.623674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511269-18eb-4029-8ab3-67eb2f7f6a33 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.624263 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511269-18eb-4029-8ab3-67eb2f7f6a33 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.624508 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511269-18eb-4029-8ab3-67eb2f7f6a33 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.627393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 431/858] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:54.632892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b900f9-ca1c-408f-b38c-fc3b40e92ff2 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.633183 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b900f9-ca1c-408f-b38c-fc3b40e92ff2 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.633450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b900f9-ca1c-408f-b38c-fc3b40e92ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.656031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-66b900f9-ca1c-408f-b38c-fc3b40e92ff2 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:54.658264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-66b900f9-ca1c-408f-b38c-fc3b40e92ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:54.667151 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a251f8f-0460-4d49-920f-d3e06820e9e3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:54.713741 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:54.714881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.715155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.715435 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.716239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:54.716551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:54.718184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:54.741490 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 428/859] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:54.743463 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:54.780376 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 432/860] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:56:54.805031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b900f9-ca1c-408f-b38c-fc3b40e92ff2 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:54.904097 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d3917-9e7a-44b1-a959-00183f03fe6b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:54.905015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:54.940810 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:54.964355 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:54.966365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:54.968099 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:54.969000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:54.970971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=f3d1c0e873e94d85b590846747152e24) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:54.971849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:54.976371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:54.990550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 429/861] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:54 2022] GET /identity/v3/role_assignments?user.id=f3d1c0e873e94d85b590846747152e24&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:54.997640 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b900f9-ca1c-408f-b38c-fc3b40e92ff2 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:55.055148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6010-0168-4983-87be-110a6f3dccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:55.055871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6010-0168-4983-87be-110a6f3dccca None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:55.056153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6010-0168-4983-87be-110a6f3dccca None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:55.056427 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6010-0168-4983-87be-110a6f3dccca None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:55.076585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f6010-0168-4983-87be-110a6f3dccca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-523f6010-0168-4983-87be-110a6f3dccca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:56:55.082085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:55.083542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:55.084753 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 433/862] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:56:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:55.090778 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:55.234753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7113a-ae07-4c12-adaf-49192db36daa None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:55.300072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:55.301200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:55.301713 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:55.302255 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:55.303662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:55.304241 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:55.306866 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:55.321947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 430/863] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:56:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:55.327508 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6010-0168-4983-87be-110a6f3dccca None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:55.443642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6010-0168-4983-87be-110a6f3dccca None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:55.460132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:55.492056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:55.493204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:55.493564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:55.493916 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:55.508111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:55.509618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:55.526504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:55.527757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:55.528918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:55.528918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:55.528918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:55.528918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:55.528918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 434/864] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:55.534313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:55.610812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:55.675966 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da5737-de96-4b8d-a2b7-3cfd4ca2d1f8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:55.726395 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:55.727087 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:55.727362 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:55.727664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:55.728419 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:55.728726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:55.730217 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:55.746921 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 431/865] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:56:55.752840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:55.886712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57dd99-9d58-4a94-ab0d-2511c130d34a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:55.942421 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:55.944142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:55.944981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:55.945805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:55.958881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:55.962001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:56:55.980718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:55.982392 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:56:55.984239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:55.988129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 435/866] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:56:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:55.999564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:56.169326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bc705-d3da-4be2-8950-251b998c1ba2 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:56.180664 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:56.209319 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbd2ef-1dca-4b72-86d1-892715678ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:56.210526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbd2ef-1dca-4b72-86d1-892715678ec0 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:56.211042 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbd2ef-1dca-4b72-86d1-892715678ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:56.211572 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbd2ef-1dca-4b72-86d1-892715678ec0 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:56.212621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbd2ef-1dca-4b72-86d1-892715678ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:56.213194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbd2ef-1dca-4b72-86d1-892715678ec0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:56.214970 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbd2ef-1dca-4b72-86d1-892715678ec0 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:56.234984 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 432/867] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:56:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:56.241542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:56.273169 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:56.275393 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab329d-1454-430a-b10a-dff1eea7fd63 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:56.276013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab329d-1454-430a-b10a-dff1eea7fd63 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:56.276558 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab329d-1454-430a-b10a-dff1eea7fd63 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:56.278505 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 433/868] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:56.287012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea3c28-8e5e-45b0-8f8a-ac7e102b9cf5 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:56.287683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea3c28-8e5e-45b0-8f8a-ac7e102b9cf5 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:56.288184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea3c28-8e5e-45b0-8f8a-ac7e102b9cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:56.310340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-b9ea3c28-8e5e-45b0-8f8a-ac7e102b9cf5 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:56.312400 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b9ea3c28-8e5e-45b0-8f8a-ac7e102b9cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:56.386039 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5b414-d153-4e60-b74a-9d92df178d0a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:56.402194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 434/869] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:56 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:56:56.417584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea3c28-8e5e-45b0-8f8a-ac7e102b9cf5 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:56.425460 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca161921-bc2b-41cd-bda4-ce76d8abd1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:56.426144 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca161921-bc2b-41cd-bda4-ce76d8abd1ef None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:56.426437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca161921-bc2b-41cd-bda4-ce76d8abd1ef None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:56.426717 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca161921-bc2b-41cd-bda4-ce76d8abd1ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:56.511308 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca161921-bc2b-41cd-bda4-ce76d8abd1ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:56.513141 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca161921-bc2b-41cd-bda4-ce76d8abd1ef None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:56.522988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:56.530199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea3c28-8e5e-45b0-8f8a-ac7e102b9cf5 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:56.554509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:56.569081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca760a-e7b4-428b-a778-d7086868c858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:56.570180 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca760a-e7b4-428b-a778-d7086868c858 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:56.570780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca760a-e7b4-428b-a778-d7086868c858 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:56.571288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca760a-e7b4-428b-a778-d7086868c858 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:56.572247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca760a-e7b4-428b-a778-d7086868c858 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:56.573926 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca760a-e7b4-428b-a778-d7086868c858 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:56.598178 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 435/870] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:56:56 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:56:56.677404 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 436/871] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:56:56 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:56:56.684755 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca760a-e7b4-428b-a778-d7086868c858 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:56.811112 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca760a-e7b4-428b-a778-d7086868c858 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:56.859484 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0154391-f2d2-4132-87c3-ede38576376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:56.860244 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0154391-f2d2-4132-87c3-ede38576376c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:56.860580 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0154391-f2d2-4132-87c3-ede38576376c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:56.860920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0154391-f2d2-4132-87c3-ede38576376c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:56.875081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0154391-f2d2-4132-87c3-ede38576376c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:56.876610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0154391-f2d2-4132-87c3-ede38576376c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-a0154391-f2d2-4132-87c3-ede38576376c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:56:56.894476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:56.895698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:56.897007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:56:56.897007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:56:56.897007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:56:56.897007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:56.897007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 436/872] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:56.903793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca161921-bc2b-41cd-bda4-ce76d8abd1ef None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:56.905626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d612a8c-ee98-4d38-8ca6-9f3058b23a08 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:56.906005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d612a8c-ee98-4d38-8ca6-9f3058b23a08 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:56.906332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d612a8c-ee98-4d38-8ca6-9f3058b23a08 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:56.907960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 437/873] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:56.914740 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924fd11-985b-49a6-9dd4-6ab5d76259e4 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:56.915049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924fd11-985b-49a6-9dd4-6ab5d76259e4 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:56.915305 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924fd11-985b-49a6-9dd4-6ab5d76259e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:56.924829 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:56.939187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-7924fd11-985b-49a6-9dd4-6ab5d76259e4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:56.941767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-7924fd11-985b-49a6-9dd4-6ab5d76259e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:57.037905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 438/874] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:56:57.064691 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924fd11-985b-49a6-9dd4-6ab5d76259e4 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:57.076266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca161921-bc2b-41cd-bda4-ce76d8abd1ef None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:57.110666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:57.122691 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:57.124219 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.124929 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.125604 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.128044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:57.128961 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:57.132071 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:57.162307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 437/875] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:57.164433 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:57.202189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924fd11-985b-49a6-9dd4-6ab5d76259e4 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:57.245064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:57.246178 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.246178 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.246491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.246952 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:57.248795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:57.260322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:57.262839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:57.265851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:57.269300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 439/876] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:56:57.269300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:57.358072 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ee7fc-17d4-4552-8346-e92fc9af75c7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:57.413125 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac54a4e-8620-4f2e-b7ea-cb5904a03466 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:57.418681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e5798-37dc-4d2a-bbcf-eb764dbc5a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:57.419775 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5798-37dc-4d2a-bbcf-eb764dbc5a5d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.420267 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5798-37dc-4d2a-bbcf-eb764dbc5a5d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.420790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5798-37dc-4d2a-bbcf-eb764dbc5a5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.433593 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e5798-37dc-4d2a-bbcf-eb764dbc5a5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:57.435448 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e5798-37dc-4d2a-bbcf-eb764dbc5a5d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-317e5798-37dc-4d2a-bbcf-eb764dbc5a5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:56:57.452985 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:57.454485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:57.456044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:56:57.456044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:56:57.456044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:56:57.456044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:56:57.456044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:56:57.456044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:56:57.456044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:56:57.456044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:57.460292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 438/877] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:56:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:57.462080 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb5dc-1447-446e-a8c8-e34722ce0411 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.462411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb5dc-1447-446e-a8c8-e34722ce0411 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.462903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb5dc-1447-446e-a8c8-e34722ce0411 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.466149 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 439/878] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:57.466149 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acb5dc-1447-446e-a8c8-e34722ce0411 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:57.469927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d99138-4219-475a-bf63-c625be42b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:57.470624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d99138-4219-475a-bf63-c625be42b1db None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.471148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d99138-4219-475a-bf63-c625be42b1db None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.471391 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d99138-4219-475a-bf63-c625be42b1db None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.472490 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d99138-4219-475a-bf63-c625be42b1db None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:57.472811 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d99138-4219-475a-bf63-c625be42b1db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:57.475564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d99138-4219-475a-bf63-c625be42b1db None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:57.481765 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 440/879] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:57.483529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf053-1e9a-45b2-aebb-f8b2dac55a5c None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.483817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf053-1e9a-45b2-aebb-f8b2dac55a5c None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.484122 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf053-1e9a-45b2-aebb-f8b2dac55a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.506963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-27dbf053-1e9a-45b2-aebb-f8b2dac55a5c None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:57.508343 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-27dbf053-1e9a-45b2-aebb-f8b2dac55a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:57.530144 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:57.588804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:57.598881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acb5dc-1447-446e-a8c8-e34722ce0411 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:57.614246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 441/880] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:57.614246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf053-1e9a-45b2-aebb-f8b2dac55a5c None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:57.651387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:57.652349 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.652690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.653373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.654467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:57.654786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:57.656307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:57.676740 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 440/881] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:56:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:57.676740 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:57.759169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf053-1e9a-45b2-aebb-f8b2dac55a5c None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:57.809453 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:57.809779 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6e1cb-b6b8-4628-a8b0-3d2f0d8da51e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:57.810175 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.810455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.810691 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.811406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:57.811711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:57.813151 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:57.819773 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 442/882] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:56:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:57.820633 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:57.832927 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:57.865713 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:57.866406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:57.866771 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:57.867056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:57.867672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:57.869096 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:57.882083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:57.884016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:56:57.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 441/883] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:56:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:57.887509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:57.913326 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:57.994556 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a447a-14df-4945-abb6-4eaf4abb7cda None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:58.040521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf3360-4ce8-45e2-a4f6-46751e784b95 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:58.076731 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:58.078344 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:58.078909 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:58.079542 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:58.081101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=ac2817fe7c654a29abe66f6e51b1dec7) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:58.081677 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:58.084520 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:58.084711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:58.085654 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:58.086261 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:58.086808 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:58.087827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] RBAC: Adding query filter params (interface=public, service_id=cc8da8ce2b4945da80a227fd94b72386, region_id=RegionOne) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:58.088393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:58.090134 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:58.101699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 442/884] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:56:57 2022] GET /identity/v3/endpoints?service_id=cc8da8ce2b4945da80a227fd94b72386&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:58.102611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 443/885] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:56:57 2022] GET /identity/v3/role_assignments?user.id=ac2817fe7c654a29abe66f6e51b1dec7&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:58.103877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:58.104044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:58.178547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:58.290090 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826330-9247-4ea4-9532-0a9441fc79bb None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:58.290382 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce581-8115-41f9-b308-574d75be9a79 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:58.323539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:58.344649 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793a321-2327-45b5-b572-718792fd580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:58.345887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793a321-2327-45b5-b572-718792fd580e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:58.346497 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793a321-2327-45b5-b572-718792fd580e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:58.346961 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793a321-2327-45b5-b572-718792fd580e None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:58.348025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793a321-2327-45b5-b572-718792fd580e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:58.348566 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793a321-2327-45b5-b572-718792fd580e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:58.350519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793a321-2327-45b5-b572-718792fd580e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:58.351663 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81609cd-32de-46f3-a4b4-e922d865d785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:58.352583 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81609cd-32de-46f3-a4b4-e922d865d785 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:58.352982 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81609cd-32de-46f3-a4b4-e922d865d785 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:58.353206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81609cd-32de-46f3-a4b4-e922d865d785 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:58.354082 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81609cd-32de-46f3-a4b4-e922d865d785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:58.357074 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81609cd-32de-46f3-a4b4-e922d865d785 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:58.359965 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 443/886] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:56:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:58.364351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 444/887] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:56:58 2022] GET /identity/v3/services => generated 1268 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:56:58.367625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793a321-2327-45b5-b572-718792fd580e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:58.538228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793a321-2327-45b5-b572-718792fd580e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:58.540528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:58.593181 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa860367-aee8-4fa7-994d-be3bc920c7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:58.595135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa860367-aee8-4fa7-994d-be3bc920c7dc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:58.596066 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa860367-aee8-4fa7-994d-be3bc920c7dc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:58.596914 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa860367-aee8-4fa7-994d-be3bc920c7dc None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:58.598720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa860367-aee8-4fa7-994d-be3bc920c7dc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:58.599674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa860367-aee8-4fa7-994d-be3bc920c7dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:58.602767 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa860367-aee8-4fa7-994d-be3bc920c7dc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:58.612854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 444/888] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:56:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:58.940318 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aae8f-00fa-4f3a-b1f6-d5748de810c9 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:58.940703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aae8f-00fa-4f3a-b1f6-d5748de810c9 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:58.941351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aae8f-00fa-4f3a-b1f6-d5748de810c9 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:58.943413 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 445/889] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:56:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:58.949880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b25b9b-8a4a-409b-8e2d-e36fa342f501 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:58.950260 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b25b9b-8a4a-409b-8e2d-e36fa342f501 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:58.950521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b25b9b-8a4a-409b-8e2d-e36fa342f501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:58.972714 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-88b25b9b-8a4a-409b-8e2d-e36fa342f501 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:56:58.974098 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-88b25b9b-8a4a-409b-8e2d-e36fa342f501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:56:59.050373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 445/890] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:56:59.075739 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aae8f-00fa-4f3a-b1f6-d5748de810c9 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:59.190622 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aae8f-00fa-4f3a-b1f6-d5748de810c9 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:59.228373 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:59.229045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:59.229331 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:59.229649 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:59.230392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:59.231837 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:56:59.241905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:56:59.243639 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:56:59.245039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 446/891] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:56:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:56:59.255251 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b25b9b-8a4a-409b-8e2d-e36fa342f501 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:59.373893 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b25b9b-8a4a-409b-8e2d-e36fa342f501 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:59.391178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:59.413139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:59.413797 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:59.414082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:59.414447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:59.415102 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:59.415411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:59.416844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:59.422871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 446/892] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:56:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:59.432893 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:59.523777 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:56:59.578671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e981c-30a9-4157-9013-0f660f2252e8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:59.616286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:59.617097 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:59.617539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:59.617833 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:59.618743 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:59.619085 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:59.620998 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:59.631343 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 447/893] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:56:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:59.639491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:59.811367 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd01b2a-8174-4478-8a87-f0fd63bc1a9d None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:59.850299 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821ced8-ead8-4063-8b80-de49fe6cc7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:56:59.851076 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821ced8-ead8-4063-8b80-de49fe6cc7ee None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:59.851486 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821ced8-ead8-4063-8b80-de49fe6cc7ee None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:59.851955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821ced8-ead8-4063-8b80-de49fe6cc7ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:59.852934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821ced8-ead8-4063-8b80-de49fe6cc7ee None admin] RBAC: Adding query filter params (interface=public, service_id=71959f44f66f40a4b66ecb0f4425e924, region_id=RegionOne) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:56:59.853253 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821ced8-ead8-4063-8b80-de49fe6cc7ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:56:59.855058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821ced8-ead8-4063-8b80-de49fe6cc7ee None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:56:59.862489 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 447/894] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:56:59 2022] GET /identity/v3/endpoints?service_id=71959f44f66f40a4b66ecb0f4425e924&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:59.869161 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:56:59.984314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bdc71-a2b8-4a95-8e13-f6b954f83da8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:56:59.999719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:00.023355 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1601cea-0d4e-46e2-b253-5d077e043384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:00.024007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1601cea-0d4e-46e2-b253-5d077e043384 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:00.024382 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1601cea-0d4e-46e2-b253-5d077e043384 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:00.024607 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1601cea-0d4e-46e2-b253-5d077e043384 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:00.025356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1601cea-0d4e-46e2-b253-5d077e043384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:00.026845 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1601cea-0d4e-46e2-b253-5d077e043384 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:00.033792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 448/895] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:56:59 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:57:00.565311 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705337e-f79a-44b6-8317-1c8f770ad1ba None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:00.567100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705337e-f79a-44b6-8317-1c8f770ad1ba None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:00.567777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705337e-f79a-44b6-8317-1c8f770ad1ba None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:00.570396 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 448/896] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:00.577742 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092563f6-0efe-4e83-b707-fe0463b7719a None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:00.579264 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092563f6-0efe-4e83-b707-fe0463b7719a None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:00.579904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092563f6-0efe-4e83-b707-fe0463b7719a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:00.601884 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-092563f6-0efe-4e83-b707-fe0463b7719a None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:00.603169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-092563f6-0efe-4e83-b707-fe0463b7719a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:00.679142 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 449/897] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:00.685025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab23985-2eaf-4dde-9c9e-f494426c938e None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:00.685368 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab23985-2eaf-4dde-9c9e-f494426c938e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:00.685570 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab23985-2eaf-4dde-9c9e-f494426c938e None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:00.687214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 449/898] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:00.696019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096ad33-4cf7-4987-a1e8-bbe97eeb1426 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:00.696327 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096ad33-4cf7-4987-a1e8-bbe97eeb1426 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:00.696594 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096ad33-4cf7-4987-a1e8-bbe97eeb1426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:00.708338 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab23985-2eaf-4dde-9c9e-f494426c938e None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:00.716834 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-9096ad33-4cf7-4987-a1e8-bbe97eeb1426 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:00.718683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-9096ad33-4cf7-4987-a1e8-bbe97eeb1426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:00.736682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:00.810237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 450/899] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:00.832389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096ad33-4cf7-4987-a1e8-bbe97eeb1426 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:00.879881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:00.880743 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab23985-2eaf-4dde-9c9e-f494426c938e None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:00.934557 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e899ad-46c8-4607-89d8-4e3ce6f96978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:00.936561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e899ad-46c8-4607-89d8-4e3ce6f96978 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:00.937082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e899ad-46c8-4607-89d8-4e3ce6f96978 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:00.937600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e899ad-46c8-4607-89d8-4e3ce6f96978 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:00.954130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e899ad-46c8-4607-89d8-4e3ce6f96978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-56e899ad-46c8-4607-89d8-4e3ce6f96978 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:00.960983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:00.962273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:00.963380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:00.966746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 450/900] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:57:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:00.976204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e899ad-46c8-4607-89d8-4e3ce6f96978 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:00.986308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096ad33-4cf7-4987-a1e8-bbe97eeb1426 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:01.027982 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371ddb1-0ee9-4cff-8aff-96fd0d1babee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:01.028656 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371ddb1-0ee9-4cff-8aff-96fd0d1babee None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.028942 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371ddb1-0ee9-4cff-8aff-96fd0d1babee None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.029221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371ddb1-0ee9-4cff-8aff-96fd0d1babee None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.030082 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371ddb1-0ee9-4cff-8aff-96fd0d1babee None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:01.033751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371ddb1-0ee9-4cff-8aff-96fd0d1babee None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-c371ddb1-0ee9-4cff-8aff-96fd0d1babee None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:01.043977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:01.048226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:01.049975 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 451/901] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:01.051776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441be65-90bb-48c4-9378-5ee610e9b51a None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.051776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441be65-90bb-48c4-9378-5ee610e9b51a None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.051776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441be65-90bb-48c4-9378-5ee610e9b51a None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.052976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 452/902] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:01.054011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441be65-90bb-48c4-9378-5ee610e9b51a None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:01.108252 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:01.141625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e899ad-46c8-4607-89d8-4e3ce6f96978 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:01.200846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00fa05-fb1d-4be9-ad91-c4c4a915c52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:01.201831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00fa05-fb1d-4be9-ad91-c4c4a915c52e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.202592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00fa05-fb1d-4be9-ad91-c4c4a915c52e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.203171 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00fa05-fb1d-4be9-ad91-c4c4a915c52e None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.204441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00fa05-fb1d-4be9-ad91-c4c4a915c52e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:01.204989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00fa05-fb1d-4be9-ad91-c4c4a915c52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:01.208088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00fa05-fb1d-4be9-ad91-c4c4a915c52e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:01.218075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:01.226165 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 451/903] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:57:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:01.227480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951fb9-5191-4958-9631-2f5528af5e77 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.227938 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951fb9-5191-4958-9631-2f5528af5e77 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.228377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951fb9-5191-4958-9631-2f5528af5e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.230943 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441be65-90bb-48c4-9378-5ee610e9b51a None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:01.260951 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-17951fb9-5191-4958-9631-2f5528af5e77 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:01.263312 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-17951fb9-5191-4958-9631-2f5528af5e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:01.278408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:01.279141 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.279486 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.279770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.280581 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:01.280959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:01.282619 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:01.288707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 453/904] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:01.295042 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:01.378633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 452/905] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:01.380721 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17951fb9-5191-4958-9631-2f5528af5e77 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:01.440272 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078f741-28b5-47ea-ba7f-29952b71a59a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:01.481343 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:01.482400 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.482695 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.483037 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.489656 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:01.496994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:01.498694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:01.514947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:01.516131 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:01.517269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:01.517269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:01.517269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:01.517269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:01.517269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 454/906] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:01.517269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:01.543022 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:01.548600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17951fb9-5191-4958-9631-2f5528af5e77 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:01.592589 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:01.593771 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.594089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.594490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.596017 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:01.596358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:01.598446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:01.604848 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 453/907] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:01.605516 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:01.663986 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab77eea-bfd1-411d-9868-d07db80ef28b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:01.704823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a037ceb-805e-4c1d-8768-46424583c817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:01.705716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a037ceb-805e-4c1d-8768-46424583c817 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.706090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a037ceb-805e-4c1d-8768-46424583c817 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.707011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a037ceb-805e-4c1d-8768-46424583c817 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.708491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a037ceb-805e-4c1d-8768-46424583c817 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:01.710498 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a037ceb-805e-4c1d-8768-46424583c817 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:01.734163 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 455/908] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:57:01 2022] POST /identity/v3/services => generated 241 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:01.734163 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a037ceb-805e-4c1d-8768-46424583c817 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:01.746861 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40852ee-225e-4319-9705-541f68e76292 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:01.796776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:01.805177 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:01.806389 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.807146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.807615 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.808638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:01.809175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:01.811760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:01.830129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 454/909] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:01.838281 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:01.878351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:01.910818 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a037ceb-805e-4c1d-8768-46424583c817 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:01.956018 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4354f8f-3bc1-4a3e-a45b-a42a5f079ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:01.956590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354f8f-3bc1-4a3e-a45b-a42a5f079ce8 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:01.956874 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354f8f-3bc1-4a3e-a45b-a42a5f079ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:01.957146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354f8f-3bc1-4a3e-a45b-a42a5f079ce8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:01.957875 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4354f8f-3bc1-4a3e-a45b-a42a5f079ce8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:01.959616 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4354f8f-3bc1-4a3e-a45b-a42a5f079ce8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:01.989151 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 456/910] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:57:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:02.001527 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae4c3a-cf45-4765-b3a0-658032cae5a7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:02.062907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75212c9b-5ee4-42e2-a795-9b3d3119ab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:02.064012 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75212c9b-5ee4-42e2-a795-9b3d3119ab38 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.064507 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75212c9b-5ee4-42e2-a795-9b3d3119ab38 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.064980 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75212c9b-5ee4-42e2-a795-9b3d3119ab38 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.076598 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75212c9b-5ee4-42e2-a795-9b3d3119ab38 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:02.078879 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75212c9b-5ee4-42e2-a795-9b3d3119ab38 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-75212c9b-5ee4-42e2-a795-9b3d3119ab38 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:57:02.094823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:02.096469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:57:02.097742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:02.100822 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 455/911] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:57:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:02.106962 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4354f8f-3bc1-4a3e-a45b-a42a5f079ce8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:02.240415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885ca9-c707-45a1-8be6-6c6c6d5b8f76 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.240634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885ca9-c707-45a1-8be6-6c6c6d5b8f76 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.240916 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885ca9-c707-45a1-8be6-6c6c6d5b8f76 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.243772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 456/912] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:02.248293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a58b24-d61a-480c-8b60-2404ad84bb78 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.248589 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a58b24-d61a-480c-8b60-2404ad84bb78 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.249237 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a58b24-d61a-480c-8b60-2404ad84bb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.257455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4354f8f-3bc1-4a3e-a45b-a42a5f079ce8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:02.273184 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-a2a58b24-d61a-480c-8b60-2404ad84bb78 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:02.274621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-a2a58b24-d61a-480c-8b60-2404ad84bb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:02.310671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:02.322029 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:02.324358 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.324358 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.324358 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.325501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:02.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:02.327976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:02.336145 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:02.353648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 457/913] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:57:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:02.360508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:02.368017 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 457/914] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:02.397045 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a58b24-d61a-480c-8b60-2404ad84bb78 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:02.506422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04deb35-dc54-4d35-b257-a0970fe92fba None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:02.552626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb46976-6ca3-4295-9023-536a1e5ecd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:02.553285 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb46976-6ca3-4295-9023-536a1e5ecd67 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.553571 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb46976-6ca3-4295-9023-536a1e5ecd67 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.554194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb46976-6ca3-4295-9023-536a1e5ecd67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.573561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a58b24-d61a-480c-8b60-2404ad84bb78 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:02.637971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:02.644982 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:02.647263 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.648089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.649023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.650696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:02.653376 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb46976-6ca3-4295-9023-536a1e5ecd67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:02.653825 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:02.655679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb46976-6ca3-4295-9023-536a1e5ecd67 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:02.675059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:02.677627 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:57:02.681188 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:02.686397 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 458/915] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 08:57:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:02.691537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:02.764441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:02.840204 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 458/916] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:02 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:02.841839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb46976-6ca3-4295-9023-536a1e5ecd67 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:02.867949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186dd972-f844-4478-996a-f4a1bf3801fc None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:02.923791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:02.925448 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.926329 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.927433 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.929164 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:02.930215 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:02.933217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:02.944114 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 459/917] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:57:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:02.951689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:02.959767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:02.980156 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb46976-6ca3-4295-9023-536a1e5ecd67 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:03.035441 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:03.036041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:03.036330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:03.036607 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:03.051449 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:03.052980 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:03.074768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:03.077158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:03.080039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:03.080039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:03.080039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:03.080039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:03.080039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 459/918] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:03.080039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:03.126063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfe5d5-8dc3-4300-b3d4-8c97b6dc2d23 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:03.134682 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:03.170255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:03.171109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:03.171391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:03.171699 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:03.173039 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:03.173039 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:03.174735 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:03.186987 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 460/919] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:57:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:03.189963 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:03.208522 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321c23f-e8b6-4d75-acd6-abf5493e868e None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:03.256458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:03.257526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:03.258489 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:03.258986 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:03.260670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:03.261199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:03.265015 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:03.290294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 460/920] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:03.297255 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:03.301830 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:03.353120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978be00-f663-419a-8bde-6b53eae7d45b None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:03.398897 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00860d4-ad1e-49a5-ba7b-7a5f39eb07f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:03.400099 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00860d4-ad1e-49a5-ba7b-7a5f39eb07f9 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:03.400612 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00860d4-ad1e-49a5-ba7b-7a5f39eb07f9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:03.401126 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00860d4-ad1e-49a5-ba7b-7a5f39eb07f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:03.402433 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00860d4-ad1e-49a5-ba7b-7a5f39eb07f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:03.404322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00860d4-ad1e-49a5-ba7b-7a5f39eb07f9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:03.434875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 461/921] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:57:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:57:03.474976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd5712-0944-4a64-840d-831d599529ca None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:03.526198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:03.527429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:03.527952 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:03.528492 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:03.537617 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:03.539600 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:57:03.554190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:03.555602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:03.556927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:57:03.556927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:57:03.556927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:57:03.556927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:57:03.556927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:57:03.556927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:57:03.556927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:03.556927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:03.559908 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 461/922] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:03.567544 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00860d4-ad1e-49a5-ba7b-7a5f39eb07f9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:03.606232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:03.693830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00860d4-ad1e-49a5-ba7b-7a5f39eb07f9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:03.731841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01e8b7-c5e0-4da9-ad29-df9b9d390c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:03.732571 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e8b7-c5e0-4da9-ad29-df9b9d390c96 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:03.732852 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e8b7-c5e0-4da9-ad29-df9b9d390c96 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:03.733122 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e8b7-c5e0-4da9-ad29-df9b9d390c96 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:03.733994 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01e8b7-c5e0-4da9-ad29-df9b9d390c96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:03.734342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01e8b7-c5e0-4da9-ad29-df9b9d390c96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:03.735855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01e8b7-c5e0-4da9-ad29-df9b9d390c96 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:03.750592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 462/923] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:57:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:03.756954 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:03.858190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:03.933438 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d4a2-4a4a-461f-b620-b7dca1b3b039 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:03.990298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:03.991473 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:03.992046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:03.992557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:03.994149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] RBAC: Adding query filter params (role.id=8dbaf90916934fa7a0670d2c504af17b, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=ac2817fe7c654a29abe66f6e51b1dec7) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:03.994712 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:03.997656 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:04.002791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42db9ec-f6b1-41c9-9514-00ee187b4b9f None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.003607 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42db9ec-f6b1-41c9-9514-00ee187b4b9f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.004114 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42db9ec-f6b1-41c9-9514-00ee187b4b9f None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.008135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 463/925] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:04.008349 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 462/925] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:03 2022] GET /identity/v3/role_assignments?user.id=ac2817fe7c654a29abe66f6e51b1dec7&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=8dbaf90916934fa7a0670d2c504af17b => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:04.015579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323745-eecd-41ac-bc17-5597b36e4b74 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.016087 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323745-eecd-41ac-bc17-5597b36e4b74 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.016586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323745-eecd-41ac-bc17-5597b36e4b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.057703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-8b323745-eecd-41ac-bc17-5597b36e4b74 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:04.059498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-8b323745-eecd-41ac-bc17-5597b36e4b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:04.171121 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 464/926] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:04.194829 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:04.285628 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfda134-aeb4-435e-932e-459b180e6f5e None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.285974 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfda134-aeb4-435e-932e-459b180e6f5e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.286313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfda134-aeb4-435e-932e-459b180e6f5e None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.289296 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 465/927] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:04.294048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754032-9263-4a87-980f-bb42570eb103 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.294407 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754032-9263-4a87-980f-bb42570eb103 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.294671 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754032-9263-4a87-980f-bb42570eb103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.314538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-21754032-9263-4a87-980f-bb42570eb103 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:04.315785 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-21754032-9263-4a87-980f-bb42570eb103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:04.331881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:04.346962 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c18cc-6631-40e4-8ebf-b50609b9631a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:04.384117 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:04.402242 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:04.403549 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.404015 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.404454 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.405448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:04.407474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:04.420787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:04.423406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:04.424751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.429599 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 466/928] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:04.429799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 463/929] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:57:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:04.434875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754032-9263-4a87-980f-bb42570eb103 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:04.460805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:04.563620 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754032-9263-4a87-980f-bb42570eb103 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:04.610260 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:04.611247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.611703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.612134 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.613012 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:04.613490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:04.615133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:04.621426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 467/930] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:57:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:04.628199 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:04.628905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e8e54-3a2b-484f-a409-e0239daa6a2c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:04.641747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:04.667631 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:04.668252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.668537 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.669022 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.669736 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:04.671464 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:04.674497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: WARNING keystone.server.flask.common [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 08:57:04.695812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.sql.core [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:57:04.695812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:57:04.695812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [parameters: {'id': '95de1446966945daa1e3d0769d6c36ff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:57:04.695812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=347147) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95de1446966945daa1e3d0769d6c36ff.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95de1446966945daa1e3d0769d6c36ff. Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:57:04.713614 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result.read() Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 08:57:04.715606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application yield resource Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise self.value Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application session.commit() Jul 28 08:57:04.719150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:57:04.723145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:57:04.727468 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result.read() Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:57:04.733009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [parameters: {'id': '95de1446966945daa1e3d0769d6c36ff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:04.737588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:04.741539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95de1446966945daa1e3d0769d6c36ff. Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 464/931] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 08:57:04 2022] POST /identity/v3/projects => generated 262 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:04.745279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:04.777190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:04.808168 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b2df1-3dc1-4aa8-910b-c99f1703d404 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:04.857111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4a6ad-e744-4e4b-a7e9-eb0b19a4e5eb None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:04.864827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:04.866080 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.866637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.867301 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.868460 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:04.869028 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:04.871485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:04.879533 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 468/932] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:57:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:04.886338 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:04.902955 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:04.903821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.904046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.904305 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.912933 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:04.914798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:57:04.928514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:04.931368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:04.933404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:57:04.933404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:57:04.933404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:57:04.933404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:57:04.933404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:57:04.933404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:57:04.933404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:04.933404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:04.934797 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 465/933] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:04.936361 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.008167 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.049154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6973e-b54a-4387-a989-e74f579e536e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.054912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79504e1-5f69-4b1d-8081-8db6459bf43b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.056648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.099551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c150a1-c8be-4660-a488-6230c64cddb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.100271 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c150a1-c8be-4660-a488-6230c64cddb2 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.100570 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c150a1-c8be-4660-a488-6230c64cddb2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.100821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c150a1-c8be-4660-a488-6230c64cddb2 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.101605 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.101792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c150a1-c8be-4660-a488-6230c64cddb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:05.101966 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c150a1-c8be-4660-a488-6230c64cddb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.103010 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.103480 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c150a1-c8be-4660-a488-6230c64cddb2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.103926 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.104455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.105455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] RBAC: Adding query filter params (interface=public, service_id=8d52fc91943d479d934bbd30cc3192d6, region_id=RegionOne) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:05.105993 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.107809 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.114647 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 469/934] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:57:04 2022] GET /identity/v3/endpoints?service_id=8d52fc91943d479d934bbd30cc3192d6&interface=public®ion_id=RegionOne => generated 194 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:05.125101 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.142807 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 466/935] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:57:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:05.312931 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f9d6c-0874-4f11-9eed-a56fc907ad3e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.370907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e7c93-6c08-4861-866a-85ae9f76492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.371747 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e7c93-6c08-4861-866a-85ae9f76492a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.372069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e7c93-6c08-4861-866a-85ae9f76492a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.372356 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e7c93-6c08-4861-866a-85ae9f76492a None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.373178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e7c93-6c08-4861-866a-85ae9f76492a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.375069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e7c93-6c08-4861-866a-85ae9f76492a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.383638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 470/936] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/services => generated 1497 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:57:05.791923 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416f1e5-79cf-4f33-a869-f831f503ca23 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.792300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416f1e5-79cf-4f33-a869-f831f503ca23 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.792499 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416f1e5-79cf-4f33-a869-f831f503ca23 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.794266 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 467/937] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:05.800859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935fc3b-6e75-4f3f-bcd4-8d37706a73b7 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.801273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935fc3b-6e75-4f3f-bcd4-8d37706a73b7 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.801511 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935fc3b-6e75-4f3f-bcd4-8d37706a73b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.809241 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.824573 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-2935fc3b-6e75-4f3f-bcd4-8d37706a73b7 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:05.825823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-2935fc3b-6e75-4f3f-bcd4-8d37706a73b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:05.909488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 471/938] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:05.932415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416f1e5-79cf-4f33-a869-f831f503ca23 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:06.059048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416f1e5-79cf-4f33-a869-f831f503ca23 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:06.114605 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:06.115351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:06.115711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:06.116062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] PATH_INFO: `/v3/services/compute` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:06.116892 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:06.118502 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:06.130535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:06.132299 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:06.133728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 468/939] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:06.143287 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935fc3b-6e75-4f3f-bcd4-8d37706a73b7 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:06.275223 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935fc3b-6e75-4f3f-bcd4-8d37706a73b7 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:06.314333 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:06.320979 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a846fa29-b3ef-4fd1-a8cd-941f9f0b66bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:06.322442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846fa29-b3ef-4fd1-a8cd-941f9f0b66bf None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:06.323171 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846fa29-b3ef-4fd1-a8cd-941f9f0b66bf None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:06.323459 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846fa29-b3ef-4fd1-a8cd-941f9f0b66bf None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:06.324334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846fa29-b3ef-4fd1-a8cd-941f9f0b66bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:06.324651 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846fa29-b3ef-4fd1-a8cd-941f9f0b66bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:06.326532 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846fa29-b3ef-4fd1-a8cd-941f9f0b66bf None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:06.333236 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 472/940] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:57:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:06.340068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:06.378762 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:06.392477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d730c11-a94b-4163-844b-66e4954f5545 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:06.392989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d730c11-a94b-4163-844b-66e4954f5545 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:06.393497 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d730c11-a94b-4163-844b-66e4954f5545 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:06.395846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 473/941] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:06.405526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722e19c-7f0b-4456-a694-cdae4ebec1d9 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:06.406200 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722e19c-7f0b-4456-a694-cdae4ebec1d9 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:06.406864 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722e19c-7f0b-4456-a694-cdae4ebec1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:06.434483 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-5722e19c-7f0b-4456-a694-cdae4ebec1d9 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:06.436214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-5722e19c-7f0b-4456-a694-cdae4ebec1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:06.481124 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07f1d6-b8b4-45b0-b625-aa2b73f7aa77 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:06.525401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:06.526398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:06.526842 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:06.527275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:06.528142 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:06.528638 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:06.531007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:06.538072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 469/942] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:57:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:06.573605 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 474/943] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:06.601226 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:06.739515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:06.754791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571bacb-ecaf-4080-ae10-fbab90a46ce5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:06.808393 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ae946-a171-4879-8ebc-3fe928448d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:06.809096 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ae946-a171-4879-8ebc-3fe928448d69 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:06.809510 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ae946-a171-4879-8ebc-3fe928448d69 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:06.809972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ae946-a171-4879-8ebc-3fe928448d69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:06.826265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ae946-a171-4879-8ebc-3fe928448d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-087ae946-a171-4879-8ebc-3fe928448d69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:06.833516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:06.834890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:06.835963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 470/944] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:57:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:06.845202 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722e19c-7f0b-4456-a694-cdae4ebec1d9 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:06.964083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722e19c-7f0b-4456-a694-cdae4ebec1d9 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:06.993545 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:07.004180 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:07.004861 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.005142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.005475 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.006293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:07.006607 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:07.008077 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:07.022290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 475/945] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:57:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:07.024958 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ae946-a171-4879-8ebc-3fe928448d69 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:07.166246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ae946-a171-4879-8ebc-3fe928448d69 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:07.207462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a03e73-4927-47ae-ade8-967bc5bec811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:07.208627 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03e73-4927-47ae-ade8-967bc5bec811 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.209181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03e73-4927-47ae-ade8-967bc5bec811 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.209696 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03e73-4927-47ae-ade8-967bc5bec811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.223051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a03e73-4927-47ae-ade8-967bc5bec811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:07.225176 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a03e73-4927-47ae-ade8-967bc5bec811 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-95a03e73-4927-47ae-ade8-967bc5bec811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:07.243207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:07.244474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:07.245931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:07.245931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:07.245931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:07.245931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:07.248433 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 471/946] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:07.257978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:07.412059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574d34-8c39-445a-a485-725ea0475e4e None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.412470 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574d34-8c39-445a-a485-725ea0475e4e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.412716 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574d34-8c39-445a-a485-725ea0475e4e None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.414543 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 472/947] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:07.420385 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f485f6-0ac9-4563-b177-dc50b71ba84e None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.420592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f485f6-0ac9-4563-b177-dc50b71ba84e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.421272 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f485f6-0ac9-4563-b177-dc50b71ba84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.429050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:07.446075 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-84f485f6-0ac9-4563-b177-dc50b71ba84e None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:07.448229 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-84f485f6-0ac9-4563-b177-dc50b71ba84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:07.454447 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede2cef-3821-4cc3-a916-bbf6f00bc338 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:07.517630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:07.518463 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.518841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.519242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.520067 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:07.520473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:07.522007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:07.536154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 476/948] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:07.544803 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:07.562255 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 473/949] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:07.587333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f485f6-0ac9-4563-b177-dc50b71ba84e None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:07.669232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:07.730595 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67b00a-959b-4b3c-87f9-309b34c4ca61 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:07.733268 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f485f6-0ac9-4563-b177-dc50b71ba84e None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:07.772288 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1847-0914-4ddf-b6f7-1be1062d8be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:07.773454 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1847-0914-4ddf-b6f7-1be1062d8be1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.774026 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1847-0914-4ddf-b6f7-1be1062d8be1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.774438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1847-0914-4ddf-b6f7-1be1062d8be1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.781776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:07.787023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec1847-0914-4ddf-b6f7-1be1062d8be1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:07.789035 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca4aae-a723-4c8b-a32f-a1da47c38dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:07.789212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec1847-0914-4ddf-b6f7-1be1062d8be1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:07.789842 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca4aae-a723-4c8b-a32f-a1da47c38dc5 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.790263 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca4aae-a723-4c8b-a32f-a1da47c38dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.790559 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca4aae-a723-4c8b-a32f-a1da47c38dc5 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.791300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca4aae-a723-4c8b-a32f-a1da47c38dc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:07.792953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca4aae-a723-4c8b-a32f-a1da47c38dc5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-7eec1847-0914-4ddf-b6f7-1be1062d8be1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:57:07.804802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:07.806084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 477/950] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:57:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165671e-fbce-4aa1-889c-c0941b4b5aed None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.808488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165671e-fbce-4aa1-889c-c0941b4b5aed None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.809953 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165671e-fbce-4aa1-889c-c0941b4b5aed None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.809953 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 478/951] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:07.813666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165671e-fbce-4aa1-889c-c0941b4b5aed None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:07.816721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.sql.core [None req-baca4aae-a723-4c8b-a32f-a1da47c38dc5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:57:07.816721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:57:07.816721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [parameters: {'user_id': 'ee24dea09ecd4c1398bb1b81476cb1c5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:57:07.816721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=347147) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-baca4aae-a723-4c8b-a32f-a1da47c38dc5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result.read() Jul 28 08:57:07.825780 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:57:07.827284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application yield resource Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise self.value Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application session.commit() Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:57:07.828973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:57:07.831939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:57:07.834804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application result.read() Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:57:07.837106 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ee24dea09ecd4c1398bb1b81476cb1c5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:07.839406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 08:57:07.841953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 474/952] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 08:57:07 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0022-c0b1-4acd-a15c-d47c602287a1 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0022-c0b1-4acd-a15c-d47c602287a1 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:07.843509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0022-c0b1-4acd-a15c-d47c602287a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:07.859055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-b9ac0022-c0b1-4acd-a15c-d47c602287a1 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:07.861267 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b9ac0022-c0b1-4acd-a15c-d47c602287a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:07.965051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165671e-fbce-4aa1-889c-c0941b4b5aed None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:07.976881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:07.983387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 475/953] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:07.984245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac0022-c0b1-4acd-a15c-d47c602287a1 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.021051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.021822 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.022368 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.022860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.024387 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:08.024823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.026398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.051362 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 479/954] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:57:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:08.052582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.142684 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac0022-c0b1-4acd-a15c-d47c602287a1 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.187686 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:08.192190 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eff574-3ac1-4a85-b495-1f7b045510ad None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.205564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.206561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.206852 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.207259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.219478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.222134 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:57:08.237586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:08.239060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 476/955] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:08.240941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.250259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.251083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.251467 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.251871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.255043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.258605 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:08.273439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:08.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:08.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 480/956] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:08.277338 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.322820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:08.395375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c630dc1-6890-43b0-a656-6018539c9e15 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.427919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235765c-db6f-409f-a453-f3772bf6976e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.453590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.454849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.455319 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.455838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.470340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.472174 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.478885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.479757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.480116 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.480498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.481490 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:08.481943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.483723 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:08.492062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:08.493392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:08.494554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:08.494554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:08.494554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:08.494554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:08.494554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 477/957] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:08.494554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.531999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 481/958] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:08.533325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.570906 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:08.692417 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97421f07-c214-42c4-b07c-f1b6a25b0618 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.703633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:08.723703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfdd07-8192-4eff-a594-5638df679510 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.749453 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.750920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.751567 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.752148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.753271 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:08.754252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.756610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.769500 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 478/959] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:08.769865 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.770558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.770839 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.771121 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.771837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:08.772157 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.773619 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.782047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.796630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 482/960] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:08.806724 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.940016 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:08.944725 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc458b-2cff-43fc-8f27-50b92d02b6a3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.955955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6d6d0-1e49-4f14-8b4d-f65e5a298890 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.986841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d00be5-81f1-433f-bad1-37db1134c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.987494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d00be5-81f1-433f-bad1-37db1134c807 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.987785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d00be5-81f1-433f-bad1-37db1134c807 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.988068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d00be5-81f1-433f-bad1-37db1134c807 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.988771 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d00be5-81f1-433f-bad1-37db1134c807 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:08.989085 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d00be5-81f1-433f-bad1-37db1134c807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.990655 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d00be5-81f1-433f-bad1-37db1134c807 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.996251 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 479/961] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:08.999794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de72ba7-0efd-44f3-a931-43c709648043 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.000113 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de72ba7-0efd-44f3-a931-43c709648043 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.000403 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de72ba7-0efd-44f3-a931-43c709648043 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.001793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 480/962] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:09.002553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de72ba7-0efd-44f3-a931-43c709648043 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.016711 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54948000-8d6b-4582-b4a9-9bef9ebe4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.017736 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54948000-8d6b-4582-b4a9-9bef9ebe4280 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.018044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54948000-8d6b-4582-b4a9-9bef9ebe4280 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.018523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54948000-8d6b-4582-b4a9-9bef9ebe4280 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.027876 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54948000-8d6b-4582-b4a9-9bef9ebe4280 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.029620 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54948000-8d6b-4582-b4a9-9bef9ebe4280 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-54948000-8d6b-4582-b4a9-9bef9ebe4280 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:57:09.048851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:09.050326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 483/963] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:09.051542 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055cb6-6d47-40b4-bf41-9238ee94b398 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.052813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055cb6-6d47-40b4-bf41-9238ee94b398 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.052813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055cb6-6d47-40b4-bf41-9238ee94b398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.071601 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-60055cb6-6d47-40b4-bf41-9238ee94b398 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:09.072843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-60055cb6-6d47-40b4-bf41-9238ee94b398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:09.075126 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:09.146125 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de72ba7-0efd-44f3-a931-43c709648043 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.156173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 484/964] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:09.157088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055cb6-6d47-40b4-bf41-9238ee94b398 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.190198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc089c-f61d-447b-99fe-5058e8cb5f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.191210 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc089c-f61d-447b-99fe-5058e8cb5f71 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.191486 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc089c-f61d-447b-99fe-5058e8cb5f71 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.191765 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc089c-f61d-447b-99fe-5058e8cb5f71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.192497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc089c-f61d-447b-99fe-5058e8cb5f71 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.193940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc089c-f61d-447b-99fe-5058e8cb5f71 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.219546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 481/965] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:57:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:09.220393 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc089c-f61d-447b-99fe-5058e8cb5f71 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.269569 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:09.310093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055cb6-6d47-40b4-bf41-9238ee94b398 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.364076 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.364752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.365069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.365370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.366339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:09.366737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.368407 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.371004 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:09.390277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 485/966] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:57:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:09.396548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.400447 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc089c-f61d-447b-99fe-5058e8cb5f71 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.455348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47fd27-42f3-4685-a5d1-ad1cf8994ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.456711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47fd27-42f3-4685-a5d1-ad1cf8994ffb None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.457118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47fd27-42f3-4685-a5d1-ad1cf8994ffb None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.457621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47fd27-42f3-4685-a5d1-ad1cf8994ffb None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.462008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47fd27-42f3-4685-a5d1-ad1cf8994ffb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.465109 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47fd27-42f3-4685-a5d1-ad1cf8994ffb None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.498156 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 482/967] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:57:09 2022] POST /identity/v3/services => generated 242 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:09.548528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57bbf-5015-440e-96c1-468251312cae None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.608867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7399536-621e-4ecb-9f9f-994435c6d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.609606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7399536-621e-4ecb-9f9f-994435c6d03d None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.609891 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7399536-621e-4ecb-9f9f-994435c6d03d None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.610206 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7399536-621e-4ecb-9f9f-994435c6d03d None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/ac2817fe7c654a29abe66f6e51b1dec7/roles/8dbaf90916934fa7a0670d2c504af17b` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.612700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7399536-621e-4ecb-9f9f-994435c6d03d None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=ac2817fe7c654a29abe66f6e51b1dec7, role_id=8dbaf90916934fa7a0670d2c504af17b)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.614248 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7399536-621e-4ecb-9f9f-994435c6d03d None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.630798 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 486/968] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:09 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/ac2817fe7c654a29abe66f6e51b1dec7/roles/8dbaf90916934fa7a0670d2c504af17b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:57:11.373893 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0268d8-7e9f-4b9d-bb7e-ccc684003361 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:11.375004 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0268d8-7e9f-4b9d-bb7e-ccc684003361 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:11.375004 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0268d8-7e9f-4b9d-bb7e-ccc684003361 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:11.378163 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 483/969] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:11.382528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e108-3b09-4ad5-acf1-dfba6b3d6f94 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:11.383069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e108-3b09-4ad5-acf1-dfba6b3d6f94 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:11.383069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e108-3b09-4ad5-acf1-dfba6b3d6f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:11.404214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-5993e108-3b09-4ad5-acf1-dfba6b3d6f94 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:11.405526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-5993e108-3b09-4ad5-acf1-dfba6b3d6f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:11.477041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:11.508774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 487/970] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:11.534174 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0268d8-7e9f-4b9d-bb7e-ccc684003361 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:11.667293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a187-0ca4-4202-ade5-6444cf268c2f None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:11.667537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a187-0ca4-4202-ade5-6444cf268c2f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:11.667724 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a187-0ca4-4202-ade5-6444cf268c2f None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:11.669372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 488/971] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:11.674900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abf207-bdb5-4a46-a946-6adeed408fd0 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:11.675181 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abf207-bdb5-4a46-a946-6adeed408fd0 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:11.675463 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abf207-bdb5-4a46-a946-6adeed408fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:11.678395 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:11.699553 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-37abf207-bdb5-4a46-a946-6adeed408fd0 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:11.701731 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-37abf207-bdb5-4a46-a946-6adeed408fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:11.718234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0268d8-7e9f-4b9d-bb7e-ccc684003361 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:11.763104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60418a-0e42-4317-9bb6-d7f00833d9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:11.763809 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60418a-0e42-4317-9bb6-d7f00833d9fa None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:11.764111 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60418a-0e42-4317-9bb6-d7f00833d9fa None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:11.764412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60418a-0e42-4317-9bb6-d7f00833d9fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:11.765039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60418a-0e42-4317-9bb6-d7f00833d9fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:11.766539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60418a-0e42-4317-9bb6-d7f00833d9fa None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1b60418a-0e42-4317-9bb6-d7f00833d9fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:11.782211 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:11.783728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:11.785043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 484/972] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:57:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:11.786061 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61495-cacb-4298-86c8-6241bdd5a321 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:11.786061 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61495-cacb-4298-86c8-6241bdd5a321 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:11.786061 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61495-cacb-4298-86c8-6241bdd5a321 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:11.787354 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 485/973] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:11.790621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b61495-cacb-4298-86c8-6241bdd5a321 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:11.797460 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 489/974] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:11.799135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5e2f5-9486-496b-b82a-99a3bf53b0e3 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:11.799562 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5e2f5-9486-496b-b82a-99a3bf53b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:11.799845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5e2f5-9486-496b-b82a-99a3bf53b0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:11.821378 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-bba5e2f5-9486-496b-b82a-99a3bf53b0e3 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:11.822664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-bba5e2f5-9486-496b-b82a-99a3bf53b0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:11.896252 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 490/975] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:11.897155 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5e2f5-9486-496b-b82a-99a3bf53b0e3 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:11.914029 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b61495-cacb-4298-86c8-6241bdd5a321 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:11.943246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:11.953421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:11.954127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:11.954411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:11.954689 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:11.955394 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:11.955706 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:11.957173 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:11.962632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 486/976] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:57:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:11.967043 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:11.980281 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.008165 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5e2f5-9486-496b-b82a-99a3bf53b0e3 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.047185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.047863 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.048148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.048426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.049071 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.050522 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:12.060401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:12.062022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:12.063446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 491/977] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:12.064983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.083325 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07ace8-a13f-42e6-b4f1-d72796a25f28 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.125433 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30d06f-c4e7-4057-9316-c15edc3b630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.126230 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30d06f-c4e7-4057-9316-c15edc3b630a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.126519 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30d06f-c4e7-4057-9316-c15edc3b630a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.126798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30d06f-c4e7-4057-9316-c15edc3b630a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.137654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30d06f-c4e7-4057-9316-c15edc3b630a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-cb30d06f-c4e7-4057-9316-c15edc3b630a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:12.142901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:12.144651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 487/978] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:57:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:12.145927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30d06f-c4e7-4057-9316-c15edc3b630a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.185559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902ceac-e5f1-4e9a-b533-a139523e45c6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.188033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.214007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.223445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.224088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.224369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.224640 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.225334 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:12.225636 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.227058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.232282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 492/979] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:12.233079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.254590 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30d06f-c4e7-4057-9316-c15edc3b630a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.291615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.292255 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.292538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.292813 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.293501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] RBAC: Adding query filter params (name=compute) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:12.293810 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.295270 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.300182 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 488/980] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:12.300987 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.335367 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230dce-09c4-4e45-8a07-fce503a2310f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.373557 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450466c-6273-4829-8503-154cce221582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.375192 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450466c-6273-4829-8503-154cce221582 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.375487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450466c-6273-4829-8503-154cce221582 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.375788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450466c-6273-4829-8503-154cce221582 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.376532 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450466c-6273-4829-8503-154cce221582 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:12.376845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450466c-6273-4829-8503-154cce221582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.378336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450466c-6273-4829-8503-154cce221582 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.387151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 493/981] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:12.387854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450466c-6273-4829-8503-154cce221582 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.411010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe58cef-e303-4934-b723-64ad6c9eab34 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.434503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.445725 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.452786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.453431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.453720 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.453993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.454799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] RBAC: Adding query filter params (interface=admin, service_id=8d52fc91943d479d934bbd30cc3192d6, region_id=RegionOne) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:12.455114 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.456545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.465838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 489/982] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/endpoints?service_id=8d52fc91943d479d934bbd30cc3192d6&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:12.466826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.519088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450466c-6273-4829-8503-154cce221582 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.562163 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.563194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.563530 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.563849 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.564583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] RBAC: Adding query filter params (type=compute) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:12.564987 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.566819 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.573283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 494/983] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:12.574051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.587320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e524e-19be-4602-a4bb-089cb62a7f38 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.626893 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.627490 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.627785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.628063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.640135 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.641645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:12.657038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:12.658237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:12.665077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:12.665077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:12.665077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:12.665077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:12.665077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 490/984] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:12.665077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.686877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.695279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4bf95d-9d8b-48a8-af56-5cf210d1fb75 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.719898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.732375 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2c738-baac-43c1-b2f2-7460da73eeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.733031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c738-baac-43c1-b2f2-7460da73eeb1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.733361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c738-baac-43c1-b2f2-7460da73eeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.733639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c738-baac-43c1-b2f2-7460da73eeb1 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.734347 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2c738-baac-43c1-b2f2-7460da73eeb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.735765 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2c738-baac-43c1-b2f2-7460da73eeb1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.742480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 495/985] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:57:12.743277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2c738-baac-43c1-b2f2-7460da73eeb1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.776485 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8a210-504b-406c-90ae-94cefcd75128 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.822841 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.823847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.824235 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.824640 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.826039 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] RBAC: Adding query filter params (interface=public, service_id=1b792bbbbea74e6999860e19cf625cdf, region_id=RegionOne) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:12.826585 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.828699 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.839540 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 491/986] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/endpoints?service_id=1b792bbbbea74e6999860e19cf625cdf&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:12.841246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.890634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2c738-baac-43c1-b2f2-7460da73eeb1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.932748 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.933780 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.934065 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.934407 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.936043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:12.936820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:12.937900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:12.952022 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 496/987] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:12.957873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.964685 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce31e-1d20-47e9-a9d8-7aa06d6b7670 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.979138 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.981631 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:13.005888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d91421-e43d-4607-8807-447842c914e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.006574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d91421-e43d-4607-8807-447842c914e6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.006860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d91421-e43d-4607-8807-447842c914e6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.007139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d91421-e43d-4607-8807-447842c914e6 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.008375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d91421-e43d-4607-8807-447842c914e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.010294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d91421-e43d-4607-8807-447842c914e6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.018172 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 492/988] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:57:13.084675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012afa-7c34-4cf8-90d3-db479f1ff317 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.124794 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.125473 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.125812 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.126065 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.134779 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.136282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:57:13.149967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:13.151461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:57:13.152738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:13.154062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 497/989] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:13.157613 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d91421-e43d-4607-8807-447842c914e6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:13.275264 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d91421-e43d-4607-8807-447842c914e6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.310806 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.311485 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.311766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.312073 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.312869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:13.313191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.314770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.333821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 493/990] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 08:57:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:13.339429 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:13.424412 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:13.457266 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da930e6-76ed-4b48-a0c8-ad3be04530e0 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.494056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e9213-7bd7-453b-a87f-6ecc13854c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.494737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e9213-7bd7-453b-a87f-6ecc13854c35 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.495025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e9213-7bd7-453b-a87f-6ecc13854c35 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.495305 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e9213-7bd7-453b-a87f-6ecc13854c35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.575123 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e9213-7bd7-453b-a87f-6ecc13854c35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.576586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e9213-7bd7-453b-a87f-6ecc13854c35 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.665557 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:13.692185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 498/991] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:13 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:13.698289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:13.734521 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:13.815461 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7587b-ce34-4ebe-b5a8-99871ebf902f None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.851833 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.852389 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.852673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.852950 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.864956 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.866532 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:13.883592 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:13.884903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:13.886072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:13.886072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:13.886072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:13.886072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:13.889939 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 494/992] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:13.891398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e9213-7bd7-453b-a87f-6ecc13854c35 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:14.005992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e9213-7bd7-453b-a87f-6ecc13854c35 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:14.049355 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:14.050005 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:14.050314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:14.050600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:14.051306 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:14.051615 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:14.053465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:14.071278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 499/993] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:14.075271 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:14.197168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:14.215123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3af553-75a8-46f1-a13b-5327bb84667b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:14.252007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:14.252683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:14.253010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:14.253193 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:14.263250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:14.263250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:57:14.277431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:14.282375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:14.283553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 495/994] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:14.287907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:14.335733 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:14.406733 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01fe65-26cf-4ff1-a073-d67f59e10409 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:14.446993 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bfb90-eee2-4c94-9f50-ac822bd804ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:14.447786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bfb90-eee2-4c94-9f50-ac822bd804ce None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:14.447976 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bfb90-eee2-4c94-9f50-ac822bd804ce None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:14.448247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bfb90-eee2-4c94-9f50-ac822bd804ce None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:14.448999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bfb90-eee2-4c94-9f50-ac822bd804ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:14.449317 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bfb90-eee2-4c94-9f50-ac822bd804ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:14.450836 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bfb90-eee2-4c94-9f50-ac822bd804ce None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:14.464877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 500/995] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:57:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:14.471665 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:14.597219 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b794a7-9c3d-4ae9-92b0-ba53f1d42ef8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:14.651875 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597df287-4fcc-4d5d-bd6b-2dfaf61ddbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:14.652522 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df287-4fcc-4d5d-bd6b-2dfaf61ddbf2 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:14.652807 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df287-4fcc-4d5d-bd6b-2dfaf61ddbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:14.653090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df287-4fcc-4d5d-bd6b-2dfaf61ddbf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:14.653912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597df287-4fcc-4d5d-bd6b-2dfaf61ddbf2 None admin] RBAC: Adding query filter params (role.id=8dbaf90916934fa7a0670d2c504af17b, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=ac2817fe7c654a29abe66f6e51b1dec7) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:14.654303 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597df287-4fcc-4d5d-bd6b-2dfaf61ddbf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:14.655880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597df287-4fcc-4d5d-bd6b-2dfaf61ddbf2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:14.662961 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 496/996] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:14 2022] GET /identity/v3/role_assignments?user.id=ac2817fe7c654a29abe66f6e51b1dec7&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=8dbaf90916934fa7a0670d2c504af17b => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:14.775445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657e28-decf-4d78-b213-7c7b96846b19 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:14.775709 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657e28-decf-4d78-b213-7c7b96846b19 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:14.775960 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657e28-decf-4d78-b213-7c7b96846b19 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:14.777491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 501/997] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:14.783386 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be088c-cf2a-46fe-be88-2628b0d3ace4 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:14.784045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be088c-cf2a-46fe-be88-2628b0d3ace4 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:14.784045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be088c-cf2a-46fe-be88-2628b0d3ace4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:14.804427 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-94be088c-cf2a-46fe-be88-2628b0d3ace4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:14.805714 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-94be088c-cf2a-46fe-be88-2628b0d3ace4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:14.809265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:14.888254 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 497/998] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:14.911054 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c657e28-decf-4d78-b213-7c7b96846b19 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.023866 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:15.030759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c657e28-decf-4d78-b213-7c7b96846b19 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.075640 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905bd7b-f93e-474b-926e-34db0fb49e96 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.075898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905bd7b-f93e-474b-926e-34db0fb49e96 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.076179 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905bd7b-f93e-474b-926e-34db0fb49e96 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.077899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 498/999] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:15.082885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.083234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424d0e8-0103-44bb-8ccb-d73126bdd8b8 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.083481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424d0e8-0103-44bb-8ccb-d73126bdd8b8 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.083662 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.083817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424d0e8-0103-44bb-8ccb-d73126bdd8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.083934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.084209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.084872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.086594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:15.096602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:15.098185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:15.099492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 502/1000] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 08:57:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:15.104163 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.107219 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-b424d0e8-0103-44bb-8ccb-d73126bdd8b8 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:15.108580 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b424d0e8-0103-44bb-8ccb-d73126bdd8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:15.187364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 499/1001] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:15.210425 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424d0e8-0103-44bb-8ccb-d73126bdd8b8 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.218918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc43ca3-e46c-482e-a2b2-7f65ce3b428e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.263453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.264280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.264498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.264774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.265475 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:15.265782 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.267736 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.277851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 503/1002] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:57:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:15.278646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.286732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:15.311189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:15.323941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424d0e8-0103-44bb-8ccb-d73126bdd8b8 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.364541 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.365218 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.365586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.365875 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] PATH_INFO: `/v3/services/compute` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.366735 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.368177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:15.377651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:15.379314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:15.380482 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 500/1003] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:15.385840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.407451 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67e5a0-7248-42ce-924b-cc31929b4d24 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.454246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.455708 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.456214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.456684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.457756 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:15.457987 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.459441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.465021 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 504/1004] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 08:57:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:15.470846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.520009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033fa34-5d46-40a1-b2ea-dce82ca233ce None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.577662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.578858 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.578858 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.579312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.580366 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] RBAC: Adding query filter params (name=compute) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:15.580890 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.583377 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.585867 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:15.599620 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 501/1005] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:57:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:15.600439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.608236 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:15.619203 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c1fd7-5e59-490c-829f-ecee1e938049 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.660023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f522c9-cc4f-4b6b-a7b6-f77d2aa673a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.661094 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f522c9-cc4f-4b6b-a7b6-f77d2aa673a7 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.661503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f522c9-cc4f-4b6b-a7b6-f77d2aa673a7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.661814 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f522c9-cc4f-4b6b-a7b6-f77d2aa673a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.662650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f522c9-cc4f-4b6b-a7b6-f77d2aa673a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.664081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f522c9-cc4f-4b6b-a7b6-f77d2aa673a7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.682401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 505/1006] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:57:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:57:15.742720 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e3b72-131b-40c1-b8d1-fea3023705be None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.792428 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f34e1-f96e-47a1-a631-7e4c3201ea35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.793264 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f34e1-f96e-47a1-a631-7e4c3201ea35 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.793397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f34e1-f96e-47a1-a631-7e4c3201ea35 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.793677 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f34e1-f96e-47a1-a631-7e4c3201ea35 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.794397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f34e1-f96e-47a1-a631-7e4c3201ea35 None admin] RBAC: Adding query filter params (type=compute) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:15.794738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f34e1-f96e-47a1-a631-7e4c3201ea35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.796342 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f34e1-f96e-47a1-a631-7e4c3201ea35 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.807493 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 502/1007] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 08:57:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:15.809924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f522c9-cc4f-4b6b-a7b6-f77d2aa673a7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.937470 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f522c9-cc4f-4b6b-a7b6-f77d2aa673a7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.978461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6cd17-9b5d-4283-ae32-e4341f3c4308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.979112 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6cd17-9b5d-4283-ae32-e4341f3c4308 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.979397 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6cd17-9b5d-4283-ae32-e4341f3c4308 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.979672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6cd17-9b5d-4283-ae32-e4341f3c4308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.980426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6cd17-9b5d-4283-ae32-e4341f3c4308 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.981833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6cd17-9b5d-4283-ae32-e4341f3c4308 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:16.008537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 506/1008] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:57:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:16.759616 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109c469-ae83-4ae0-b01d-827f8ebbc03d None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:16.759903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109c469-ae83-4ae0-b01d-827f8ebbc03d None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:16.760174 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109c469-ae83-4ae0-b01d-827f8ebbc03d None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:16.761596 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 503/1009] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:16.767468 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97276d78-aec0-40a0-a4c6-f6da3dd74acd None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:16.767984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97276d78-aec0-40a0-a4c6-f6da3dd74acd None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:16.768118 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97276d78-aec0-40a0-a4c6-f6da3dd74acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:16.785617 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:16.792196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-97276d78-aec0-40a0-a4c6-f6da3dd74acd None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:16.794614 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-97276d78-aec0-40a0-a4c6-f6da3dd74acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:16.885518 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 507/1010] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:16 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:16.907548 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109c469-ae83-4ae0-b01d-827f8ebbc03d None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:16.972340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:17.033114 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109c469-ae83-4ae0-b01d-827f8ebbc03d None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:17.072286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:17.073278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:17.073565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:17.073846 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] PATH_INFO: `/v3/services/image` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:17.074524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:17.075938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:17.085333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:17.086662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:17.088247 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:17.089537 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 504/1011] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:57:16 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:17.092923 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97276d78-aec0-40a0-a4c6-f6da3dd74acd None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:17.196586 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97276d78-aec0-40a0-a4c6-f6da3dd74acd None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:17.228287 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:17.235118 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b36a60-6b17-42ed-9b2b-19b8c421daca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:17.235768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b36a60-6b17-42ed-9b2b-19b8c421daca None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:17.236056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b36a60-6b17-42ed-9b2b-19b8c421daca None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:17.236335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b36a60-6b17-42ed-9b2b-19b8c421daca None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:17.237041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b36a60-6b17-42ed-9b2b-19b8c421daca None admin] RBAC: Adding query filter params (name=image) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:17.237355 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b36a60-6b17-42ed-9b2b-19b8c421daca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:17.238804 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b36a60-6b17-42ed-9b2b-19b8c421daca None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:17.244256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 508/1012] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:17.249993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:17.356457 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018648b-77c7-42c8-98ff-fe26b4995724 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:17.380052 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:17.393771 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b2351-0ee5-4d57-b709-1ba950727893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:17.394441 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b2351-0ee5-4d57-b709-1ba950727893 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:17.394728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b2351-0ee5-4d57-b709-1ba950727893 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:17.395007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b2351-0ee5-4d57-b709-1ba950727893 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:17.395699 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b2351-0ee5-4d57-b709-1ba950727893 None admin] RBAC: Adding query filter params (type=image) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:17.396010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b2351-0ee5-4d57-b709-1ba950727893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:17.397431 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b2351-0ee5-4d57-b709-1ba950727893 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:17.402497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 505/1013] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:17 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:17.799826 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11066601-40a5-4c98-ac38-5b57d2ce847c None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:17.800413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11066601-40a5-4c98-ac38-5b57d2ce847c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:17.800413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11066601-40a5-4c98-ac38-5b57d2ce847c None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:17.801927 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 509/1014] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:17.807270 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306cf93-6380-4006-987e-9180127439e2 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:17.807539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306cf93-6380-4006-987e-9180127439e2 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:17.807824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306cf93-6380-4006-987e-9180127439e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:17.827874 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-1306cf93-6380-4006-987e-9180127439e2 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:17.829114 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-1306cf93-6380-4006-987e-9180127439e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:17.904946 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 506/1015] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:17 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:17.928132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11066601-40a5-4c98-ac38-5b57d2ce847c None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:18.045851 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11066601-40a5-4c98-ac38-5b57d2ce847c None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:18.091121 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164ea2e-10b1-418d-bd6f-461d0f4af422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:18.092731 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164ea2e-10b1-418d-bd6f-461d0f4af422 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.093541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164ea2e-10b1-418d-bd6f-461d0f4af422 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.094605 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164ea2e-10b1-418d-bd6f-461d0f4af422 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:18.096593 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164ea2e-10b1-418d-bd6f-461d0f4af422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:18.098382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164ea2e-10b1-418d-bd6f-461d0f4af422 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:18.100964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 510/1016] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:18.107346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306cf93-6380-4006-987e-9180127439e2 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:18.147106 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64a0d1-b452-402d-808a-9e6da4b3c1aa None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.147378 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64a0d1-b452-402d-808a-9e6da4b3c1aa None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.147650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64a0d1-b452-402d-808a-9e6da4b3c1aa None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:18.149177 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 511/1017] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:18.154173 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ee38d-e473-46da-96cb-90ddd54e5d09 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.154468 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ee38d-e473-46da-96cb-90ddd54e5d09 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.154793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ee38d-e473-46da-96cb-90ddd54e5d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:18.174428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-558ee38d-e473-46da-96cb-90ddd54e5d09 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:18.175646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-558ee38d-e473-46da-96cb-90ddd54e5d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:18.225751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306cf93-6380-4006-987e-9180127439e2 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:18.242790 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:18.247801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:18.254130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 512/1018] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:18 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:18.277258 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ee38d-e473-46da-96cb-90ddd54e5d09 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:18.282506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0425-e831-4f57-aa56-c641dd493c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:18.283155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0425-e831-4f57-aa56-c641dd493c14 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.283495 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0425-e831-4f57-aa56-c641dd493c14 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.283704 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0425-e831-4f57-aa56-c641dd493c14 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:18.284401 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0425-e831-4f57-aa56-c641dd493c14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:18.285883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0425-e831-4f57-aa56-c641dd493c14 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:18.316556 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 507/1019] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:57:18 2022] POST /identity/v3/projects => generated 311 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:57:18.418942 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ee38d-e473-46da-96cb-90ddd54e5d09 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:18.460279 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84769917-76c2-4820-a339-b838b2821e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:18.460971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84769917-76c2-4820-a339-b838b2821e95 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.461260 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84769917-76c2-4820-a339-b838b2821e95 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.461550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84769917-76c2-4820-a339-b838b2821e95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:18.475103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84769917-76c2-4820-a339-b838b2821e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-84769917-76c2-4820-a339-b838b2821e95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:18.481550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:18.482887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:18.484030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 513/1020] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:57:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:18.489063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0425-e831-4f57-aa56-c641dd493c14 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:18.618333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0425-e831-4f57-aa56-c641dd493c14 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:18.658687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:18.659329 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.659648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.659946 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:18.660618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:18.660922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:18.662445 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:18.671948 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 508/1021] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:57:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:18.678051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84769917-76c2-4820-a339-b838b2821e95 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:18.743201 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:18.792037 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84769917-76c2-4820-a339-b838b2821e95 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:18.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:18.830891 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.831233 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.831457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:18.842695 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:18.844204 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:18.860352 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:18.862650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:18.864835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:18.864835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:18.864835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:18.864835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:18.864835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 514/1022] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:18.875742 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:18.918236 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:18.999289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039bd26-0c6f-46d8-8366-1225f0214fd7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.050149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.051078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.051208 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.051395 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.052092 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:19.052402 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:19.053906 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:19.078139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 509/1023] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:19.081936 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.252688 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802581b-a891-4a9d-af30-63301ad11964 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.263544 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.300093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7c685-6cd7-45a9-81d3-944c08cac474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.300752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7c685-6cd7-45a9-81d3-944c08cac474 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.301086 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7c685-6cd7-45a9-81d3-944c08cac474 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.301336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7c685-6cd7-45a9-81d3-944c08cac474 None admin] PATH_INFO: `/v3/users/nova` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.310025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7c685-6cd7-45a9-81d3-944c08cac474 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:19.311640 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7c685-6cd7-45a9-81d3-944c08cac474 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-c1d7c685-6cd7-45a9-81d3-944c08cac474 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:57:19.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:19.327348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:57:19.328548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:19.330502 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 515/1024] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:19.333068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.434576 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.446284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301c0ba-c097-484a-8f53-16e6956257ef None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.461248 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb843fb-448d-4ccb-b62b-7c6ad9e93d40 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.461571 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb843fb-448d-4ccb-b62b-7c6ad9e93d40 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.461837 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb843fb-448d-4ccb-b62b-7c6ad9e93d40 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.463444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 516/1025] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:19.468598 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ed82-0824-4b09-babf-3a9b43784dac None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.468873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ed82-0824-4b09-babf-3a9b43784dac None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.469156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ed82-0824-4b09-babf-3a9b43784dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.484711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.485357 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.485643 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.485922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.486770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:19.487083 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:19.488358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-4a69ed82-0824-4b09-babf-3a9b43784dac None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:19.488580 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:19.489593 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-4a69ed82-0824-4b09-babf-3a9b43784dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:19.510687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 510/1026] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:19.516777 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.572470 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 517/1027] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:19 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:19.594813 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69ed82-0824-4b09-babf-3a9b43784dac None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.635998 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933fc66-3b13-48a1-8038-8b1bfc1d24e4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.672817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd89b9-660e-445a-9bf2-f4cb65878327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.673446 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd89b9-660e-445a-9bf2-f4cb65878327 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.673726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd89b9-660e-445a-9bf2-f4cb65878327 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.673995 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd89b9-660e-445a-9bf2-f4cb65878327 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.694164 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.699367 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.705666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69ed82-0824-4b09-babf-3a9b43784dac None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.744695 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16110592-7d67-44df-bcd5-d6954f378617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.745348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16110592-7d67-44df-bcd5-d6954f378617 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.745636 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16110592-7d67-44df-bcd5-d6954f378617 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.745918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16110592-7d67-44df-bcd5-d6954f378617 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.746675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16110592-7d67-44df-bcd5-d6954f378617 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:19.748100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16110592-7d67-44df-bcd5-d6954f378617 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:19.750456 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd89b9-660e-445a-9bf2-f4cb65878327 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:19.752285 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd89b9-660e-445a-9bf2-f4cb65878327 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:19.765109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 518/1028] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:57:19 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:57:19.866630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 511/1029] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:19.872984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16110592-7d67-44df-bcd5-d6954f378617 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.997937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16110592-7d67-44df-bcd5-d6954f378617 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.038815 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.039513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.039811 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.040108 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.052934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.054457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:20.070249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:20.071469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:20.072594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:20.072594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:20.072594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:20.072594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:20.072594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 519/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:20.077785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd89b9-660e-445a-9bf2-f4cb65878327 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.132337 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.189586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd89b9-660e-445a-9bf2-f4cb65878327 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.230931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.231573 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.231858 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.232135 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.232998 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:20.233221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.234728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.248152 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 512/1031] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:20.254045 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.259474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.362803 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b391b4-6307-4f43-bfaf-9f4c5312d2a6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.398648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6e7e7-6d45-495e-8484-f33979ca7067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.399393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6e7e7-6d45-495e-8484-f33979ca7067 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.399683 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6e7e7-6d45-495e-8484-f33979ca7067 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.399949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6e7e7-6d45-495e-8484-f33979ca7067 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.407249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6e7e7-6d45-495e-8484-f33979ca7067 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.408739 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6e7e7-6d45-495e-8484-f33979ca7067 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-e9f6e7e7-6d45-495e-8484-f33979ca7067 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:57:20.420648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:20.421831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:20.423040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 520/1032] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:20.427741 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.453258 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d406e-5262-41bf-8d2e-20ced445b437 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.453520 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d406e-5262-41bf-8d2e-20ced445b437 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.453779 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d406e-5262-41bf-8d2e-20ced445b437 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.455344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 521/1033] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:20.460252 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a89d5-af20-4d99-bc7b-527d854b6cc7 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.460538 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a89d5-af20-4d99-bc7b-527d854b6cc7 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.460832 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a89d5-af20-4d99-bc7b-527d854b6cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.481875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-1e0a89d5-af20-4d99-bc7b-527d854b6cc7 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:20.483201 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-1e0a89d5-af20-4d99-bc7b-527d854b6cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:20.538412 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352c7f1-e6b1-4645-b40c-bfe72c880887 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.546377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.548481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.566185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 522/1034] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:20.581703 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.582368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.582650 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.582922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.583673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:20.583981 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.585454 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.587509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a89d5-af20-4d99-bc7b-527d854b6cc7 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.598859 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 513/1035] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:20.606538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.700780 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a89d5-af20-4d99-bc7b-527d854b6cc7 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.716020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1074e-b1ce-4648-9502-c949edb9fb73 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.741120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634f9f-d9c1-4951-a2c0-0b5352e239a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.741800 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634f9f-d9c1-4951-a2c0-0b5352e239a2 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.742085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634f9f-d9c1-4951-a2c0-0b5352e239a2 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.742393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634f9f-d9c1-4951-a2c0-0b5352e239a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.743611 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88634f9f-d9c1-4951-a2c0-0b5352e239a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.745089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88634f9f-d9c1-4951-a2c0-0b5352e239a2 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.750133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 523/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:20.753389 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634f9f-d9c1-4951-a2c0-0b5352e239a2 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.759814 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8099a-dc5b-4051-a720-692742b389de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.760439 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8099a-dc5b-4051-a720-692742b389de None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.760722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8099a-dc5b-4051-a720-692742b389de None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.760993 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8099a-dc5b-4051-a720-692742b389de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.761790 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8099a-dc5b-4051-a720-692742b389de None admin] RBAC: Adding query filter params (role.id=8dbaf90916934fa7a0670d2c504af17b, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=4fe070acb9a84d90baaa3b08a4fd3191) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:20.762118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8099a-dc5b-4051-a720-692742b389de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.763603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8099a-dc5b-4051-a720-692742b389de None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.769761 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 514/1037] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/role_assignments?user.id=4fe070acb9a84d90baaa3b08a4fd3191&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=8dbaf90916934fa7a0670d2c504af17b => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:20.826949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.874081 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634f9f-d9c1-4951-a2c0-0b5352e239a2 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.913499 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0251f60-61d5-46c0-9cb5-ee7da7d592cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.914191 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251f60-61d5-46c0-9cb5-ee7da7d592cc None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.914465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251f60-61d5-46c0-9cb5-ee7da7d592cc None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.914743 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251f60-61d5-46c0-9cb5-ee7da7d592cc None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.915479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0251f60-61d5-46c0-9cb5-ee7da7d592cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.917020 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0251f60-61d5-46c0-9cb5-ee7da7d592cc None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.946767 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 524/1038] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:20 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:21.870343 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638ce64-8f3b-40f3-8839-ec2c4b9172ea None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.870934 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638ce64-8f3b-40f3-8839-ec2c4b9172ea None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.870934 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638ce64-8f3b-40f3-8839-ec2c4b9172ea None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.872422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 515/1039] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:21.878136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656103-715f-4cad-8fdf-cb0715e51f8b None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.878625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656103-715f-4cad-8fdf-cb0715e51f8b None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.878717 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656103-715f-4cad-8fdf-cb0715e51f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.898639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-56656103-715f-4cad-8fdf-cb0715e51f8b None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:21.899900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-56656103-715f-4cad-8fdf-cb0715e51f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:21.977071 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 525/1040] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:22.004919 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638ce64-8f3b-40f3-8839-ec2c4b9172ea None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:22.043948 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:22.126761 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638ce64-8f3b-40f3-8839-ec2c4b9172ea None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.165786 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.166508 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.166762 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.167048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] PATH_INFO: `/v3/services/image` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.167689 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.169107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:22.178660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:22.179960 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:22.181767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 516/1041] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:57:22 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:22.186010 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56656103-715f-4cad-8fdf-cb0715e51f8b None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:22.240899 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:22.296732 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56656103-715f-4cad-8fdf-cb0715e51f8b None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.332386 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.333025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.333307 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.333581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.334297 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] RBAC: Adding query filter params (name=image) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:22.334604 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.336060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.346160 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 526/1042] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:22.347455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:22.452888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:22.460626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac1b2d-33b6-4978-9bdb-d944304f14a7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.496433 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.497064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.497344 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.497619 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.498331 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] RBAC: Adding query filter params (type=image) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:22.498672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.500077 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.505562 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 517/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:22.511598 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:22.618925 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec610fd-32ed-4bc9-8608-a66971b25978 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.633095 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:22.659273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fba95-3cf9-4804-bf17-a10e4e223a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.659913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fba95-3cf9-4804-bf17-a10e4e223a4f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.660196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fba95-3cf9-4804-bf17-a10e4e223a4f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.660471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fba95-3cf9-4804-bf17-a10e4e223a4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.661244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fba95-3cf9-4804-bf17-a10e4e223a4f None admin] RBAC: Adding query filter params (interface=public, service_id=d9ec229a441044099af9957d756c2c36, region_id=RegionOne) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:22.661556 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fba95-3cf9-4804-bf17-a10e4e223a4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.662997 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fba95-3cf9-4804-bf17-a10e4e223a4f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.668669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 527/1044] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:57:22 2022] GET /identity/v3/endpoints?service_id=d9ec229a441044099af9957d756c2c36&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:22.674540 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:22.780631 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbccc41-3444-4a41-b6e7-9c25ec72babb None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.788582 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa87a64-b375-46bd-a537-80f8fb769b9e None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.788961 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa87a64-b375-46bd-a537-80f8fb769b9e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.789323 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa87a64-b375-46bd-a537-80f8fb769b9e None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.790873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 528/1045] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:22.796768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b5549-eb4c-4fc2-9193-37517dbe5e08 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.797017 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b5549-eb4c-4fc2-9193-37517dbe5e08 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.797287 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b5549-eb4c-4fc2-9193-37517dbe5e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.816487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-390b5549-eb4c-4fc2-9193-37517dbe5e08 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:22.817686 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-390b5549-eb4c-4fc2-9193-37517dbe5e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:22.820084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178e529-fc85-4645-b902-ba1dd84860cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.820721 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178e529-fc85-4645-b902-ba1dd84860cf None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.821002 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178e529-fc85-4645-b902-ba1dd84860cf None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.821281 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178e529-fc85-4645-b902-ba1dd84860cf None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.821937 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178e529-fc85-4645-b902-ba1dd84860cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.823365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178e529-fc85-4645-b902-ba1dd84860cf None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.830122 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 518/1046] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:57:22 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:57:22.897087 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 529/1047] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:22.920020 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178e529-fc85-4645-b902-ba1dd84860cf None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:22.951922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:23.036138 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081953b6-436a-4c1b-a74c-7092e447e902 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.036449 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081953b6-436a-4c1b-a74c-7092e447e902 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.036723 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081953b6-436a-4c1b-a74c-7092e447e902 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.038128 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 530/1048] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:23.043532 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d5c2f-c688-42ad-ba96-0784ea450126 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.043823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d5c2f-c688-42ad-ba96-0784ea450126 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.044111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d5c2f-c688-42ad-ba96-0784ea450126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.047014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178e529-fc85-4645-b902-ba1dd84860cf None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.062830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-eb1d5c2f-c688-42ad-ba96-0784ea450126 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:23.064041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-eb1d5c2f-c688-42ad-ba96-0784ea450126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:23.084880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb48d6-33f5-482c-b256-d231614155c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.085544 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb48d6-33f5-482c-b256-d231614155c4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.085834 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb48d6-33f5-482c-b256-d231614155c4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.086139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb48d6-33f5-482c-b256-d231614155c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.099629 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb48d6-33f5-482c-b256-d231614155c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-8bcb48d6-33f5-482c-b256-d231614155c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:23.107183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:23.108487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:23.109668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:23.110530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 519/1049] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:57:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:23.115665 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb48d6-33f5-482c-b256-d231614155c4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.150288 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 531/1050] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:23.168049 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d5c2f-c688-42ad-ba96-0784ea450126 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.207860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:23.208346 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:23.225294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb48d6-33f5-482c-b256-d231614155c4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.264989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.265633 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.265914 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.266217 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.266949 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:23.267248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:23.268668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:23.277945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 520/1051] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:23.283654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.284595 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d5c2f-c688-42ad-ba96-0784ea450126 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.323583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65205956-572a-4bba-94b3-7eacc75f7007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.324166 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205956-572a-4bba-94b3-7eacc75f7007 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.324476 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205956-572a-4bba-94b3-7eacc75f7007 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.324765 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205956-572a-4bba-94b3-7eacc75f7007 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.336842 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65205956-572a-4bba-94b3-7eacc75f7007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-65205956-572a-4bba-94b3-7eacc75f7007 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:23.341828 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:23.343063 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:23.344142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 532/1052] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:23.349086 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65205956-572a-4bba-94b3-7eacc75f7007 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.396093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce7405-7bd4-47ce-b2d6-5ccbf911829d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.437055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.437730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.438038 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.438567 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.448876 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:23.450587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:23.452012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:23.456205 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65205956-572a-4bba-94b3-7eacc75f7007 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.459450 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:23.470584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:23.471764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:23.472896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:23.472896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:23.472896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:23.472896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:23.476693 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 521/1053] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:23.477503 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.497907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.498722 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.498995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.499274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.500052 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:23.500367 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:23.501799 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:23.516298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 533/1054] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:23.517403 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.602396 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f779-8989-4161-9b40-ef9a62e6317d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.648485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dadef-3333-4e22-a54f-1c9d0781b1c2 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.661595 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.662697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.663199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.663701 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.664888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:23.665478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:23.668093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:23.690078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9e268-d9c9-485d-9d6e-8020eed7fa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.690383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 522/1055] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:23.691619 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e268-d9c9-485d-9d6e-8020eed7fa4e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.692089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e268-d9c9-485d-9d6e-8020eed7fa4e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.692226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e268-d9c9-485d-9d6e-8020eed7fa4e None admin] PATH_INFO: `/v3/users/cd9615a0c09241eeae1f27d858a19d3c` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.696560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.711061 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9e268-d9c9-485d-9d6e-8020eed7fa4e None admin] RBAC: Authorizing `identity:get_user(user_id=cd9615a0c09241eeae1f27d858a19d3c)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:23.712554 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9e268-d9c9-485d-9d6e-8020eed7fa4e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:23.714983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 534/1056] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/users/cd9615a0c09241eeae1f27d858a19d3c => generated 285 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:23.720831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9e268-d9c9-485d-9d6e-8020eed7fa4e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.740584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:23.772025 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:23.829190 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c758cb5-acbb-4c50-843b-085e34815c19 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.841916 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9e268-d9c9-485d-9d6e-8020eed7fa4e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:23.868606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.869540 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.870121 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.870579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] PATH_INFO: `/v3/users/nova` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.879025 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:23.880540 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:23.882008 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee99069-781f-49aa-ad28-70259d4ea35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:23.882752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee99069-781f-49aa-ad28-70259d4ea35f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.883044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee99069-781f-49aa-ad28-70259d4ea35f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.883327 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee99069-781f-49aa-ad28-70259d4ea35f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:57:23.893787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:23.895844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:23.897293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 523/1057] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:23.901078 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:23.986135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee99069-781f-49aa-ad28-70259d4ea35f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:23.988321 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee99069-781f-49aa-ad28-70259d4ea35f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.010531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.028799 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d9e8e-4035-4695-90bf-0019ecec5083 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.049538 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.070618 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.071269 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.071550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.071829 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.072636 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:24.072951 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.074462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.099611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 524/1058] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:24.101033 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.118793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 535/1059] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:24.119802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee99069-781f-49aa-ad28-70259d4ea35f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.221335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f2bf6-869d-45c5-a213-e2f30b500981 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.228983 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee99069-781f-49aa-ad28-70259d4ea35f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.251001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.262953 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.263514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.263794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.264367 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.273377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8a1a2-2435-42dd-a9d2-0e325dc62f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.274257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8a1a2-2435-42dd-a9d2-0e325dc62f4f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.274655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8a1a2-2435-42dd-a9d2-0e325dc62f4f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.274995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8a1a2-2435-42dd-a9d2-0e325dc62f4f None admin] PATH_INFO: `/v3/projects/38667033cc1c45a69ced74bc92424c80` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.275652 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.276505 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8a1a2-2435-42dd-a9d2-0e325dc62f4f None admin] RBAC: Authorizing `identity:get_project(project_id=38667033cc1c45a69ced74bc92424c80)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.277171 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.278428 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8a1a2-2435-42dd-a9d2-0e325dc62f4f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.285644 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 536/1060] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/projects/38667033cc1c45a69ced74bc92424c80 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:24.286621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8a1a2-2435-42dd-a9d2-0e325dc62f4f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:24.296306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:24.297635 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:24.298793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:24.298793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:24.298793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:24.298793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:24.298793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 525/1061] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:24.303117 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.340768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.411994 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8a1a2-2435-42dd-a9d2-0e325dc62f4f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.425375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ac5da-063c-4317-843e-c66d2252d00a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.453009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.453680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.453965 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.454278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.455122 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=38667033cc1c45a69ced74bc92424c80, user.id=cd9615a0c09241eeae1f27d858a19d3c) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:24.455440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.456978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.464564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 537/1062] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/role_assignments?user.id=cd9615a0c09241eeae1f27d858a19d3c&scope.project.id=38667033cc1c45a69ced74bc92424c80&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:24.469808 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.470887 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.471373 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.471824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.472781 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:24.473276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.475034 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.490798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 526/1063] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:24.496660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.559141 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.620935 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bda412-e2a9-4c98-b2cd-53209add8997 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.663922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.664530 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.664820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.665098 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.673411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.674941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:57:24.688655 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:24.691280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:24.692917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 538/1064] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:24.698235 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.816947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.831938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1f09-5be7-40a3-b598-f94d06615ef5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.880463 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ced364-2831-4bb8-bfa7-b7d969343791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.881166 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ced364-2831-4bb8-bfa7-b7d969343791 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.881450 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ced364-2831-4bb8-bfa7-b7d969343791 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.881778 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ced364-2831-4bb8-bfa7-b7d969343791 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.882598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ced364-2831-4bb8-bfa7-b7d969343791 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:24.882910 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ced364-2831-4bb8-bfa7-b7d969343791 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.884391 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ced364-2831-4bb8-bfa7-b7d969343791 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.897499 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 527/1065] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:24.903704 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.917338 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff8f41-e9ca-42b6-8391-6ce326b56632 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.917553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff8f41-e9ca-42b6-8391-6ce326b56632 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.917810 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff8f41-e9ca-42b6-8391-6ce326b56632 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.919430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 528/1066] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:24.924728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b2496-3923-435b-a84f-2a858f096bb5 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.925006 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b2496-3923-435b-a84f-2a858f096bb5 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.925294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b2496-3923-435b-a84f-2a858f096bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.944659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-987b2496-3923-435b-a84f-2a858f096bb5 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:24.945883 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-987b2496-3923-435b-a84f-2a858f096bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:25.019550 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b84c7-8caf-4396-9a1a-4fbcf1c01965 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.020612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 529/1067] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:25.036715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.045977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b2496-3923-435b-a84f-2a858f096bb5 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.065477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd865e63-96d4-4b1c-b27a-5c9083d09768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.066920 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd865e63-96d4-4b1c-b27a-5c9083d09768 None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.067179 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd865e63-96d4-4b1c-b27a-5c9083d09768 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.067487 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd865e63-96d4-4b1c-b27a-5c9083d09768 None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/4fe070acb9a84d90baaa3b08a4fd3191/roles/8dbaf90916934fa7a0670d2c504af17b` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.069918 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd865e63-96d4-4b1c-b27a-5c9083d09768 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=4fe070acb9a84d90baaa3b08a4fd3191, role_id=8dbaf90916934fa7a0670d2c504af17b)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.071436 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd865e63-96d4-4b1c-b27a-5c9083d09768 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.098218 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 539/1068] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:24 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/4fe070acb9a84d90baaa3b08a4fd3191/roles/8dbaf90916934fa7a0670d2c504af17b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:25.148441 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.175746 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b2496-3923-435b-a84f-2a858f096bb5 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.221364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.222052 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.222379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.222658 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] PATH_INFO: `/v3/services/image` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.223308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.224762 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:25.235051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:25.236527 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:57:25.237737 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:25.241750 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 530/1069] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/services/image => generated 86 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:25.243151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd865e63-96d4-4b1c-b27a-5c9083d09768 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.351858 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd865e63-96d4-4b1c-b27a-5c9083d09768 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.392191 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.392840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.393130 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.393412 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.394131 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] RBAC: Adding query filter params (name=image) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:25.394451 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.395877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.405776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 540/1070] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:25.407187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.519698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1ae8-dc5e-47d1-a056-dcb5d5aba1f8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.559988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3f59b-ab96-4a0c-a722-9c26d3a14ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.560649 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f59b-ab96-4a0c-a722-9c26d3a14ffe None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.560931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f59b-ab96-4a0c-a722-9c26d3a14ffe None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.561266 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f59b-ab96-4a0c-a722-9c26d3a14ffe None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.561972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3f59b-ab96-4a0c-a722-9c26d3a14ffe None admin] RBAC: Adding query filter params (type=image) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:25.562312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3f59b-ab96-4a0c-a722-9c26d3a14ffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.563784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3f59b-ab96-4a0c-a722-9c26d3a14ffe None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.571350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 531/1071] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:25.576921 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.615248 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.691919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8af48d-8804-487a-94d4-001673b2faf4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.728523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e468a6f-55ec-4268-ad4f-af1d701f35d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.729172 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468a6f-55ec-4268-ad4f-af1d701f35d0 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.729457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468a6f-55ec-4268-ad4f-af1d701f35d0 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.729735 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468a6f-55ec-4268-ad4f-af1d701f35d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.730502 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e468a6f-55ec-4268-ad4f-af1d701f35d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.731909 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e468a6f-55ec-4268-ad4f-af1d701f35d0 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.757079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 541/1072] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:57:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:26.526985 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a014ee-0237-4788-8b90-7b3d42ad90c5 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.527600 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a014ee-0237-4788-8b90-7b3d42ad90c5 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.527600 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a014ee-0237-4788-8b90-7b3d42ad90c5 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.529139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 532/1073] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:26.534689 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d70ca-450d-4bad-b283-9b9a6e0232d4 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.535200 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d70ca-450d-4bad-b283-9b9a6e0232d4 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.535200 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d70ca-450d-4bad-b283-9b9a6e0232d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.554916 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-8e8d70ca-450d-4bad-b283-9b9a6e0232d4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:26.556303 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-8e8d70ca-450d-4bad-b283-9b9a6e0232d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:26.634834 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 542/1074] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:26.657245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a014ee-0237-4788-8b90-7b3d42ad90c5 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.667193 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.775554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a014ee-0237-4788-8b90-7b3d42ad90c5 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.815032 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd3ee5-a678-4e74-b6e9-ad2bfbca0854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.815598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd3ee5-a678-4e74-b6e9-ad2bfbca0854 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.816250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd3ee5-a678-4e74-b6e9-ad2bfbca0854 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.816487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd3ee5-a678-4e74-b6e9-ad2bfbca0854 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.828847 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd3ee5-a678-4e74-b6e9-ad2bfbca0854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-20cd3ee5-a678-4e74-b6e9-ad2bfbca0854 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:26.833821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:26.835220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:26.836335 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 533/1075] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:26.843724 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d70ca-450d-4bad-b283-9b9a6e0232d4 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.860554 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.963189 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d70ca-450d-4bad-b283-9b9a6e0232d4 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.000618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1e4c5-e862-49eb-8b4f-c1ce72e819f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.001288 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1e4c5-e862-49eb-8b4f-c1ce72e819f1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.001557 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1e4c5-e862-49eb-8b4f-c1ce72e819f1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.001835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1e4c5-e862-49eb-8b4f-c1ce72e819f1 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.002606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1e4c5-e862-49eb-8b4f-c1ce72e819f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:27.002912 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1e4c5-e862-49eb-8b4f-c1ce72e819f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.004317 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1e4c5-e862-49eb-8b4f-c1ce72e819f1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.013906 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 543/1076] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:27.020135 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd3ee5-a678-4e74-b6e9-ad2bfbca0854 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.100517 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.120106 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3217f-cefd-47f9-b84a-a22c4b76db4a None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.120397 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3217f-cefd-47f9-b84a-a22c4b76db4a None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.120671 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3217f-cefd-47f9-b84a-a22c4b76db4a None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.122152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 544/1077] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:27.128140 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a54ef-ce2c-4b47-b7af-196dbdd6ec66 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.128449 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a54ef-ce2c-4b47-b7af-196dbdd6ec66 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.128727 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a54ef-ce2c-4b47-b7af-196dbdd6ec66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.146081 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd3ee5-a678-4e74-b6e9-ad2bfbca0854 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.148147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ec5a54ef-ce2c-4b47-b7af-196dbdd6ec66 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:27.149400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ec5a54ef-ce2c-4b47-b7af-196dbdd6ec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:27.186350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b5453-9996-4a04-a999-477fef9b8fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.187034 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5453-9996-4a04-a999-477fef9b8fe4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.187323 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5453-9996-4a04-a999-477fef9b8fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.187602 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5453-9996-4a04-a999-477fef9b8fe4 None admin] PATH_INFO: `/v3/users/cd9615a0c09241eeae1f27d858a19d3c` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.188833 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b5453-9996-4a04-a999-477fef9b8fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=cd9615a0c09241eeae1f27d858a19d3c)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.190514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b5453-9996-4a04-a999-477fef9b8fe4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.193004 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 534/1078] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/users/cd9615a0c09241eeae1f27d858a19d3c => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:27.198783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b5453-9996-4a04-a999-477fef9b8fe4 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.230616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 545/1079] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:27.254344 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a54ef-ce2c-4b47-b7af-196dbdd6ec66 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.263941 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.321003 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b5453-9996-4a04-a999-477fef9b8fe4 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.363997 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454241e-38d6-4c37-aa7d-e867fb765d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.364750 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454241e-38d6-4c37-aa7d-e867fb765d4e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.365192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454241e-38d6-4c37-aa7d-e867fb765d4e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.365661 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454241e-38d6-4c37-aa7d-e867fb765d4e None admin] PATH_INFO: `/v3/projects/38667033cc1c45a69ced74bc92424c80` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.369300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454241e-38d6-4c37-aa7d-e867fb765d4e None admin] RBAC: Authorizing `identity:get_project(project_id=38667033cc1c45a69ced74bc92424c80)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.370011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454241e-38d6-4c37-aa7d-e867fb765d4e None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.374234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 535/1080] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/projects/38667033cc1c45a69ced74bc92424c80 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:27.378897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454241e-38d6-4c37-aa7d-e867fb765d4e None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.381835 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.406886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a54ef-ce2c-4b47-b7af-196dbdd6ec66 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.453731 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662921c-86b5-4265-8cf6-1de500b97da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.454592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662921c-86b5-4265-8cf6-1de500b97da6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.454876 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662921c-86b5-4265-8cf6-1de500b97da6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.455154 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662921c-86b5-4265-8cf6-1de500b97da6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.477468 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662921c-86b5-4265-8cf6-1de500b97da6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-f662921c-86b5-4265-8cf6-1de500b97da6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:27.487621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:27.488999 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:57:27.490092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:27.491858 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 546/1081] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:27.497952 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662921c-86b5-4265-8cf6-1de500b97da6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.520810 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454241e-38d6-4c37-aa7d-e867fb765d4e None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722dd19b-313d-4aec-b837-5bb4ed10dd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.564157 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dd19b-313d-4aec-b837-5bb4ed10dd00 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.564947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dd19b-313d-4aec-b837-5bb4ed10dd00 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.564947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dd19b-313d-4aec-b837-5bb4ed10dd00 None admin] PATH_INFO: `/v3/projects/38667033cc1c45a69ced74bc92424c80/users/cd9615a0c09241eeae1f27d858a19d3c/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.567557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722dd19b-313d-4aec-b837-5bb4ed10dd00 None admin] RBAC: Authorizing `identity:create_grant(project_id=38667033cc1c45a69ced74bc92424c80, user_id=cd9615a0c09241eeae1f27d858a19d3c, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.569668 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722dd19b-313d-4aec-b837-5bb4ed10dd00 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.578805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.598297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 536/1082] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:27 2022] PUT /identity/v3/projects/38667033cc1c45a69ced74bc92424c80/users/cd9615a0c09241eeae1f27d858a19d3c/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:57:27.624839 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662921c-86b5-4265-8cf6-1de500b97da6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.670488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bb58e-f955-4493-805e-f732e0a21d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.671156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bb58e-f955-4493-805e-f732e0a21d78 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.671445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bb58e-f955-4493-805e-f732e0a21d78 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.671724 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bb58e-f955-4493-805e-f732e0a21d78 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.672517 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bb58e-f955-4493-805e-f732e0a21d78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:27.672892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bb58e-f955-4493-805e-f732e0a21d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.674393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bb58e-f955-4493-805e-f732e0a21d78 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.684924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 547/1083] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:27.691387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722dd19b-313d-4aec-b837-5bb4ed10dd00 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.770992 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.825995 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722dd19b-313d-4aec-b837-5bb4ed10dd00 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.828532 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82facf-db5f-4319-98c1-1cdb78faccf3 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.828820 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82facf-db5f-4319-98c1-1cdb78faccf3 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.829110 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82facf-db5f-4319-98c1-1cdb78faccf3 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.830797 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 548/1084] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:27.836277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1c5a3-c7b0-496b-9261-b10437626911 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.836623 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1c5a3-c7b0-496b-9261-b10437626911 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.836830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1c5a3-c7b0-496b-9261-b10437626911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.857058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ecb1c5a3-c7b0-496b-9261-b10437626911 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:27.858697 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ecb1c5a3-c7b0-496b-9261-b10437626911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:27.877525 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.878283 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.878650 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.879319 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.892977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.894671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:27.912730 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:27.914239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:27.915611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:27.915611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:27.915611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:27.915611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:27.916682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 537/1085] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:27.924187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.953327 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 549/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:27.975904 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1c5a3-c7b0-496b-9261-b10437626911 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.044672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.060047 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02cd0a-b7c0-4483-94b7-359456327c89 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.078191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.100262 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1c5a3-c7b0-496b-9261-b10437626911 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.106661 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.107314 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.107599 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.107874 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.108588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:28.108901 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.110419 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.127940 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 538/1087] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:28.136880 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.160048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e2df7-34ff-4c9f-ae3b-014f602348be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.160873 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e2df7-34ff-4c9f-ae3b-014f602348be None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.161162 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e2df7-34ff-4c9f-ae3b-014f602348be None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.161752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e2df7-34ff-4c9f-ae3b-014f602348be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.255927 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e2df7-34ff-4c9f-ae3b-014f602348be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.257429 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e2df7-34ff-4c9f-ae3b-014f602348be None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.276688 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d2589-684b-4e4d-8a28-23ed23d73a06 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.321271 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.322344 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.322818 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.323284 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] PATH_INFO: `/v3/users/nova` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.327176 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.334971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.336840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:57:28.362093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:28.363647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:28.365009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 539/1088] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:28.372398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.401612 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 550/1089] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:57:28.407904 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e2df7-34ff-4c9f-ae3b-014f602348be None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.423139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.512692 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23dc6f-66be-4684-962d-f93d62e66d36 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.552269 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e2df7-34ff-4c9f-ae3b-014f602348be None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.566741 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.567831 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.568330 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.568732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.569803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:28.570310 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.571826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.594445 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 540/1090] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:28.602094 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.606514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.607512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.607998 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.608295 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.623177 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.625246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:28.644340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:28.645634 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:28.646881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:28.646881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:28.646881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:28.646881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:28.646881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 551/1091] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:28.653336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.669196 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.734930 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.752271 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f508634-a0a4-402a-903f-03d78b157bef None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.798183 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610b41e-3cb1-4457-a2f2-1745e2f842ea None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.799626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb8149-b084-47a0-a83b-ce3254f533ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.800327 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb8149-b084-47a0-a83b-ce3254f533ee None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.800709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb8149-b084-47a0-a83b-ce3254f533ee None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.801023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb8149-b084-47a0-a83b-ce3254f533ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.853393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.854268 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.854563 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.854843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.855702 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:28.856031 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.857481 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.874778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 552/1092] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:28.901781 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb8149-b084-47a0-a83b-ce3254f533ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.903808 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb8149-b084-47a0-a83b-ce3254f533ee None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:29.039863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:29.056137 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 541/1093] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:57:29.063000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:29.170372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:29.216170 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650b152-a3a5-4dfc-bd74-9f299cbab561 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:29.270670 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:29.271814 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.272311 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.272806 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.289474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:29.291735 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:29.320681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:29.322602 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:29.324059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:29.324059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:29.324059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:29.324059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:29.324059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 553/1094] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:29.330795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb8149-b084-47a0-a83b-ce3254f533ee None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:29.451729 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb8149-b084-47a0-a83b-ce3254f533ee None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:29.492071 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb14537-ac1a-42d7-afb8-fd7d38e73ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:29.492715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14537-ac1a-42d7-afb8-fd7d38e73ea7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.493000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14537-ac1a-42d7-afb8-fd7d38e73ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.493350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14537-ac1a-42d7-afb8-fd7d38e73ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.494230 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb14537-ac1a-42d7-afb8-fd7d38e73ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:29.494505 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb14537-ac1a-42d7-afb8-fd7d38e73ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:29.496056 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb14537-ac1a-42d7-afb8-fd7d38e73ea7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:29.516776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 542/1095] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:29.523362 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:29.654000 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddbea4-5022-43c8-86f6-e847bd7295e2 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:29.680009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:29.693629 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea7a2e-1d72-43b7-b7f6-63fdd79da03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:29.694340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea7a2e-1d72-43b7-b7f6-63fdd79da03f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.694613 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea7a2e-1d72-43b7-b7f6-63fdd79da03f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.694888 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea7a2e-1d72-43b7-b7f6-63fdd79da03f None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.702544 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea7a2e-1d72-43b7-b7f6-63fdd79da03f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:29.704008 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea7a2e-1d72-43b7-b7f6-63fdd79da03f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:29.716827 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde228-851c-414d-951f-6e53e9283efb None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-4aea7a2e-1d72-43b7-b7f6-63fdd79da03f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:57:29.717700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:29.719365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:29.721493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:57:29.721493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:57:29.721493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:57:29.721493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:57:29.721493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:57:29.721493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:57:29.721493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:57:29.721493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:29.722387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde228-851c-414d-951f-6e53e9283efb None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.722387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde228-851c-414d-951f-6e53e9283efb None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.722700 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 554/1096] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:29.722848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 543/1097] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:29.725461 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebde228-851c-414d-951f-6e53e9283efb None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:29.725639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdb9c8-4210-4eb7-a37c-7d1a70ed2740 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.725807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdb9c8-4210-4eb7-a37c-7d1a70ed2740 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.726246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdb9c8-4210-4eb7-a37c-7d1a70ed2740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.747141 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-9ffdb9c8-4210-4eb7-a37c-7d1a70ed2740 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:29.748465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9ffdb9c8-4210-4eb7-a37c-7d1a70ed2740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:29.797049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:29.830695 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 555/1098] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:29.853860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebde228-851c-414d-951f-6e53e9283efb None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:29.858687 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdb9c8-4210-4eb7-a37c-7d1a70ed2740 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:29.896931 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:29.898026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.898558 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.899061 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.900206 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:29.900598 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:29.902307 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:29.921382 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 544/1099] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:29.928242 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:30.029346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdb9c8-4210-4eb7-a37c-7d1a70ed2740 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:30.052327 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5573cd-4077-49e8-90f2-cf46d5166481 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:30.056962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:30.072672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:30.094765 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017421a-3590-4ae0-b44d-e8826a330229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:30.095865 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017421a-3590-4ae0-b44d-e8826a330229 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.096363 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017421a-3590-4ae0-b44d-e8826a330229 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.096882 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017421a-3590-4ae0-b44d-e8826a330229 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.099845 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:30.100501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.100779 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.101082 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.101918 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] RBAC: Adding query filter params (role.id=8dbaf90916934fa7a0670d2c504af17b, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=4fe070acb9a84d90baaa3b08a4fd3191) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:30.102302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:30.103990 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:30.111660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 545/1100] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/role_assignments?user.id=4fe070acb9a84d90baaa3b08a4fd3191&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=8dbaf90916934fa7a0670d2c504af17b => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:30.117995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017421a-3590-4ae0-b44d-e8826a330229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-6017421a-3590-4ae0-b44d-e8826a330229 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:30.127890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:30.130452 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:30.133060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 556/1101] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:30.139611 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:30.278178 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa06503-9b37-4bf1-84a5-6b5e68082842 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:30.328497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:30.329062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.329340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.329620 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.330376 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:30.330682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:30.332133 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:30.342718 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 546/1102] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:30.349172 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017421a-3590-4ae0-b44d-e8826a330229 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:30.467125 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017421a-3590-4ae0-b44d-e8826a330229 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:30.506559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a91a94-6edb-48a4-88a5-6ea6bb00155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:30.507249 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a91a94-6edb-48a4-88a5-6ea6bb00155f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.507522 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a91a94-6edb-48a4-88a5-6ea6bb00155f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.507802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a91a94-6edb-48a4-88a5-6ea6bb00155f None admin] PATH_INFO: `/v3/users/cd9615a0c09241eeae1f27d858a19d3c` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.509216 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a91a94-6edb-48a4-88a5-6ea6bb00155f None admin] RBAC: Authorizing `identity:get_user(user_id=cd9615a0c09241eeae1f27d858a19d3c)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:30.510737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a91a94-6edb-48a4-88a5-6ea6bb00155f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:30.512446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 557/1103] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:30 2022] GET /identity/v3/users/cd9615a0c09241eeae1f27d858a19d3c => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:30.520760 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:30.575227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:30.661712 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf988f-d102-4a12-87fb-e56242644c12 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:30.702507 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90c64-d5e3-4f08-abef-02e14363c8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:30.703165 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90c64-d5e3-4f08-abef-02e14363c8e8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.703458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90c64-d5e3-4f08-abef-02e14363c8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.703731 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90c64-d5e3-4f08-abef-02e14363c8e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.782855 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd90c64-d5e3-4f08-abef-02e14363c8e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:30.784353 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd90c64-d5e3-4f08-abef-02e14363c8e8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:30.827494 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:30.902181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 547/1104] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:30.909346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a91a94-6edb-48a4-88a5-6ea6bb00155f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:30.960779 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:31.045960 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a91a94-6edb-48a4-88a5-6ea6bb00155f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.089179 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382221ee-65e3-4af0-8bea-be0eed20e0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.090254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382221ee-65e3-4af0-8bea-be0eed20e0a5 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.090651 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382221ee-65e3-4af0-8bea-be0eed20e0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.091117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382221ee-65e3-4af0-8bea-be0eed20e0a5 None admin] PATH_INFO: `/v3/projects/38667033cc1c45a69ced74bc92424c80` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.092887 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382221ee-65e3-4af0-8bea-be0eed20e0a5 None admin] RBAC: Authorizing `identity:get_project(project_id=38667033cc1c45a69ced74bc92424c80)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:31.094791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382221ee-65e3-4af0-8bea-be0eed20e0a5 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:31.097405 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 558/1105] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:30 2022] GET /identity/v3/projects/38667033cc1c45a69ced74bc92424c80 => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:31.105200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90c64-d5e3-4f08-abef-02e14363c8e8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.215662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90c64-d5e3-4f08-abef-02e14363c8e8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.254455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d104e0d-f891-400a-8000-2ecafc24aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.255107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104e0d-f891-400a-8000-2ecafc24aebf None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.255421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104e0d-f891-400a-8000-2ecafc24aebf None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.255704 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104e0d-f891-400a-8000-2ecafc24aebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.256515 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d104e0d-f891-400a-8000-2ecafc24aebf None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=38667033cc1c45a69ced74bc92424c80, user.id=cd9615a0c09241eeae1f27d858a19d3c) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:31.256832 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d104e0d-f891-400a-8000-2ecafc24aebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:31.258350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d104e0d-f891-400a-8000-2ecafc24aebf None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:31.264915 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 548/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:31 2022] GET /identity/v3/role_assignments?user.id=cd9615a0c09241eeae1f27d858a19d3c&scope.project.id=38667033cc1c45a69ced74bc92424c80&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:33.334270 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b1279-ace3-4c83-ac1b-605eb7bf2a0e None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.334850 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b1279-ace3-4c83-ac1b-605eb7bf2a0e None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.334850 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b1279-ace3-4c83-ac1b-605eb7bf2a0e None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.336145 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 559/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:33.343305 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985a12-dd7e-413a-aa26-0a1b4441813d None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.344066 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985a12-dd7e-413a-aa26-0a1b4441813d None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.344066 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985a12-dd7e-413a-aa26-0a1b4441813d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.351286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:33.365316 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-85985a12-dd7e-413a-aa26-0a1b4441813d None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:33.366558 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-85985a12-dd7e-413a-aa26-0a1b4441813d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:33.458803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 549/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:33.483717 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b1279-ace3-4c83-ac1b-605eb7bf2a0e None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:33.600236 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b1279-ace3-4c83-ac1b-605eb7bf2a0e None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.611051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:33.638884 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33e679-250d-486a-8934-4c54ae8683ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.639561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33e679-250d-486a-8934-4c54ae8683ca None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.639848 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33e679-250d-486a-8934-4c54ae8683ca None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.640127 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33e679-250d-486a-8934-4c54ae8683ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.641347 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33e679-250d-486a-8934-4c54ae8683ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:33.642868 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33e679-250d-486a-8934-4c54ae8683ca None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:33.644526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 560/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:33.651381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985a12-dd7e-413a-aa26-0a1b4441813d None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:33.762505 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985a12-dd7e-413a-aa26-0a1b4441813d None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.803864 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:33.812551 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e45655-4187-4965-bc2e-56bdd86af3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.813624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e45655-4187-4965-bc2e-56bdd86af3e6 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.814130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e45655-4187-4965-bc2e-56bdd86af3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.814752 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e45655-4187-4965-bc2e-56bdd86af3e6 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.816666 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e45655-4187-4965-bc2e-56bdd86af3e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:33.819728 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e45655-4187-4965-bc2e-56bdd86af3e6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:33.849909 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 550/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:33 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:35.954553 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f0dbb-ae2a-430e-aa7e-792e942f24ed None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.955185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f0dbb-ae2a-430e-aa7e-792e942f24ed None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.955185 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f0dbb-ae2a-430e-aa7e-792e942f24ed None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.956568 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 561/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:35.963009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0f4a8-8135-40b3-809e-2c2101992531 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.963715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0f4a8-8135-40b3-809e-2c2101992531 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.963715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0f4a8-8135-40b3-809e-2c2101992531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.983872 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-bcb0f4a8-8135-40b3-809e-2c2101992531 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:35.985237 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-bcb0f4a8-8135-40b3-809e-2c2101992531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:36.073121 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 551/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:36.097706 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f0dbb-ae2a-430e-aa7e-792e942f24ed None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.228345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f0dbb-ae2a-430e-aa7e-792e942f24ed None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.267780 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520c6a4-28f4-4542-9104-9fb5d88d4d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.268475 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c6a4-28f4-4542-9104-9fb5d88d4d26 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.268757 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c6a4-28f4-4542-9104-9fb5d88d4d26 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.269055 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c6a4-28f4-4542-9104-9fb5d88d4d26 None admin] PATH_INFO: `/v3/roles/167df0c7f94644578aff0abc3a0c8e84` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.270690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520c6a4-28f4-4542-9104-9fb5d88d4d26 None admin] RBAC: Authorizing `identity:get_role(role_id=167df0c7f94644578aff0abc3a0c8e84)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.272147 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520c6a4-28f4-4542-9104-9fb5d88d4d26 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.273619 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 562/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/roles/167df0c7f94644578aff0abc3a0c8e84 => generated 223 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:36.280026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0f4a8-8135-40b3-809e-2c2101992531 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.425593 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0f4a8-8135-40b3-809e-2c2101992531 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.455025 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.483349 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1bb3b-fb8d-4ba3-bf9d-d93ea662f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.484274 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1bb3b-fb8d-4ba3-bf9d-d93ea662f9aa None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.484624 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1bb3b-fb8d-4ba3-bf9d-d93ea662f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.485120 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1bb3b-fb8d-4ba3-bf9d-d93ea662f9aa None admin] PATH_INFO: `/v3/users/46c62a901bd641d8a723ad2e8fa1f969` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.508378 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1bb3b-fb8d-4ba3-bf9d-d93ea662f9aa None admin] RBAC: Authorizing `identity:get_user(user_id=46c62a901bd641d8a723ad2e8fa1f969)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.510545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1bb3b-fb8d-4ba3-bf9d-d93ea662f9aa None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.513414 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 552/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/users/46c62a901bd641d8a723ad2e8fa1f969 => generated 286 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:36.520078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520c6a4-28f4-4542-9104-9fb5d88d4d26 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.586629 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.655032 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520c6a4-28f4-4542-9104-9fb5d88d4d26 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.691995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9c0a-56c2-4fb1-997b-8b892bad38e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.692650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9c0a-56c2-4fb1-997b-8b892bad38e9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.693023 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9c0a-56c2-4fb1-997b-8b892bad38e9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.693309 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9c0a-56c2-4fb1-997b-8b892bad38e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.775157 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b9c0a-56c2-4fb1-997b-8b892bad38e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.776617 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b9c0a-56c2-4fb1-997b-8b892bad38e9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.830070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.897950 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 563/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:36.907103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1bb3b-fb8d-4ba3-bf9d-d93ea662f9aa None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:37.042411 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1bb3b-fb8d-4ba3-bf9d-d93ea662f9aa None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:37.079876 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333993a-b40b-4a8e-870b-5275ad1e244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:37.080550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333993a-b40b-4a8e-870b-5275ad1e244e None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:37.080838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333993a-b40b-4a8e-870b-5275ad1e244e None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:37.081118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333993a-b40b-4a8e-870b-5275ad1e244e None admin] PATH_INFO: `/v3/projects/38667033cc1c45a69ced74bc92424c80` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:37.082392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333993a-b40b-4a8e-870b-5275ad1e244e None admin] RBAC: Authorizing `identity:get_project(project_id=38667033cc1c45a69ced74bc92424c80)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:37.083887 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333993a-b40b-4a8e-870b-5275ad1e244e None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:37.085675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 553/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/projects/38667033cc1c45a69ced74bc92424c80 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:37.092345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9c0a-56c2-4fb1-997b-8b892bad38e9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:37.250567 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9c0a-56c2-4fb1-997b-8b892bad38e9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:37.290707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b57cf-6e30-4fc8-a4c9-89e7f404eefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:37.291257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b57cf-6e30-4fc8-a4c9-89e7f404eefa None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:37.291537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b57cf-6e30-4fc8-a4c9-89e7f404eefa None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:37.291811 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b57cf-6e30-4fc8-a4c9-89e7f404eefa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:37.292670 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b57cf-6e30-4fc8-a4c9-89e7f404eefa None admin] RBAC: Adding query filter params (role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=38667033cc1c45a69ced74bc92424c80, user.id=46c62a901bd641d8a723ad2e8fa1f969) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:37.292935 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b57cf-6e30-4fc8-a4c9-89e7f404eefa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:37.294493 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b57cf-6e30-4fc8-a4c9-89e7f404eefa None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:37.300491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 564/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:37 2022] GET /identity/v3/role_assignments?user.id=46c62a901bd641d8a723ad2e8fa1f969&scope.project.id=38667033cc1c45a69ced74bc92424c80&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:39.572812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b98b2-28c3-45b4-878d-ab01b9696dc3 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.573409 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b98b2-28c3-45b4-878d-ab01b9696dc3 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.573518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b98b2-28c3-45b4-878d-ab01b9696dc3 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.575391 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 554/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:39.582437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51444937-c1f4-41c3-8bd5-f11ae398d610 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.583660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51444937-c1f4-41c3-8bd5-f11ae398d610 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.583660 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51444937-c1f4-41c3-8bd5-f11ae398d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.607138 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-51444937-c1f4-41c3-8bd5-f11ae398d610 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:39.608722 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-51444937-c1f4-41c3-8bd5-f11ae398d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:39.611301 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.689881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 565/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:57:39.714759 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b98b2-28c3-45b4-878d-ab01b9696dc3 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.770080 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.835894 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b98b2-28c3-45b4-878d-ab01b9696dc3 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:39.875497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc36a-6ebe-4551-8e88-b4e2c73e22d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:39.876194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc36a-6ebe-4551-8e88-b4e2c73e22d0 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.876484 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc36a-6ebe-4551-8e88-b4e2c73e22d0 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.876754 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc36a-6ebe-4551-8e88-b4e2c73e22d0 None admin] PATH_INFO: `/v3/roles/167df0c7f94644578aff0abc3a0c8e84` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.878289 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cc36a-6ebe-4551-8e88-b4e2c73e22d0 None admin] RBAC: Authorizing `identity:get_role(role_id=167df0c7f94644578aff0abc3a0c8e84)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:39.879693 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cc36a-6ebe-4551-8e88-b4e2c73e22d0 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:39.881090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 555/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/roles/167df0c7f94644578aff0abc3a0c8e84 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:39.887325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51444937-c1f4-41c3-8bd5-f11ae398d610 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:40.015156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51444937-c1f4-41c3-8bd5-f11ae398d610 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.056875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a569535-0f76-4a9c-8289-d775f03e4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.057566 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a569535-0f76-4a9c-8289-d775f03e4607 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.057844 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a569535-0f76-4a9c-8289-d775f03e4607 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.058150 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a569535-0f76-4a9c-8289-d775f03e4607 None admin] PATH_INFO: `/v3/users/46c62a901bd641d8a723ad2e8fa1f969` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.059405 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a569535-0f76-4a9c-8289-d775f03e4607 None admin] RBAC: Authorizing `identity:get_user(user_id=46c62a901bd641d8a723ad2e8fa1f969)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.061007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a569535-0f76-4a9c-8289-d775f03e4607 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.062738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 566/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/users/46c62a901bd641d8a723ad2e8fa1f969 => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:40.069743 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc36a-6ebe-4551-8e88-b4e2c73e22d0 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:40.202357 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc36a-6ebe-4551-8e88-b4e2c73e22d0 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.218023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:40.241653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603112e4-b683-4045-b4c6-056f0455f054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.242359 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603112e4-b683-4045-b4c6-056f0455f054 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.242651 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603112e4-b683-4045-b4c6-056f0455f054 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.242936 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603112e4-b683-4045-b4c6-056f0455f054 None admin] PATH_INFO: `/v3/projects/38667033cc1c45a69ced74bc92424c80` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.244148 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603112e4-b683-4045-b4c6-056f0455f054 None admin] RBAC: Authorizing `identity:get_project(project_id=38667033cc1c45a69ced74bc92424c80)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.245644 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603112e4-b683-4045-b4c6-056f0455f054 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.247423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 556/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:40 2022] GET /identity/v3/projects/38667033cc1c45a69ced74bc92424c80 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:40.253802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a569535-0f76-4a9c-8289-d775f03e4607 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:40.263543 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:40.375448 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a569535-0f76-4a9c-8289-d775f03e4607 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.413276 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2b3de-93f3-4f53-a7aa-ec82f1349533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.413984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2b3de-93f3-4f53-a7aa-ec82f1349533 None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.414306 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2b3de-93f3-4f53-a7aa-ec82f1349533 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.414592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2b3de-93f3-4f53-a7aa-ec82f1349533 None admin] PATH_INFO: `/v3/projects/38667033cc1c45a69ced74bc92424c80/users/46c62a901bd641d8a723ad2e8fa1f969/roles/167df0c7f94644578aff0abc3a0c8e84` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.417101 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2b3de-93f3-4f53-a7aa-ec82f1349533 None admin] RBAC: Authorizing `identity:create_grant(project_id=38667033cc1c45a69ced74bc92424c80, user_id=46c62a901bd641d8a723ad2e8fa1f969, role_id=167df0c7f94644578aff0abc3a0c8e84)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.418727 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2b3de-93f3-4f53-a7aa-ec82f1349533 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.433963 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 567/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:40 2022] PUT /identity/v3/projects/38667033cc1c45a69ced74bc92424c80/users/46c62a901bd641d8a723ad2e8fa1f969/roles/167df0c7f94644578aff0abc3a0c8e84 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:42.547079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6074-cc29-43cc-9c0f-21fff99779d1 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.547709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6074-cc29-43cc-9c0f-21fff99779d1 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.547709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6074-cc29-43cc-9c0f-21fff99779d1 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.549012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 557/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:42.554804 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d6eb-aacd-4e44-a1ad-6236fbf1a494 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.555283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d6eb-aacd-4e44-a1ad-6236fbf1a494 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.555400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d6eb-aacd-4e44-a1ad-6236fbf1a494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.575283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-0261d6eb-aacd-4e44-a1ad-6236fbf1a494 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:42.576623 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-0261d6eb-aacd-4e44-a1ad-6236fbf1a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:42.640648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:42.650705 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 568/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:42.675002 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6074-cc29-43cc-9c0f-21fff99779d1 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:42.785132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6074-cc29-43cc-9c0f-21fff99779d1 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:42.822896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d876cb-f7fd-46c7-80d9-3351c033921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:42.823535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d876cb-f7fd-46c7-80d9-3351c033921c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.823824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d876cb-f7fd-46c7-80d9-3351c033921c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.824107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d876cb-f7fd-46c7-80d9-3351c033921c None admin] PATH_INFO: `/v3/roles/167df0c7f94644578aff0abc3a0c8e84` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.825615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d876cb-f7fd-46c7-80d9-3351c033921c None admin] RBAC: Authorizing `identity:get_role(role_id=167df0c7f94644578aff0abc3a0c8e84)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:42.827151 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d876cb-f7fd-46c7-80d9-3351c033921c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:42.828557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 558/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:42 2022] GET /identity/v3/roles/167df0c7f94644578aff0abc3a0c8e84 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:42.835065 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d6eb-aacd-4e44-a1ad-6236fbf1a494 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:42.939225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d6eb-aacd-4e44-a1ad-6236fbf1a494 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:42.977445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185326d-0fb3-486d-8b1c-2830be0f1267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:42.978140 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185326d-0fb3-486d-8b1c-2830be0f1267 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.978420 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185326d-0fb3-486d-8b1c-2830be0f1267 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.978706 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185326d-0fb3-486d-8b1c-2830be0f1267 None admin] PATH_INFO: `/v3/users/46c62a901bd641d8a723ad2e8fa1f969` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.979962 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185326d-0fb3-486d-8b1c-2830be0f1267 None admin] RBAC: Authorizing `identity:get_user(user_id=46c62a901bd641d8a723ad2e8fa1f969)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:42.981448 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185326d-0fb3-486d-8b1c-2830be0f1267 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:42.983141 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 569/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:42 2022] GET /identity/v3/users/46c62a901bd641d8a723ad2e8fa1f969 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:42.989325 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d876cb-f7fd-46c7-80d9-3351c033921c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.043458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.097475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d876cb-f7fd-46c7-80d9-3351c033921c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.133505 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7073-75b9-46ee-8574-09c1c77fcddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.134151 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7073-75b9-46ee-8574-09c1c77fcddb None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.134434 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7073-75b9-46ee-8574-09c1c77fcddb None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.134707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7073-75b9-46ee-8574-09c1c77fcddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.206147 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f7073-75b9-46ee-8574-09c1c77fcddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:43.207641 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f7073-75b9-46ee-8574-09c1c77fcddb None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:43.257386 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.315331 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 559/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:43.321947 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185326d-0fb3-486d-8b1c-2830be0f1267 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.388325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.427967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185326d-0fb3-486d-8b1c-2830be0f1267 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.463895 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e613d-0001-479d-bffb-ee3053161f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.464541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e613d-0001-479d-bffb-ee3053161f6c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.464825 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e613d-0001-479d-bffb-ee3053161f6c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.465106 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e613d-0001-479d-bffb-ee3053161f6c None admin] PATH_INFO: `/v3/projects/38667033cc1c45a69ced74bc92424c80` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.466372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e613d-0001-479d-bffb-ee3053161f6c None admin] RBAC: Authorizing `identity:get_project(project_id=38667033cc1c45a69ced74bc92424c80)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:43.467875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e613d-0001-479d-bffb-ee3053161f6c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:43.469576 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 570/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/projects/38667033cc1c45a69ced74bc92424c80 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:43.476079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7073-75b9-46ee-8574-09c1c77fcddb None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.587897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7073-75b9-46ee-8574-09c1c77fcddb None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.624320 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221e465-3448-48a4-9635-5cde2a3ba172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.624955 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e465-3448-48a4-9635-5cde2a3ba172 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.625238 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e465-3448-48a4-9635-5cde2a3ba172 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.625521 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e465-3448-48a4-9635-5cde2a3ba172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.626363 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221e465-3448-48a4-9635-5cde2a3ba172 None admin] RBAC: Adding query filter params (role.id=167df0c7f94644578aff0abc3a0c8e84, scope.project.id=38667033cc1c45a69ced74bc92424c80, user.id=46c62a901bd641d8a723ad2e8fa1f969) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:43.626680 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221e465-3448-48a4-9635-5cde2a3ba172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:43.628173 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221e465-3448-48a4-9635-5cde2a3ba172 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:43.635302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 560/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/role_assignments?user.id=46c62a901bd641d8a723ad2e8fa1f969&scope.project.id=38667033cc1c45a69ced74bc92424c80&role.id=167df0c7f94644578aff0abc3a0c8e84 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:45.770733 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfc2e2-aa3f-4f3a-b05b-b145ec54074d None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.771372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfc2e2-aa3f-4f3a-b05b-b145ec54074d None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.771372 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfc2e2-aa3f-4f3a-b05b-b145ec54074d None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.772663 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 571/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:45.778712 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027df390-6c2a-45ae-a2ef-46bca7359698 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.779313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027df390-6c2a-45ae-a2ef-46bca7359698 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.779313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027df390-6c2a-45ae-a2ef-46bca7359698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.800197 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-027df390-6c2a-45ae-a2ef-46bca7359698 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:45.801434 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-027df390-6c2a-45ae-a2ef-46bca7359698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:45.810022 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:45.881991 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 561/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:45.906357 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfc2e2-aa3f-4f3a-b05b-b145ec54074d None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:46.026773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfc2e2-aa3f-4f3a-b05b-b145ec54074d None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.062464 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:46.070259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38131f9a-e0e3-4b51-ab0f-b1fd3b5a464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.070943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f9a-e0e3-4b51-ab0f-b1fd3b5a464b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.071215 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f9a-e0e3-4b51-ab0f-b1fd3b5a464b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.071494 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f9a-e0e3-4b51-ab0f-b1fd3b5a464b None admin] PATH_INFO: `/v3/domains/default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.072830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38131f9a-e0e3-4b51-ab0f-b1fd3b5a464b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:46.074361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38131f9a-e0e3-4b51-ab0f-b1fd3b5a464b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:46.076042 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 572/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:46.084670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027df390-6c2a-45ae-a2ef-46bca7359698 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:46.198071 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027df390-6c2a-45ae-a2ef-46bca7359698 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.234897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1108536-eec1-4ec4-af1f-c531198a1cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.235530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1108536-eec1-4ec4-af1f-c531198a1cee None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.235812 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1108536-eec1-4ec4-af1f-c531198a1cee None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.236091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1108536-eec1-4ec4-af1f-c531198a1cee None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.236789 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1108536-eec1-4ec4-af1f-c531198a1cee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:46.238316 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1108536-eec1-4ec4-af1f-c531198a1cee None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:46.253573 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:46.267826 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 562/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:57:46 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:48.348812 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ee81-21a2-41b4-a13a-53f6ab7e96df None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:48.349393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ee81-21a2-41b4-a13a-53f6ab7e96df None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:48.349393 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ee81-21a2-41b4-a13a-53f6ab7e96df None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:48.350716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 573/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:48.356015 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe650-4c4f-4088-9543-d2896870b01f None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:48.356478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe650-4c4f-4088-9543-d2896870b01f None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:48.356605 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe650-4c4f-4088-9543-d2896870b01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:48.376626 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-a4ebe650-4c4f-4088-9543-d2896870b01f None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:48.377895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-a4ebe650-4c4f-4088-9543-d2896870b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:48.452092 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 563/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:48.477762 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453ee81-21a2-41b4-a13a-53f6ab7e96df None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:48.593539 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453ee81-21a2-41b4-a13a-53f6ab7e96df None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:48.631095 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba294e-120b-4d08-97ee-60731c728b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:48.631759 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba294e-120b-4d08-97ee-60731c728b8c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:48.632040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba294e-120b-4d08-97ee-60731c728b8c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:48.632332 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba294e-120b-4d08-97ee-60731c728b8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:48.633581 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba294e-120b-4d08-97ee-60731c728b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:48.635267 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba294e-120b-4d08-97ee-60731c728b8c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:48.637111 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 574/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:48.645366 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebe650-4c4f-4088-9543-d2896870b01f None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:48.749807 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebe650-4c4f-4088-9543-d2896870b01f None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:48.774458 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:48.786776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a8cd-fba2-4241-b299-4795b5051938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:48.787409 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a8cd-fba2-4241-b299-4795b5051938 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:48.787695 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a8cd-fba2-4241-b299-4795b5051938 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:48.787971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a8cd-fba2-4241-b299-4795b5051938 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:48.788659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98a8cd-fba2-4241-b299-4795b5051938 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:48.790168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98a8cd-fba2-4241-b299-4795b5051938 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:48.812905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 564/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:48 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:50.881595 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4256-f240-42dc-a28e-cc45d32b100b None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.882215 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4256-f240-42dc-a28e-cc45d32b100b None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.882215 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4256-f240-42dc-a28e-cc45d32b100b None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.883552 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 575/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:50.889276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017871b5-cf1b-42dc-a1b4-dea04c4e442e None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.889800 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017871b5-cf1b-42dc-a1b4-dea04c4e442e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.889924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017871b5-cf1b-42dc-a1b4-dea04c4e442e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.908893 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-017871b5-cf1b-42dc-a1b4-dea04c4e442e None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:50.910150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-017871b5-cf1b-42dc-a1b4-dea04c4e442e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:50.982929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 565/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:51.007332 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e4256-f240-42dc-a28e-cc45d32b100b None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:51.089579 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:51.126445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e4256-f240-42dc-a28e-cc45d32b100b None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:51.171559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b0f54-448e-482d-b25d-eb538dc618c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:51.172137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b0f54-448e-482d-b25d-eb538dc618c6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:51.172416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b0f54-448e-482d-b25d-eb538dc618c6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:51.172680 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b0f54-448e-482d-b25d-eb538dc618c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:51.185468 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b0f54-448e-482d-b25d-eb538dc618c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-370b0f54-448e-482d-b25d-eb538dc618c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:51.190592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:51.192358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:51.193547 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 576/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:51.199091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017871b5-cf1b-42dc-a1b4-dea04c4e442e None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:51.308662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017871b5-cf1b-42dc-a1b4-dea04c4e442e None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:51.320537 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:51.351421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:51.352062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:51.352355 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:51.352634 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:51.353365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:51.353676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:51.355168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:51.364392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 566/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:51.373892 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b0f54-448e-482d-b25d-eb538dc618c6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:51.492919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b0f54-448e-482d-b25d-eb538dc618c6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:51.522760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:51.532622 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03242fc8-25af-417d-8928-964743afd044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:51.533316 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03242fc8-25af-417d-8928-964743afd044 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:51.533599 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03242fc8-25af-417d-8928-964743afd044 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:51.533881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03242fc8-25af-417d-8928-964743afd044 None admin] PATH_INFO: `/v3/users/ccc82311ac0e4eed811377b8565ef556` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:51.549802 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03242fc8-25af-417d-8928-964743afd044 None admin] RBAC: Authorizing `identity:get_user(user_id=ccc82311ac0e4eed811377b8565ef556)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:51.551391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03242fc8-25af-417d-8928-964743afd044 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:51.553338 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 577/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:51 2022] GET /identity/v3/users/ccc82311ac0e4eed811377b8565ef556 => generated 286 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:51.560759 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:51.679238 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44343349-df41-4210-a1aa-1ec5bd310177 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:51.717285 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095f649-85a6-42c0-9b09-237115222d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:51.717930 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095f649-85a6-42c0-9b09-237115222d4a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:51.718253 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095f649-85a6-42c0-9b09-237115222d4a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:51.718535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095f649-85a6-42c0-9b09-237115222d4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:51.770182 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:51.800175 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095f649-85a6-42c0-9b09-237115222d4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:51.801697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095f649-85a6-42c0-9b09-237115222d4a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:51.918312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 567/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:51.925343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03242fc8-25af-417d-8928-964743afd044 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:52.030569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03242fc8-25af-417d-8928-964743afd044 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:52.070570 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b6102-6bfd-4d66-8d96-6f9001f89a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:52.071637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b6102-6bfd-4d66-8d96-6f9001f89a68 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.071934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b6102-6bfd-4d66-8d96-6f9001f89a68 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.072329 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b6102-6bfd-4d66-8d96-6f9001f89a68 None admin] PATH_INFO: `/v3/projects/3e4dec88aa9e4aae9f8418de1f416a51` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.074360 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b6102-6bfd-4d66-8d96-6f9001f89a68 None admin] RBAC: Authorizing `identity:get_project(project_id=3e4dec88aa9e4aae9f8418de1f416a51)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:52.076750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b6102-6bfd-4d66-8d96-6f9001f89a68 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:52.079569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 578/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:51 2022] GET /identity/v3/projects/3e4dec88aa9e4aae9f8418de1f416a51 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:52.086068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095f649-85a6-42c0-9b09-237115222d4a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:52.162638 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:52.205300 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095f649-85a6-42c0-9b09-237115222d4a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:52.241992 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346314e-0da4-4f4e-be5e-708e846012d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:52.242643 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346314e-0da4-4f4e-be5e-708e846012d0 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.242924 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346314e-0da4-4f4e-be5e-708e846012d0 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.243199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346314e-0da4-4f4e-be5e-708e846012d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.243987 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346314e-0da4-4f4e-be5e-708e846012d0 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=3e4dec88aa9e4aae9f8418de1f416a51, user.id=ccc82311ac0e4eed811377b8565ef556) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:52.244297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346314e-0da4-4f4e-be5e-708e846012d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:52.245770 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346314e-0da4-4f4e-be5e-708e846012d0 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:52.251543 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 568/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:52 2022] GET /identity/v3/role_assignments?user.id=ccc82311ac0e4eed811377b8565ef556&scope.project.id=3e4dec88aa9e4aae9f8418de1f416a51&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:54.306770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b0301-2c7d-49d1-b13b-f7908ccb3151 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:54.307343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b0301-2c7d-49d1-b13b-f7908ccb3151 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:54.307343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b0301-2c7d-49d1-b13b-f7908ccb3151 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:54.308646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 579/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:54.314476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7910f2c-b900-4a7a-9edb-2b7fcc319574 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:54.314982 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7910f2c-b900-4a7a-9edb-2b7fcc319574 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:54.314982 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7910f2c-b900-4a7a-9edb-2b7fcc319574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:54.334895 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-a7910f2c-b900-4a7a-9edb-2b7fcc319574 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:54.336136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-a7910f2c-b900-4a7a-9edb-2b7fcc319574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:54.416221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 569/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:54.439109 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b0301-2c7d-49d1-b13b-f7908ccb3151 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:54.507491 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:54.562040 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b0301-2c7d-49d1-b13b-f7908ccb3151 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:54.602033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0f3f1-af60-4669-8eba-9b6b9e287ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:54.602750 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0f3f1-af60-4669-8eba-9b6b9e287ec1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:54.603036 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0f3f1-af60-4669-8eba-9b6b9e287ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:54.603382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0f3f1-af60-4669-8eba-9b6b9e287ec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:54.615094 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0f3f1-af60-4669-8eba-9b6b9e287ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-58e0f3f1-af60-4669-8eba-9b6b9e287ec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:54.620209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:54.621633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:57:54.622978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 580/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:54.628365 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7910f2c-b900-4a7a-9edb-2b7fcc319574 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:54.698208 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:54.734831 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7910f2c-b900-4a7a-9edb-2b7fcc319574 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:54.771423 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:54.772068 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:54.772358 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:54.772634 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:54.773371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:54.773698 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:54.775140 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:54.784477 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 570/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:54.790653 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0f3f1-af60-4669-8eba-9b6b9e287ec1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:54.898625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0f3f1-af60-4669-8eba-9b6b9e287ec1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:54.913958 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:54.936565 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20133fa9-ce7b-4372-b2a6-cbac96e32c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:54.937296 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133fa9-ce7b-4372-b2a6-cbac96e32c56 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:54.937629 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133fa9-ce7b-4372-b2a6-cbac96e32c56 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:54.937955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133fa9-ce7b-4372-b2a6-cbac96e32c56 None admin] PATH_INFO: `/v3/users/ccc82311ac0e4eed811377b8565ef556` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:54.939283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20133fa9-ce7b-4372-b2a6-cbac96e32c56 None admin] RBAC: Authorizing `identity:get_user(user_id=ccc82311ac0e4eed811377b8565ef556)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:54.941138 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20133fa9-ce7b-4372-b2a6-cbac96e32c56 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:54.942915 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 581/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:54 2022] GET /identity/v3/users/ccc82311ac0e4eed811377b8565ef556 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:54.949231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:55.082387 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2911e-28ab-4b34-b02a-dbb441961ed7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:55.110795 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:55.120103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7cbd6-2899-4b99-93a4-db119b4e39eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:55.120777 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7cbd6-2899-4b99-93a4-db119b4e39eb None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:55.121063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7cbd6-2899-4b99-93a4-db119b4e39eb None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:55.121341 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7cbd6-2899-4b99-93a4-db119b4e39eb None admin] PATH_INFO: `/v3/projects/3e4dec88aa9e4aae9f8418de1f416a51` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:55.122702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7cbd6-2899-4b99-93a4-db119b4e39eb None admin] RBAC: Authorizing `identity:get_project(project_id=3e4dec88aa9e4aae9f8418de1f416a51)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:55.124224 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7cbd6-2899-4b99-93a4-db119b4e39eb None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:55.125955 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 571/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:54 2022] GET /identity/v3/projects/3e4dec88aa9e4aae9f8418de1f416a51 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:55.132583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20133fa9-ce7b-4372-b2a6-cbac96e32c56 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:55.241509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20133fa9-ce7b-4372-b2a6-cbac96e32c56 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:55.277035 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942811e-ff7e-4fa4-8014-9f4ebd8bbfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:55.277723 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942811e-ff7e-4fa4-8014-9f4ebd8bbfdf None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:55.278011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942811e-ff7e-4fa4-8014-9f4ebd8bbfdf None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:55.278325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942811e-ff7e-4fa4-8014-9f4ebd8bbfdf None admin] PATH_INFO: `/v3/projects/3e4dec88aa9e4aae9f8418de1f416a51/users/ccc82311ac0e4eed811377b8565ef556/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:55.280488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942811e-ff7e-4fa4-8014-9f4ebd8bbfdf None admin] RBAC: Authorizing `identity:create_grant(project_id=3e4dec88aa9e4aae9f8418de1f416a51, user_id=ccc82311ac0e4eed811377b8565ef556, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:55.281959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942811e-ff7e-4fa4-8014-9f4ebd8bbfdf None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:55.296483 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 582/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:55 2022] PUT /identity/v3/projects/3e4dec88aa9e4aae9f8418de1f416a51/users/ccc82311ac0e4eed811377b8565ef556/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:57.385938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e77dd-7923-4d95-8c01-c89ccfc57567 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.386607 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e77dd-7923-4d95-8c01-c89ccfc57567 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.386607 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e77dd-7923-4d95-8c01-c89ccfc57567 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.387973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 572/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:57.394056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbee781-1770-4d8d-924f-c0fe48848ebd None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.394696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbee781-1770-4d8d-924f-c0fe48848ebd None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.394696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbee781-1770-4d8d-924f-c0fe48848ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.414894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ecbee781-1770-4d8d-924f-c0fe48848ebd None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:57.416140 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ecbee781-1770-4d8d-924f-c0fe48848ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:57.456541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:57.491404 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 583/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:57:57.514891 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e77dd-7923-4d95-8c01-c89ccfc57567 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.637501 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e77dd-7923-4d95-8c01-c89ccfc57567 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.678923 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc8ca5-c855-40a9-9154-96c2bd9942d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.679632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8ca5-c855-40a9-9154-96c2bd9942d5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.679912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8ca5-c855-40a9-9154-96c2bd9942d5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.680181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8ca5-c855-40a9-9154-96c2bd9942d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.694059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc8ca5-c855-40a9-9154-96c2bd9942d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-88cc8ca5-c855-40a9-9154-96c2bd9942d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:57.699234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:57.700783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:57:57.702050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 573/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:57.710140 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbee781-1770-4d8d-924f-c0fe48848ebd None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.830738 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbee781-1770-4d8d-924f-c0fe48848ebd None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.869891 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.870569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.870830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.871141 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.874067 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:57.874414 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:57.875840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:57.886718 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 584/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:57.892982 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc8ca5-c855-40a9-9154-96c2bd9942d5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.957369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.007903 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc8ca5-c855-40a9-9154-96c2bd9942d5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.044444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce7ea4-d816-424b-a3ad-9f95fc53cc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.045110 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce7ea4-d816-424b-a3ad-9f95fc53cc47 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.045396 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce7ea4-d816-424b-a3ad-9f95fc53cc47 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.045675 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce7ea4-d816-424b-a3ad-9f95fc53cc47 None admin] PATH_INFO: `/v3/users/ccc82311ac0e4eed811377b8565ef556` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.046948 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce7ea4-d816-424b-a3ad-9f95fc53cc47 None admin] RBAC: Authorizing `identity:get_user(user_id=ccc82311ac0e4eed811377b8565ef556)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.048511 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce7ea4-d816-424b-a3ad-9f95fc53cc47 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:58.050219 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 574/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/users/ccc82311ac0e4eed811377b8565ef556 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:58.057233 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.101136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.189358 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884ce65-7db6-4a23-ae34-894ed21d0c86 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.236251 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bac39-055f-40e1-b24c-e7871b575247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.236884 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bac39-055f-40e1-b24c-e7871b575247 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.237165 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bac39-055f-40e1-b24c-e7871b575247 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.237439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bac39-055f-40e1-b24c-e7871b575247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.322554 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bac39-055f-40e1-b24c-e7871b575247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.324062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bac39-055f-40e1-b24c-e7871b575247 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:58.359716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.429351 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 585/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:57:58.435560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce7ea4-d816-424b-a3ad-9f95fc53cc47 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.543870 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce7ea4-d816-424b-a3ad-9f95fc53cc47 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.571045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.583629 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf8c6b-078b-4dd9-bf9a-87c134663db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.584309 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8c6b-078b-4dd9-bf9a-87c134663db5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.584596 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8c6b-078b-4dd9-bf9a-87c134663db5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.584910 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8c6b-078b-4dd9-bf9a-87c134663db5 None admin] PATH_INFO: `/v3/projects/3e4dec88aa9e4aae9f8418de1f416a51` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.586221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf8c6b-078b-4dd9-bf9a-87c134663db5 None admin] RBAC: Authorizing `identity:get_project(project_id=3e4dec88aa9e4aae9f8418de1f416a51)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.587821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf8c6b-078b-4dd9-bf9a-87c134663db5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:58.589646 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 575/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/projects/3e4dec88aa9e4aae9f8418de1f416a51 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:58.596332 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bac39-055f-40e1-b24c-e7871b575247 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.703498 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bac39-055f-40e1-b24c-e7871b575247 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.740027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dba5e8-7428-43a1-ab72-416fe4c60352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.740669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dba5e8-7428-43a1-ab72-416fe4c60352 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.740949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dba5e8-7428-43a1-ab72-416fe4c60352 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.741221 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dba5e8-7428-43a1-ab72-416fe4c60352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.742029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dba5e8-7428-43a1-ab72-416fe4c60352 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=3e4dec88aa9e4aae9f8418de1f416a51, user.id=ccc82311ac0e4eed811377b8565ef556) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:58.742503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dba5e8-7428-43a1-ab72-416fe4c60352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.743969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dba5e8-7428-43a1-ab72-416fe4c60352 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:58.745888 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.751351 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 586/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/role_assignments?user.id=ccc82311ac0e4eed811377b8565ef556&scope.project.id=3e4dec88aa9e4aae9f8418de1f416a51&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:00.847731 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d318b-54b5-4904-9e31-634d290d219b None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.848370 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d318b-54b5-4904-9e31-634d290d219b None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.848370 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d318b-54b5-4904-9e31-634d290d219b None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.849688 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 576/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:00.855163 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fc735-a720-4d24-a350-d15c4026c6c7 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.855648 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fc735-a720-4d24-a350-d15c4026c6c7 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.855760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fc735-a720-4d24-a350-d15c4026c6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.875847 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-c12fc735-a720-4d24-a350-d15c4026c6c7 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:00.877078 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-c12fc735-a720-4d24-a350-d15c4026c6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:00.950298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 587/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:00.973191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d318b-54b5-4904-9e31-634d290d219b None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:01.088704 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d318b-54b5-4904-9e31-634d290d219b None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:01.126676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc803398-f3ed-44af-a04c-8eb35919f8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:01.127316 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc803398-f3ed-44af-a04c-8eb35919f8c3 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.127601 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc803398-f3ed-44af-a04c-8eb35919f8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.127878 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc803398-f3ed-44af-a04c-8eb35919f8c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.189245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:01.200802 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc803398-f3ed-44af-a04c-8eb35919f8c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:01.202350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc803398-f3ed-44af-a04c-8eb35919f8c3 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:01.307337 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 577/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:58:01.313880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fc735-a720-4d24-a350-d15c4026c6c7 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:01.417406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fc735-a720-4d24-a350-d15c4026c6c7 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:01.445418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:01.455564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde90b-df3f-449e-bb31-13eb5884c03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:01.456247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde90b-df3f-449e-bb31-13eb5884c03e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.456531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde90b-df3f-449e-bb31-13eb5884c03e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.456814 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde90b-df3f-449e-bb31-13eb5884c03e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.464676 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde90b-df3f-449e-bb31-13eb5884c03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:01.466194 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde90b-df3f-449e-bb31-13eb5884c03e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-7dfde90b-df3f-449e-bb31-13eb5884c03e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:01.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:01.480046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:58:01.481133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 588/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:58:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:01.486469 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc803398-f3ed-44af-a04c-8eb35919f8c3 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:01.583288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:01.592861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc803398-f3ed-44af-a04c-8eb35919f8c3 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:01.629025 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc1c1a-1e04-4a8e-bde0-752b33ec4df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:01.629648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1c1a-1e04-4a8e-bde0-752b33ec4df8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.629929 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1c1a-1e04-4a8e-bde0-752b33ec4df8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.630261 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1c1a-1e04-4a8e-bde0-752b33ec4df8 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.631074 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc1c1a-1e04-4a8e-bde0-752b33ec4df8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:01.631380 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc1c1a-1e04-4a8e-bde0-752b33ec4df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:01.632800 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc1c1a-1e04-4a8e-bde0-752b33ec4df8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:01.640334 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 578/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 08:58:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:03.646577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdc397-bcf8-47d0-8d23-c764ad29b3be None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.647226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdc397-bcf8-47d0-8d23-c764ad29b3be None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.647226 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdc397-bcf8-47d0-8d23-c764ad29b3be None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.648509 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 589/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:03.654063 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13b5d2-2997-40e4-82cd-53c47866432f None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.654573 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13b5d2-2997-40e4-82cd-53c47866432f None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.654688 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13b5d2-2997-40e4-82cd-53c47866432f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.675046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-df13b5d2-2997-40e4-82cd-53c47866432f None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:03.676306 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-df13b5d2-2997-40e4-82cd-53c47866432f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:03.754133 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 579/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:03.779105 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdc397-bcf8-47d0-8d23-c764ad29b3be None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:03.894132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdc397-bcf8-47d0-8d23-c764ad29b3be None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:03.930694 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485bd17-4dca-42fb-8eda-2ef1b6d5a466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:03.931248 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485bd17-4dca-42fb-8eda-2ef1b6d5a466 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.931533 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485bd17-4dca-42fb-8eda-2ef1b6d5a466 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.931811 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485bd17-4dca-42fb-8eda-2ef1b6d5a466 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.932551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485bd17-4dca-42fb-8eda-2ef1b6d5a466 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:03.933970 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485bd17-4dca-42fb-8eda-2ef1b6d5a466 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:03.954958 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 590/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jul 28 08:58:03 2022] POST /identity/v3/domains => generated 267 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:58:06.043006 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072778-fc4a-4b7d-aa7b-f72cfe5c25bf None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.043491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072778-fc4a-4b7d-aa7b-f72cfe5c25bf None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.043628 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072778-fc4a-4b7d-aa7b-f72cfe5c25bf None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.045191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 580/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:06.050981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c653867-4cac-4695-a7e2-17f237b583eb None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.051514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c653867-4cac-4695-a7e2-17f237b583eb None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.051514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c653867-4cac-4695-a7e2-17f237b583eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.070956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-9c653867-4cac-4695-a7e2-17f237b583eb None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:06.072175 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9c653867-4cac-4695-a7e2-17f237b583eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:06.118945 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:06.144054 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 591/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:58:06.167806 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2072778-fc4a-4b7d-aa7b-f72cfe5c25bf None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:06.257984 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:06.275227 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2072778-fc4a-4b7d-aa7b-f72cfe5c25bf None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.311124 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fb93e-690d-425c-a0fa-5fb9b01ed7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.311676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fb93e-690d-425c-a0fa-5fb9b01ed7a4 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.311977 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fb93e-690d-425c-a0fa-5fb9b01ed7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.312250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fb93e-690d-425c-a0fa-5fb9b01ed7a4 None admin] PATH_INFO: `/v3/domains/d8e19d9e6263454780fa018569e3e16e` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.324182 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fb93e-690d-425c-a0fa-5fb9b01ed7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d8e19d9e6263454780fa018569e3e16e)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:06.325694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fb93e-690d-425c-a0fa-5fb9b01ed7a4 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:06.327440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 581/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:58:06 2022] GET /identity/v3/domains/d8e19d9e6263454780fa018569e3e16e => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:06.333860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c653867-4cac-4695-a7e2-17f237b583eb None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:06.438411 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c653867-4cac-4695-a7e2-17f237b583eb None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.475219 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6ecd7-9477-4ffc-90f5-f92c72758f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.475932 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6ecd7-9477-4ffc-90f5-f92c72758f89 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.476153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6ecd7-9477-4ffc-90f5-f92c72758f89 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.476429 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6ecd7-9477-4ffc-90f5-f92c72758f89 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.477131 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6ecd7-9477-4ffc-90f5-f92c72758f89 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:06.478659 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6ecd7-9477-4ffc-90f5-f92c72758f89 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:06.507949 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 592/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:58:06 2022] POST /identity/v3/projects => generated 361 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:08.535118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c20db7-1a2c-46e4-82cf-ec507a9e83a9 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.535742 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c20db7-1a2c-46e4-82cf-ec507a9e83a9 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.535742 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c20db7-1a2c-46e4-82cf-ec507a9e83a9 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.537050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 582/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:08.542551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3769-a824-43db-afda-e8a5777f401c None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.543034 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3769-a824-43db-afda-e8a5777f401c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.543150 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3769-a824-43db-afda-e8a5777f401c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.563263 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-02ac3769-a824-43db-afda-e8a5777f401c None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:08.564526 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-02ac3769-a824-43db-afda-e8a5777f401c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:08.590816 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:08.640315 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 593/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:08.664834 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c20db7-1a2c-46e4-82cf-ec507a9e83a9 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.773783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c20db7-1a2c-46e4-82cf-ec507a9e83a9 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.809824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad6c1d-db54-4c46-8f7b-565d4986566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.810442 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad6c1d-db54-4c46-8f7b-565d4986566c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.810731 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad6c1d-db54-4c46-8f7b-565d4986566c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.811014 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad6c1d-db54-4c46-8f7b-565d4986566c None admin] PATH_INFO: `/v3/domains/d8e19d9e6263454780fa018569e3e16e` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.812229 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad6c1d-db54-4c46-8f7b-565d4986566c None admin] RBAC: Authorizing `identity:get_domain(domain_id=d8e19d9e6263454780fa018569e3e16e)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.813723 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad6c1d-db54-4c46-8f7b-565d4986566c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.815391 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 583/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 08:58:08 2022] GET /identity/v3/domains/d8e19d9e6263454780fa018569e3e16e => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:08.821907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac3769-a824-43db-afda-e8a5777f401c None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.925248 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac3769-a824-43db-afda-e8a5777f401c None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.961468 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d2252-45f3-4060-acc6-1d60961f2768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.962136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2252-45f3-4060-acc6-1d60961f2768 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.962421 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2252-45f3-4060-acc6-1d60961f2768 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.962703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2252-45f3-4060-acc6-1d60961f2768 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.963404 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d2252-45f3-4060-acc6-1d60961f2768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.964883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d2252-45f3-4060-acc6-1d60961f2768 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.986395 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 594/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:58:08 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:11.052838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7f7f-c599-4f17-8f26-324c3423a726 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.053488 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7f7f-c599-4f17-8f26-324c3423a726 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.053488 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7f7f-c599-4f17-8f26-324c3423a726 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.055003 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 584/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:11.062286 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fbd48-7cf5-4e2e-aa33-0cadd7bb152c None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.062951 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fbd48-7cf5-4e2e-aa33-0cadd7bb152c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.063064 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fbd48-7cf5-4e2e-aa33-0cadd7bb152c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.089391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-a58fbd48-7cf5-4e2e-aa33-0cadd7bb152c None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:11.090846 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-a58fbd48-7cf5-4e2e-aa33-0cadd7bb152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:11.106142 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:11.177041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 595/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:11.201012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e7f7f-c599-4f17-8f26-324c3423a726 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:11.222719 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:11.317307 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e7f7f-c599-4f17-8f26-324c3423a726 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:11.353756 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a872f9f-201f-4a41-a11b-0b296949ca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:11.354444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a872f9f-201f-4a41-a11b-0b296949ca09 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.354732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a872f9f-201f-4a41-a11b-0b296949ca09 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.355010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a872f9f-201f-4a41-a11b-0b296949ca09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.367166 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a872f9f-201f-4a41-a11b-0b296949ca09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0a872f9f-201f-4a41-a11b-0b296949ca09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:11.372160 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:11.373513 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:58:11.374869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 585/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:11.379957 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fbd48-7cf5-4e2e-aa33-0cadd7bb152c None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:11.492558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fbd48-7cf5-4e2e-aa33-0cadd7bb152c None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:11.532929 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:11.533574 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.533855 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.534153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.534902 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:11.535236 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:11.536659 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:11.545398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 596/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:11.551885 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a872f9f-201f-4a41-a11b-0b296949ca09 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:11.636804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:11.664396 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a872f9f-201f-4a41-a11b-0b296949ca09 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:11.705252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717f9f-4212-46be-9c38-682a262b47c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:11.705922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717f9f-4212-46be-9c38-682a262b47c9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.706245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717f9f-4212-46be-9c38-682a262b47c9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.706528 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717f9f-4212-46be-9c38-682a262b47c9 None admin] PATH_INFO: `/v3/users/d5d0f766399043f18f8610c27febb46b` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.722221 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c717f9f-4212-46be-9c38-682a262b47c9 None admin] RBAC: Authorizing `identity:get_user(user_id=d5d0f766399043f18f8610c27febb46b)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:11.723752 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c717f9f-4212-46be-9c38-682a262b47c9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:11.725529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 586/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:58:11 2022] GET /identity/v3/users/d5d0f766399043f18f8610c27febb46b => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:11.731971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:11.744283 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:11.843329 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee2e13-4c2a-4f3e-8086-679a2504afd8 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:11.883932 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee8d27-607a-4fb7-8ce8-72aba6a173f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:11.884587 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee8d27-607a-4fb7-8ce8-72aba6a173f6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.884881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee8d27-607a-4fb7-8ce8-72aba6a173f6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.885133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee8d27-607a-4fb7-8ce8-72aba6a173f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.959276 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee8d27-607a-4fb7-8ce8-72aba6a173f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:11.960687 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee8d27-607a-4fb7-8ce8-72aba6a173f6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:11.971333 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.081401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 597/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:58:12.087774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717f9f-4212-46be-9c38-682a262b47c9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.197963 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717f9f-4212-46be-9c38-682a262b47c9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.228031 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.233787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55ced2-ef20-4b6a-8fff-b16ac3ea002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:12.234530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55ced2-ef20-4b6a-8fff-b16ac3ea002b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.234816 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55ced2-ef20-4b6a-8fff-b16ac3ea002b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.235093 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55ced2-ef20-4b6a-8fff-b16ac3ea002b None admin] PATH_INFO: `/v3/projects/7fced7c8bbe54fd6a7c03d553d283392` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.236361 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55ced2-ef20-4b6a-8fff-b16ac3ea002b None admin] RBAC: Authorizing `identity:get_project(project_id=7fced7c8bbe54fd6a7c03d553d283392)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:12.237869 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55ced2-ef20-4b6a-8fff-b16ac3ea002b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:12.239631 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 587/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:12 2022] GET /identity/v3/projects/7fced7c8bbe54fd6a7c03d553d283392 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:12.246776 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee8d27-607a-4fb7-8ce8-72aba6a173f6 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.356798 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee8d27-607a-4fb7-8ce8-72aba6a173f6 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.366097 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.392787 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07bd15-a00b-4650-a0bb-3a0c3d284170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:12.393424 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07bd15-a00b-4650-a0bb-3a0c3d284170 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.393707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07bd15-a00b-4650-a0bb-3a0c3d284170 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.393986 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07bd15-a00b-4650-a0bb-3a0c3d284170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.394830 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07bd15-a00b-4650-a0bb-3a0c3d284170 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=7fced7c8bbe54fd6a7c03d553d283392, user.id=d5d0f766399043f18f8610c27febb46b) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:12.395133 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07bd15-a00b-4650-a0bb-3a0c3d284170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:12.396591 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07bd15-a00b-4650-a0bb-3a0c3d284170 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:12.402237 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 598/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:12 2022] GET /identity/v3/role_assignments?user.id=d5d0f766399043f18f8610c27febb46b&scope.project.id=7fced7c8bbe54fd6a7c03d553d283392&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:14.381145 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cd49f-f8b8-4651-83ad-4c5228d1a659 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:14.381751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cd49f-f8b8-4651-83ad-4c5228d1a659 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:14.381751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cd49f-f8b8-4651-83ad-4c5228d1a659 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:14.383127 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 588/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:14.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c195-4fa8-4372-92cf-545d8ed72f4f None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:14.389107 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c195-4fa8-4372-92cf-545d8ed72f4f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:14.389107 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c195-4fa8-4372-92cf-545d8ed72f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:14.408042 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-9fc4c195-4fa8-4372-92cf-545d8ed72f4f None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:14.409242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9fc4c195-4fa8-4372-92cf-545d8ed72f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:14.479925 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 599/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:14.502642 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cd49f-f8b8-4651-83ad-4c5228d1a659 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:14.613615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cd49f-f8b8-4651-83ad-4c5228d1a659 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:14.652548 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7223d-434e-4b30-878a-d60d5f5b8afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:14.653116 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7223d-434e-4b30-878a-d60d5f5b8afa None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:14.653506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7223d-434e-4b30-878a-d60d5f5b8afa None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:14.653789 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7223d-434e-4b30-878a-d60d5f5b8afa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:14.664744 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7223d-434e-4b30-878a-d60d5f5b8afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-add7223d-434e-4b30-878a-d60d5f5b8afa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:14.669753 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:14.674475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:58:14.675767 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 589/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:14.677449 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4c195-4fa8-4372-92cf-545d8ed72f4f None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:14.780911 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4c195-4fa8-4372-92cf-545d8ed72f4f None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:14.782533 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:14.819104 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:14.819740 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:14.820024 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:14.820278 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:14.821014 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:14.821316 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:14.822752 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:14.831365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 600/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:14.837555 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7223d-434e-4b30-878a-d60d5f5b8afa None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:14.903162 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:14.948778 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7223d-434e-4b30-878a-d60d5f5b8afa None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:14.989012 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade95712-3909-4bcb-8e77-ceb7b15c6486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:14.989678 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade95712-3909-4bcb-8e77-ceb7b15c6486 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:14.989965 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade95712-3909-4bcb-8e77-ceb7b15c6486 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:14.990275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade95712-3909-4bcb-8e77-ceb7b15c6486 None admin] PATH_INFO: `/v3/users/d5d0f766399043f18f8610c27febb46b` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:14.991572 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade95712-3909-4bcb-8e77-ceb7b15c6486 None admin] RBAC: Authorizing `identity:get_user(user_id=d5d0f766399043f18f8610c27febb46b)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:14.993074 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade95712-3909-4bcb-8e77-ceb7b15c6486 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:14.994838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 590/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:58:14 2022] GET /identity/v3/users/d5d0f766399043f18f8610c27febb46b => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:15.001298 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:15.110585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91cc31-04d3-4ea8-8892-87169a2d6e1e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:15.145160 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d560c6-907a-4ff9-8016-cc9a24de8417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:15.145785 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d560c6-907a-4ff9-8016-cc9a24de8417 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:15.146069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d560c6-907a-4ff9-8016-cc9a24de8417 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:15.146382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d560c6-907a-4ff9-8016-cc9a24de8417 None admin] PATH_INFO: `/v3/projects/7fced7c8bbe54fd6a7c03d553d283392` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:15.147574 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d560c6-907a-4ff9-8016-cc9a24de8417 None admin] RBAC: Authorizing `identity:get_project(project_id=7fced7c8bbe54fd6a7c03d553d283392)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:15.149038 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d560c6-907a-4ff9-8016-cc9a24de8417 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:15.150773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 601/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:15 2022] GET /identity/v3/projects/7fced7c8bbe54fd6a7c03d553d283392 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:15.157327 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade95712-3909-4bcb-8e77-ceb7b15c6486 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:15.268008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade95712-3909-4bcb-8e77-ceb7b15c6486 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:15.295690 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:15.304718 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a51f30-76bd-4602-8c9d-7d1ca3cd99b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:15.305405 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a51f30-76bd-4602-8c9d-7d1ca3cd99b6 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:15.305679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a51f30-76bd-4602-8c9d-7d1ca3cd99b6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:15.305972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a51f30-76bd-4602-8c9d-7d1ca3cd99b6 None admin] PATH_INFO: `/v3/projects/7fced7c8bbe54fd6a7c03d553d283392/users/d5d0f766399043f18f8610c27febb46b/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:15.308040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a51f30-76bd-4602-8c9d-7d1ca3cd99b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fced7c8bbe54fd6a7c03d553d283392, user_id=d5d0f766399043f18f8610c27febb46b, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:15.309627 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a51f30-76bd-4602-8c9d-7d1ca3cd99b6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:15.324455 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 591/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:15 2022] PUT /identity/v3/projects/7fced7c8bbe54fd6a7c03d553d283392/users/d5d0f766399043f18f8610c27febb46b/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:17.300231 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a5143-7e68-4a02-a74a-3cdd56f0e46a None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.300835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a5143-7e68-4a02-a74a-3cdd56f0e46a None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.300835 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a5143-7e68-4a02-a74a-3cdd56f0e46a None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.302079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 602/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:17.307645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03531021-fd95-43df-a91a-36ed3e603092 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.308071 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03531021-fd95-43df-a91a-36ed3e603092 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.308183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03531021-fd95-43df-a91a-36ed3e603092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.328589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-03531021-fd95-43df-a91a-36ed3e603092 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:17.329803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-03531021-fd95-43df-a91a-36ed3e603092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:17.407840 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 592/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:17.431860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a5143-7e68-4a02-a74a-3cdd56f0e46a None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.469701 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.544169 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a5143-7e68-4a02-a74a-3cdd56f0e46a None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.579742 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeb4d9-3296-4256-aa7b-078ad8663d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.580345 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeb4d9-3296-4256-aa7b-078ad8663d09 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.580621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeb4d9-3296-4256-aa7b-078ad8663d09 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.580897 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeb4d9-3296-4256-aa7b-078ad8663d09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.592749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeb4d9-3296-4256-aa7b-078ad8663d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-4cdeb4d9-3296-4256-aa7b-078ad8663d09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:17.597675 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:17.599088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:58:17.600394 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 603/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:17.605701 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03531021-fd95-43df-a91a-36ed3e603092 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.714831 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03531021-fd95-43df-a91a-36ed3e603092 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.733244 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.753714 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.754383 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.754661 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.754985 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.755726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:17.756030 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.757889 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.768521 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 593/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:17.775713 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeb4d9-3296-4256-aa7b-078ad8663d09 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.873025 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.882310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeb4d9-3296-4256-aa7b-078ad8663d09 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.917514 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5813027-6d45-4165-b868-6625cdd707f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.918215 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813027-6d45-4165-b868-6625cdd707f4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.918500 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813027-6d45-4165-b868-6625cdd707f4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.918781 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813027-6d45-4165-b868-6625cdd707f4 None admin] PATH_INFO: `/v3/users/d5d0f766399043f18f8610c27febb46b` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.920083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5813027-6d45-4165-b868-6625cdd707f4 None admin] RBAC: Authorizing `identity:get_user(user_id=d5d0f766399043f18f8610c27febb46b)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.921562 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5813027-6d45-4165-b868-6625cdd707f4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.923251 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 604/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/users/d5d0f766399043f18f8610c27febb46b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:17.929549 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.039750 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d1321-1943-434d-babc-8afd3a9712c8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.074984 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc5660-10db-4004-8723-2c91838565d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.075824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5660-10db-4004-8723-2c91838565d7 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.076112 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5660-10db-4004-8723-2c91838565d7 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.076392 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5660-10db-4004-8723-2c91838565d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.130042 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.152631 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc5660-10db-4004-8723-2c91838565d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.154163 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc5660-10db-4004-8723-2c91838565d7 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.256492 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 594/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:58:18.262931 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5813027-6d45-4165-b868-6625cdd707f4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.372217 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5813027-6d45-4165-b868-6625cdd707f4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.410440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c38e-95fa-4974-a3a6-12d8aa6c2208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.411010 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c38e-95fa-4974-a3a6-12d8aa6c2208 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.411293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c38e-95fa-4974-a3a6-12d8aa6c2208 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.411567 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c38e-95fa-4974-a3a6-12d8aa6c2208 None admin] PATH_INFO: `/v3/projects/7fced7c8bbe54fd6a7c03d553d283392` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.412819 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c38e-95fa-4974-a3a6-12d8aa6c2208 None admin] RBAC: Authorizing `identity:get_project(project_id=7fced7c8bbe54fd6a7c03d553d283392)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.414325 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c38e-95fa-4974-a3a6-12d8aa6c2208 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.416051 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 605/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/projects/7fced7c8bbe54fd6a7c03d553d283392 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:18.422968 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc5660-10db-4004-8723-2c91838565d7 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.512870 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.535254 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc5660-10db-4004-8723-2c91838565d7 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.571708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85361b-d379-4396-8a51-5fa90cdb65c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.572408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85361b-d379-4396-8a51-5fa90cdb65c5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.572701 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85361b-d379-4396-8a51-5fa90cdb65c5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.573051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85361b-d379-4396-8a51-5fa90cdb65c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.573862 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85361b-d379-4396-8a51-5fa90cdb65c5 None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=7fced7c8bbe54fd6a7c03d553d283392, user.id=d5d0f766399043f18f8610c27febb46b) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:18.574276 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85361b-d379-4396-8a51-5fa90cdb65c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.575839 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85361b-d379-4396-8a51-5fa90cdb65c5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.581933 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 595/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/role_assignments?user.id=d5d0f766399043f18f8610c27febb46b&scope.project.id=7fced7c8bbe54fd6a7c03d553d283392&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:28.405304 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=86 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:28.579298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=28 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:15.167670 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a256b-7914-4b73-b9d9-e206b7cac64f None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:15.168770 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a256b-7914-4b73-b9d9-e206b7cac64f None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:15.169381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a256b-7914-4b73-b9d9-e206b7cac64f None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:15.171630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 606/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:15.177565 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339701f-ce1f-430a-a44a-d4348334a5e7 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:15.179518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339701f-ce1f-430a-a44a-d4348334a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:15.179518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339701f-ce1f-430a-a44a-d4348334a5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:15.201661 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-8339701f-ce1f-430a-a44a-d4348334a5e7 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:15.204134 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-8339701f-ce1f-430a-a44a-d4348334a5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:15.318979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 596/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:15.343720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a256b-7914-4b73-b9d9-e206b7cac64f None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:15.505158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a256b-7914-4b73-b9d9-e206b7cac64f None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:15.545341 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:15.546029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:15.546348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:15.546630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:15.559719 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:15.561244 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:15.579058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:15.580346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:15.581612 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:15.581612 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:15.581612 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:15.581612 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:15.581612 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 607/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:15.588620 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339701f-ce1f-430a-a44a-d4348334a5e7 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:15.728561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339701f-ce1f-430a-a44a-d4348334a5e7 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:15.760997 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:15.769381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6feac9d-f639-4cf3-8d05-a8de5a8aa833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:15.770046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feac9d-f639-4cf3-8d05-a8de5a8aa833 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:15.770425 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feac9d-f639-4cf3-8d05-a8de5a8aa833 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:15.770710 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feac9d-f639-4cf3-8d05-a8de5a8aa833 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:15.771447 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feac9d-f639-4cf3-8d05-a8de5a8aa833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:15.771757 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feac9d-f639-4cf3-8d05-a8de5a8aa833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:15.773312 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feac9d-f639-4cf3-8d05-a8de5a8aa833 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:15.788337 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 597/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:15.794471 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:15.862329 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:15.919659 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe7546-733e-424a-9240-1a343bb00576 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:15.959058 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243307ec-7243-4326-b834-35c2a02c22ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:15.960085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243307ec-7243-4326-b834-35c2a02c22ce None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:15.960400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243307ec-7243-4326-b834-35c2a02c22ce None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:15.960735 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243307ec-7243-4326-b834-35c2a02c22ce None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:15.961446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243307ec-7243-4326-b834-35c2a02c22ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:15.963033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243307ec-7243-4326-b834-35c2a02c22ce None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:15.991980 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 608/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 08:59:15 2022] POST /identity/v3/users => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:18.445158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6a7d3-dd63-4950-9827-a77d99967e3a None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:18.445849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6a7d3-dd63-4950-9827-a77d99967e3a None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:18.445849 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6a7d3-dd63-4950-9827-a77d99967e3a None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:18.447220 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 598/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:18.456137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ddcdc-3218-4744-b573-ede0be0cc7d4 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:18.456712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ddcdc-3218-4744-b573-ede0be0cc7d4 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:18.456712 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ddcdc-3218-4744-b573-ede0be0cc7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:18.477600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-711ddcdc-3218-4744-b573-ede0be0cc7d4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:18.478883 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-711ddcdc-3218-4744-b573-ede0be0cc7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:18.588521 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 609/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:18.617462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6a7d3-dd63-4950-9827-a77d99967e3a None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:18.791545 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6a7d3-dd63-4950-9827-a77d99967e3a None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:18.846943 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1269f3-45c6-4d50-ba15-f6a9f347b411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:18.847620 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1269f3-45c6-4d50-ba15-f6a9f347b411 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:18.847904 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1269f3-45c6-4d50-ba15-f6a9f347b411 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:18.848181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1269f3-45c6-4d50-ba15-f6a9f347b411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:18.862530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1269f3-45c6-4d50-ba15-f6a9f347b411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-4b1269f3-45c6-4d50-ba15-f6a9f347b411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:18.867648 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:18.870023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:18.871717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 599/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:59:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:18.876564 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ddcdc-3218-4744-b573-ede0be0cc7d4 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:18.926228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:19.043937 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ddcdc-3218-4744-b573-ede0be0cc7d4 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:19.101008 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:19.101739 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:19.102029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:19.102340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:19.103108 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:19.103423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:19.105339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:19.115998 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 610/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:59:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:19.123292 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1269f3-45c6-4d50-ba15-f6a9f347b411 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:19.185135 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:19.242025 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1269f3-45c6-4d50-ba15-f6a9f347b411 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:19.280045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:19.280710 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:19.281045 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:19.281283 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:19.293345 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:19.295302 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:19.313364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:19.315044 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:19.317010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:19.317010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:19.317010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:19.317010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:19.318672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 600/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:19.330136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:19.451335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:19.455540 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3598dff-ad61-40a9-addd-ce7df8f9988c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:19.508017 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:19.508816 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:19.509241 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:19.509610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:19.510628 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:19.511125 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:19.513253 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:19.537599 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 611/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:19.545181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:19.706023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b7315-eff6-4298-92eb-78a6980995c6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:19.708084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:19.754182 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:19.755009 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:19.755293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:19.755994 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] PATH_INFO: `/v3/users/placement` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:19.765561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:19.767183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:19.780231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:19.787011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:19.788384 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 601/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:19.789549 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:19.917361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db15876-c83a-4825-8fde-29b02eccd5ff None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:19.960459 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:19.961282 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:19.961647 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:19.962057 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:19.963534 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:19.964085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:19.966995 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:19.992518 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 612/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:59:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:19.999474 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.166406 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d21a1-8b07-4530-8325-cffe30cb90de None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:20.230369 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39261644-585f-4230-8df0-fb1128e0abbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:20.231357 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39261644-585f-4230-8df0-fb1128e0abbd None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.231905 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39261644-585f-4230-8df0-fb1128e0abbd None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.232471 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39261644-585f-4230-8df0-fb1128e0abbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.286732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:20.363548 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39261644-585f-4230-8df0-fb1128e0abbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:20.366057 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39261644-585f-4230-8df0-fb1128e0abbd None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:20.546033 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 602/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:59:20.552528 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.580933 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:20.761696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ee52f-0f8a-46f4-bfbc-d93713d6889c None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:20.823905 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:20.824733 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.825125 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.825558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.840867 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:20.842681 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:20.871070 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:20.873577 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:20.876009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:20.876009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:20.876009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:20.876009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:20.876009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 613/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:20.881845 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39261644-585f-4230-8df0-fb1128e0abbd None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.985710 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:21.065141 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39261644-585f-4230-8df0-fb1128e0abbd None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.122768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.124207 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.124917 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.125491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.126584 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:21.127189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.129962 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.151086 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 603/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:21.160664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:21.300161 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:21.366440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc458-7f99-44d4-81ca-db128c1aff8a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.444200 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.445195 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.445638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.446159 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.460695 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.463474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:21.489234 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:21.491254 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:21.493400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 614/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:21.500290 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:21.645686 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784926f-791d-43ba-8644-da39d98f8eae None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.675433 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:21.695375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59822c2d-9c68-42de-a73a-74fc05a79e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.696128 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59822c2d-9c68-42de-a73a-74fc05a79e04 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.696564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59822c2d-9c68-42de-a73a-74fc05a79e04 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.697169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59822c2d-9c68-42de-a73a-74fc05a79e04 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.698475 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59822c2d-9c68-42de-a73a-74fc05a79e04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:21.699086 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59822c2d-9c68-42de-a73a-74fc05a79e04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.701140 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59822c2d-9c68-42de-a73a-74fc05a79e04 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.718351 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 604/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:21.727374 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:21.875589 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36316-f059-438c-9f30-9dd01fd0d78a None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.921481 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:21.930098 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15bc20-4c88-4acb-b183-0c81e7cb3674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.931046 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15bc20-4c88-4acb-b183-0c81e7cb3674 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.931512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15bc20-4c88-4acb-b183-0c81e7cb3674 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.932014 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15bc20-4c88-4acb-b183-0c81e7cb3674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.934191 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15bc20-4c88-4acb-b183-0c81e7cb3674 None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=9ecd4b2f7b4c49c895aba69111e604d8) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:21.934646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15bc20-4c88-4acb-b183-0c81e7cb3674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.936437 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15bc20-4c88-4acb-b183-0c81e7cb3674 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.945827 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 615/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:21 2022] GET /identity/v3/role_assignments?user.id=9ecd4b2f7b4c49c895aba69111e604d8&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:24.237409 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddaf55-0bbe-46c9-8951-4ce040be9c8e None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:24.238026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddaf55-0bbe-46c9-8951-4ce040be9c8e None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:24.238026 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddaf55-0bbe-46c9-8951-4ce040be9c8e None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:24.239738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 605/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:24.247132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda591d-fb3f-4d45-8f9c-e4d13d957945 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:24.247787 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda591d-fb3f-4d45-8f9c-e4d13d957945 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:24.247787 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda591d-fb3f-4d45-8f9c-e4d13d957945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:24.268607 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-2dda591d-fb3f-4d45-8f9c-e4d13d957945 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:24.270068 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-2dda591d-fb3f-4d45-8f9c-e4d13d957945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:24.366228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 616/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:24.386192 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddaf55-0bbe-46c9-8951-4ce040be9c8e None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:24.512751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddaf55-0bbe-46c9-8951-4ce040be9c8e None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:24.564140 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523227ca-3439-4f51-9ed2-09a166cd50a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:24.565249 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523227ca-3439-4f51-9ed2-09a166cd50a8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:24.566058 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523227ca-3439-4f51-9ed2-09a166cd50a8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:24.566615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523227ca-3439-4f51-9ed2-09a166cd50a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:24.583572 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523227ca-3439-4f51-9ed2-09a166cd50a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-523227ca-3439-4f51-9ed2-09a166cd50a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:24.588673 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:24.590922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:59:24.592681 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:24.595258 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 606/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:59:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:24.596297 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda591d-fb3f-4d45-8f9c-e4d13d957945 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:24.722788 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda591d-fb3f-4d45-8f9c-e4d13d957945 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:24.756943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:24.777442 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:24.778631 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:24.779129 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:24.779765 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:24.781097 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] RBAC: Adding query filter params (name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:24.781656 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:24.785492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:24.801259 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 617/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:59:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:24.808287 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523227ca-3439-4f51-9ed2-09a166cd50a8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:24.879462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:24.982560 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523227ca-3439-4f51-9ed2-09a166cd50a8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.050367 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.051050 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.051333 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.051615 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.066447 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.068708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:25.092524 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:25.093971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:25.095397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:25.095397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:25.095397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:25.095397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:25.095397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 607/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:25.101638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646c8a-b855-4822-9248-151159f6ebe9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.319367 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.320036 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.320315 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.320591 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.321368 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:25.321711 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.323720 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.340705 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 618/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:25.347865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.517629 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:25.529195 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bac115-c344-4feb-ae3d-2b71af18eef8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.576809 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.577491 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.577794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.578049 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.588432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.590800 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:25.609348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:25.612046 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:25.613804 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 608/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:25.618322 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.695415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:25.783503 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca423-0d00-4ee7-9f30-53d91f14a6d7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.828190 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.829005 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.829391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.829871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.831021 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:25.831432 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.833137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.855127 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 619/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:25.865782 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.993566 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91eb27-05bf-4e76-b602-05032fade1e5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.039567 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.040408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.040798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.041250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.055470 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.057364 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:26.077170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:26.085983 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:26.088430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:26.088430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:26.088430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:26.088430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:26.088430 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 609/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:26.088922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.231286 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6a150-4c51-42c1-b7f0-c1cf0fbeae66 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.234560 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.283886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.284633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.284633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.284818 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.285546 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:26.285914 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.287336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.300881 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 620/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:26.307727 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.318807 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.446489 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a0dba-da9e-4647-add7-566d01e76997 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.488015 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.488695 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.488981 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.489259 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.498421 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.500123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:26.515589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:26.517920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:26.519979 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 610/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:26.524251 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.692744 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114a2d8-5f32-40d2-8325-554c70ee8d13 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.754316 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67a5fe-79d8-4a34-b97c-d769306f8e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.755339 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67a5fe-79d8-4a34-b97c-d769306f8e9e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.755561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67a5fe-79d8-4a34-b97c-d769306f8e9e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.755954 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67a5fe-79d8-4a34-b97c-d769306f8e9e None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.756791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67a5fe-79d8-4a34-b97c-d769306f8e9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:26.757103 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67a5fe-79d8-4a34-b97c-d769306f8e9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.758719 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67a5fe-79d8-4a34-b97c-d769306f8e9e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.776618 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 621/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:26.784829 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.874505 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.922452 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b9c66-e8d4-4406-ac27-210862654d80 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.970084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97cd35-d44b-49bc-ad74-abab4863b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.970955 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97cd35-d44b-49bc-ad74-abab4863b310 None admin] REQUEST_METHOD: `PUT` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.971534 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97cd35-d44b-49bc-ad74-abab4863b310 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.971820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97cd35-d44b-49bc-ad74-abab4863b310 None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/9ecd4b2f7b4c49c895aba69111e604d8/roles/e5f9b7b560ca414994499e84782fa22d` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.992240 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97cd35-d44b-49bc-ad74-abab4863b310 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=9ecd4b2f7b4c49c895aba69111e604d8, role_id=e5f9b7b560ca414994499e84782fa22d)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.994028 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97cd35-d44b-49bc-ad74-abab4863b310 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:27.013853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 611/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:59:26 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/9ecd4b2f7b4c49c895aba69111e604d8/roles/e5f9b7b560ca414994499e84782fa22d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:59:29.459218 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac82ce-ae46-4946-9848-56b209ab675d None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.460008 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac82ce-ae46-4946-9848-56b209ab675d None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.460150 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac82ce-ae46-4946-9848-56b209ab675d None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.462856 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 622/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:29.470089 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b80579-edae-40ac-b0e3-c857f3729e81 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.471130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b80579-edae-40ac-b0e3-c857f3729e81 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.471130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b80579-edae-40ac-b0e3-c857f3729e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.496183 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-a0b80579-edae-40ac-b0e3-c857f3729e81 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:29.498136 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-a0b80579-edae-40ac-b0e3-c857f3729e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:29.591749 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 612/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:29.623667 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac82ce-ae46-4946-9848-56b209ab675d None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.657105 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:29.739301 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac82ce-ae46-4946-9848-56b209ab675d None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:29.777170 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137242ac-b767-43bc-bf17-24d275d103d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:29.777859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137242ac-b767-43bc-bf17-24d275d103d9 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.778152 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137242ac-b767-43bc-bf17-24d275d103d9 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.778431 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137242ac-b767-43bc-bf17-24d275d103d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.791695 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137242ac-b767-43bc-bf17-24d275d103d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-137242ac-b767-43bc-bf17-24d275d103d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:29.797009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:29.798512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:29.800117 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 623/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:29.805591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b80579-edae-40ac-b0e3-c857f3729e81 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.867244 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:29.932357 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b80579-edae-40ac-b0e3-c857f3729e81 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:29.980870 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:29.981944 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.982354 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.982745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.984040 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] RBAC: Adding query filter params (name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:29.984593 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:29.986764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:29.999388 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 613/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:30.005444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137242ac-b767-43bc-bf17-24d275d103d9 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.122691 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.144171 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137242ac-b767-43bc-bf17-24d275d103d9 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.185869 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.186592 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.186877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.187156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.200225 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.201749 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:30.217833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:30.219678 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:30.221110 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:30.221110 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:30.221110 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:30.221110 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:30.221110 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 624/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:30.226071 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.358208 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993d33e-6d30-4683-ab7e-7b5a5ca4bbbd None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.369896 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.400487 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.401065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.401361 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.401831 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.402453 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:30.402766 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.404191 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.420205 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 614/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:30.426319 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.569840 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d548bc0-d6ef-4e42-9807-6d4d176fe9e4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.616455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.617065 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.617350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.617625 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.621252 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.628910 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.630984 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:30.645280 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:30.647314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:30.649600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 625/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:30.653444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.781403 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca38d8-6172-4b18-ba99-c22aeb58d07a None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.822529 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.823175 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.823454 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.823945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.825294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:30.825782 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.827654 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.856775 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 615/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:30.863069 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:31.003361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b6cd7-7a7e-4c2f-9e70-dfb0ad141bb4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.055441 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a56a6b-330e-40b2-b005-dc94e2496e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.056728 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56a6b-330e-40b2-b005-dc94e2496e90 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.057256 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56a6b-330e-40b2-b005-dc94e2496e90 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.057822 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56a6b-330e-40b2-b005-dc94e2496e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.162869 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:31.174183 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a56a6b-330e-40b2-b005-dc94e2496e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.174871 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a56a6b-330e-40b2-b005-dc94e2496e90 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.311994 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 626/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:31.313260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:31.350418 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:31.460749 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e177ae-fdad-4475-8d0c-fde292389dc9 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.518747 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.519490 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.519854 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.520199 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.532530 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.535343 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:31.568107 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:31.569574 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:31.571564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:31.571564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:31.571564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:31.571564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:31.571564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 616/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:31.576501 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a56a6b-330e-40b2-b005-dc94e2496e90 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:31.679328 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:31.686293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a56a6b-330e-40b2-b005-dc94e2496e90 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.727035 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.727793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.728093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.728435 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.729123 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:31.729433 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.730958 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.743805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 627/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:31.750403 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:31.839397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:31.865006 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99ddc0-53bc-4195-8519-3d61c9304d70 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.915072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.915948 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.916540 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.916974 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.927876 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.929670 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:31.943091 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:31.944711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:31.946019 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 617/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:31.951594 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:32.082153 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb1db-885e-40fe-b616-ee525b1ba6f7 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:32.135781 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cf764-33fd-4ac7-9a50-2ddf4d84acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:32.137385 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cf764-33fd-4ac7-9a50-2ddf4d84acf6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:32.138009 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cf764-33fd-4ac7-9a50-2ddf4d84acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:32.138482 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cf764-33fd-4ac7-9a50-2ddf4d84acf6 None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:32.139879 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cf764-33fd-4ac7-9a50-2ddf4d84acf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:32.140454 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cf764-33fd-4ac7-9a50-2ddf4d84acf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:32.143302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cf764-33fd-4ac7-9a50-2ddf4d84acf6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:32.168525 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 628/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:32.176514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:32.328764 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034f3f-7f17-4607-8c8a-0a2205d87ae8 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:32.376118 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996a127-127a-4d56-b8f5-b3210378a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:32.376952 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996a127-127a-4d56-b8f5-b3210378a45b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:32.377379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996a127-127a-4d56-b8f5-b3210378a45b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:32.377817 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996a127-127a-4d56-b8f5-b3210378a45b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:32.378865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996a127-127a-4d56-b8f5-b3210378a45b None admin] RBAC: Adding query filter params (role.id=e5f9b7b560ca414994499e84782fa22d, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=9ecd4b2f7b4c49c895aba69111e604d8) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:32.379292 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996a127-127a-4d56-b8f5-b3210378a45b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:32.381028 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996a127-127a-4d56-b8f5-b3210378a45b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:32.389147 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 618/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:32 2022] GET /identity/v3/role_assignments?user.id=9ecd4b2f7b4c49c895aba69111e604d8&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=e5f9b7b560ca414994499e84782fa22d => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:34.597485 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef219-fa33-430e-bca1-dbf137c53969 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:34.598495 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef219-fa33-430e-bca1-dbf137c53969 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:34.598698 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef219-fa33-430e-bca1-dbf137c53969 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:34.600600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 629/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:34.607051 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e018c2-5556-4b6a-b38a-ac04733cbdb2 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:34.607621 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e018c2-5556-4b6a-b38a-ac04733cbdb2 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:34.608008 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e018c2-5556-4b6a-b38a-ac04733cbdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:34.619185 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:34.637528 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-69e018c2-5556-4b6a-b38a-ac04733cbdb2 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:34.639304 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-69e018c2-5556-4b6a-b38a-ac04733cbdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:34.718805 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 619/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:34.743042 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beef219-fa33-430e-bca1-dbf137c53969 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:34.771014 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:34.895630 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beef219-fa33-430e-bca1-dbf137c53969 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:34.933144 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aa26e-8813-40c0-9310-e185052f0776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:34.933868 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aa26e-8813-40c0-9310-e185052f0776 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:34.934158 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aa26e-8813-40c0-9310-e185052f0776 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:34.934439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aa26e-8813-40c0-9310-e185052f0776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:34.951488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01aa26e-8813-40c0-9310-e185052f0776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d01aa26e-8813-40c0-9310-e185052f0776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:34.958760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:34.960274 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:34.963373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 630/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:34.967938 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e018c2-5556-4b6a-b38a-ac04733cbdb2 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:35.092870 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e018c2-5556-4b6a-b38a-ac04733cbdb2 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:35.131629 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:35.139868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:35.141645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:35.142288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:35.142761 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:35.144037 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:35.144659 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:35.147310 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:35.168101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 620/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:35.176391 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aa26e-8813-40c0-9310-e185052f0776 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:35.278578 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:35.317907 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aa26e-8813-40c0-9310-e185052f0776 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:35.360488 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:35.361257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:35.361548 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:35.361815 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:35.373771 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:35.375351 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:35.392382 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:35.394934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:35.397672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:35.397672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:35.397672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:35.397672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:35.397672 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 631/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:35.402258 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:35.535731 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f391a-202f-4dbc-b534-887948ee5aa6 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:35.589198 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:35.590315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:35.590801 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:35.591301 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:35.592495 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:35.593024 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:35.595168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:35.611242 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 621/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:35.617807 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:35.768446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f598-0097-4c2c-b483-500d077605b1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:35.808257 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:35.822033 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:35.823135 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:35.823574 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:35.824029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] PATH_INFO: `/v3/users/placement` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:35.835919 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:35.837894 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:35.857146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:35.858737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:35.860148 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 632/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:35.868647 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:35.947399 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:36.011705 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5b9b1-9558-4810-b421-42cdd32898af None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:36.055676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:36.056418 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:36.056660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:36.056959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:36.057861 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:36.058337 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:36.059740 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:36.086694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 622/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:59:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:36.093436 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:36.263886 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6196a3b-e2b6-409d-a1fe-b0896be56a83 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:36.309206 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18316cb5-1b5b-43b9-a890-2757e421e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:36.309965 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18316cb5-1b5b-43b9-a890-2757e421e7ae None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:36.310435 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18316cb5-1b5b-43b9-a890-2757e421e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:36.310845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18316cb5-1b5b-43b9-a890-2757e421e7ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:36.388821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:36.410769 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18316cb5-1b5b-43b9-a890-2757e421e7ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:36.412313 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18316cb5-1b5b-43b9-a890-2757e421e7ae None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:36.557008 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 633/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:36.565352 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:36.721307 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fde3e0-ba86-4432-bfe8-e4b4f047a72b None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:36.743228 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:36.782079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:36.783308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:36.783829 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:36.784339 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:36.807641 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:36.810662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:36.846053 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:36.847610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:36.848927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:36.848927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:36.848927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:36.848927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:36.849328 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 623/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:36.857369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18316cb5-1b5b-43b9-a890-2757e421e7ae None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:36.980139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:36.996162 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18316cb5-1b5b-43b9-a890-2757e421e7ae None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:37.046276 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:37.046946 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:37.047246 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:37.047522 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:37.048415 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:37.048729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:37.050319 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:37.065388 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 634/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:37.072440 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:37.236687 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33557d-2e31-4947-8298-91f395bd41ab None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:37.290622 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:37.291288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:37.291577 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:37.291851 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] PATH_INFO: `/v3/projects/service` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:37.301897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:37.303597 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:37.320278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:37.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:37.325298 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 624/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:37.328380 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:37.468413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184a09-9129-45b8-a78e-dd8e36a49069 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:37.530815 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a583957-dc4f-41dc-bf74-2331e9d0389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:37.531555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a583957-dc4f-41dc-bf74-2331e9d0389c None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:37.532015 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a583957-dc4f-41dc-bf74-2331e9d0389c None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:37.532465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a583957-dc4f-41dc-bf74-2331e9d0389c None admin] PATH_INFO: `/v3/projects` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:37.533792 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a583957-dc4f-41dc-bf74-2331e9d0389c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:37.534409 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a583957-dc4f-41dc-bf74-2331e9d0389c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:37.537255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a583957-dc4f-41dc-bf74-2331e9d0389c None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:37.564178 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 635/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:37.566379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:37.609149 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:37.711971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7cc9-45b6-47db-bc97-a0d6b24fa148 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:37.758997 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d96e40-98fe-4d87-94c1-a417728808ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:37.759788 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d96e40-98fe-4d87-94c1-a417728808ca None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:37.759907 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d96e40-98fe-4d87-94c1-a417728808ca None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:37.760181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d96e40-98fe-4d87-94c1-a417728808ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:37.760999 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d96e40-98fe-4d87-94c1-a417728808ca None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=9ecd4b2f7b4c49c895aba69111e604d8) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:37.761315 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d96e40-98fe-4d87-94c1-a417728808ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:37.763004 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d96e40-98fe-4d87-94c1-a417728808ca None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:37.786355 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 625/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:37 2022] GET /identity/v3/role_assignments?user.id=9ecd4b2f7b4c49c895aba69111e604d8&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:40.168214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94fd6-6a35-4f5d-a1dc-f3ff581df0f4 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:40.168939 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94fd6-6a35-4f5d-a1dc-f3ff581df0f4 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:40.168939 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94fd6-6a35-4f5d-a1dc-f3ff581df0f4 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:40.170723 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 636/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:40.178906 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968244fb-3d4c-4a00-a6db-2e4cac7f4213 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:40.179816 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968244fb-3d4c-4a00-a6db-2e4cac7f4213 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:40.179816 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968244fb-3d4c-4a00-a6db-2e4cac7f4213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:40.203653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-968244fb-3d4c-4a00-a6db-2e4cac7f4213 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:40.205141 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-968244fb-3d4c-4a00-a6db-2e4cac7f4213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:40.302079 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 626/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:40.333263 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe94fd6-6a35-4f5d-a1dc-f3ff581df0f4 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:40.347077 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:40.473085 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe94fd6-6a35-4f5d-a1dc-f3ff581df0f4 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:40.521713 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee653938-60e5-4d8a-b393-15a98c839ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:40.523056 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653938-60e5-4d8a-b393-15a98c839ce1 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:40.523481 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653938-60e5-4d8a-b393-15a98c839ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:40.524367 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653938-60e5-4d8a-b393-15a98c839ce1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:40.540229 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee653938-60e5-4d8a-b393-15a98c839ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-ee653938-60e5-4d8a-b393-15a98c839ce1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:40.545616 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:40.547969 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:40.549763 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 637/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:40.555167 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968244fb-3d4c-4a00-a6db-2e4cac7f4213 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:40.579265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:40.685349 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968244fb-3d4c-4a00-a6db-2e4cac7f4213 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:40.733606 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:40.734655 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:40.735090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:40.735516 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] PATH_INFO: `/v3/roles` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:40.736700 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:40.737168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:40.739571 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:40.754595 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 627/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:40.760978 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee653938-60e5-4d8a-b393-15a98c839ce1 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:40.853875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:40.900273 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee653938-60e5-4d8a-b393-15a98c839ce1 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:40.942186 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:40.942882 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:40.943166 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:40.943438 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:40.955943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:40.957460 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:40.974558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:40.976924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:40.979088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:40.979088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:40.979088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:40.979088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:40.979088 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 638/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:40.983505 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:41.088708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:41.115453 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3c89f-7daa-4994-a199-cf08e90df9ca None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:41.164671 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:41.165711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:41.166187 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:41.166557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:41.167825 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:41.168249 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:41.170379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:41.187971 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 628/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:41.196310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:41.311642 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0672bfa-404c-44b5-bbe1-ea67abe5382e None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:41.333019 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:41.349104 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:41.349778 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:41.350062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:41.350340 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] PATH_INFO: `/v3/users/placement` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:41.359400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:41.360917 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:41.374610 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:41.376089 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:41.377793 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 639/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:41.384863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:41.498023 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936762a5-f88b-439b-8a34-41242619a842 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:41.535845 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:41.536486 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:41.536768 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:41.537041 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] PATH_INFO: `/v3/users` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:41.537868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:41.538201 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:41.539697 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:41.550007 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:41.561372 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 629/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:59:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:41.567943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:41.679151 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d94b93-1457-456a-bf7a-e7bae8e67ada None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:41.716212 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:41.716888 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:41.717177 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:41.717455 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:41.729239 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:41.730975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:41.746453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:41.748584 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:41.750559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:41.750559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:41.750559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:41.750559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:41.750559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 640/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:41.754134 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:41.875099 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c194-5e43-4c75-8180-d7af4df3a9ec None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:41.913553 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:41.914215 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:41.914507 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:41.914789 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:41.915481 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:41.915787 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:41.917283 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:41.929811 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 630/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:41.935865 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:41.970620 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:42.051725 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df111a15-aef9-4d4c-ab8b-3bb26d0f1a68 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:42.096336 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:42.097030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.097317 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.097568 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.105899 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:42.107551 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:42.121981 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:42.123519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:42.124821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 641/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:42.131462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:42.211774 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:42.257348 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d070c-8360-4c6d-b554-694eb8a5901c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:42.292711 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f251eb-7b6e-4378-a320-b84a3ab28883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:42.293347 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f251eb-7b6e-4378-a320-b84a3ab28883 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.293632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f251eb-7b6e-4378-a320-b84a3ab28883 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.293914 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f251eb-7b6e-4378-a320-b84a3ab28883 None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.294707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f251eb-7b6e-4378-a320-b84a3ab28883 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:42.295022 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f251eb-7b6e-4378-a320-b84a3ab28883 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:42.296522 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f251eb-7b6e-4378-a320-b84a3ab28883 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:42.309793 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 631/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:42.316591 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:42.417690 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:42.428028 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb309b0-2e81-446f-98ca-726f648f3c6b None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:42.465172 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185cbdb-6cef-40e8-a1d3-4dc2b739c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:42.465772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185cbdb-6cef-40e8-a1d3-4dc2b739c2d7 None admin] REQUEST_METHOD: `PUT` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.466059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185cbdb-6cef-40e8-a1d3-4dc2b739c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.466342 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185cbdb-6cef-40e8-a1d3-4dc2b739c2d7 None admin] PATH_INFO: `/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/9ecd4b2f7b4c49c895aba69111e604d8/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.468453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185cbdb-6cef-40e8-a1d3-4dc2b739c2d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd917707f724449cbc3cffdb9cd5ca68, user_id=9ecd4b2f7b4c49c895aba69111e604d8, role_id=1f7e5ce3ed3847cca9b2b1c967bf0f38)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:42.469924 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185cbdb-6cef-40e8-a1d3-4dc2b739c2d7 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:42.485959 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 642/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 08:59:42 2022] PUT /identity/v3/projects/dd917707f724449cbc3cffdb9cd5ca68/users/9ecd4b2f7b4c49c895aba69111e604d8/roles/1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:59:44.547785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40209789-0f98-4646-8d34-fe1d647a36e0 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:44.548471 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40209789-0f98-4646-8d34-fe1d647a36e0 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:44.548471 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40209789-0f98-4646-8d34-fe1d647a36e0 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:44.549748 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 632/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:44.556290 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4307a-f25d-4e38-84f6-14e40a86511c None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:44.557064 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4307a-f25d-4e38-84f6-14e40a86511c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:44.557235 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4307a-f25d-4e38-84f6-14e40a86511c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:44.579098 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-e9a4307a-f25d-4e38-84f6-14e40a86511c None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:44.580512 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-e9a4307a-f25d-4e38-84f6-14e40a86511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:44.658740 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 643/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:44.691952 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40209789-0f98-4646-8d34-fe1d647a36e0 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:44.803213 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40209789-0f98-4646-8d34-fe1d647a36e0 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:44.823920 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:44.851301 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0fe1c-df9a-439c-a83a-c86320bef9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:44.852265 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0fe1c-df9a-439c-a83a-c86320bef9e5 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:44.852632 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0fe1c-df9a-439c-a83a-c86320bef9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:44.852932 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0fe1c-df9a-439c-a83a-c86320bef9e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:44.864789 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0fe1c-df9a-439c-a83a-c86320bef9e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-b9c0fe1c-df9a-439c-a83a-c86320bef9e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:44.869745 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:44.871843 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:44.873978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 633/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:44.877789 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4307a-f25d-4e38-84f6-14e40a86511c None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:44.965595 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:44.987664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4307a-f25d-4e38-84f6-14e40a86511c None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.030664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.031310 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.031606 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.031882 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] PATH_INFO: `/v3/roles` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.032628 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:45.032940 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.034398 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.046743 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 644/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:45.053010 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0fe1c-df9a-439c-a83a-c86320bef9e5 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.176589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0fe1c-df9a-439c-a83a-c86320bef9e5 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.217067 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.217724 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.218072 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.218379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.229076 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.230630 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:45.245853 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:45.247129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:45.248519 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:45.248519 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:45.248519 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:45.248519 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:45.248519 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 634/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:45.254003 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.369516 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd38b70-3dd5-40c7-99c2-98a62115b9aa None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.405430 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.406050 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.406369 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.406638 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] PATH_INFO: `/v3/domains` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.407360 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:45.407667 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.409082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.421160 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 645/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:45.431297 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.468184 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:45.542907 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d971ee5-c1c6-45d8-a1a6-2be9f6a90ffc None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.583030 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.583715 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.583947 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.584231 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] PATH_INFO: `/v3/users/placement` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.592682 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.594213 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:45.606497 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:45.607888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:45.609104 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 635/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:45.614292 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.619064 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:45.721956 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9932e81-7ac4-45b3-93f4-31f4dc1773e4 None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.758553 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.759179 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.759463 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.759737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] PATH_INFO: `/v3/users` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.760555 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:45.760932 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.762767 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.787791 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 646/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:45.794288 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.893564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:45.910824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0243721-b757-460b-bb11-cdea0756a890 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.950550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206522da-aa4b-4564-8c3b-8d1a88f18057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.951167 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206522da-aa4b-4564-8c3b-8d1a88f18057 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.951450 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206522da-aa4b-4564-8c3b-8d1a88f18057 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.951722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206522da-aa4b-4564-8c3b-8d1a88f18057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:46.028794 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206522da-aa4b-4564-8c3b-8d1a88f18057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:46.030250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206522da-aa4b-4564-8c3b-8d1a88f18057 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:46.136872 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:46.150898 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 636/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:46.157013 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:46.225272 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:46.268987 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e8019-3b36-4285-93c2-33a910c5693f None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:46.307405 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:46.307992 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:46.308276 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:46.308558 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:46.320831 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:46.322445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:46.337885 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:46.339242 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:46.340805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:46.340805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:46.340805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:46.340805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:46.340805 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 647/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:46.347274 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206522da-aa4b-4564-8c3b-8d1a88f18057 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:46.449526 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206522da-aa4b-4564-8c3b-8d1a88f18057 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:46.485170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:46.485785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:46.486066 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:46.486375 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] PATH_INFO: `/v3/domains` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:46.487065 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] RBAC: Adding query filter params (name=Default) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:46.487372 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:46.488783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:46.500436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 637/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:46.507139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:46.616007 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f48fd4-e041-410f-b1fa-bc64ee05f5bd None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:46.629967 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:46.652857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:46.653525 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:46.653814 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:46.654122 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] PATH_INFO: `/v3/projects/service` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:46.661243 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:46.662823 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:46.675082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:46.676715 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: ERROR keystone.server.flask.application Jul 28 08:59:46.677859 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 648/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:46.682783 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:46.737518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:46.789273 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6cb3f-7126-4c3d-8fcd-2bc2b10fe30d None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:46.827062 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ecb41-7585-4f3c-ae0e-2fa2bb19dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:46.827683 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ecb41-7585-4f3c-ae0e-2fa2bb19dd6d None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:46.827962 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ecb41-7585-4f3c-ae0e-2fa2bb19dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:46.828239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ecb41-7585-4f3c-ae0e-2fa2bb19dd6d None admin] PATH_INFO: `/v3/projects` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:46.828997 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ecb41-7585-4f3c-ae0e-2fa2bb19dd6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:46.829303 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ecb41-7585-4f3c-ae0e-2fa2bb19dd6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:46.830803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ecb41-7585-4f3c-ae0e-2fa2bb19dd6d None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:46.844322 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 638/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:46.850927 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:46.974669 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e636d-403c-4c1b-b76e-1e218cfeb28b None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:47.012106 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8303a-dc55-42ce-adf4-84a354986aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:47.012774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8303a-dc55-42ce-adf4-84a354986aad None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:47.013060 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8303a-dc55-42ce-adf4-84a354986aad None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:47.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8303a-dc55-42ce-adf4-84a354986aad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:47.014146 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8303a-dc55-42ce-adf4-84a354986aad None admin] RBAC: Adding query filter params (role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38, scope.project.id=dd917707f724449cbc3cffdb9cd5ca68, user.id=9ecd4b2f7b4c49c895aba69111e604d8) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:47.014480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8303a-dc55-42ce-adf4-84a354986aad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:47.015954 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8303a-dc55-42ce-adf4-84a354986aad None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:47.022799 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 649/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:46 2022] GET /identity/v3/role_assignments?user.id=9ecd4b2f7b4c49c895aba69111e604d8&scope.project.id=dd917707f724449cbc3cffdb9cd5ca68&role.id=1f7e5ce3ed3847cca9b2b1c967bf0f38 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:49.070809 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232e3b0-4577-45df-b348-312aed15abaa None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:49.071408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232e3b0-4577-45df-b348-312aed15abaa None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:49.071408 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232e3b0-4577-45df-b348-312aed15abaa None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:49.072744 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 639/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:49.078451 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883cda-8983-4bef-9170-a7a2537e5d37 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:49.078905 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883cda-8983-4bef-9170-a7a2537e5d37 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:49.079029 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883cda-8983-4bef-9170-a7a2537e5d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:49.098637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-55883cda-8983-4bef-9170-a7a2537e5d37 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:49.099903 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-55883cda-8983-4bef-9170-a7a2537e5d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:49.137955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:49.183772 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 650/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:49.214653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232e3b0-4577-45df-b348-312aed15abaa None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:49.326250 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232e3b0-4577-45df-b348-312aed15abaa None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:49.337347 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:49.363788 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:49.364445 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:49.364732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:49.365013 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] PATH_INFO: `/v3/services/placement` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:49.365649 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:49.367113 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:49.376295 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:49.377702 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:49.378848 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 640/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:59:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:49.383858 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55883cda-8983-4bef-9170-a7a2537e5d37 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:49.489751 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55883cda-8983-4bef-9170-a7a2537e5d37 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:49.526026 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab6d73-3ed8-4afa-bb62-ba1199f78557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:49.526729 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6d73-3ed8-4afa-bb62-ba1199f78557 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:49.527011 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6d73-3ed8-4afa-bb62-ba1199f78557 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:49.527285 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6d73-3ed8-4afa-bb62-ba1199f78557 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:49.527990 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab6d73-3ed8-4afa-bb62-ba1199f78557 None admin] RBAC: Adding query filter params (name=placement) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:49.528296 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab6d73-3ed8-4afa-bb62-ba1199f78557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:49.529707 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab6d73-3ed8-4afa-bb62-ba1199f78557 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:49.535413 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 651/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:59:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:49.541758 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:49.652278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75116-668b-461e-afa9-18daacc34e56 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:49.689432 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4328bc2-7e73-4605-a2ec-99838cc78027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:49.690067 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328bc2-7e73-4605-a2ec-99838cc78027 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:49.690381 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328bc2-7e73-4605-a2ec-99838cc78027 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:49.690660 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328bc2-7e73-4605-a2ec-99838cc78027 None admin] PATH_INFO: `/v3/services` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:49.691742 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4328bc2-7e73-4605-a2ec-99838cc78027 None admin] RBAC: Adding query filter params (type=placement) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:49.692048 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4328bc2-7e73-4605-a2ec-99838cc78027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:49.693478 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4328bc2-7e73-4605-a2ec-99838cc78027 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:49.698476 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 641/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:59:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:51.716137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed3f8-0dae-454b-8dbe-c2c786141842 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:51.716782 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed3f8-0dae-454b-8dbe-c2c786141842 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:51.716782 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed3f8-0dae-454b-8dbe-c2c786141842 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:51.718048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 652/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:51.724090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b3b5-11a0-4fd7-954f-8d320ed765d5 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:51.724564 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b3b5-11a0-4fd7-954f-8d320ed765d5 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:51.724679 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b3b5-11a0-4fd7-954f-8d320ed765d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:51.744820 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-aa32b3b5-11a0-4fd7-954f-8d320ed765d5 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:51.746189 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-aa32b3b5-11a0-4fd7-954f-8d320ed765d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:51.786033 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:51.829003 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 642/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:51.856639 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ed3f8-0dae-454b-8dbe-c2c786141842 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:51.891400 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:51.976878 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ed3f8-0dae-454b-8dbe-c2c786141842 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:52.017418 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff120ad-e0f2-4de6-b840-6a2292be7991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:52.018072 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff120ad-e0f2-4de6-b840-6a2292be7991 None admin] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:52.018389 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff120ad-e0f2-4de6-b840-6a2292be7991 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:52.018661 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff120ad-e0f2-4de6-b840-6a2292be7991 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:52.019377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff120ad-e0f2-4de6-b840-6a2292be7991 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:52.020780 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff120ad-e0f2-4de6-b840-6a2292be7991 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:52.041097 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 653/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 08:59:51 2022] POST /identity/v3/services => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:54.126535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680004ee-3f8c-429f-9b41-bd5db253f788 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:54.127422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680004ee-3f8c-429f-9b41-bd5db253f788 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:54.127422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680004ee-3f8c-429f-9b41-bd5db253f788 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:54.128612 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 643/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:54.135030 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ad7b1-13f4-4afe-a2a6-6b012d68d0b4 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:54.135684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ad7b1-13f4-4afe-a2a6-6b012d68d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:54.135684 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ad7b1-13f4-4afe-a2a6-6b012d68d0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:54.156371 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-377ad7b1-13f4-4afe-a2a6-6b012d68d0b4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:54.157663 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-377ad7b1-13f4-4afe-a2a6-6b012d68d0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:54.249449 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 654/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:54.273554 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680004ee-3f8c-429f-9b41-bd5db253f788 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:54.402308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680004ee-3f8c-429f-9b41-bd5db253f788 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:54.446722 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:54.447422 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:54.447714 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:54.448005 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] PATH_INFO: `/v3/services/placement` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:54.448726 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:54.450234 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:54.460514 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:54.461899 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:54.463000 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 644/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:59:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:54.469012 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ad7b1-13f4-4afe-a2a6-6b012d68d0b4 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:54.499156 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:54.585209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ad7b1-13f4-4afe-a2a6-6b012d68d0b4 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:54.621374 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:54.622044 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:54.622363 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:54.622650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:54.623350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] RBAC: Adding query filter params (name=placement) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:54.623663 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:54.625077 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:54.630746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 655/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:59:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:54.637279 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:54.670587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:54.748865 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea22ca-3ae3-49c3-802c-126804a6b41c None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:54.788978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba78855-2e38-40fa-81d3-94d351259a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:54.789594 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78855-2e38-40fa-81d3-94d351259a27 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:54.789873 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78855-2e38-40fa-81d3-94d351259a27 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:54.790144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78855-2e38-40fa-81d3-94d351259a27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:54.790941 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba78855-2e38-40fa-81d3-94d351259a27 None admin] RBAC: Adding query filter params (interface=public, service_id=68d32ef6e86c4d6fab6b149505b82761, region_id=RegionOne) {{(pid=347147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:54.791252 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba78855-2e38-40fa-81d3-94d351259a27 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:54.792738 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba78855-2e38-40fa-81d3-94d351259a27 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:54.798021 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 645/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 08:59:54 2022] GET /identity/v3/endpoints?service_id=68d32ef6e86c4d6fab6b149505b82761&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:54.804323 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:54.894869 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:54.920928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed02e8-cfca-4963-bb75-40aa5696deca None admin] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:54.964027 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160602-43ba-4ed9-8692-16cb577e43a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:54.964673 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160602-43ba-4ed9-8692-16cb577e43a6 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:54.964968 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160602-43ba-4ed9-8692-16cb577e43a6 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:54.965247 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160602-43ba-4ed9-8692-16cb577e43a6 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:54.965911 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09160602-43ba-4ed9-8692-16cb577e43a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:54.967361 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09160602-43ba-4ed9-8692-16cb577e43a6 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:54.974814 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 656/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 08:59:54 2022] GET /identity/v3/services => generated 2191 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:59:57.018563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e54af-7d82-486e-9ee1-84c19186003f None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:57.019170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e54af-7d82-486e-9ee1-84c19186003f None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:57.019170 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e54af-7d82-486e-9ee1-84c19186003f None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:57.020486 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 646/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 08:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:57.026241 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c094-9f76-4e66-8ada-109966602750 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:57.026708 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c094-9f76-4e66-8ada-109966602750 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:57.026845 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c094-9f76-4e66-8ada-109966602750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:57.047125 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-7637c094-9f76-4e66-8ada-109966602750 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:57.048368 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-7637c094-9f76-4e66-8ada-109966602750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:57.126640 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 657/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:57.149594 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e54af-7d82-486e-9ee1-84c19186003f None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:57.257586 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:57.265448 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e54af-7d82-486e-9ee1-84c19186003f None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:57.301653 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:57.302262 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:57.302549 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:57.302827 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] PATH_INFO: `/v3/services/placement` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:57.303462 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:57.304887 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:57.314694 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:57.317308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: ERROR keystone.server.flask.application Jul 28 08:59:57.318550 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 647/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 08:59:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:57.326723 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637c094-9f76-4e66-8ada-109966602750 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:57.429101 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:57.454302 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637c094-9f76-4e66-8ada-109966602750 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:57.501824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d65bb1-6a69-4dec-b227-ca21b49db987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:57.502492 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d65bb1-6a69-4dec-b227-ca21b49db987 None admin] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:57.502777 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d65bb1-6a69-4dec-b227-ca21b49db987 None admin] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:57.503047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d65bb1-6a69-4dec-b227-ca21b49db987 None admin] PATH_INFO: `/v3/services` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:57.503741 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d65bb1-6a69-4dec-b227-ca21b49db987 None admin] RBAC: Adding query filter params (name=placement) {{(pid=347146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:57.504043 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d65bb1-6a69-4dec-b227-ca21b49db987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:57.505440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d65bb1-6a69-4dec-b227-ca21b49db987 None admin] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:57.510964 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 658/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 08:59:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:57.517102 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:57.618420 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45237a-1b69-42aa-9ade-d8f494023271 None admin] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:57.654059 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc30c1c-fd5a-41af-8a06-f60bb1e65ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27091c013a8c447a81380abde9ae7c70', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:57.654709 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc30c1c-fd5a-41af-8a06-f60bb1e65ec2 None admin] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:57.654991 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc30c1c-fd5a-41af-8a06-f60bb1e65ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:57.655260 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc30c1c-fd5a-41af-8a06-f60bb1e65ec2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:57.655966 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc30c1c-fd5a-41af-8a06-f60bb1e65ec2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:57.657379 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc30c1c-fd5a-41af-8a06-f60bb1e65ec2 None admin] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:57.679006 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 648/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 08:59:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:00:07.507647 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=30 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:07.674303 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:06.442348 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e96a87-f970-4882-a8c7-c9fe5c5a2a2a None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:06.442908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e96a87-f970-4882-a8c7-c9fe5c5a2a2a None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:06.442908 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e96a87-f970-4882-a8c7-c9fe5c5a2a2a None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:06.444795 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 659/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:06.450146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385fe70-e295-4928-a9f0-47442df54941 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:06.452169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385fe70-e295-4928-a9f0-47442df54941 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:06.452169 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385fe70-e295-4928-a9f0-47442df54941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:06.614587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-a385fe70-e295-4928-a9f0-47442df54941 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:06.615871 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-a385fe70-e295-4928-a9f0-47442df54941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:06.847139 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 649/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:06.918765 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583191f-5b92-4159-bcad-5b11792e2bf7 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:06.919228 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583191f-5b92-4159-bcad-5b11792e2bf7 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:06.919620 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583191f-5b92-4159-bcad-5b11792e2bf7 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:06.921913 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 660/1309] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 09:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:06.929200 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f716-944c-4da0-a80f-afdb8ea8c45b None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:06.929672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f716-944c-4da0-a80f-afdb8ea8c45b None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:06.930094 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f716-944c-4da0-a80f-afdb8ea8c45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:07.119879 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-be94f716-944c-4da0-a80f-afdb8ea8c45b None None] MFA Rules not processed for user `2f8958e3b5dd46bb9e5c835214654ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:07.124150 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-be94f716-944c-4da0-a80f-afdb8ea8c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:07.184429 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:07.301011 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 650/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 09:01:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:07.311854 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583191f-5b92-4159-bcad-5b11792e2bf7 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:07.313950 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-7583191f-5b92-4159-bcad-5b11792e2bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:07.522631 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583191f-5b92-4159-bcad-5b11792e2bf7 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:07.615971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16659f31-9289-4703-9d62-22a8696f6276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:07.616541 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16659f31-9289-4703-9d62-22a8696f6276 service neutron] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:07.616824 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16659f31-9289-4703-9d62-22a8696f6276 service neutron] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:07.617267 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16659f31-9289-4703-9d62-22a8696f6276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:07.618993 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-16659f31-9289-4703-9d62-22a8696f6276 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:07.675974 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:07.736431 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16659f31-9289-4703-9d62-22a8696f6276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:07.737916 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16659f31-9289-4703-9d62-22a8696f6276 service neutron] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:07.758439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 661/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:10.136757 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748019b-35ee-45a0-a182-117791166b00 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:10.137239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748019b-35ee-45a0-a182-117791166b00 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:10.137239 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748019b-35ee-45a0-a182-117791166b00 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:10.138744 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 651/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:01:10.144482 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f34d84-5bf7-4684-94e0-6ad87e666365 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:10.145191 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f34d84-5bf7-4684-94e0-6ad87e666365 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:10.145191 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f34d84-5bf7-4684-94e0-6ad87e666365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:10.238762 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-03f34d84-5bf7-4684-94e0-6ad87e666365 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:10.240083 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-03f34d84-5bf7-4684-94e0-6ad87e666365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:10.343095 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 662/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:10.397084 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbc547-0b4f-4326-b7da-8ca9ab8e3359 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:10.397281 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbc547-0b4f-4326-b7da-8ca9ab8e3359 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:10.397557 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbc547-0b4f-4326-b7da-8ca9ab8e3359 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:10.399027 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 652/1314] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 09:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:10.405468 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a61eee-0816-44ab-9e83-aa6741f6e659 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:10.405767 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a61eee-0816-44ab-9e83-aa6741f6e659 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:10.406087 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a61eee-0816-44ab-9e83-aa6741f6e659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:10.418760 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:10.489091 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-25a61eee-0816-44ab-9e83-aa6741f6e659 None None] MFA Rules not processed for user `2f8958e3b5dd46bb9e5c835214654ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:10.490365 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-25a61eee-0816-44ab-9e83-aa6741f6e659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:10.571516 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 663/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 09:01:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:10.580996 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbc547-0b4f-4326-b7da-8ca9ab8e3359 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:10.582591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-d9dbc547-0b4f-4326-b7da-8ca9ab8e3359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:10.695708 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbc547-0b4f-4326-b7da-8ca9ab8e3359 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:10.780469 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:10.783313 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd39e6-051e-48e3-8443-a3d28e3f46e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:10.784164 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd39e6-051e-48e3-8443-a3d28e3f46e9 service neutron] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:10.784717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd39e6-051e-48e3-8443-a3d28e3f46e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:10.786450 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd39e6-051e-48e3-8443-a3d28e3f46e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:10.788332 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-60bd39e6-051e-48e3-8443-a3d28e3f46e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:10.907742 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd39e6-051e-48e3-8443-a3d28e3f46e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:10.909246 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd39e6-051e-48e3-8443-a3d28e3f46e9 service neutron] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:10.928563 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 653/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:13.526603 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d4b4d-bb3b-4507-8df8-97b8cbf8e7a0 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:13.527238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d4b4d-bb3b-4507-8df8-97b8cbf8e7a0 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:13.527238 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d4b4d-bb3b-4507-8df8-97b8cbf8e7a0 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:13.528583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 664/1317] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:13.533988 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b60bc-2f8f-465f-9bfa-9606de870d70 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:13.534483 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b60bc-2f8f-465f-9bfa-9606de870d70 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:13.534610 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b60bc-2f8f-465f-9bfa-9606de870d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:13.608868 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-1e8b60bc-2f8f-465f-9bfa-9606de870d70 None None] MFA Rules not processed for user `596bd6a71df14947b131621dfe20517f`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:13.610158 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-1e8b60bc-2f8f-465f-9bfa-9606de870d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:13.669972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:13.730467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 654/1318] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:13.763002 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d4b4d-bb3b-4507-8df8-97b8cbf8e7a0 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:13.764583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-1e5d4b4d-bb3b-4507-8df8-97b8cbf8e7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:13.872989 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d4b4d-bb3b-4507-8df8-97b8cbf8e7a0 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:13.876005 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:13.952326 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff323699-fba6-42aa-8ae6-163257c8551e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:13.952975 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff323699-fba6-42aa-8ae6-163257c8551e service neutron] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:13.953260 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff323699-fba6-42aa-8ae6-163257c8551e service neutron] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:13.953537 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff323699-fba6-42aa-8ae6-163257c8551e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:13.955272 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ff323699-fba6-42aa-8ae6-163257c8551e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:14.078650 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff323699-fba6-42aa-8ae6-163257c8551e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:14.080139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff323699-fba6-42aa-8ae6-163257c8551e service neutron] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:14.100436 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 665/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:16.742932 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22e337-611e-473c-8d34-300b6dbf93cc None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:16.743587 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22e337-611e-473c-8d34-300b6dbf93cc None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:16.743731 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22e337-611e-473c-8d34-300b6dbf93cc None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:16.745356 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 655/1320] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:16.751406 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1602441-4891-4e5d-890a-2815a774c211 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:16.751857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1602441-4891-4e5d-890a-2815a774c211 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:16.751960 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1602441-4891-4e5d-890a-2815a774c211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:16.797465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:16.826041 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-c1602441-4891-4e5d-890a-2815a774c211 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:16.827346 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-c1602441-4891-4e5d-890a-2815a774c211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:16.938780 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 666/1321] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:16.980959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22e337-611e-473c-8d34-300b6dbf93cc None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:16.983509 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-bd22e337-611e-473c-8d34-300b6dbf93cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:17.103397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22e337-611e-473c-8d34-300b6dbf93cc None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:17.185021 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f831bb5-779d-48e2-985d-cc39828aee64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:17.185662 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f831bb5-779d-48e2-985d-cc39828aee64 service neutron] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:17.185950 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f831bb5-779d-48e2-985d-cc39828aee64 service neutron] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:17.186257 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f831bb5-779d-48e2-985d-cc39828aee64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:17.187922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-2f831bb5-779d-48e2-985d-cc39828aee64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:17.196203 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:17.307485 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f831bb5-779d-48e2-985d-cc39828aee64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:17.308927 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f831bb5-779d-48e2-985d-cc39828aee64 service neutron] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:17.329275 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 656/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:20.309900 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a023-0a34-43d6-8500-b174f2549a60 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:20.310477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a023-0a34-43d6-8500-b174f2549a60 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:20.310477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a023-0a34-43d6-8500-b174f2549a60 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:20.311768 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 667/1323] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:20.317467 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d4334-667b-4f3a-8eab-16d49e036d35 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:20.317922 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d4334-667b-4f3a-8eab-16d49e036d35 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:20.318025 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d4334-667b-4f3a-8eab-16d49e036d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:20.390129 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-510d4334-667b-4f3a-8eab-16d49e036d35 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:20.391436 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-510d4334-667b-4f3a-8eab-16d49e036d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:20.462714 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:20.506274 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 657/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:20.543458 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17a023-0a34-43d6-8500-b174f2549a60 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:20.545048 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-4d17a023-0a34-43d6-8500-b174f2549a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:20.667774 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17a023-0a34-43d6-8500-b174f2549a60 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:20.682293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:20.759482 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21461b39-5469-4ae0-a5cd-9019c7cf9527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:20.760139 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21461b39-5469-4ae0-a5cd-9019c7cf9527 service neutron] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:20.760425 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21461b39-5469-4ae0-a5cd-9019c7cf9527 service neutron] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:20.760703 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21461b39-5469-4ae0-a5cd-9019c7cf9527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:20.762504 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-21461b39-5469-4ae0-a5cd-9019c7cf9527 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:20.892716 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21461b39-5469-4ae0-a5cd-9019c7cf9527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:20.894293 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21461b39-5469-4ae0-a5cd-9019c7cf9527 service neutron] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:20.915343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 668/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:01:23.013842 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8d152-e219-4970-b426-5404a6622b6d None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:23.013842 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8d152-e219-4970-b426-5404a6622b6d None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:23.014521 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8d152-e219-4970-b426-5404a6622b6d None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:23.015500 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 658/1326] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:23.021664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7b479-b9e1-43c8-b900-5d2cca3febbc None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:23.022128 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7b479-b9e1-43c8-b900-5d2cca3febbc None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:23.022277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7b479-b9e1-43c8-b900-5d2cca3febbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:23.080240 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:23.101381 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-a8b7b479-b9e1-43c8-b900-5d2cca3febbc None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:23.102698 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-a8b7b479-b9e1-43c8-b900-5d2cca3febbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:23.213102 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 669/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:23.249973 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8d152-e219-4970-b426-5404a6622b6d None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:23.251588 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-c5f8d152-e219-4970-b426-5404a6622b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:23.365881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8d152-e219-4970-b426-5404a6622b6d None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:23.448144 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e4884-4b73-4d2c-82f5-12c0d4b5e6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:23.448789 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e4884-4b73-4d2c-82f5-12c0d4b5e6aa service neutron] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:23.449074 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e4884-4b73-4d2c-82f5-12c0d4b5e6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:23.449349 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e4884-4b73-4d2c-82f5-12c0d4b5e6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:23.451066 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-fb2e4884-4b73-4d2c-82f5-12c0d4b5e6aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:23.471645 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:23.573633 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e4884-4b73-4d2c-82f5-12c0d4b5e6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:23.575101 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e4884-4b73-4d2c-82f5-12c0d4b5e6aa service neutron] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:23.594655 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 659/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:25.651316 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47034c07-5e5d-42d9-8c66-29feb3f47732 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:25.651880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47034c07-5e5d-42d9-8c66-29feb3f47732 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:25.651880 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47034c07-5e5d-42d9-8c66-29feb3f47732 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:25.653301 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 670/1329] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:25.658978 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff059-61ef-4872-944c-6daada109f38 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:25.659404 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff059-61ef-4872-944c-6daada109f38 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:25.659518 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff059-61ef-4872-944c-6daada109f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:25.730579 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-18fff059-61ef-4872-944c-6daada109f38 None None] MFA Rules not processed for user `596bd6a71df14947b131621dfe20517f`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:25.731824 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-18fff059-61ef-4872-944c-6daada109f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:25.799672 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:25.833561 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 660/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:25.871132 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47034c07-5e5d-42d9-8c66-29feb3f47732 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:25.872773 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-47034c07-5e5d-42d9-8c66-29feb3f47732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:25.984890 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47034c07-5e5d-42d9-8c66-29feb3f47732 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:26.005207 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:26.062370 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdbe02-0202-4850-afe0-359ec3a2fb55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:26.063019 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdbe02-0202-4850-afe0-359ec3a2fb55 service neutron] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:26.063300 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdbe02-0202-4850-afe0-359ec3a2fb55 service neutron] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:26.063569 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdbe02-0202-4850-afe0-359ec3a2fb55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:26.065220 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-1dfdbe02-0202-4850-afe0-359ec3a2fb55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:26.178264 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdbe02-0202-4850-afe0-359ec3a2fb55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:26.179731 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdbe02-0202-4850-afe0-359ec3a2fb55 service neutron] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:26.198600 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 671/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:28.453493 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5347e6b-65c1-4f88-904b-00f4fb775e25 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:28.454060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5347e6b-65c1-4f88-904b-00f4fb775e25 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:28.454060 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5347e6b-65c1-4f88-904b-00f4fb775e25 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:28.455546 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 661/1332] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:28.461439 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e37fbe-9f04-4561-a6b5-e2dce92536a8 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:28.461991 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e37fbe-9f04-4561-a6b5-e2dce92536a8 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:28.461991 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e37fbe-9f04-4561-a6b5-e2dce92536a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:28.532170 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:28.535508 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-31e37fbe-9f04-4561-a6b5-e2dce92536a8 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:28.537587 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-31e37fbe-9f04-4561-a6b5-e2dce92536a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:28.644510 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 672/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:28.684168 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5347e6b-65c1-4f88-904b-00f4fb775e25 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:28.685791 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-f5347e6b-65c1-4f88-904b-00f4fb775e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:28.816522 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5347e6b-65c1-4f88-904b-00f4fb775e25 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:28.903860 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3282f-7c78-44f2-b2ba-d126289064c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:28.904507 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3282f-7c78-44f2-b2ba-d126289064c2 service neutron] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:28.904784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3282f-7c78-44f2-b2ba-d126289064c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:28.905055 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3282f-7c78-44f2-b2ba-d126289064c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:28.906776 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-ddf3282f-7c78-44f2-b2ba-d126289064c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:28.936547 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:29.044324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3282f-7c78-44f2-b2ba-d126289064c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:29.045838 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3282f-7c78-44f2-b2ba-d126289064c2 service neutron] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:29.067912 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 662/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:31.137877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b604-7de2-4136-9917-12e15d1f55d0 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:31.138465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b604-7de2-4136-9917-12e15d1f55d0 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:31.138465 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b604-7de2-4136-9917-12e15d1f55d0 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:31.139872 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 673/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:31.145398 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102beac-085b-4832-a32d-7ea7308e7ade None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:31.145863 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102beac-085b-4832-a32d-7ea7308e7ade None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:31.145959 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102beac-085b-4832-a32d-7ea7308e7ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:31.218925 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-c102beac-085b-4832-a32d-7ea7308e7ade None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:31.220177 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-c102beac-085b-4832-a32d-7ea7308e7ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:31.299510 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:31.324846 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 663/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:31.365193 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300b604-7de2-4136-9917-12e15d1f55d0 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:31.366808 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-0300b604-7de2-4136-9917-12e15d1f55d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:31.481477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300b604-7de2-4136-9917-12e15d1f55d0 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:31.513943 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:31.564583 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccba4ae-5777-4ab9-80af-2b8c9a45f607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:31.565120 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba4ae-5777-4ab9-80af-2b8c9a45f607 service neutron] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:31.565401 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba4ae-5777-4ab9-80af-2b8c9a45f607 service neutron] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:31.565673 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba4ae-5777-4ab9-80af-2b8c9a45f607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:31.567735 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9ccba4ae-5777-4ab9-80af-2b8c9a45f607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:31.680958 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccba4ae-5777-4ab9-80af-2b8c9a45f607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:31.682674 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccba4ae-5777-4ab9-80af-2b8c9a45f607 service neutron] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:31.703137 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 674/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:34.147340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b3d0-18f3-4adc-81a7-78c2d5377e0f None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:34.147888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b3d0-18f3-4adc-81a7-78c2d5377e0f None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:34.147888 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b3d0-18f3-4adc-81a7-78c2d5377e0f None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:34.149293 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 664/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:34.155459 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06269f-2a16-40f0-aa2f-e12956c90cc4 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:34.155934 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06269f-2a16-40f0-aa2f-e12956c90cc4 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:34.156131 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06269f-2a16-40f0-aa2f-e12956c90cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:34.224559 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-fa06269f-2a16-40f0-aa2f-e12956c90cc4 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:34.225801 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-fa06269f-2a16-40f0-aa2f-e12956c90cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:34.234821 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:34.332478 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 675/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:34.369185 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3b3d0-18f3-4adc-81a7-78c2d5377e0f None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:34.370803 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-f8f3b3d0-18f3-4adc-81a7-78c2d5377e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:34.481844 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3b3d0-18f3-4adc-81a7-78c2d5377e0f None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:34.557454 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28cf43-3244-4775-a7a8-3d8cdbabdace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:34.558090 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28cf43-3244-4775-a7a8-3d8cdbabdace service neutron] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:34.558402 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28cf43-3244-4775-a7a8-3d8cdbabdace service neutron] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:34.558676 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28cf43-3244-4775-a7a8-3d8cdbabdace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:34.560324 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-6e28cf43-3244-4775-a7a8-3d8cdbabdace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:34.597419 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:34.678409 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28cf43-3244-4775-a7a8-3d8cdbabdace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:34.679914 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28cf43-3244-4775-a7a8-3d8cdbabdace service neutron] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:34.699589 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 665/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:38.700666 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605aa690-10dc-4d82-95b6-3c467c9fd2bc None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:38.701136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605aa690-10dc-4d82-95b6-3c467c9fd2bc None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:38.701232 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605aa690-10dc-4d82-95b6-3c467c9fd2bc None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:38.708278 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d585c1-93a8-43e9-9a07-1b78b97cd9f3 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:38.708696 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 676/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:38.708792 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d585c1-93a8-43e9-9a07-1b78b97cd9f3 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:38.708881 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d585c1-93a8-43e9-9a07-1b78b97cd9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:38.795897 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-36d585c1-93a8-43e9-9a07-1b78b97cd9f3 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:38.797248 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-36d585c1-93a8-43e9-9a07-1b78b97cd9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:38.893341 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:38.906486 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 666/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:01:38.946059 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605aa690-10dc-4d82-95b6-3c467c9fd2bc None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:38.947860 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-605aa690-10dc-4d82-95b6-3c467c9fd2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:39.061082 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605aa690-10dc-4d82-95b6-3c467c9fd2bc None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:39.103482 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:39.148444 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf586e-eacb-4977-9a39-45d7150679a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:39.149093 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf586e-eacb-4977-9a39-45d7150679a1 service neutron] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:39.149384 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf586e-eacb-4977-9a39-45d7150679a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:39.149637 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf586e-eacb-4977-9a39-45d7150679a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:39.151450 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-8eaf586e-eacb-4977-9a39-45d7150679a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:39.279973 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf586e-eacb-4977-9a39-45d7150679a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:39.281477 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf586e-eacb-4977-9a39-45d7150679a1 service neutron] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:39.300289 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 677/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:42.122539 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da0856-c48b-4885-9f74-5d5cbd174eb6 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.123132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da0856-c48b-4885-9f74-5d5cbd174eb6 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.123132 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da0856-c48b-4885-9f74-5d5cbd174eb6 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.124628 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 667/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:42.130857 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32660a-3b4b-41e1-8862-d34b53e7e056 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.131474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32660a-3b4b-41e1-8862-d34b53e7e056 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.131474 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32660a-3b4b-41e1-8862-d34b53e7e056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.211817 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-9a32660a-3b4b-41e1-8862-d34b53e7e056 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:42.213172 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9a32660a-3b4b-41e1-8862-d34b53e7e056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:42.234136 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:42.323350 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 678/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:42.361942 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da0856-c48b-4885-9f74-5d5cbd174eb6 None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:42.363603 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-e1da0856-c48b-4885-9f74-5d5cbd174eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:42.492361 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da0856-c48b-4885-9f74-5d5cbd174eb6 None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:42.575784 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1469d47-8be1-40f6-9ff7-2a7706f37f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:42.576308 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469d47-8be1-40f6-9ff7-2a7706f37f2f service neutron] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.576591 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469d47-8be1-40f6-9ff7-2a7706f37f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.576877 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469d47-8be1-40f6-9ff7-2a7706f37f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.578638 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-b1469d47-8be1-40f6-9ff7-2a7706f37f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:42.631119 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:42.701194 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1469d47-8be1-40f6-9ff7-2a7706f37f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:42.702717 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1469d47-8be1-40f6-9ff7-2a7706f37f2f service neutron] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:42.723506 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 668/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:47.058047 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca9f3b-6bf1-48c4-8e3b-24100389251d None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:47.058649 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca9f3b-6bf1-48c4-8e3b-24100389251d None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:47.058649 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca9f3b-6bf1-48c4-8e3b-24100389251d None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:47.060064 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 679/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:01:47.065765 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1d86-23d0-4ea9-8e09-a0da113286a5 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:47.066686 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1d86-23d0-4ea9-8e09-a0da113286a5 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:47.067155 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1d86-23d0-4ea9-8e09-a0da113286a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:47.143444 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-e2fb1d86-23d0-4ea9-8e09-a0da113286a5 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:47.144798 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-e2fb1d86-23d0-4ea9-8e09-a0da113286a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:47.245153 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:47.248944 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 669/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:47.287948 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca9f3b-6bf1-48c4-8e3b-24100389251d None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:47.289522 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-32ca9f3b-6bf1-48c4-8e3b-24100389251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:47.402955 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca9f3b-6bf1-48c4-8e3b-24100389251d None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:47.453922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:47.481607 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6e918-a7d2-4df8-883e-af05ee1aa3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8958e3b5dd46bb9e5c835214654ca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:47.482285 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e918-a7d2-4df8-883e-af05ee1aa3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:47.482568 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e918-a7d2-4df8-883e-af05ee1aa3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:47.482843 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e918-a7d2-4df8-883e-af05ee1aa3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:47.484466 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-0ca6e918-a7d2-4df8-883e-af05ee1aa3a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:47.601092 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6e918-a7d2-4df8-883e-af05ee1aa3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:47.602523 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6e918-a7d2-4df8-883e-af05ee1aa3a2 service neutron] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:47.621098 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 680/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:01:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:01:52.741821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a8a58-fd07-461e-879f-4902ee3436d7 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:52.742415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a8a58-fd07-461e-879f-4902ee3436d7 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:52.742415 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a8a58-fd07-461e-879f-4902ee3436d7 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:52.743976 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 670/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 09:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:52.749786 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb73b-9e3e-4d7c-9891-755105ab70be None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:52.750272 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb73b-9e3e-4d7c-9891-755105ab70be None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:52.751412 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb73b-9e3e-4d7c-9891-755105ab70be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:52.853961 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-ebcfb73b-9e3e-4d7c-9891-755105ab70be None None] MFA Rules not processed for user `9ecd4b2f7b4c49c895aba69111e604d8`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:52.855421 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-ebcfb73b-9e3e-4d7c-9891-755105ab70be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:52.881335 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:53.017079 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 681/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 09:01:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:57.650368 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4786b7-11b3-44c9-af8c-f52c5d984f35 None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:57.650961 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4786b7-11b3-44c9-af8c-f52c5d984f35 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:57.651251 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4786b7-11b3-44c9-af8c-f52c5d984f35 None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:57.652748 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 671/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 09:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:57.658801 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55917-75ce-4805-9c33-41ea8e704c70 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:57.659373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55917-75ce-4805-9c33-41ea8e704c70 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:57.659373 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55917-75ce-4805-9c33-41ea8e704c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:57.745847 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-52c55917-75ce-4805-9c33-41ea8e704c70 None None] MFA Rules not processed for user `9ecd4b2f7b4c49c895aba69111e604d8`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:57.747531 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-52c55917-75ce-4805-9c33-41ea8e704c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:57.826480 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:57.866065 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 682/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 09:01:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:01:59.282785 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18907db8-8724-4e1f-9b52-5b683e9103ec None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:59.283310 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18907db8-8724-4e1f-9b52-5b683e9103ec None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:59.283559 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18907db8-8724-4e1f-9b52-5b683e9103ec None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:59.285146 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 672/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 09:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:59.290440 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdd389-5040-4487-9852-da8b87e1c473 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:59.290877 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdd389-5040-4487-9852-da8b87e1c473 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:59.290971 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdd389-5040-4487-9852-da8b87e1c473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:59.371199 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-70cdd389-5040-4487-9852-da8b87e1c473 None None] MFA Rules not processed for user `9ecd4b2f7b4c49c895aba69111e604d8`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:59.372446 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-70cdd389-5040-4487-9852-da8b87e1c473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:59.492677 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 683/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 09:01:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:02:02.812915 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2ca3d-9a10-4d08-9a9c-b7d4164cff3a None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:02.813463 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2ca3d-9a10-4d08-9a9c-b7d4164cff3a None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:02.813463 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2ca3d-9a10-4d08-9a9c-b7d4164cff3a None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:02.814823 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 673/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:02:02.820746 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729c720-84e5-40ba-9f56-003a6fe33658 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:02.821165 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729c720-84e5-40ba-9f56-003a6fe33658 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:02.821277 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729c720-84e5-40ba-9f56-003a6fe33658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:02.859513 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:02.893713 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-2729c720-84e5-40ba-9f56-003a6fe33658 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:02:02.895001 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-2729c720-84e5-40ba-9f56-003a6fe33658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:03.009479 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 684/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:02:03.084556 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121f0a-cec7-4884-9d5c-12e6b0bcddba None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:03.084837 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121f0a-cec7-4884-9d5c-12e6b0bcddba None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:03.085294 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121f0a-cec7-4884-9d5c-12e6b0bcddba None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:03.087162 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 674/1358] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 09:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:02:03.094633 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fa22-146e-4ae6-b601-b1c90e0f8207 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:03.094950 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fa22-146e-4ae6-b601-b1c90e0f8207 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:03.095275 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fa22-146e-4ae6-b601-b1c90e0f8207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:03.202241 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-9df3fa22-146e-4ae6-b601-b1c90e0f8207 None None] MFA Rules not processed for user `4fe070acb9a84d90baaa3b08a4fd3191`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:02:03.203426 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-9df3fa22-146e-4ae6-b601-b1c90e0f8207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:03.257664 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:03.332737 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 685/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 09:02:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:02:03.340972 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87121f0a-cec7-4884-9d5c-12e6b0bcddba None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:02:03.342677 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-87121f0a-cec7-4884-9d5c-12e6b0bcddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:03.517913 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87121f0a-cec7-4884-9d5c-12e6b0bcddba None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:02:03.600103 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:03.645119 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f0326-4624-4063-a7d3-adbef3e93c83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fe070acb9a84d90baaa3b08a4fd3191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:02:03.645945 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f0326-4624-4063-a7d3-adbef3e93c83 service nova] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:03.646309 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f0326-4624-4063-a7d3-adbef3e93c83 service nova] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:03.646663 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f0326-4624-4063-a7d3-adbef3e93c83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:03.648535 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-604f0326-4624-4063-a7d3-adbef3e93c83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:03.749076 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f0326-4624-4063-a7d3-adbef3e93c83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:02:03.751204 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f0326-4624-4063-a7d3-adbef3e93c83 service nova] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:02:03.772340 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 675/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:02:06.001453 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4820d9a-48e3-4e1a-b4cb-b13f81a54238 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:06.002308 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4820d9a-48e3-4e1a-b4cb-b13f81a54238 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:06.002308 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4820d9a-48e3-4e1a-b4cb-b13f81a54238 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:06.004164 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 686/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:02:06.010064 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f4591-9698-43b1-87bf-fda292369583 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:06.010821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f4591-9698-43b1-87bf-fda292369583 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:06.010821 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f4591-9698-43b1-87bf-fda292369583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:06.098130 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-7a7f4591-9698-43b1-87bf-fda292369583 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:02:06.099819 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-7a7f4591-9698-43b1-87bf-fda292369583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:06.147989 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:06.245732 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 676/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:02:06.288445 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4820d9a-48e3-4e1a-b4cb-b13f81a54238 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:02:06.290087 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-d4820d9a-48e3-4e1a-b4cb-b13f81a54238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:06.471457 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4820d9a-48e3-4e1a-b4cb-b13f81a54238 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:02:06.482210 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:06.631875 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bcd20-0db6-4df3-8b63-af7a52de9d61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fe070acb9a84d90baaa3b08a4fd3191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:02:06.632779 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bcd20-0db6-4df3-8b63-af7a52de9d61 service nova] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:06.633209 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bcd20-0db6-4df3-8b63-af7a52de9d61 service nova] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:06.634053 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bcd20-0db6-4df3-8b63-af7a52de9d61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:06.637062 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-908bcd20-0db6-4df3-8b63-af7a52de9d61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:06.766100 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bcd20-0db6-4df3-8b63-af7a52de9d61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:02:06.769549 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bcd20-0db6-4df3-8b63-af7a52de9d61 service nova] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:02:06.792019 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 687/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:02:09.176209 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f626-5370-465e-a4e7-08757cff512d None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:09.176741 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f626-5370-465e-a4e7-08757cff512d None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:09.176831 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f626-5370-465e-a4e7-08757cff512d None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:09.178249 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 677/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:02:09.183922 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec836ea-4d43-4979-b2e5-b8d13636a8d7 None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:09.184461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec836ea-4d43-4979-b2e5-b8d13636a8d7 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:09.184461 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec836ea-4d43-4979-b2e5-b8d13636a8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:09.202084 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:09.269852 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-2ec836ea-4d43-4979-b2e5-b8d13636a8d7 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:02:09.271255 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-2ec836ea-4d43-4979-b2e5-b8d13636a8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:09.391519 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 688/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:02:09.437807 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb2b35-bb11-4fd6-8b8c-7c2b3907c26f None None] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:09.438162 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb2b35-bb11-4fd6-8b8c-7c2b3907c26f None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:09.438437 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb2b35-bb11-4fd6-8b8c-7c2b3907c26f None None] PATH_INFO: `/` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:09.440099 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 678/1366] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 09:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:02:09.445932 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de026fb-e28d-4dc9-93bb-200bb992407c None None] REQUEST_METHOD: `POST` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:09.446311 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de026fb-e28d-4dc9-93bb-200bb992407c None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:09.446611 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de026fb-e28d-4dc9-93bb-200bb992407c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:09.530343 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.auth.core [None req-6de026fb-e28d-4dc9-93bb-200bb992407c None None] MFA Rules not processed for user `4fe070acb9a84d90baaa3b08a4fd3191`. Rule list: `[]` (Enabled: `True`). {{(pid=347146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:02:09.531615 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-6de026fb-e28d-4dc9-93bb-200bb992407c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:09.572423 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:09.668621 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 689/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 09:02:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:02:09.676707 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb2b35-bb11-4fd6-8b8c-7c2b3907c26f None None] Authenticating user token {{(pid=347147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:02:09.678371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-4deb2b35-bb11-4fd6-8b8c-7c2b3907c26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:09.846751 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb2b35-bb11-4fd6-8b8c-7c2b3907c26f None None] Validating token access rules against request {{(pid=347147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:02:09.869286 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:09.972245 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81650826-d936-45be-8fc4-aed0ecadd5b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fe070acb9a84d90baaa3b08a4fd3191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:02:09.972827 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650826-d936-45be-8fc4-aed0ecadd5b5 service nova] REQUEST_METHOD: `GET` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:09.973123 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650826-d936-45be-8fc4-aed0ecadd5b5 service nova] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:09.973397 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650826-d936-45be-8fc4-aed0ecadd5b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:09.975179 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-81650826-d936-45be-8fc4-aed0ecadd5b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:10.083533 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81650826-d936-45be-8fc4-aed0ecadd5b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:02:10.085205 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81650826-d936-45be-8fc4-aed0ecadd5b5 service nova] RBAC: Authorization granted {{(pid=347147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:02:10.106305 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 679/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:02:11.076284 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6446a2a-a54e-4afb-99a3-900adc2e0216 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:11.076841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6446a2a-a54e-4afb-99a3-900adc2e0216 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:11.076841 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6446a2a-a54e-4afb-99a3-900adc2e0216 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:11.078561 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 690/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:02:11.083740 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7454a53-f6e4-46a6-8522-fe1a459e2717 None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:11.084181 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7454a53-f6e4-46a6-8522-fe1a459e2717 None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:11.084285 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7454a53-f6e4-46a6-8522-fe1a459e2717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:11.119277 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:11.163216 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-e7454a53-f6e4-46a6-8522-fe1a459e2717 None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:02:11.166372 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-e7454a53-f6e4-46a6-8522-fe1a459e2717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:11.290142 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 680/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:02:12.204314 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c6e5-137b-4143-a53f-7f01e5b96c60 None None] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:12.204833 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c6e5-137b-4143-a53f-7f01e5b96c60 None None] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:12.205214 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c6e5-137b-4143-a53f-7f01e5b96c60 None None] PATH_INFO: `/` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:12.208177 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 691/1371] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:02:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:02:12.215366 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd2df-c911-453e-a6e5-3e2f01daafac None None] REQUEST_METHOD: `POST` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:12.215366 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd2df-c911-453e-a6e5-3e2f01daafac None None] SCRIPT_NAME: `/identity` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:12.215366 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd2df-c911-453e-a6e5-3e2f01daafac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=347147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:12.310076 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.auth.core [None req-26fdd2df-c911-453e-a6e5-3e2f01daafac None None] MFA Rules not processed for user `27091c013a8c447a81380abde9ae7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=347147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:02:12.311453 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG keystone.common.fernet_utils [None req-26fdd2df-c911-453e-a6e5-3e2f01daafac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:12.387371 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=100 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:12.449317 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: [pid: 347147|app: 0|req: 681/1372] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:02:12.497668 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c6e5-137b-4143-a53f-7f01e5b96c60 None None] Authenticating user token {{(pid=347146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:02:12.499585 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-cbc5c6e5-137b-4143-a53f-7f01e5b96c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:12.674928 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:12.682679 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c6e5-137b-4143-a53f-7f01e5b96c60 None None] Validating token access rules against request {{(pid=347146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:02:12.807469 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a8245-5267-472d-aac3-b48400f85442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fe070acb9a84d90baaa3b08a4fd3191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd917707f724449cbc3cffdb9cd5ca68', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=347146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:02:12.808016 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a8245-5267-472d-aac3-b48400f85442 service nova] REQUEST_METHOD: `GET` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:12.808300 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a8245-5267-472d-aac3-b48400f85442 service nova] SCRIPT_NAME: `/identity` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:12.808578 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a8245-5267-472d-aac3-b48400f85442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=347146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:12.810272 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.fernet_utils [None req-fe7a8245-5267-472d-aac3-b48400f85442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=347146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:02:12.908646 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a8245-5267-472d-aac3-b48400f85442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:02:12.910377 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a8245-5267-472d-aac3-b48400f85442 service nova] RBAC: Authorization granted {{(pid=347146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:02:12.934160 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: [pid: 347146|app: 0|req: 692/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:02:12.934416 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: DEBUG dbcounter [-] [347146] Writing DB stats keystone:SELECT=100 {{(pid=347146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:22.447309 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: DEBUG dbcounter [-] [347147] Writing DB stats keystone:SELECT=19 {{(pid=347147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:33.676808 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: Thu Jul 28 09:02:33 2022 - graceful shutdown triggered... Jul 28 09:02:33.677350 dv-cinder-851351-1-iscsi devstack@keystone.service[347147]: Gracefully killing worker 2 (pid: 347147)... Jul 28 09:02:33.677713 dv-cinder-851351-1-iscsi devstack@keystone.service[347146]: Gracefully killing worker 1 (pid: 347146)... Jul 28 09:02:33.677636 dv-cinder-851351-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Jul 28 09:02:34.678501 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: worker 1 buried after 1 seconds Jul 28 09:02:34.678501 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: worker 2 buried after 1 seconds Jul 28 09:02:34.678501 dv-cinder-851351-1-iscsi devstack@keystone.service[347133]: goodbye to uWSGI. Jul 28 09:02:34.680333 dv-cinder-851351-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Jul 28 09:02:34.680897 dv-cinder-851351-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. Jul 28 09:07:24.768772 dv-cinder-851351-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 09:07:24.783004 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 09:07:24 2022] *** Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: nodename: dv-cinder-851351-1-iscsi Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: machine: x86_64 Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: clock source: unix Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: pcre jit disabled Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: detected number of CPU cores: 4 Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: current working directory: / Jul 28 09:07:24.786226 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: detected binary path: /usr/bin/uwsgi-core Jul 28 09:07:24.787023 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: your processes number limit is 47680 Jul 28 09:07:24.787023 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: your memory page size is 4096 bytes Jul 28 09:07:24.787023 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: detected max file descriptor number: 2048 Jul 28 09:07:24.787023 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: lock engine: pthread robust mutexes Jul 28 09:07:24.787023 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: thunder lock: enabled Jul 28 09:07:24.787023 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 09:07:24.787023 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 09:07:24.829470 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: Python main interpreter initialized at 0x5563a78c29c0 Jul 28 09:07:24.829470 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: python threads support enabled Jul 28 09:07:24.829470 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: your server socket listen backlog is limited to 100 connections Jul 28 09:07:24.829470 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: your mercy for graceful operations on workers is 90 seconds Jul 28 09:07:24.830064 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 09:07:24.830064 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: *** Operational MODE: preforking *** Jul 28 09:07:24.830360 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: *** uWSGI is running in multiple interpreter mode *** Jul 28 09:07:24.830360 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: spawned uWSGI master process (pid: 411097) Jul 28 09:07:24.830488 dv-cinder-851351-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 09:07:24.832020 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: spawned uWSGI worker 1 (pid: 411107, cores: 1) Jul 28 09:07:24.832020 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: spawned uWSGI worker 2 (pid: 411108, cores: 1) Jul 28 09:07:24.832020 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 09:07:30.765857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 09:07:30.765857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: from cryptography.utils import int_from_bytes Jul 28 09:07:30.766845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 09:07:30.766845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: from cryptography.utils import int_from_bytes Jul 28 09:07:30.767314 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 09:07:30.767314 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: from cryptography.utils import int_from_bytes Jul 28 09:07:30.767447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 09:07:30.767447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: from cryptography.utils import int_from_bytes Jul 28 09:07:30.817739 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 09:07:30.817994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 09:07:30.818125 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 09:07:30.818238 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 09:07:30.818323 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] command line args: [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 09:07:30.818400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] command line args: [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 09:07:30.818476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 09:07:30.818555 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 09:07:30.818631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ================================================================================ {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 09:07:30.818706 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ================================================================================ {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 09:07:30.818884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] admin_token = **** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.818984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] admin_token = **** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.819072 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] config_dir = [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.819152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] config_dir = [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.819289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.819489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.819628 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] config_source = [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.819774 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] config_source = [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.819873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] debug = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.819983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] debug = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.820061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.820370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.820370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.820641 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.820641 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] insecure_debug = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.820869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] insecure_debug = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.820954 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821291 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821370 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] list_limit = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821446 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] list_limit = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] log_config_append = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] log_config_append = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821817 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] log_dir = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821891 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] log_dir = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.821975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] log_file = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] log_file = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822171 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822427 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822507 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822585 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822744 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.822928 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823154 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823463 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823624 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823891 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.823982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.824108 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.824239 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.824331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.824495 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.824586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.824682 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.824769 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.824899 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] notification_format = cadf {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.825002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] notification_format = cadf {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.825194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.825730 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.825877 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] publish_errors = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826168 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] publish_errors = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826681 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826901 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.826996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827090 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827226 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] standard_threads = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827427 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] standard_threads = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] strict_password_check = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] strict_password_check = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827682 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827772 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827858 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] use_eventlog = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.827947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] use_eventlog = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828034 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] use_journal = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828125 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] use_journal = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828212 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] use_json = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] use_json = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] use_stderr = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828472 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] use_stderr = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828553 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] use_syslog = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828637 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] use_syslog = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828772 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] watch_log_file = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828871 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] watch_log_file = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:07:30.828959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.829055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.829130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.829297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.829447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.829544 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.829634 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.829807 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.829961 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.830058 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.830244 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.830340 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.830427 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.830519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.830606 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.830785 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.830920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831269 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831840 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.831990 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.832079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.832233 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.832329 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.832480 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.832579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.832729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.832866 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.833001 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.833131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.833256 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.833384 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.833521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.833654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.833785 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.833925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.834055 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.834216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.834353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.834483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.834615 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.834744 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.834873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.835004 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.835140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.835271 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.835395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.835523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.835654 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.835790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.835921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.836045 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.836183 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.836308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.836437 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.836573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.836688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.836794 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.836911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.837024 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.837163 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.837319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.837548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.837685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.837819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.837950 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.838081 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.838257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.838375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.838489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.838596 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.838724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.838855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.839000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.839131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.839262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.839392 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.839522 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.839660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.839800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.839932 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.840063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.840194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.840324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.840453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.840575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.840689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.840804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.840920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.841041 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.841155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.841274 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.841388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.841502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.841615 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.841818 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.841936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842545 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842659 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.842994 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.843112 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.843262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.843380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.843380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.843629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.843629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.843880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.844012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.844136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.844136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.844389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.844519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.844654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.844784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.844912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.845048 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.845172 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.845301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.845437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.845568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.845716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.845847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.845978 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.846148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.846293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.846429 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.846565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.846697 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.846838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.846969 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.847091 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.847225 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.847356 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.847486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.847621 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.847746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.847874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.848004 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.enabled = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.848134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.848264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.848440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.848604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.848745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.enabled = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.848875 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.848998 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.849121 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.849256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.849397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.849528 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.849658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.849787 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.849917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.850054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.850216 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.850344 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.850474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.850605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.850736 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.850873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.851004 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.851134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.851264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.851397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.851538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.851686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.851817 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.851947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.852078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.852078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.852336 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.852475 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.852606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.852736 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.852866 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.852996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.853126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.853264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.853394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.853524 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.853654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.853785 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.853915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.854049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.854290 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.854417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.854546 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.854676 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.854812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.854943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.855072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.855201 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.855325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.855455 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.855593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.855723 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] auth.external = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.855857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.855981 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] auth.mapped = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.856109 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.856239 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.856376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.856507 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.856638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] auth.external = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.856775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] auth.password = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.856912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] auth.mapped = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.857043 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] auth.token = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.857171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.857296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.857457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.857595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] catalog.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.857744 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] auth.password = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.857876 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.858006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] auth.token = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.858147 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.858289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.858421 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.858551 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] catalog.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.858675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.858804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.858939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.859063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.859192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] credential.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.859328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.859459 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] credential.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.859589 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.859721 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.859862 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.859992 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.860121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] credential.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.860250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.860369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] credential.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.860488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.860619 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.860750 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.860884 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.861014 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.861143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.861278 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.861408 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.861545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.861678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.861801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.861924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.862055 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.862218 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.862349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.862473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.862628 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.862766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.862986 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.863119 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.863243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.863243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] federation.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.863481 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.863601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.863735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.863866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] federation.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.863991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.864105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.864219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.864351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.864497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] federation.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.864622 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.864744 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.864873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.864998 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] federation.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.865126 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.865262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.865386 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.865507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.865636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.865766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.865889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.866020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.866208 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.866334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.866463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.866596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.866720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.866854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.866975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.867089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.867219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.867348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.867477 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.867610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.867732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.867872 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.868002 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.868124 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.868256 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.868386 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.868516 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.868640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.868807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.868930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.869066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.869198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.869322 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.869445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.869574 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.869704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.869839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.869839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.870080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.870080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.870340 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.870470 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.870606 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.870730 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.870868 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.870999 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.871129 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.871259 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.871394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.871518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.871640 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.871640 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.871896 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.872018 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.872150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.872281 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.872411 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.872541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.872672 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.872801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.872937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.873071 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.873204 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.873334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.873459 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.873587 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.873722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.873846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.873974 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.874144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.874280 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.874412 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.874545 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.874668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.874790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.874911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.875033 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.875155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.875289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.875413 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.875536 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.875665 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.password = **** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.875794 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.875923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.876061 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.876191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.876315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.876436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.876566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.876689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.876824 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.876948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.877069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.password = **** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.877200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.877329 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.877458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.877594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.877724 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.877724 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.877953 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.878081 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.878219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.878219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.878482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.878482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.878722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.878844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.878972 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.879253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.879410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.879541 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.879672 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.879808 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.879947 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.880079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.880209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.880333 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.880456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.880591 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.880716 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.880845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.880975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.881104 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.881227 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.881360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.881491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.881621 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.881744 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.881873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.881996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.882187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.882324 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.882463 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.882587 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.882710 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.882839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.882975 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.882975 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.883222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.883352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.883482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.883612 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.883749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.883873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.883996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.884132 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.884288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.884422 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.884558 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.884558 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.884804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.884935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.885065 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.885195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.885337 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.885471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.885602 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.885771 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.885906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.886039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.886177 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.886302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.886425 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.886548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] policy.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.886683 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.886809 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.886938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.887069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.887198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.887321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.887321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] receipt.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.887565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.887694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.887818 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.887948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.888079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.888212 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.888336 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] policy.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.888459 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.888580 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.888710 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.888834 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.888966 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] resource.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.889090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.889213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.889344 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] receipt.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.889473 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] resource.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.889596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.889731 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.889855 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.889978 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.890100 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.890281 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.890408 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.890541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] revoke.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.890666 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.890797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.890929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] resource.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.891152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.891289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.891420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] role.cache_time = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.891550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] resource.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.891673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] role.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.891802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.891926 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] role.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.892063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.892195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] role.list_limit = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.892326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.892468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.892599 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] revoke.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.892729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.892863 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.892993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.893123 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.893247 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.893370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] role.cache_time = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.893500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.893642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] role.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.893772 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.893896 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] role.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.894026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.894164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] role.list_limit = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.894288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.894429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.894555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.894677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.894811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.894946 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.895140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.895274 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.895400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.895530 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.895661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.895800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.895937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.896070 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.896195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.896318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.896448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.896579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.896709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.896842 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.896967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.897093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.897235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.897366 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.897497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.897630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.897755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.897878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.898007 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.898198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.898347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.898347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.898590 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.898714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.898843 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.898974 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.899107 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.899232 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.899356 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.899480 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.899610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.899740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.899908 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.900064 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.900187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.900311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.900440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.900570 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] token.cache_time = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.900708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.900833 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] token.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.900973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.901104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.901228 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.901351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] token.provider = fernet {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.901486 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.901611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.901735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.901864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.901995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.902134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.902260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.902379 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.902500 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.902623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.902753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] token.cache_time = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.902876 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.903051 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] token.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.903182 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] trust.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.903306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.903428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.903550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] token.provider = fernet {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.903673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.903806 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.903938 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904314 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904443 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904953 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.904953 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.905184 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.905323 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] trust.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.905453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.connection = **** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.905577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.905716 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.905841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.905971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.906096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.906286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.906418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.906555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.906555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.906796 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.906925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.907050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.907175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.907307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.907430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.907553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.907674 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.907796 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.connection = **** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.907919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.908055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.908180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.908303 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.908441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.908579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.908703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.908826 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.908955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.909079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.909208 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.909341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.909465 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.909588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.909710 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.909832 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.909961 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910095 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=411107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.910363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:07:30.911444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=411108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 09:07:30.932021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.932273 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.932463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.933702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.933942 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.934206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.941191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.943351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.972486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.972766 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.974185 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:30.974433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.026828 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.027318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.079123 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.079465 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.080010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.080252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.146372 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.146665 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.146775 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.146880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.146973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.147068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.147215 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.147308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.147405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.147531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.147617 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.147701 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.171156 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.171409 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.171516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.171606 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.171692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.171775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.171872 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.171963 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.172054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.172145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.172242 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.172332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.172460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.172562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.183913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.184193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.197460 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.197749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.197862 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.197952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.198045 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.198245 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.198441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.198441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.198668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.198765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.198857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.198941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=411107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.199050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.199050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=411108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:07:31.217359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.217603 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 09:07:31.217900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.218196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 09:07:31.218330 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.218428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.218611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 09:07:31.218709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.218814 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.218901 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.219040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 09:07:31.219137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.219272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.219460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.219725 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.219986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.228923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.229204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.229476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.229760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.229901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.230215 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.230496 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.230787 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.236811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.237861 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.241836 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.242145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.242427 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.242708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.242958 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.243068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.243240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.243347 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.243509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.243823 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.243970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.244115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.244275 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.244436 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.244590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.244746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.244889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.244983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.244983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.245164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.245441 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.245713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.268037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.268901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.269540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.270474 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.270762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.274455 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.274735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.275057 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.275494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.275785 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.276074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.281735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.281951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.282762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.283052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.286149 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.286443 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.286725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.287004 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.287293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.287516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.287662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.287797 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.287939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.288077 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.288215 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.288344 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.288642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.288944 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.289219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.289498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.304069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.304343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.305609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.305888 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.308316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.309925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.311484 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.311775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.313138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.313439 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.316212 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.318283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.319782 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.321494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.323110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.323409 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.323693 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.323980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.324263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.324560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.324889 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.325194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.325493 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.325786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.326071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.326404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.340542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.340827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.341090 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.341352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.342511 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.342820 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.343096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.343367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.348276 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.349822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.350086 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.350382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.351973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.352247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.352957 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.353283 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.353595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.353872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.354179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.355224 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.355513 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.355825 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.356128 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.356422 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.365649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.365929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.366238 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.366507 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.366799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.367072 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.367334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.368011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.368286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.368590 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.368918 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.369204 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.369485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.369758 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.382916 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.385568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.386342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.388704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.389298 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.391766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.392015 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.393521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.393823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.394161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.394523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.396087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.396380 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.396668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.400694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.402358 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.403281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.404923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.405265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.405555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.405845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.406157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.407909 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.408201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.408493 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.408784 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.414227 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.414525 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.414816 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.417049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.417356 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.417659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.421752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=411107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.422007 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.422320 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.422601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.422910 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.423209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.423457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.423734 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.424135 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.424415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.424645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=411108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:07:31.424838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.424962 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.425052 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.425216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.425404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.425576 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=411107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.425746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.425746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.426065 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.426369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.426649 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.427031 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.427281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.427595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.427859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.428137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.428406 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=411108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:07:31.462591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5563a78c29c0 pid: 411107 (default app) Jul 28 09:07:31.463878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5563a78c29c0 pid: 411108 (default app) Jul 28 09:07:31.466203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068538a-ad51-456a-846d-b8d51ac91f2c None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:31.466484 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068538a-ad51-456a-846d-b8d51ac91f2c None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:31.466770 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068538a-ad51-456a-846d-b8d51ac91f2c None None] PATH_INFO: `/v3` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:31.471159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jul 28 09:07:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:39.258985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de17b-3272-43d6-b19e-ca0d04228d09 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:39.259596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de17b-3272-43d6-b19e-ca0d04228d09 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:39.259596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de17b-3272-43d6-b19e-ca0d04228d09 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:39.264170 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:07:39.269777 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:39.270254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:39.270367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:39.342173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: INFO dbcounter [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] Registered counter for database keystone Jul 28 09:07:39.364692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG oslo_db.sqlalchemy.engines [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=411107) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 09:07:39.372567 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writer thread running {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 09:07:39.556745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG passlib.handlers.bcrypt [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=411107) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 09:07:39.557135 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG passlib.handlers.bcrypt [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=411107) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 09:07:39.612994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:39.614678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:39.754643 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 09:07:39.778997 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15de17b-3272-43d6-b19e-ca0d04228d09 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:39.859813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: INFO dbcounter [None req-c15de17b-3272-43d6-b19e-ca0d04228d09 None None] Registered counter for database keystone Jul 28 09:07:39.881089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG oslo_db.sqlalchemy.engines [None req-c15de17b-3272-43d6-b19e-ca0d04228d09 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=411108) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 09:07:39.889146 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writer thread running {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 09:07:40.012318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15de17b-3272-43d6-b19e-ca0d04228d09 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:40.017155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa763820-aef4-4492-9d40-1dfc6c52b92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2203339adc34a0499886c3bb35876aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:40.017836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa763820-aef4-4492-9d40-1dfc6c52b92a admin admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:40.018143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa763820-aef4-4492-9d40-1dfc6c52b92a admin admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:40.018421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa763820-aef4-4492-9d40-1dfc6c52b92a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:40.026930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa763820-aef4-4492-9d40-1dfc6c52b92a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:40.078950 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fa763820-aef4-4492-9d40-1dfc6c52b92a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 09:07:40.079306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa763820-aef4-4492-9d40-1dfc6c52b92a admin admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:40.090701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 09:07:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:07:40.098058 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:40.114362 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dc4e3-e853-49d3-9c1f-82fea45bba9c None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:40.120206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7d785-d068-452c-b48b-73b7386d51a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2203339adc34a0499886c3bb35876aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:40.121373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7d785-d068-452c-b48b-73b7386d51a7 admin admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:40.121662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7d785-d068-452c-b48b-73b7386d51a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:40.121946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7d785-d068-452c-b48b-73b7386d51a7 admin admin] PATH_INFO: `/v3/projects/b2203339adc34a0499886c3bb35876aa` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:40.123659 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7d785-d068-452c-b48b-73b7386d51a7 admin admin] RBAC: Authorizing `identity:get_project(project_id=b2203339adc34a0499886c3bb35876aa)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:40.173777 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fed7d785-d068-452c-b48b-73b7386d51a7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 09:07:40.174155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7d785-d068-452c-b48b-73b7386d51a7 admin admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:40.179038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jul 28 09:07:40 2022] GET /identity/v3/projects/b2203339adc34a0499886c3bb35876aa => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:42.187173 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f0d17-c48c-44ab-b3db-c3761549fb35 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:42.187783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f0d17-c48c-44ab-b3db-c3761549fb35 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:42.187783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f0d17-c48c-44ab-b3db-c3761549fb35 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:42.192401 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:42.197921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389fdda-812b-458d-8f9b-537dc0de2d7a None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:42.198413 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389fdda-812b-458d-8f9b-537dc0de2d7a None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:42.198519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389fdda-812b-458d-8f9b-537dc0de2d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:42.220451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-8389fdda-812b-458d-8f9b-537dc0de2d7a None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:42.222056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-8389fdda-812b-458d-8f9b-537dc0de2d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:42.231659 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 09:07:42.254063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f0d17-c48c-44ab-b3db-c3761549fb35 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:42.268657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f0d17-c48c-44ab-b3db-c3761549fb35 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:42.273457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d828e1b-7591-437e-9e4b-c86980bb6a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2203339adc34a0499886c3bb35876aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:42.274153 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d828e1b-7591-437e-9e4b-c86980bb6a10 admin admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:42.274429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d828e1b-7591-437e-9e4b-c86980bb6a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:42.274756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d828e1b-7591-437e-9e4b-c86980bb6a10 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:42.283579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d828e1b-7591-437e-9e4b-c86980bb6a10 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:42.285410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d828e1b-7591-437e-9e4b-c86980bb6a10 admin admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:42.294441 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 09:07:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:07:42.300561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389fdda-812b-458d-8f9b-537dc0de2d7a None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:42.310303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389fdda-812b-458d-8f9b-537dc0de2d7a None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:42.315288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f4da0-c854-4bd4-9963-9bb8ac5c4534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2203339adc34a0499886c3bb35876aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:42.315955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f4da0-c854-4bd4-9963-9bb8ac5c4534 admin admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:42.316245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f4da0-c854-4bd4-9963-9bb8ac5c4534 admin admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:42.316528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f4da0-c854-4bd4-9963-9bb8ac5c4534 admin admin] PATH_INFO: `/v3/users/65605145c2d84cce8746bb3e047916a8` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:42.318375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f4da0-c854-4bd4-9963-9bb8ac5c4534 admin admin] RBAC: Authorizing `identity:get_user(user_id=65605145c2d84cce8746bb3e047916a8)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:42.320250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f4da0-c854-4bd4-9963-9bb8ac5c4534 admin admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:42.321865 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jul 28 09:07:42 2022] GET /identity/v3/users/65605145c2d84cce8746bb3e047916a8 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:44.401746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b28b3-d3ec-4c2a-ad35-3181a4c73c24 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:44.402383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b28b3-d3ec-4c2a-ad35-3181a4c73c24 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:44.402383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b28b3-d3ec-4c2a-ad35-3181a4c73c24 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:44.403713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:44.409348 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddb90-bbe5-4cba-8684-50214ada223e None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:44.409830 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddb90-bbe5-4cba-8684-50214ada223e None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:44.409943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddb90-bbe5-4cba-8684-50214ada223e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:44.425658 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2fde3-11c1-4458-84b6-e2ab1192f36e None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:44.425928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2fde3-11c1-4458-84b6-e2ab1192f36e None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:44.426273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2fde3-11c1-4458-84b6-e2ab1192f36e None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:44.427591 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:07:44.430140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-0e5ddb90-bbe5-4cba-8684-50214ada223e None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:44.431473 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-0e5ddb90-bbe5-4cba-8684-50214ada223e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:44.432982 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:44.433249 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:44.433542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:44.475880 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG passlib.handlers.bcrypt [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=411108) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 09:07:44.476199 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG passlib.handlers.bcrypt [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=411108) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 09:07:44.521200 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:44.522861 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:44.525314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:44.549181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ddb90-bbe5-4cba-8684-50214ada223e None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:44.580427 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:44.627915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:07:44.649706 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:44.666978 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ddb90-bbe5-4cba-8684-50214ada223e None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:44.702255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb492f-1d87-426f-a9e9-74611878967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:44.702906 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb492f-1d87-426f-a9e9-74611878967e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:44.703197 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb492f-1d87-426f-a9e9-74611878967e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:44.703482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb492f-1d87-426f-a9e9-74611878967e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:44.715171 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb492f-1d87-426f-a9e9-74611878967e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-32eb492f-1d87-426f-a9e9-74611878967e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:07:44.724353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:44.725469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:07:44.726456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:07:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:07:44.732082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb492f-1d87-426f-a9e9-74611878967e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:44.757045 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dd9c1-c0ea-4783-a307-130106fae768 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:44.760561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:44.797223 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f9676-fd58-472d-af18-bffe1ff37bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:44.798144 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f9676-fd58-472d-af18-bffe1ff37bc8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:44.798445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f9676-fd58-472d-af18-bffe1ff37bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:44.798722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f9676-fd58-472d-af18-bffe1ff37bc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:44.818520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:44.841052 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb492f-1d87-426f-a9e9-74611878967e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:44.869996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f9676-fd58-472d-af18-bffe1ff37bc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:44.871619 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f9676-fd58-472d-af18-bffe1ff37bc8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:44.875403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:44.876199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:44.876484 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:44.876761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:44.877506 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:44.877817 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:44.879400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:44.892767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:07:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:44.898721 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:44.982912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 09:07:44.989155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f9676-fd58-472d-af18-bffe1ff37bc8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:44.992735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:45.103981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f9676-fd58-472d-af18-bffe1ff37bc8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:45.119716 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66398c-1398-4f2d-b9d2-6380bab0087c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:45.140330 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:45.141220 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:45.141507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:45.141784 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:45.154535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c8c1c-ecbb-4990-a048-a280dc6d420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:45.155204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8c1c-ecbb-4990-a048-a280dc6d420b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:45.155493 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8c1c-ecbb-4990-a048-a280dc6d420b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:45.155775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8c1c-ecbb-4990-a048-a280dc6d420b None admin] PATH_INFO: `/v3/users/65605145c2d84cce8746bb3e047916a8` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:45.156992 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c8c1c-ecbb-4990-a048-a280dc6d420b None admin] RBAC: Authorizing `identity:get_user(user_id=65605145c2d84cce8746bb3e047916a8)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:45.158514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c8c1c-ecbb-4990-a048-a280dc6d420b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:45.160157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:07:44 2022] GET /identity/v3/users/65605145c2d84cce8746bb3e047916a8 => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:45.160601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:45.162129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:45.165808 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c8c1c-ecbb-4990-a048-a280dc6d420b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:45.180620 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:07:45.181235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:45.182304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:45.183219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:07:45.183219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:07:45.183219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:07:45.183219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:07:45.183219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:07:45.188821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:45.254757 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:45.271882 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c8c1c-ecbb-4990-a048-a280dc6d420b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:45.304243 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54b48b-af68-4c24-8800-f0e5aedff15d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:45.307240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c6818-6b6a-4c29-9581-0dbb7ecb76df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:45.307963 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c6818-6b6a-4c29-9581-0dbb7ecb76df None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:45.308243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c6818-6b6a-4c29-9581-0dbb7ecb76df None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:45.308524 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c6818-6b6a-4c29-9581-0dbb7ecb76df None admin] PATH_INFO: `/v3/domains/default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:45.309749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c6818-6b6a-4c29-9581-0dbb7ecb76df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:45.311267 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c6818-6b6a-4c29-9581-0dbb7ecb76df None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:45.312897 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:45.319281 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c6818-6b6a-4c29-9581-0dbb7ecb76df None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:45.340245 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dba9c-cf28-4bef-805d-82123f67dad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:45.340894 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dba9c-cf28-4bef-805d-82123f67dad5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:45.341184 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dba9c-cf28-4bef-805d-82123f67dad5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:45.341462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dba9c-cf28-4bef-805d-82123f67dad5 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:45.342227 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dba9c-cf28-4bef-805d-82123f67dad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:45.342537 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dba9c-cf28-4bef-805d-82123f67dad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:45.343980 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dba9c-cf28-4bef-805d-82123f67dad5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:45.361931 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:45.411302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:45.435452 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c6818-6b6a-4c29-9581-0dbb7ecb76df None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:45.475245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31921e05-1ceb-442f-98cc-21ab01354cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:45.475909 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e05-1ceb-442f-98cc-21ab01354cc7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:45.476198 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e05-1ceb-442f-98cc-21ab01354cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:45.476483 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e05-1ceb-442f-98cc-21ab01354cc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:45.477400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31921e05-1ceb-442f-98cc-21ab01354cc7 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.domain.id=default, user.id=65605145c2d84cce8746bb3e047916a8) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:45.477763 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31921e05-1ceb-442f-98cc-21ab01354cc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:45.479365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31921e05-1ceb-442f-98cc-21ab01354cc7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:45.487453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 09:07:45 2022] GET /identity/v3/role_assignments?user.id=65605145c2d84cce8746bb3e047916a8&scope.domain.id=default&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 233 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:47.857119 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987058-1b29-4965-b717-9fb0b600a16d None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:47.858291 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987058-1b29-4965-b717-9fb0b600a16d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:47.858799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987058-1b29-4965-b717-9fb0b600a16d None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:47.860817 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:47.867212 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48b639-a7fc-4435-a087-073485e390ae None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:47.867715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48b639-a7fc-4435-a087-073485e390ae None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:47.867822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48b639-a7fc-4435-a087-073485e390ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:47.895144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-1a48b639-a7fc-4435-a087-073485e390ae None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:47.896369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-1a48b639-a7fc-4435-a087-073485e390ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:47.906376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdf607-131f-4f23-95f7-78ddc95858c1 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:47.907030 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdf607-131f-4f23-95f7-78ddc95858c1 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:47.907523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdf607-131f-4f23-95f7-78ddc95858c1 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:47.909138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:07:47.917387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cf532-7ddc-4e9b-9637-705fc606cff2 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:47.918050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cf532-7ddc-4e9b-9637-705fc606cff2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:47.918582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cf532-7ddc-4e9b-9637-705fc606cff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:47.944477 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-560cf532-7ddc-4e9b-9637-705fc606cff2 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:47.946059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-560cf532-7ddc-4e9b-9637-705fc606cff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:48.025429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 13/24] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:07:48.030351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 12/25] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:48.030351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950c44-d9bd-4d7b-a837-4bbebdcc1c88 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:48.030351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950c44-d9bd-4d7b-a837-4bbebdcc1c88 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:48.030351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950c44-d9bd-4d7b-a837-4bbebdcc1c88 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:48.030754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 13/26] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:48.043779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec612-cec8-476f-a987-1dd9d48d8836 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:48.044067 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec612-cec8-476f-a987-1dd9d48d8836 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:48.044353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec612-cec8-476f-a987-1dd9d48d8836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:48.050387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cf532-7ddc-4e9b-9637-705fc606cff2 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:48.079263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-8a2ec612-cec8-476f-a987-1dd9d48d8836 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:48.081667 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-8a2ec612-cec8-476f-a987-1dd9d48d8836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:48.141988 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:48.182094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 14/27] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:48.182977 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ec612-cec8-476f-a987-1dd9d48d8836 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:48.186595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:48.246551 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cf532-7ddc-4e9b-9637-705fc606cff2 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:48.316399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:48.317743 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:48.318921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:48.319358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:48.338039 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:48.344607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:07:48.367057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:48.368248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:48.369389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:07:48.369389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:07:48.369389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:07:48.369389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:07:48.377147 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 14/28] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:07:48.379306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:48.406902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ec612-cec8-476f-a987-1dd9d48d8836 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:48.475720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8909-3e14-4dbb-8eff-2bb9e367c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:48.476738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8909-3e14-4dbb-8eff-2bb9e367c55f None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:48.477205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8909-3e14-4dbb-8eff-2bb9e367c55f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:48.477786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8909-3e14-4dbb-8eff-2bb9e367c55f None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:48.478745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8909-3e14-4dbb-8eff-2bb9e367c55f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:48.480869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8909-3e14-4dbb-8eff-2bb9e367c55f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:48.550359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:48.555578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52a532-0935-4e20-a00d-92ed3118d350 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:48.639622 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 15/29] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:07:48 2022] POST /identity/v3/roles => generated 219 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:07:48.640718 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8909-3e14-4dbb-8eff-2bb9e367c55f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:48.811568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:48.811755 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c203d-00ac-49ab-9d0b-ef92b31c4e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:48.812687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c203d-00ac-49ab-9d0b-ef92b31c4e54 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:48.812687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c203d-00ac-49ab-9d0b-ef92b31c4e54 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:48.812983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c203d-00ac-49ab-9d0b-ef92b31c4e54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:48.845516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c203d-00ac-49ab-9d0b-ef92b31c4e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-dd9c203d-00ac-49ab-9d0b-ef92b31c4e54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:07:48.852532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:48.853634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:07:48.854575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:07:48.865458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 15/30] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:07:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 486 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:07:48.867049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08367c14-7057-4ccc-891d-aca60cfa1f66 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:48.867345 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08367c14-7057-4ccc-891d-aca60cfa1f66 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:48.867627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08367c14-7057-4ccc-891d-aca60cfa1f66 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:48.875207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 16/31] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:48.876654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73447aa5-e4a6-4870-ab2c-2e70cd12883b None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:48.877115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73447aa5-e4a6-4870-ab2c-2e70cd12883b None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:48.877401 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73447aa5-e4a6-4870-ab2c-2e70cd12883b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:48.900899 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8909-3e14-4dbb-8eff-2bb9e367c55f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:48.909701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-73447aa5-e4a6-4870-ab2c-2e70cd12883b None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:48.911383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-73447aa5-e4a6-4870-ab2c-2e70cd12883b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:48.943039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:48.943901 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:48.944197 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:48.944639 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:48.945392 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:48.945709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:48.947385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:48.967804 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 16/32] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:48.968707 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:49.014743 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 17/33] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:49.019117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccb9db-642b-4d94-ad55-6f932a7f44b7 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.019550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccb9db-642b-4d94-ad55-6f932a7f44b7 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.020101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccb9db-642b-4d94-ad55-6f932a7f44b7 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.022888 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 18/34] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:49.023824 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccb9db-642b-4d94-ad55-6f932a7f44b7 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:49.104002 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7049a94-9389-4ffe-ab09-9af2888ea1d7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:49.145513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:49.147486 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:49.162077 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61b226-fed5-46a2-8e8c-8a533604834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:49.162792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61b226-fed5-46a2-8e8c-8a533604834c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.163086 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61b226-fed5-46a2-8e8c-8a533604834c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.163373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61b226-fed5-46a2-8e8c-8a533604834c None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.164216 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61b226-fed5-46a2-8e8c-8a533604834c None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:49.164534 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61b226-fed5-46a2-8e8c-8a533604834c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:49.166090 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61b226-fed5-46a2-8e8c-8a533604834c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:49.169729 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccb9db-642b-4d94-ad55-6f932a7f44b7 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:49.180562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 17/35] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:49.180562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16281a29-8ab4-4fc3-bfd5-571973b10290 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.180562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16281a29-8ab4-4fc3-bfd5-571973b10290 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.180562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16281a29-8ab4-4fc3-bfd5-571973b10290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.199199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-16281a29-8ab4-4fc3-bfd5-571973b10290 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:49.201111 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-16281a29-8ab4-4fc3-bfd5-571973b10290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:49.230382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe855f73-90de-4c09-8877-046cb3a072ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:49.231055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe855f73-90de-4c09-8877-046cb3a072ef None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.231350 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe855f73-90de-4c09-8877-046cb3a072ef None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.231623 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe855f73-90de-4c09-8877-046cb3a072ef None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.232615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe855f73-90de-4c09-8877-046cb3a072ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:49.234145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe855f73-90de-4c09-8877-046cb3a072ef None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:49.290176 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 18/36] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:49.298790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16281a29-8ab4-4fc3-bfd5-571973b10290 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:49.378534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 19/37] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:07:49 2022] POST /identity/v3/projects => generated 301 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:07:49.379604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe855f73-90de-4c09-8877-046cb3a072ef None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:49.427352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16281a29-8ab4-4fc3-bfd5-571973b10290 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:49.449679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:49.473177 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f9a02-a32a-4ab9-9914-1a1761063b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:49.473848 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f9a02-a32a-4ab9-9914-1a1761063b4c None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.474184 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f9a02-a32a-4ab9-9914-1a1761063b4c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.474471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f9a02-a32a-4ab9-9914-1a1761063b4c None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.475261 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f9a02-a32a-4ab9-9914-1a1761063b4c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:49.476969 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f9a02-a32a-4ab9-9914-1a1761063b4c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:49.498084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 19/38] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:07:49 2022] POST /identity/v3/roles => generated 225 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:07:49.498765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f9a02-a32a-4ab9-9914-1a1761063b4c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:49.520330 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe855f73-90de-4c09-8877-046cb3a072ef None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:49.574948 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92f756-d766-40c3-9e44-0067fc092340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:49.575759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92f756-d766-40c3-9e44-0067fc092340 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.576057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92f756-d766-40c3-9e44-0067fc092340 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.576372 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92f756-d766-40c3-9e44-0067fc092340 None admin] PATH_INFO: `/v3/users/65605145c2d84cce8746bb3e047916a8` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.578981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92f756-d766-40c3-9e44-0067fc092340 None admin] RBAC: Authorizing `identity:get_user(user_id=65605145c2d84cce8746bb3e047916a8)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:49.580664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92f756-d766-40c3-9e44-0067fc092340 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:49.582598 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 20/39] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:07:49 2022] GET /identity/v3/users/65605145c2d84cce8746bb3e047916a8 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:49.584043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e02bd3-dac3-4661-8c87-131a13cd103f None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.584329 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e02bd3-dac3-4661-8c87-131a13cd103f None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.584615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e02bd3-dac3-4661-8c87-131a13cd103f None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.586024 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 21/40] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:49.591382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e02bd3-dac3-4661-8c87-131a13cd103f None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:49.609383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:49.667533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f9a02-a32a-4ab9-9914-1a1761063b4c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:49.714789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dd85b-37b4-4b28-9bc5-31198dbeadfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:49.715266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dd85b-37b4-4b28-9bc5-31198dbeadfc None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.715552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dd85b-37b4-4b28-9bc5-31198dbeadfc None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.715838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dd85b-37b4-4b28-9bc5-31198dbeadfc None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.716573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dd85b-37b4-4b28-9bc5-31198dbeadfc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:49.718001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dd85b-37b4-4b28-9bc5-31198dbeadfc None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:49.732213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e02bd3-dac3-4661-8c87-131a13cd103f None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:49.737705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 20/41] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:07:49 2022] POST /identity/v3/roles => generated 223 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:07:49.739308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369c2bb-ac20-45bb-b7a9-57ce55485598 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.739308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369c2bb-ac20-45bb-b7a9-57ce55485598 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.739487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369c2bb-ac20-45bb-b7a9-57ce55485598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.761571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-2369c2bb-ac20-45bb-b7a9-57ce55485598 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:49.762864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-2369c2bb-ac20-45bb-b7a9-57ce55485598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:49.773022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac93ff7-ed34-4b92-af35-6fb7ede7c61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:49.773720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac93ff7-ed34-4b92-af35-6fb7ede7c61c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.774010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac93ff7-ed34-4b92-af35-6fb7ede7c61c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.774325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac93ff7-ed34-4b92-af35-6fb7ede7c61c None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.775864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac93ff7-ed34-4b92-af35-6fb7ede7c61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:49.777381 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac93ff7-ed34-4b92-af35-6fb7ede7c61c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:49.779241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 22/42] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:49.785660 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac93ff7-ed34-4b92-af35-6fb7ede7c61c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:49.805363 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:49.849342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 21/43] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:49.873341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369c2bb-ac20-45bb-b7a9-57ce55485598 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:49.883779 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:49.912937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac93ff7-ed34-4b92-af35-6fb7ede7c61c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:49.954441 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c56dc-6b35-40b2-9db1-c3bada0e7cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:49.955163 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c56dc-6b35-40b2-9db1-c3bada0e7cdf None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:49.955450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c56dc-6b35-40b2-9db1-c3bada0e7cdf None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:49.955737 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c56dc-6b35-40b2-9db1-c3bada0e7cdf None admin] PATH_INFO: `/v3/domains/default/users/65605145c2d84cce8746bb3e047916a8/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:49.958671 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c56dc-6b35-40b2-9db1-c3bada0e7cdf None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=65605145c2d84cce8746bb3e047916a8, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:49.960562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c56dc-6b35-40b2-9db1-c3bada0e7cdf None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:49.977910 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 23/44] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jul 28 09:07:49 2022] PUT /identity/v3/domains/default/users/65605145c2d84cce8746bb3e047916a8/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:07:49.988574 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369c2bb-ac20-45bb-b7a9-57ce55485598 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:50.027361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfa741-49cf-4e01-86bd-3eede4848202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:50.028090 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa741-49cf-4e01-86bd-3eede4848202 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:50.028381 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa741-49cf-4e01-86bd-3eede4848202 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:50.028663 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa741-49cf-4e01-86bd-3eede4848202 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:50.029971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfa741-49cf-4e01-86bd-3eede4848202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:50.031534 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfa741-49cf-4e01-86bd-3eede4848202 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:50.033281 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 22/45] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:50.039732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c56dc-6b35-40b2-9db1-c3bada0e7cdf None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:50.150876 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c56dc-6b35-40b2-9db1-c3bada0e7cdf None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:50.186335 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6d454-8836-4ff7-9ad1-004f133dfcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:50.187022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d454-8836-4ff7-9ad1-004f133dfcdc None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:50.187310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d454-8836-4ff7-9ad1-004f133dfcdc None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:50.187623 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d454-8836-4ff7-9ad1-004f133dfcdc None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:50.188336 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6d454-8836-4ff7-9ad1-004f133dfcdc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:50.189841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6d454-8836-4ff7-9ad1-004f133dfcdc None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:50.196369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:50.224445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:07:50 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:07:52.003220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315b4a0-d0f8-46f0-a09f-3bd9b52ea7b9 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.003791 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315b4a0-d0f8-46f0-a09f-3bd9b52ea7b9 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.003791 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315b4a0-d0f8-46f0-a09f-3bd9b52ea7b9 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.005179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:52.010916 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbef91-a8cb-4fc5-b38b-f65fb2935010 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.011418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbef91-a8cb-4fc5-b38b-f65fb2935010 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.011512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbef91-a8cb-4fc5-b38b-f65fb2935010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.030582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-5ddbef91-a8cb-4fc5-b38b-f65fb2935010 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:52.032122 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-5ddbef91-a8cb-4fc5-b38b-f65fb2935010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:52.112999 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 25/48] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:52.135845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315b4a0-d0f8-46f0-a09f-3bd9b52ea7b9 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:52.175539 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:52.251109 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315b4a0-d0f8-46f0-a09f-3bd9b52ea7b9 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:52.252588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052770-1f87-4a40-982b-20a8b5574290 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.252904 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052770-1f87-4a40-982b-20a8b5574290 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.253190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052770-1f87-4a40-982b-20a8b5574290 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.254776 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 26/49] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:52.260060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed699e-21a2-42c5-911e-ab8eef70da4a None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.260443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed699e-21a2-42c5-911e-ab8eef70da4a None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.260805 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed699e-21a2-42c5-911e-ab8eef70da4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.282922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-14ed699e-21a2-42c5-911e-ab8eef70da4a None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:52.284162 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-14ed699e-21a2-42c5-911e-ab8eef70da4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:52.290056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bced6-aeb4-4e1f-80c7-3d1d17312563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:52.290860 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bced6-aeb4-4e1f-80c7-3d1d17312563 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.291150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bced6-aeb4-4e1f-80c7-3d1d17312563 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.291426 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bced6-aeb4-4e1f-80c7-3d1d17312563 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.302078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bced6-aeb4-4e1f-80c7-3d1d17312563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-6c1bced6-aeb4-4e1f-80c7-3d1d17312563 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:07:52.307181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:52.308157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:07:52.309236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 24/50] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:07:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:07:52.314454 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bced6-aeb4-4e1f-80c7-3d1d17312563 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:52.355699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 27/51] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:52.378425 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed699e-21a2-42c5-911e-ab8eef70da4a None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:52.412205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:52.419506 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bced6-aeb4-4e1f-80c7-3d1d17312563 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:52.433036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:52.455875 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:52.456521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.456808 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.457084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.457822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:52.458123 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:52.459589 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:52.468683 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 25/52] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:07:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:52.474464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:52.489937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed699e-21a2-42c5-911e-ab8eef70da4a None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:52.523661 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4730eb-7e06-4da1-ae27-56ef967dd063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:52.524255 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4730eb-7e06-4da1-ae27-56ef967dd063 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.524652 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4730eb-7e06-4da1-ae27-56ef967dd063 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.524935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4730eb-7e06-4da1-ae27-56ef967dd063 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.526222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4730eb-7e06-4da1-ae27-56ef967dd063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:52.527762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4730eb-7e06-4da1-ae27-56ef967dd063 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:52.529592 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 28/53] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:52.535184 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4730eb-7e06-4da1-ae27-56ef967dd063 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:52.575047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b73a5-e2da-4560-8b41-d26b6985b060 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:52.612409 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125851e-97e1-4f51-8589-cf257ca8c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:52.613079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125851e-97e1-4f51-8589-cf257ca8c478 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.613372 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125851e-97e1-4f51-8589-cf257ca8c478 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.613655 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125851e-97e1-4f51-8589-cf257ca8c478 None admin] PATH_INFO: `/v3/users/65605145c2d84cce8746bb3e047916a8` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.614917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125851e-97e1-4f51-8589-cf257ca8c478 None admin] RBAC: Authorizing `identity:get_user(user_id=65605145c2d84cce8746bb3e047916a8)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:52.616451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125851e-97e1-4f51-8589-cf257ca8c478 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:52.618397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 26/54] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:07:52 2022] GET /identity/v3/users/65605145c2d84cce8746bb3e047916a8 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:52.623809 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125851e-97e1-4f51-8589-cf257ca8c478 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:52.637716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4730eb-7e06-4da1-ae27-56ef967dd063 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:52.649985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:52.655886 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:52.675527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4e38f-5a03-40ca-a888-48eb1fa9b7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:52.676176 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4e38f-5a03-40ca-a888-48eb1fa9b7b2 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.676465 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4e38f-5a03-40ca-a888-48eb1fa9b7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.676746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4e38f-5a03-40ca-a888-48eb1fa9b7b2 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.677443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4e38f-5a03-40ca-a888-48eb1fa9b7b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:52.678956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4e38f-5a03-40ca-a888-48eb1fa9b7b2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:52.708026 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 29/55] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 09:07:52 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:07:52.727742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125851e-97e1-4f51-8589-cf257ca8c478 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:52.768068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b9c22-48fb-4d3d-9095-d450f7166aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:52.768767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9c22-48fb-4d3d-9095-d450f7166aaf None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.769063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9c22-48fb-4d3d-9095-d450f7166aaf None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.769774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9c22-48fb-4d3d-9095-d450f7166aaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.771174 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b9c22-48fb-4d3d-9095-d450f7166aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:52.772701 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b9c22-48fb-4d3d-9095-d450f7166aaf None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:52.774597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 27/56] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:52.780937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4e38f-5a03-40ca-a888-48eb1fa9b7b2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:52.905386 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4e38f-5a03-40ca-a888-48eb1fa9b7b2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:52.939068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84098-10e5-4892-89a8-5bd42f419040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:52.939716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84098-10e5-4892-89a8-5bd42f419040 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:52.940006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84098-10e5-4892-89a8-5bd42f419040 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:52.940286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84098-10e5-4892-89a8-5bd42f419040 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:52.941113 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed84098-10e5-4892-89a8-5bd42f419040 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.domain.id=default, user.id=65605145c2d84cce8746bb3e047916a8) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:52.941424 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed84098-10e5-4892-89a8-5bd42f419040 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:52.943275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed84098-10e5-4892-89a8-5bd42f419040 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:52.962141 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 30/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 09:07:52 2022] GET /identity/v3/role_assignments?user.id=65605145c2d84cce8746bb3e047916a8&scope.domain.id=default&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 530 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:07:54.766385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b2308-8a81-49a6-b2f7-d43d5afb701d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:54.766965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b2308-8a81-49a6-b2f7-d43d5afb701d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:54.766965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b2308-8a81-49a6-b2f7-d43d5afb701d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:54.768347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 28/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:54.775290 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474df5c-9d0b-424d-b068-1888dd63e015 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:54.776078 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474df5c-9d0b-424d-b068-1888dd63e015 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:54.776078 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474df5c-9d0b-424d-b068-1888dd63e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:54.795428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-8474df5c-9d0b-424d-b068-1888dd63e015 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:54.796697 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-8474df5c-9d0b-424d-b068-1888dd63e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:54.826856 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:54.872407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 31/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:54.899892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b2308-8a81-49a6-b2f7-d43d5afb701d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:55.007965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:55.021194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b2308-8a81-49a6-b2f7-d43d5afb701d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:55.062578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529ffea-be23-48a3-b13d-5e627817c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:55.063260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529ffea-be23-48a3-b13d-5e627817c81a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:55.063553 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529ffea-be23-48a3-b13d-5e627817c81a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:55.063837 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529ffea-be23-48a3-b13d-5e627817c81a None admin] PATH_INFO: `/v3/domains/default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:55.065130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529ffea-be23-48a3-b13d-5e627817c81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:55.066707 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529ffea-be23-48a3-b13d-5e627817c81a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:55.068846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 29/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:55.074913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474df5c-9d0b-424d-b068-1888dd63e015 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:55.184000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474df5c-9d0b-424d-b068-1888dd63e015 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:55.220912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db40e90-7797-488d-a344-17cbec7813df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:55.221563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40e90-7797-488d-a344-17cbec7813df None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:55.221851 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40e90-7797-488d-a344-17cbec7813df None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:55.222129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40e90-7797-488d-a344-17cbec7813df None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:55.222865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db40e90-7797-488d-a344-17cbec7813df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:55.224363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db40e90-7797-488d-a344-17cbec7813df None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:55.254474 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 32/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:07:55 2022] POST /identity/v3/users => generated 275 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:07:58.535404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cf178-4aa3-4613-a05b-7dc990fe7371 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:58.536452 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cf178-4aa3-4613-a05b-7dc990fe7371 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:58.536952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cf178-4aa3-4613-a05b-7dc990fe7371 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:58.538604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 30/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:58.540159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b463e-c448-4fee-9490-6bb0216dd6d1 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:58.540563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b463e-c448-4fee-9490-6bb0216dd6d1 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:58.540724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b463e-c448-4fee-9490-6bb0216dd6d1 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:58.542289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 33/63] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:07:58.544200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5857e-e031-484e-ac93-79c289fc8901 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:58.544501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5857e-e031-484e-ac93-79c289fc8901 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:58.544790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5857e-e031-484e-ac93-79c289fc8901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:58.548553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf1aca-a6f4-4465-8ac5-c9b02d948f81 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:58.549078 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf1aca-a6f4-4465-8ac5-c9b02d948f81 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:58.549645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf1aca-a6f4-4465-8ac5-c9b02d948f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:58.578450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-e2bf1aca-a6f4-4465-8ac5-c9b02d948f81 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:58.579731 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-d7a5857e-e031-484e-ac93-79c289fc8901 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:58.581206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-d7a5857e-e031-484e-ac93-79c289fc8901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:58.584771 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-e2bf1aca-a6f4-4465-8ac5-c9b02d948f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:58.614730 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:58.677024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 31/64] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:58.707763 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 34/65] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:07:58.712832 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5857e-e031-484e-ac93-79c289fc8901 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:58.750313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf1aca-a6f4-4465-8ac5-c9b02d948f81 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:58.827257 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:58.839035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5857e-e031-484e-ac93-79c289fc8901 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:58.863425 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf1aca-a6f4-4465-8ac5-c9b02d948f81 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:58.877000 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772a41c-fec1-4d7c-abb4-ec4aa477cb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:58.878171 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772a41c-fec1-4d7c-abb4-ec4aa477cb6c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:58.878695 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772a41c-fec1-4d7c-abb4-ec4aa477cb6c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:58.879233 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772a41c-fec1-4d7c-abb4-ec4aa477cb6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:58.890892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772a41c-fec1-4d7c-abb4-ec4aa477cb6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-4772a41c-fec1-4d7c-abb4-ec4aa477cb6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:07:58.896690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:58.897636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:07:58.898532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:07:58.899871 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcda74-2a10-461d-a810-ac3c4ef5b4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:58.900544 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcda74-2a10-461d-a810-ac3c4ef5b4cc None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:58.900829 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcda74-2a10-461d-a810-ac3c4ef5b4cc None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:58.901103 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcda74-2a10-461d-a810-ac3c4ef5b4cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:58.901906 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 32/66] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:07:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:07:58.908435 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa88ab1-a472-4c5b-aff8-f7fee1882024 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:58.909066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa88ab1-a472-4c5b-aff8-f7fee1882024 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:58.909669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa88ab1-a472-4c5b-aff8-f7fee1882024 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:58.911582 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 33/67] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:58.912513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa88ab1-a472-4c5b-aff8-f7fee1882024 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:58.912867 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcda74-2a10-461d-a810-ac3c4ef5b4cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-c9dcda74-2a10-461d-a810-ac3c4ef5b4cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:07:58.918606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:58.919582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:07:58.920435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 35/68] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:07:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:07:58.921498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84dd1c-1809-47a8-b80d-54bc63f97b22 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:58.922003 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84dd1c-1809-47a8-b80d-54bc63f97b22 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:58.927372 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84dd1c-1809-47a8-b80d-54bc63f97b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:58.946872 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-1a84dd1c-1809-47a8-b80d-54bc63f97b22 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:58.948612 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-1a84dd1c-1809-47a8-b80d-54bc63f97b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:58.951131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.023358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 36/69] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:59.024274 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84dd1c-1809-47a8-b80d-54bc63f97b22 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.031747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa88ab1-a472-4c5b-aff8-f7fee1882024 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.072978 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b3108-2f62-4d05-9f4d-0afecca9bc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.073703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b3108-2f62-4d05-9f4d-0afecca9bc2d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.073919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b3108-2f62-4d05-9f4d-0afecca9bc2d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.074231 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b3108-2f62-4d05-9f4d-0afecca9bc2d None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.074964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b3108-2f62-4d05-9f4d-0afecca9bc2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:59.075263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b3108-2f62-4d05-9f4d-0afecca9bc2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:59.076683 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b3108-2f62-4d05-9f4d-0afecca9bc2d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:59.089714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 34/70] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:07:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:59.089714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc4ed1-db6c-4cc1-b2bc-cf579158cb5e None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.089714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc4ed1-db6c-4cc1-b2bc-cf579158cb5e None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.089714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc4ed1-db6c-4cc1-b2bc-cf579158cb5e None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.089714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 35/71] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:59.089714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc4ed1-db6c-4cc1-b2bc-cf579158cb5e None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.108321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.141704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84dd1c-1809-47a8-b80d-54bc63f97b22 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.177853 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.178540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.178849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.179222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.179881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:59.180190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:59.181659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:59.190725 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.198347 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 37/72] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:59.199307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.202267 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc4ed1-db6c-4cc1-b2bc-cf579158cb5e None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.239822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f49fe5-089f-46b0-9e8b-ae373609f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.240490 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f49fe5-089f-46b0-9e8b-ae373609f31b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.240779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f49fe5-089f-46b0-9e8b-ae373609f31b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.241059 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f49fe5-089f-46b0-9e8b-ae373609f31b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.251480 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f49fe5-089f-46b0-9e8b-ae373609f31b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-b3f49fe5-089f-46b0-9e8b-ae373609f31b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:07:59.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:59.258060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:07:59.259134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 36/73] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:07:59.263451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8958-38e9-40e4-8e43-9596c5da0908 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.263719 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8958-38e9-40e4-8e43-9596c5da0908 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.264010 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8958-38e9-40e4-8e43-9596c5da0908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.282270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-05bf8958-38e9-40e4-8e43-9596c5da0908 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:59.283509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-05bf8958-38e9-40e4-8e43-9596c5da0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:59.307987 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f2f4f-15dd-4384-b9d6-9a407c4edc11 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.343630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a54f79-ec10-4368-a21e-65ccabcb0363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.344218 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54f79-ec10-4368-a21e-65ccabcb0363 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.344513 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54f79-ec10-4368-a21e-65ccabcb0363 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.344799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54f79-ec10-4368-a21e-65ccabcb0363 None admin] PATH_INFO: `/v3/users/65605145c2d84cce8746bb3e047916a8` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.346035 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a54f79-ec10-4368-a21e-65ccabcb0363 None admin] RBAC: Authorizing `identity:get_user(user_id=65605145c2d84cce8746bb3e047916a8)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:59.347550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a54f79-ec10-4368-a21e-65ccabcb0363 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:59.349206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 38/74] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/users/65605145c2d84cce8746bb3e047916a8 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:59.350413 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d116c88-b35f-465b-b17e-d9265de41eaf None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.350684 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d116c88-b35f-465b-b17e-d9265de41eaf None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.350972 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d116c88-b35f-465b-b17e-d9265de41eaf None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.352402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 39/75] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:07:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:59.352949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d116c88-b35f-465b-b17e-d9265de41eaf None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.366325 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 37/76] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:59.366325 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf8958-38e9-40e4-8e43-9596c5da0908 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.366325 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.423189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.459152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d116c88-b35f-465b-b17e-d9265de41eaf None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.466872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf8958-38e9-40e4-8e43-9596c5da0908 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.494638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e2ed2-9644-4340-9608-c5c8714537a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.495321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e2ed2-9644-4340-9608-c5c8714537a5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.495638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e2ed2-9644-4340-9608-c5c8714537a5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.495917 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e2ed2-9644-4340-9608-c5c8714537a5 None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.501229 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.501867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.502175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.502455 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.503178 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:59.503469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:59.504904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:59.511773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e2ed2-9644-4340-9608-c5c8714537a5 None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:59.513230 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 38/77] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:59.518294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e2ed2-9644-4340-9608-c5c8714537a5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:59.518397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.518475 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 40/78] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:59.518475 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d91c6-ffc7-4494-8534-4790120477fc None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.518475 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d91c6-ffc7-4494-8534-4790120477fc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.518475 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d91c6-ffc7-4494-8534-4790120477fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.535152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-835d91c6-ffc7-4494-8534-4790120477fc None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:59.536407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-835d91c6-ffc7-4494-8534-4790120477fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:59.582544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.604517 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 41/79] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:07:59.605394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d91c6-ffc7-4494-8534-4790120477fc None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.624848 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a8712-a3ec-4a10-8237-02ee9f39b10c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.659817 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f30a3-12e6-47a0-852b-87db2b885d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.660465 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f30a3-12e6-47a0-852b-87db2b885d7b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.660755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f30a3-12e6-47a0-852b-87db2b885d7b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.661044 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f30a3-12e6-47a0-852b-87db2b885d7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.662295 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.707123 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d91c6-ffc7-4494-8534-4790120477fc None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.731136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f30a3-12e6-47a0-852b-87db2b885d7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:59.732907 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f30a3-12e6-47a0-852b-87db2b885d7b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:59.741964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60615d6e-7548-4b97-bab9-69844ccc4e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.742678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60615d6e-7548-4b97-bab9-69844ccc4e45 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.742989 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60615d6e-7548-4b97-bab9-69844ccc4e45 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.743259 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60615d6e-7548-4b97-bab9-69844ccc4e45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.753937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60615d6e-7548-4b97-bab9-69844ccc4e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-60615d6e-7548-4b97-bab9-69844ccc4e45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:07:59.759326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:07:59.760319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:07:59.761161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 42/80] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:07:59.761880 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60615d6e-7548-4b97-bab9-69844ccc4e45 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.799834 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.844255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 39/81] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:07:59.845142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f30a3-12e6-47a0-852b-87db2b885d7b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.866962 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60615d6e-7548-4b97-bab9-69844ccc4e45 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.885553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:59.904271 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcd803-b10d-4fbe-81e3-21e82e54d392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.904945 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcd803-b10d-4fbe-81e3-21e82e54d392 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.905231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcd803-b10d-4fbe-81e3-21e82e54d392 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.905514 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcd803-b10d-4fbe-81e3-21e82e54d392 None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:59.906750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcd803-b10d-4fbe-81e3-21e82e54d392 None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:59.908289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcd803-b10d-4fbe-81e3-21e82e54d392 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:59.910137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 43/82] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:59.910137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcd803-b10d-4fbe-81e3-21e82e54d392 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:59.953594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f30a3-12e6-47a0-852b-87db2b885d7b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:59.991918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368919e-ff82-4292-ab5a-fff3922f8622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:59.992591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368919e-ff82-4292-ab5a-fff3922f8622 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:59.992889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368919e-ff82-4292-ab5a-fff3922f8622 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:59.993250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368919e-ff82-4292-ab5a-fff3922f8622 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.015710 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcd803-b10d-4fbe-81e3-21e82e54d392 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.025106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:00.051718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a40ea-ea87-4c0c-acc2-41cda589f0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.052402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a40ea-ea87-4c0c-acc2-41cda589f0d5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.052686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a40ea-ea87-4c0c-acc2-41cda589f0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.052968 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a40ea-ea87-4c0c-acc2-41cda589f0d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.054258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a40ea-ea87-4c0c-acc2-41cda589f0d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.055827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a40ea-ea87-4c0c-acc2-41cda589f0d5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.057469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 44/83] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:00.058284 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a40ea-ea87-4c0c-acc2-41cda589f0d5 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.070476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368919e-ff82-4292-ab5a-fff3922f8622 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.071579 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368919e-ff82-4292-ab5a-fff3922f8622 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.116621 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:00.172262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a40ea-ea87-4c0c-acc2-41cda589f0d5 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.180893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 40/84] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:00.181766 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368919e-ff82-4292-ab5a-fff3922f8622 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.215065 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.215638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.215927 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.216207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.216948 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:00.217255 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.218758 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.227243 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 45/85] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:00.228134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.254378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:00.293797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368919e-ff82-4292-ab5a-fff3922f8622 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.330629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e22715-0313-4cf3-906f-851de397f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.331294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e22715-0313-4cf3-906f-851de397f734 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.331578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e22715-0313-4cf3-906f-851de397f734 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.331855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e22715-0313-4cf3-906f-851de397f734 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.333056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e22715-0313-4cf3-906f-851de397f734 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.334576 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e22715-0313-4cf3-906f-851de397f734 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.336299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 41/86] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:00.337134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e22715-0313-4cf3-906f-851de397f734 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.337255 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd7d01-c855-463e-95d7-b795b17d8559 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.352071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:00.378738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e661e-761c-49c2-aed2-56346bd9510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.379394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e661e-761c-49c2-aed2-56346bd9510d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.379690 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e661e-761c-49c2-aed2-56346bd9510d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.379977 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e661e-761c-49c2-aed2-56346bd9510d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.442933 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e22715-0313-4cf3-906f-851de397f734 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.456996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e661e-761c-49c2-aed2-56346bd9510d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.458713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e661e-761c-49c2-aed2-56346bd9510d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.481595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:00.487600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865aab86-6558-4be8-8ecc-15a96b805782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.488252 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865aab86-6558-4be8-8ecc-15a96b805782 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.488540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865aab86-6558-4be8-8ecc-15a96b805782 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.488826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865aab86-6558-4be8-8ecc-15a96b805782 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.489537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865aab86-6558-4be8-8ecc-15a96b805782 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.491058 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865aab86-6558-4be8-8ecc-15a96b805782 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.519284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 42/87] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:00 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:00.520156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865aab86-6558-4be8-8ecc-15a96b805782 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.580253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 46/88] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:00.581461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e661e-761c-49c2-aed2-56346bd9510d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.599697 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:00.650335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865aab86-6558-4be8-8ecc-15a96b805782 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.698217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527a66d-712b-4d2e-9587-6c50187a3ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.699344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527a66d-712b-4d2e-9587-6c50187a3ea6 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.699344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527a66d-712b-4d2e-9587-6c50187a3ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.699535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527a66d-712b-4d2e-9587-6c50187a3ea6 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.700632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527a66d-712b-4d2e-9587-6c50187a3ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.702241 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527a66d-712b-4d2e-9587-6c50187a3ea6 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.703994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 43/89] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:00.706185 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527a66d-712b-4d2e-9587-6c50187a3ea6 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.709780 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e661e-761c-49c2-aed2-56346bd9510d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.758583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab8887-5984-4287-9fe5-353906f84303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.759312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab8887-5984-4287-9fe5-353906f84303 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.759598 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab8887-5984-4287-9fe5-353906f84303 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.759873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab8887-5984-4287-9fe5-353906f84303 None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.761130 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab8887-5984-4287-9fe5-353906f84303 None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.762739 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab8887-5984-4287-9fe5-353906f84303 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.765542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 47/90] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:00.765542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab8887-5984-4287-9fe5-353906f84303 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.780823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:00.846245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527a66d-712b-4d2e-9587-6c50187a3ea6 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.878659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:00.894894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.895859 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.896157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.896448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.897287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=6a427cb96a904a66aca0187281bdad57, user.id=65605145c2d84cce8746bb3e047916a8) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:00.897611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.899155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.905732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 44/91] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/role_assignments?user.id=65605145c2d84cce8746bb3e047916a8&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:00.906475 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:00.907603 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab8887-5984-4287-9fe5-353906f84303 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:00.956530 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033401fd-3155-4f92-bcea-d90bda83a7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:00.957371 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033401fd-3155-4f92-bcea-d90bda83a7c7 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:00.957863 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033401fd-3155-4f92-bcea-d90bda83a7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:00.958368 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033401fd-3155-4f92-bcea-d90bda83a7c7 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:00.960525 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033401fd-3155-4f92-bcea-d90bda83a7c7 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:00.963718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033401fd-3155-4f92-bcea-d90bda83a7c7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:00.965873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 48/92] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:00.966812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033401fd-3155-4f92-bcea-d90bda83a7c7 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:01.088671 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47822f4e-14c8-4fe1-9cb0-53f3544ccd60 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:01.107805 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033401fd-3155-4f92-bcea-d90bda83a7c7 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:01.126481 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:01.144358 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:01.145019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:01.145313 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:01.145600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:01.146487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] RBAC: Adding query filter params (role.id=2935a7836c864df48e32fa734f020882, scope.project.id=6a427cb96a904a66aca0187281bdad57, user.id=a5086738a1284cb6a9c74c681b71749c) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:01.146808 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:01.148342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:01.157492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 45/93] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/role_assignments?user.id=a5086738a1284cb6a9c74c681b71749c&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=2935a7836c864df48e32fa734f020882 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:01.158442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:01.160747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0c7ee-2870-46c5-8a24-8c82d2f3abad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:01.162007 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0c7ee-2870-46c5-8a24-8c82d2f3abad None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:01.162540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0c7ee-2870-46c5-8a24-8c82d2f3abad None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:01.162989 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0c7ee-2870-46c5-8a24-8c82d2f3abad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:01.218497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:01.300206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0c7ee-2870-46c5-8a24-8c82d2f3abad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:01.302181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0c7ee-2870-46c5-8a24-8c82d2f3abad None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:01.323747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841c88f-34b2-4570-9686-d6b4c7e8e8fa None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:01.366400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:01.367067 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:01.367356 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:01.367637 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:01.368511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] RBAC: Adding query filter params (role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=6a427cb96a904a66aca0187281bdad57, user.id=a5086738a1284cb6a9c74c681b71749c) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:01.368863 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:01.370364 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:01.382258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 46/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:01 2022] GET /identity/v3/role_assignments?user.id=a5086738a1284cb6a9c74c681b71749c&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=601939cb90d641e08b97775c0c12d374 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:01.430667 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 49/95] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:01.439360 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:01.522636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:01.589612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5ab8d-29f7-442d-815a-17edc0f5715d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:01.635410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1b219-80b1-4c17-ab44-456c6ab36cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:01.636561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b219-80b1-4c17-ab44-456c6ab36cf8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:01.637227 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b219-80b1-4c17-ab44-456c6ab36cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:01.637728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b219-80b1-4c17-ab44-456c6ab36cf8 None admin] PATH_INFO: `/v3/projects/2608c6d855fa478e95db30be9aab107a` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:01.639420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1b219-80b1-4c17-ab44-456c6ab36cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=2608c6d855fa478e95db30be9aab107a)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:01.641181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1b219-80b1-4c17-ab44-456c6ab36cf8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:01.643274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 47/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:01 2022] GET /identity/v3/projects/2608c6d855fa478e95db30be9aab107a => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:01.650341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0c7ee-2870-46c5-8a24-8c82d2f3abad None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:01.710398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:01.778718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0c7ee-2870-46c5-8a24-8c82d2f3abad None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:01.814985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36792ce4-c27f-47da-8ad5-7e1bf20e80af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:01.815644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36792ce4-c27f-47da-8ad5-7e1bf20e80af None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:01.815936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36792ce4-c27f-47da-8ad5-7e1bf20e80af None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:01.816220 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36792ce4-c27f-47da-8ad5-7e1bf20e80af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:01.817402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36792ce4-c27f-47da-8ad5-7e1bf20e80af None admin] RBAC: Adding query filter params (role.id=2935a7836c864df48e32fa734f020882, scope.project.id=2608c6d855fa478e95db30be9aab107a, user.id=a5086738a1284cb6a9c74c681b71749c) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:01.817722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36792ce4-c27f-47da-8ad5-7e1bf20e80af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:01.819231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36792ce4-c27f-47da-8ad5-7e1bf20e80af None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:01.830297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 50/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:01 2022] GET /identity/v3/role_assignments?user.id=a5086738a1284cb6a9c74c681b71749c&scope.project.id=2608c6d855fa478e95db30be9aab107a&role.id=2935a7836c864df48e32fa734f020882 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:03.167470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97208-680a-4970-857e-012a7fadf109 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:03.168547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97208-680a-4970-857e-012a7fadf109 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:03.169048 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97208-680a-4970-857e-012a7fadf109 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:03.170691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 48/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:03.176231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d17d3b-62a6-4e00-be0b-a6d721fadf19 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:03.177087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d17d3b-62a6-4e00-be0b-a6d721fadf19 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:03.177580 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d17d3b-62a6-4e00-be0b-a6d721fadf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:03.198016 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-b5d17d3b-62a6-4e00-be0b-a6d721fadf19 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:03.199626 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-b5d17d3b-62a6-4e00-be0b-a6d721fadf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:03.272795 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 51/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:03.295900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f97208-680a-4970-857e-012a7fadf109 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:03.431274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f97208-680a-4970-857e-012a7fadf109 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:03.454810 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:03.472307 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eefff-a673-4bca-b72c-2069e9d0b9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:03.473339 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eefff-a673-4bca-b72c-2069e9d0b9ac None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:03.473874 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eefff-a673-4bca-b72c-2069e9d0b9ac None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:03.474421 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eefff-a673-4bca-b72c-2069e9d0b9ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:03.485776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eefff-a673-4bca-b72c-2069e9d0b9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-911eefff-a673-4bca-b72c-2069e9d0b9ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:03.491554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:03.492542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:03.493448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:03.495898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 49/100] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:03.502135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d17d3b-62a6-4e00-be0b-a6d721fadf19 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:03.558769 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:03.624548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d17d3b-62a6-4e00-be0b-a6d721fadf19 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:03.671949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206a2b5-12a4-42c5-bf89-8978dc5ee26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:03.672597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a2b5-12a4-42c5-bf89-8978dc5ee26a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:03.672881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a2b5-12a4-42c5-bf89-8978dc5ee26a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:03.673157 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a2b5-12a4-42c5-bf89-8978dc5ee26a None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:03.673910 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206a2b5-12a4-42c5-bf89-8978dc5ee26a None admin] RBAC: Adding query filter params (name=reader) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:03.674245 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206a2b5-12a4-42c5-bf89-8978dc5ee26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:03.675672 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206a2b5-12a4-42c5-bf89-8978dc5ee26a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:03.693643 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 52/101] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:03.700527 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eefff-a673-4bca-b72c-2069e9d0b9ac None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:03.828001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eefff-a673-4bca-b72c-2069e9d0b9ac None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:03.863538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645ebfc-0316-47e4-9f35-db9caae04205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:03.864211 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645ebfc-0316-47e4-9f35-db9caae04205 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:03.864496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645ebfc-0316-47e4-9f35-db9caae04205 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:03.864776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645ebfc-0316-47e4-9f35-db9caae04205 None admin] PATH_INFO: `/v3/users/44969c9e86984ff38dbc5d664aa8ade2` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:03.869428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6adbf-c62f-4488-ab4a-65629d0125e8 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:03.869713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6adbf-c62f-4488-ab4a-65629d0125e8 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:03.869955 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6adbf-c62f-4488-ab4a-65629d0125e8 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:03.871361 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 53/102] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:03.877053 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7d10e-f570-44b0-82b0-8a4b5aab8391 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:03.877321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7d10e-f570-44b0-82b0-8a4b5aab8391 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:03.877606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7d10e-f570-44b0-82b0-8a4b5aab8391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:03.901923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645ebfc-0316-47e4-9f35-db9caae04205 None admin] RBAC: Authorizing `identity:get_user(user_id=44969c9e86984ff38dbc5d664aa8ade2)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:03.903465 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645ebfc-0316-47e4-9f35-db9caae04205 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:03.905247 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 50/103] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:03 2022] GET /identity/v3/users/44969c9e86984ff38dbc5d664aa8ade2 => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:03.909254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-7ee7d10e-f570-44b0-82b0-8a4b5aab8391 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:03.910497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-7ee7d10e-f570-44b0-82b0-8a4b5aab8391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:03.914450 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645ebfc-0316-47e4-9f35-db9caae04205 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:03.982383 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:04.040087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 54/104] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:04.079698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7d10e-f570-44b0-82b0-8a4b5aab8391 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:04.112396 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645ebfc-0316-47e4-9f35-db9caae04205 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:04.143753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:04.202028 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eee862-1c59-499f-a1f8-586018e2ef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:04.202730 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee862-1c59-499f-a1f8-586018e2ef0c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.203190 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee862-1c59-499f-a1f8-586018e2ef0c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.203641 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee862-1c59-499f-a1f8-586018e2ef0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.278006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7d10e-f570-44b0-82b0-8a4b5aab8391 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:04.374421 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eee862-1c59-499f-a1f8-586018e2ef0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:04.376197 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eee862-1c59-499f-a1f8-586018e2ef0c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:04.383882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e85d68-82c5-4fa7-a48c-69e137175890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:04.384839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85d68-82c5-4fa7-a48c-69e137175890 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.385311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85d68-82c5-4fa7-a48c-69e137175890 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.385756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85d68-82c5-4fa7-a48c-69e137175890 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.389005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e85d68-82c5-4fa7-a48c-69e137175890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:04.390731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e85d68-82c5-4fa7-a48c-69e137175890 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:04.394506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 55/105] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:04.397398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752dba-2120-4c33-8d44-e163de01edc2 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.398607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752dba-2120-4c33-8d44-e163de01edc2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.399376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752dba-2120-4c33-8d44-e163de01edc2 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.402503 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 56/106] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:04.404366 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05752dba-2120-4c33-8d44-e163de01edc2 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:04.439344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:04.524085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 51/107] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:04.525552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804af38-9063-434f-9657-a6dc6dace754 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.525813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804af38-9063-434f-9657-a6dc6dace754 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.526126 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804af38-9063-434f-9657-a6dc6dace754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.548042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-1804af38-9063-434f-9657-a6dc6dace754 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:04.549221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-1804af38-9063-434f-9657-a6dc6dace754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:04.577175 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05752dba-2120-4c33-8d44-e163de01edc2 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:04.580103 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:04.616976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f33dc-2d86-4011-a51b-e39f79480f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:04.617636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f33dc-2d86-4011-a51b-e39f79480f5e None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.617992 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f33dc-2d86-4011-a51b-e39f79480f5e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.618358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f33dc-2d86-4011-a51b-e39f79480f5e None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.619083 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f33dc-2d86-4011-a51b-e39f79480f5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:04.620597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f33dc-2d86-4011-a51b-e39f79480f5e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:04.643349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 52/108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:04.643349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e1328-1017-44c2-9d1b-63d2849999e7 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.643349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e1328-1017-44c2-9d1b-63d2849999e7 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.643349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e1328-1017-44c2-9d1b-63d2849999e7 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.643349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 53/109] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:04.643349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e1328-1017-44c2-9d1b-63d2849999e7 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:04.651310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 57/110] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:08:04 2022] POST /identity/v3/projects => generated 302 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:04.652517 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b913-e4f3-4452-be9c-d2d4ef665f6c None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.652870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b913-e4f3-4452-be9c-d2d4ef665f6c None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.653165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b913-e4f3-4452-be9c-d2d4ef665f6c None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.654605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 58/111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:04.660638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77a5ce-4722-4554-86c2-6f76963ae91d None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.660922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77a5ce-4722-4554-86c2-6f76963ae91d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.661226 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77a5ce-4722-4554-86c2-6f76963ae91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.690052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-5d77a5ce-4722-4554-86c2-6f76963ae91d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:04.693237 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-5d77a5ce-4722-4554-86c2-6f76963ae91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:04.728841 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:04.778198 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e1328-1017-44c2-9d1b-63d2849999e7 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:04.808874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 59/112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:04.819718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bad00-1fcd-4d8c-a44e-a78106b5efa7 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.820437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bad00-1fcd-4d8c-a44e-a78106b5efa7 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.820718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bad00-1fcd-4d8c-a44e-a78106b5efa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.827146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c63a2-3dcd-4f05-ac17-38dec8562016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:04.828038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c63a2-3dcd-4f05-ac17-38dec8562016 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:04.828329 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c63a2-3dcd-4f05-ac17-38dec8562016 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:04.829167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c63a2-3dcd-4f05-ac17-38dec8562016 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:04.832599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c63a2-3dcd-4f05-ac17-38dec8562016 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:04.834218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c63a2-3dcd-4f05-ac17-38dec8562016 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:04.836348 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 54/113] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:04 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:04.837164 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c63a2-3dcd-4f05-ac17-38dec8562016 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:04.849749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-897bad00-1fcd-4d8c-a44e-a78106b5efa7 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:04.851485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-897bad00-1fcd-4d8c-a44e-a78106b5efa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:04.935472 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 60/114] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:04.936195 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bad00-1fcd-4d8c-a44e-a78106b5efa7 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:04.961762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:04.972703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c63a2-3dcd-4f05-ac17-38dec8562016 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.010500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:05.013105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff16f0b-7bd7-49e2-a079-9dc0cb9afcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.013816 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff16f0b-7bd7-49e2-a079-9dc0cb9afcc5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.014431 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff16f0b-7bd7-49e2-a079-9dc0cb9afcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.014739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff16f0b-7bd7-49e2-a079-9dc0cb9afcc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.032735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff16f0b-7bd7-49e2-a079-9dc0cb9afcc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-aff16f0b-7bd7-49e2-a079-9dc0cb9afcc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:05.037885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:05.039152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 55/115] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:05.040243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff16f0b-7bd7-49e2-a079-9dc0cb9afcc5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.083675 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bad00-1fcd-4d8c-a44e-a78106b5efa7 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.123909 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd40e98-a9a7-40ee-a4e0-b25a786a5164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.124765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd40e98-a9a7-40ee-a4e0-b25a786a5164 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.125162 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd40e98-a9a7-40ee-a4e0-b25a786a5164 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.125548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd40e98-a9a7-40ee-a4e0-b25a786a5164 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.137859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd40e98-a9a7-40ee-a4e0-b25a786a5164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-2fd40e98-a9a7-40ee-a4e0-b25a786a5164 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:05.143000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:05.143976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:05.144854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 61/116] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:08:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:05.145555 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd40e98-a9a7-40ee-a4e0-b25a786a5164 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.170279 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff16f0b-7bd7-49e2-a079-9dc0cb9afcc5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.210736 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.211405 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.211739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.212030 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.212847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] RBAC: Adding query filter params (role.id=73552b9e126d49b188a201b0e02d1011, scope.project.id=6a427cb96a904a66aca0187281bdad57, user.id=44969c9e86984ff38dbc5d664aa8ade2) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:05.213165 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:05.214684 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:05.221137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 56/117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/role_assignments?user.id=44969c9e86984ff38dbc5d664aa8ade2&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=73552b9e126d49b188a201b0e02d1011 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:05.222085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.247285 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:05.283318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd40e98-a9a7-40ee-a4e0-b25a786a5164 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.293204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:05.341096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe3aa-0588-4cfb-b85e-e7159ba2818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.342524 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe3aa-0588-4cfb-b85e-e7159ba2818a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.343587 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe3aa-0588-4cfb-b85e-e7159ba2818a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.344398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe3aa-0588-4cfb-b85e-e7159ba2818a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.353120 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a820239-4f94-4f3f-a06b-40e1cc8abe2d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.361187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afe3aa-0588-4cfb-b85e-e7159ba2818a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-55afe3aa-0588-4cfb-b85e-e7159ba2818a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:05.369963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:05.371679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:05.372518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:05.376064 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 62/118] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:05.379209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89567f9-998a-4a41-9011-b29e3d769ea7 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.379740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89567f9-998a-4a41-9011-b29e3d769ea7 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.380186 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89567f9-998a-4a41-9011-b29e3d769ea7 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.386809 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 63/119] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:05.387442 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89567f9-998a-4a41-9011-b29e3d769ea7 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.406197 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.406954 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.407249 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.407533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.408286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:05.408599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:05.410063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:05.423260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 57/120] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:05.426583 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.522811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89567f9-998a-4a41-9011-b29e3d769ea7 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.546398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dd79c-1014-4827-82d1-9095ea7370e2 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.559705 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a4559-87dc-48e0-ac84-63c70fd1c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.560361 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a4559-87dc-48e0-ac84-63c70fd1c51a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.560766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a4559-87dc-48e0-ac84-63c70fd1c51a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.561112 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a4559-87dc-48e0-ac84-63c70fd1c51a None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.561952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a4559-87dc-48e0-ac84-63c70fd1c51a None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:05.562289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a4559-87dc-48e0-ac84-63c70fd1c51a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:05.563720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a4559-87dc-48e0-ac84-63c70fd1c51a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:05.565473 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:05.567978 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:05.581822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 64/121] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:05.583485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01429-8c33-4950-9553-12b73f15c3f5 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.583723 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01429-8c33-4950-9553-12b73f15c3f5 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.584062 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01429-8c33-4950-9553-12b73f15c3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.588280 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.589153 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.589621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.590053 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.591087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:05.591624 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:05.593319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:05.602846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 58/122] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:05.604009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-f2a01429-8c33-4950-9553-12b73f15c3f5 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:05.604258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.605339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-f2a01429-8c33-4950-9553-12b73f15c3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:05.680147 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 65/123] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:05.681094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a01429-8c33-4950-9553-12b73f15c3f5 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.717331 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4bbbb-1175-4f35-a9b8-6031958196ea None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.754769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc96daf-d0c9-4070-97c0-a8e1eead96bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.755707 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc96daf-d0c9-4070-97c0-a8e1eead96bf None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.756300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc96daf-d0c9-4070-97c0-a8e1eead96bf None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.756846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc96daf-d0c9-4070-97c0-a8e1eead96bf None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.758297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc96daf-d0c9-4070-97c0-a8e1eead96bf None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:05.760038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc96daf-d0c9-4070-97c0-a8e1eead96bf None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:05.761928 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 59/124] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:05.762817 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc96daf-d0c9-4070-97c0-a8e1eead96bf None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.800241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a01429-8c33-4950-9553-12b73f15c3f5 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.813650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:05.846765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:05.852652 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18460c1-d700-4233-9fe9-1b19e3fd3766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.853348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18460c1-d700-4233-9fe9-1b19e3fd3766 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.853640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18460c1-d700-4233-9fe9-1b19e3fd3766 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.853922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18460c1-d700-4233-9fe9-1b19e3fd3766 None admin] PATH_INFO: `/v3/users/65605145c2d84cce8746bb3e047916a8` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.855185 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18460c1-d700-4233-9fe9-1b19e3fd3766 None admin] RBAC: Authorizing `identity:get_user(user_id=65605145c2d84cce8746bb3e047916a8)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:05.856783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18460c1-d700-4233-9fe9-1b19e3fd3766 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:05.858488 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 66/125] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/users/65605145c2d84cce8746bb3e047916a8 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:05.859373 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18460c1-d700-4233-9fe9-1b19e3fd3766 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.876439 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc96daf-d0c9-4070-97c0-a8e1eead96bf None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:05.913055 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e30db-7510-4d18-a8e5-556289cd076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:05.914153 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e30db-7510-4d18-a8e5-556289cd076a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:05.914662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e30db-7510-4d18-a8e5-556289cd076a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:05.915436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e30db-7510-4d18-a8e5-556289cd076a None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:05.916886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e30db-7510-4d18-a8e5-556289cd076a None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:05.918584 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e30db-7510-4d18-a8e5-556289cd076a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:05.920431 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 60/126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:05.921386 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e30db-7510-4d18-a8e5-556289cd076a None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:05.977020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18460c1-d700-4233-9fe9-1b19e3fd3766 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.013477 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4d418-3803-4d43-81b0-b7f4b3d7d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.015089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4d418-3803-4d43-81b0-b7f4b3d7d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.015089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4d418-3803-4d43-81b0-b7f4b3d7d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.015089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4d418-3803-4d43-81b0-b7f4b3d7d4b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.026432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4d418-3803-4d43-81b0-b7f4b3d7d4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-ceb4d418-3803-4d43-81b0-b7f4b3d7d4b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:06.032562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:06.033520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:06.034440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 67/127] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:06.035216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4d418-3803-4d43-81b0-b7f4b3d7d4b3 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:06.050510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e30db-7510-4d18-a8e5-556289cd076a None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.068605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:06.089658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9313eb7-4b28-42f5-9d44-04485ee50722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.090794 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9313eb7-4b28-42f5-9d44-04485ee50722 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.091249 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9313eb7-4b28-42f5-9d44-04485ee50722 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.091679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9313eb7-4b28-42f5-9d44-04485ee50722 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.093240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9313eb7-4b28-42f5-9d44-04485ee50722 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:06.094892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9313eb7-4b28-42f5-9d44-04485ee50722 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:06.097305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 61/128] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:05 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:06.098356 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9313eb7-4b28-42f5-9d44-04485ee50722 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:06.105470 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:06.154135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4d418-3803-4d43-81b0-b7f4b3d7d4b3 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.190960 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318fdd7-0810-44e5-9058-aedcc9d8a8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.192383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318fdd7-0810-44e5-9058-aedcc9d8a8c2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.192658 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318fdd7-0810-44e5-9058-aedcc9d8a8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.193042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318fdd7-0810-44e5-9058-aedcc9d8a8c2 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.194382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318fdd7-0810-44e5-9058-aedcc9d8a8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:06.196118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318fdd7-0810-44e5-9058-aedcc9d8a8c2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:06.198389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 68/129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:06 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:06.199305 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318fdd7-0810-44e5-9058-aedcc9d8a8c2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:06.219208 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9313eb7-4b28-42f5-9d44-04485ee50722 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.261441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efb392-382b-467e-af77-d1ce2674650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.262491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efb392-382b-467e-af77-d1ce2674650b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.263013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efb392-382b-467e-af77-d1ce2674650b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.263453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efb392-382b-467e-af77-d1ce2674650b None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.264994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efb392-382b-467e-af77-d1ce2674650b None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:06.267316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efb392-382b-467e-af77-d1ce2674650b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:06.270488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 62/130] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:06 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:06.272049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efb392-382b-467e-af77-d1ce2674650b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:06.314459 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318fdd7-0810-44e5-9058-aedcc9d8a8c2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.336292 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:06.341816 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:06.354875 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.355753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.355884 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.356117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.356879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:06.357189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:06.358652 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:06.368319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 69/131] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:06.369253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:06.395113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efb392-382b-467e-af77-d1ce2674650b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.434317 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f3107-7279-48c9-b7c9-2711dd949635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.435471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f3107-7279-48c9-b7c9-2711dd949635 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.436061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f3107-7279-48c9-b7c9-2711dd949635 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.436575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f3107-7279-48c9-b7c9-2711dd949635 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57/users/a5086738a1284cb6a9c74c681b71749c/roles/2935a7836c864df48e32fa734f020882` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.438976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f3107-7279-48c9-b7c9-2711dd949635 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a427cb96a904a66aca0187281bdad57, user_id=a5086738a1284cb6a9c74c681b71749c, role_id=2935a7836c864df48e32fa734f020882)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:06.440939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f3107-7279-48c9-b7c9-2711dd949635 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:06.466942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 63/132] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:06 2022] PUT /identity/v3/projects/6a427cb96a904a66aca0187281bdad57/users/a5086738a1284cb6a9c74c681b71749c/roles/2935a7836c864df48e32fa734f020882 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:06.469139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f3107-7279-48c9-b7c9-2711dd949635 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:06.519925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2487d0-04e6-4103-b2b1-4a807a361366 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.611657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2071d39-557f-414b-88f8-ccc90b14ae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.612340 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2071d39-557f-414b-88f8-ccc90b14ae01 None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.612634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2071d39-557f-414b-88f8-ccc90b14ae01 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.612916 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2071d39-557f-414b-88f8-ccc90b14ae01 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57/users/65605145c2d84cce8746bb3e047916a8/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.615009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2071d39-557f-414b-88f8-ccc90b14ae01 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a427cb96a904a66aca0187281bdad57, user_id=65605145c2d84cce8746bb3e047916a8, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:06.616605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2071d39-557f-414b-88f8-ccc90b14ae01 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:06.655429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 70/133] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:06 2022] PUT /identity/v3/projects/6a427cb96a904a66aca0187281bdad57/users/65605145c2d84cce8746bb3e047916a8/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:06.656313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2071d39-557f-414b-88f8-ccc90b14ae01 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:06.723317 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f3107-7279-48c9-b7c9-2711dd949635 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.735043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:06.761438 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:06.807306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2071d39-557f-414b-88f8-ccc90b14ae01 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.810226 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b86e4-7399-4b6f-87e8-b64e1978457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.811784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b86e4-7399-4b6f-87e8-b64e1978457d None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.812599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b86e4-7399-4b6f-87e8-b64e1978457d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.813452 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b86e4-7399-4b6f-87e8-b64e1978457d None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57/users/a5086738a1284cb6a9c74c681b71749c/roles/601939cb90d641e08b97775c0c12d374` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.818176 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b86e4-7399-4b6f-87e8-b64e1978457d None admin] RBAC: Authorizing `identity:create_grant(project_id=6a427cb96a904a66aca0187281bdad57, user_id=a5086738a1284cb6a9c74c681b71749c, role_id=601939cb90d641e08b97775c0c12d374)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:06.821629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b86e4-7399-4b6f-87e8-b64e1978457d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:06.847026 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36c5ef-782d-4343-984f-9c28da9f85a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:06.847740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36c5ef-782d-4343-984f-9c28da9f85a5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.848037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36c5ef-782d-4343-984f-9c28da9f85a5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.848308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36c5ef-782d-4343-984f-9c28da9f85a5 None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.849675 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36c5ef-782d-4343-984f-9c28da9f85a5 None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:06.851263 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36c5ef-782d-4343-984f-9c28da9f85a5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:06.852281 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 64/134] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:06 2022] PUT /identity/v3/projects/6a427cb96a904a66aca0187281bdad57/users/a5086738a1284cb6a9c74c681b71749c/roles/601939cb90d641e08b97775c0c12d374 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:06.853285 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 71/135] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:06 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:06.859202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b86e4-7399-4b6f-87e8-b64e1978457d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:06.973801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c26856-66e9-484a-ab3c-f3257fc2b7ec None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.974489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c26856-66e9-484a-ab3c-f3257fc2b7ec None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.974489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c26856-66e9-484a-ab3c-f3257fc2b7ec None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:06.976041 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 72/136] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:08:06.995054 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b86e4-7399-4b6f-87e8-b64e1978457d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:06.996131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b8433-00b9-420b-ab72-21d4fa1ac513 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:06.996434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b8433-00b9-420b-ab72-21d4fa1ac513 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:06.996723 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b8433-00b9-420b-ab72-21d4fa1ac513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:07.031955 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-846b8433-00b9-420b-ab72-21d4fa1ac513 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:07.038683 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-846b8433-00b9-420b-ab72-21d4fa1ac513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:07.041210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dcea-f1e0-4ed8-abaa-a84da3e44cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:07.042358 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dcea-f1e0-4ed8-abaa-a84da3e44cd8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:07.042936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dcea-f1e0-4ed8-abaa-a84da3e44cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:07.043499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dcea-f1e0-4ed8-abaa-a84da3e44cd8 None admin] PATH_INFO: `/v3/projects/2608c6d855fa478e95db30be9aab107a` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:07.045089 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087dcea-f1e0-4ed8-abaa-a84da3e44cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=2608c6d855fa478e95db30be9aab107a)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:07.046895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087dcea-f1e0-4ed8-abaa-a84da3e44cd8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:07.049487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 65/137] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:06 2022] GET /identity/v3/projects/2608c6d855fa478e95db30be9aab107a => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:07.056661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dcea-f1e0-4ed8-abaa-a84da3e44cd8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:07.118130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:07.128126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 73/138] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:07.157461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b8433-00b9-420b-ab72-21d4fa1ac513 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:07.216960 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dcea-f1e0-4ed8-abaa-a84da3e44cd8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:07.258458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5eb70-5025-4159-8921-a6e5a03dc710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:07.259334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5eb70-5025-4159-8921-a6e5a03dc710 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:07.259728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5eb70-5025-4159-8921-a6e5a03dc710 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:07.260113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5eb70-5025-4159-8921-a6e5a03dc710 None admin] PATH_INFO: `/v3/projects/2608c6d855fa478e95db30be9aab107a/users/a5086738a1284cb6a9c74c681b71749c/roles/2935a7836c864df48e32fa734f020882` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:07.262496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5eb70-5025-4159-8921-a6e5a03dc710 None admin] RBAC: Authorizing `identity:create_grant(project_id=2608c6d855fa478e95db30be9aab107a, user_id=a5086738a1284cb6a9c74c681b71749c, role_id=2935a7836c864df48e32fa734f020882)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:07.264180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5eb70-5025-4159-8921-a6e5a03dc710 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:07.285537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 66/139] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:07 2022] PUT /identity/v3/projects/2608c6d855fa478e95db30be9aab107a/users/a5086738a1284cb6a9c74c681b71749c/roles/2935a7836c864df48e32fa734f020882 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:08:07.301179 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:07.373148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b8433-00b9-420b-ab72-21d4fa1ac513 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:07.414540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed990f3-2e5e-4985-ae8e-f466da00076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:07.415732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed990f3-2e5e-4985-ae8e-f466da00076e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:07.416266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed990f3-2e5e-4985-ae8e-f466da00076e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:07.416786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed990f3-2e5e-4985-ae8e-f466da00076e None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:07.418412 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed990f3-2e5e-4985-ae8e-f466da00076e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:07.420701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed990f3-2e5e-4985-ae8e-f466da00076e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:07.424658 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 74/140] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:07.432064 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5eb70-5025-4159-8921-a6e5a03dc710 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:07.568495 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:07.571027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5eb70-5025-4159-8921-a6e5a03dc710 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:07.610865 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899cd00-5695-471f-a2b9-a028f8c1bd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:07.611532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cd00-5695-471f-a2b9-a028f8c1bd13 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:07.611824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cd00-5695-471f-a2b9-a028f8c1bd13 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:07.612108 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cd00-5695-471f-a2b9-a028f8c1bd13 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:07.612825 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899cd00-5695-471f-a2b9-a028f8c1bd13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:07.614644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899cd00-5695-471f-a2b9-a028f8c1bd13 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:07.638845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 67/141] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:07 2022] POST /identity/v3/users => generated 283 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:07.775004 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c38688-af78-48ce-a7a3-a80579c790cb None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:07.775620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c38688-af78-48ce-a7a3-a80579c790cb None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:07.776165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c38688-af78-48ce-a7a3-a80579c790cb None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:07.778002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 75/142] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:07.783785 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624de6-d676-4262-b8a8-fe877fa1ddbf None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:07.784239 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624de6-d676-4262-b8a8-fe877fa1ddbf None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:07.784621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624de6-d676-4262-b8a8-fe877fa1ddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:07.807173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-f1624de6-d676-4262-b8a8-fe877fa1ddbf None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:07.808458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-f1624de6-d676-4262-b8a8-fe877fa1ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:07.917643 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 68/143] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:07.957100 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c38688-af78-48ce-a7a3-a80579c790cb None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:08.097318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c38688-af78-48ce-a7a3-a80579c790cb None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:08.116138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:08.151699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f99dd-f04a-4d39-a4d8-4564c436c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:08.152380 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f99dd-f04a-4d39-a4d8-4564c436c070 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:08.152665 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f99dd-f04a-4d39-a4d8-4564c436c070 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:08.152940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f99dd-f04a-4d39-a4d8-4564c436c070 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:08.170040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f99dd-f04a-4d39-a4d8-4564c436c070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-021f99dd-f04a-4d39-a4d8-4564c436c070 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:08.175076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:08.176056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:08.176940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 76/144] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:08.184944 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1624de6-d676-4262-b8a8-fe877fa1ddbf None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:08.318561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:08.336645 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1624de6-d676-4262-b8a8-fe877fa1ddbf None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:08.381210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:08.381916 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:08.382244 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:08.382525 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:08.383266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:08.383605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:08.385011 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:08.395017 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 69/145] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:08.403263 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f99dd-f04a-4d39-a4d8-4564c436c070 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:08.527508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f99dd-f04a-4d39-a4d8-4564c436c070 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:08.566385 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c799fe-54a5-484b-b9df-182c7309b276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:08.567460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c799fe-54a5-484b-b9df-182c7309b276 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:08.567747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c799fe-54a5-484b-b9df-182c7309b276 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:08.568025 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c799fe-54a5-484b-b9df-182c7309b276 None admin] PATH_INFO: `/v3/users/44969c9e86984ff38dbc5d664aa8ade2` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:08.569212 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c799fe-54a5-484b-b9df-182c7309b276 None admin] RBAC: Authorizing `identity:get_user(user_id=44969c9e86984ff38dbc5d664aa8ade2)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:08.570739 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c799fe-54a5-484b-b9df-182c7309b276 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:08.572435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 77/146] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:08 2022] GET /identity/v3/users/44969c9e86984ff38dbc5d664aa8ade2 => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:08.578353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:08.689325 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc2c94-d531-4786-ba30-4044603843f9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:08.726631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ce8a1-a216-4901-9cac-0c92a8f51e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:08.727470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ce8a1-a216-4901-9cac-0c92a8f51e88 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:08.727757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ce8a1-a216-4901-9cac-0c92a8f51e88 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:08.728034 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ce8a1-a216-4901-9cac-0c92a8f51e88 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:08.729438 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ce8a1-a216-4901-9cac-0c92a8f51e88 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:08.731079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ce8a1-a216-4901-9cac-0c92a8f51e88 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:08.732827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 70/147] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:08 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:08.738902 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c799fe-54a5-484b-b9df-182c7309b276 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:08.788170 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:08.865668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c799fe-54a5-484b-b9df-182c7309b276 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:08.905938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2ace2-010a-4873-bd00-1d3032b21554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:08.906676 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2ace2-010a-4873-bd00-1d3032b21554 None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:08.906962 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2ace2-010a-4873-bd00-1d3032b21554 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:08.907243 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2ace2-010a-4873-bd00-1d3032b21554 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57/users/44969c9e86984ff38dbc5d664aa8ade2/roles/73552b9e126d49b188a201b0e02d1011` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:08.910276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2ace2-010a-4873-bd00-1d3032b21554 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a427cb96a904a66aca0187281bdad57, user_id=44969c9e86984ff38dbc5d664aa8ade2, role_id=73552b9e126d49b188a201b0e02d1011)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:08.911862 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2ace2-010a-4873-bd00-1d3032b21554 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:08.950864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 78/148] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:08 2022] PUT /identity/v3/projects/6a427cb96a904a66aca0187281bdad57/users/44969c9e86984ff38dbc5d664aa8ade2/roles/73552b9e126d49b188a201b0e02d1011 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:09.863235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743881f-467f-4238-8a03-02a36ab10214 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:09.864338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743881f-467f-4238-8a03-02a36ab10214 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:09.864851 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743881f-467f-4238-8a03-02a36ab10214 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:09.866950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 71/149] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:09.873703 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881b23b-2469-4ac6-a49e-cccad151c5c8 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:09.874659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881b23b-2469-4ac6-a49e-cccad151c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:09.875155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881b23b-2469-4ac6-a49e-cccad151c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:09.903581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-0881b23b-2469-4ac6-a49e-cccad151c5c8 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:09.905928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-0881b23b-2469-4ac6-a49e-cccad151c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:09.998190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 79/150] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:10.056069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743881f-467f-4238-8a03-02a36ab10214 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:10.076444 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:10.175705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743881f-467f-4238-8a03-02a36ab10214 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:10.232008 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e154c-6182-4577-9af7-2ad95c110376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:10.232691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e154c-6182-4577-9af7-2ad95c110376 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:10.232977 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e154c-6182-4577-9af7-2ad95c110376 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:10.233256 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e154c-6182-4577-9af7-2ad95c110376 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:10.250169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e154c-6182-4577-9af7-2ad95c110376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-2a2e154c-6182-4577-9af7-2ad95c110376 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:10.255660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:10.257021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:10.257902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 72/151] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:10.264987 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881b23b-2469-4ac6-a49e-cccad151c5c8 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:10.317312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:10.411276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881b23b-2469-4ac6-a49e-cccad151c5c8 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:10.455849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3053188-80f1-47c5-b198-220fec163912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:10.456822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3053188-80f1-47c5-b198-220fec163912 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:10.457310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3053188-80f1-47c5-b198-220fec163912 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:10.457797 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3053188-80f1-47c5-b198-220fec163912 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:10.458793 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3053188-80f1-47c5-b198-220fec163912 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:10.459294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3053188-80f1-47c5-b198-220fec163912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:10.460926 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3053188-80f1-47c5-b198-220fec163912 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:10.476572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 80/152] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:10.482708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e154c-6182-4577-9af7-2ad95c110376 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:10.610783 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:10.610989 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a444-217a-4609-b661-f560d7ef4f3b None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:10.611077 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a444-217a-4609-b661-f560d7ef4f3b None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:10.611358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a444-217a-4609-b661-f560d7ef4f3b None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:10.612992 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 81/153] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:10.619243 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4f298-910f-4184-b5e4-f203c3991534 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:10.619540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4f298-910f-4184-b5e4-f203c3991534 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:10.619821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4f298-910f-4184-b5e4-f203c3991534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:10.650789 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-53f4f298-910f-4184-b5e4-f203c3991534 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:10.652021 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-53f4f298-910f-4184-b5e4-f203c3991534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:10.652268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e154c-6182-4577-9af7-2ad95c110376 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:10.754356 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 82/154] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:10.755877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957622fa-36a1-4972-b642-2758a1b87443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:10.756567 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957622fa-36a1-4972-b642-2758a1b87443 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:10.756855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957622fa-36a1-4972-b642-2758a1b87443 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:10.757127 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957622fa-36a1-4972-b642-2758a1b87443 None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:10.758878 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957622fa-36a1-4972-b642-2758a1b87443 None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:10.760370 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957622fa-36a1-4972-b642-2758a1b87443 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:10.762027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 73/155] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:10 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:10.767867 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4f298-910f-4184-b5e4-f203c3991534 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:10.799782 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5e96a-5aa4-466b-a281-bdacb97cbf1c None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:10.800116 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5e96a-5aa4-466b-a281-bdacb97cbf1c None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:10.800347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5e96a-5aa4-466b-a281-bdacb97cbf1c None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:10.802035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 74/156] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:10.802604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:10.811659 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236eed7-c859-4d5e-be12-7fb8896a5f7a None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:10.812009 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236eed7-c859-4d5e-be12-7fb8896a5f7a None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:10.812218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236eed7-c859-4d5e-be12-7fb8896a5f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:10.861201 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-0236eed7-c859-4d5e-be12-7fb8896a5f7a None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:10.861201 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-0236eed7-c859-4d5e-be12-7fb8896a5f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:10.944875 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4f298-910f-4184-b5e4-f203c3991534 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:11.017431 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a82298-40fe-4a43-89ee-6205c84fcf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:11.018508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a82298-40fe-4a43-89ee-6205c84fcf9f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:11.019036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a82298-40fe-4a43-89ee-6205c84fcf9f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:11.019539 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a82298-40fe-4a43-89ee-6205c84fcf9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:11.052626 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 75/157] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:11.053458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236eed7-c859-4d5e-be12-7fb8896a5f7a None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:11.132905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a82298-40fe-4a43-89ee-6205c84fcf9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:11.134863 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a82298-40fe-4a43-89ee-6205c84fcf9f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:11.170513 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:11.262462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 83/158] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:11.265064 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b548fa-c128-4c9d-bdfc-33ad4fe0e419 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:11.265376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b548fa-c128-4c9d-bdfc-33ad4fe0e419 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:11.265660 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b548fa-c128-4c9d-bdfc-33ad4fe0e419 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:11.267054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 84/159] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:11.267839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b548fa-c128-4c9d-bdfc-33ad4fe0e419 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:11.276758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:11.330064 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236eed7-c859-4d5e-be12-7fb8896a5f7a None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:11.367522 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed4fe1-8321-4b28-a9fb-9a4fd5aac764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:11.369162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed4fe1-8321-4b28-a9fb-9a4fd5aac764 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:11.369448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed4fe1-8321-4b28-a9fb-9a4fd5aac764 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:11.369728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed4fe1-8321-4b28-a9fb-9a4fd5aac764 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:11.389291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed4fe1-8321-4b28-a9fb-9a4fd5aac764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-40ed4fe1-8321-4b28-a9fb-9a4fd5aac764 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:11.402983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:11.403996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:11.404929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 76/160] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:11.405659 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed4fe1-8321-4b28-a9fb-9a4fd5aac764 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:11.481953 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b548fa-c128-4c9d-bdfc-33ad4fe0e419 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:11.508260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:11.520832 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9165e1-0eed-46f4-8e52-e7296c66a436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:11.521455 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9165e1-0eed-46f4-8e52-e7296c66a436 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:11.521752 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9165e1-0eed-46f4-8e52-e7296c66a436 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:11.522030 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9165e1-0eed-46f4-8e52-e7296c66a436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:11.534987 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9165e1-0eed-46f4-8e52-e7296c66a436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-bc9165e1-0eed-46f4-8e52-e7296c66a436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:11.540947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:11.542257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:11.543165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:11.543832 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 85/161] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:08:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:11.546487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb07a5-7d24-42f2-8dfa-1b46436d80f2 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:11.547334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb07a5-7d24-42f2-8dfa-1b46436d80f2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:11.547858 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb07a5-7d24-42f2-8dfa-1b46436d80f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:11.573973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-f4bb07a5-7d24-42f2-8dfa-1b46436d80f2 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:11.575261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-f4bb07a5-7d24-42f2-8dfa-1b46436d80f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:11.668052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 86/162] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:11.669533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb07a5-7d24-42f2-8dfa-1b46436d80f2 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:11.772200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed4fe1-8321-4b28-a9fb-9a4fd5aac764 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:11.871938 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a6f27-eca8-444b-b962-16f78a23c289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:11.872633 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a6f27-eca8-444b-b962-16f78a23c289 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:11.872919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a6f27-eca8-444b-b962-16f78a23c289 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:11.873197 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a6f27-eca8-444b-b962-16f78a23c289 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:11.874499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a6f27-eca8-444b-b962-16f78a23c289 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:11.876497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a6f27-eca8-444b-b962-16f78a23c289 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:11.880594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb07a5-7d24-42f2-8dfa-1b46436d80f2 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:11.887050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 77/163] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:11 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:11.888083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a6f27-eca8-444b-b962-16f78a23c289 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:11.891482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:11.910733 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:11.955800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:11.957338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:11.957857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:11.958405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:11.959911 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:11.962780 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:11.964580 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:11.981638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 87/164] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:11.983072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:12.119445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1584fe0-2c24-45f4-b904-abb372c6e1c7 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:12.166211 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805e6d5-494b-4f4c-aa82-2c042eff8aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:12.167082 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e6d5-494b-4f4c-aa82-2c042eff8aac None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.167378 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e6d5-494b-4f4c-aa82-2c042eff8aac None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.167716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e6d5-494b-4f4c-aa82-2c042eff8aac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.210830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805e6d5-494b-4f4c-aa82-2c042eff8aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-a805e6d5-494b-4f4c-aa82-2c042eff8aac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:12.217396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:12.218434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 88/165] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:12.219311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805e6d5-494b-4f4c-aa82-2c042eff8aac None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:12.240299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a6f27-eca8-444b-b962-16f78a23c289 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:12.330531 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:12.342072 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:12.343254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.343762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.344269 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.345397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:12.345923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:12.347673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:12.359111 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 78/166] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:12.365449 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:12.419904 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805e6d5-494b-4f4c-aa82-2c042eff8aac None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:12.436594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:12.464109 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:12.465239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.465721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.466512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.467525 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] RBAC: Adding query filter params (role.id=2935a7836c864df48e32fa734f020882, scope.project.id=6a427cb96a904a66aca0187281bdad57, user.id=a5086738a1284cb6a9c74c681b71749c) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:12.468168 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:12.469687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:12.480707 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 89/167] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:12 2022] GET /identity/v3/role_assignments?user.id=a5086738a1284cb6a9c74c681b71749c&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=2935a7836c864df48e32fa734f020882 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:12.481426 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:12.483194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff12ae-6cfd-471a-83d4-1e1fcde93997 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:12.546172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873021c-c229-4c81-81cb-548f1c091256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:12.547307 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873021c-c229-4c81-81cb-548f1c091256 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.547814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873021c-c229-4c81-81cb-548f1c091256 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.548307 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873021c-c229-4c81-81cb-548f1c091256 None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.550031 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873021c-c229-4c81-81cb-548f1c091256 None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:12.551904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873021c-c229-4c81-81cb-548f1c091256 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:12.554398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 79/168] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:12 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:12.556294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b16b2-876a-4392-bf39-7c2b6b2e654d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.556860 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b16b2-876a-4392-bf39-7c2b6b2e654d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.557343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b16b2-876a-4392-bf39-7c2b6b2e654d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.559223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 80/169] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:12.560764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79eb9-7503-42e7-b5ed-0a83a066791f None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.561283 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79eb9-7503-42e7-b5ed-0a83a066791f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.561761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79eb9-7503-42e7-b5ed-0a83a066791f None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.563396 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 81/170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:12.564380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79eb9-7503-42e7-b5ed-0a83a066791f None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:12.642487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:12.647512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76422550-4522-4574-886e-420bc3fc4e6f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:12.689875 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d9143-317f-4ed7-bc7a-0a1bb4dc329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:12.690499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d9143-317f-4ed7-bc7a-0a1bb4dc329f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.690790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d9143-317f-4ed7-bc7a-0a1bb4dc329f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.691070 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d9143-317f-4ed7-bc7a-0a1bb4dc329f None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.691821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d9143-317f-4ed7-bc7a-0a1bb4dc329f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:12.692498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d9143-317f-4ed7-bc7a-0a1bb4dc329f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:12.693936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d9143-317f-4ed7-bc7a-0a1bb4dc329f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:12.696439 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79eb9-7503-42e7-b5ed-0a83a066791f None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:12.703979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 90/171] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:12.705095 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987c716-74b6-40bc-ac8d-b6a337d0ba05 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.705446 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987c716-74b6-40bc-ac8d-b6a337d0ba05 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.705720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987c716-74b6-40bc-ac8d-b6a337d0ba05 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.707165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 91/172] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:12.711921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987c716-74b6-40bc-ac8d-b6a337d0ba05 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:12.732367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:12.738727 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa85614-4fbe-4fb6-a44d-46b5d9658885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:12.741410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85614-4fbe-4fb6-a44d-46b5d9658885 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.741410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85614-4fbe-4fb6-a44d-46b5d9658885 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.741410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85614-4fbe-4fb6-a44d-46b5d9658885 None admin] PATH_INFO: `/v3/users/65605145c2d84cce8746bb3e047916a8` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.741410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa85614-4fbe-4fb6-a44d-46b5d9658885 None admin] RBAC: Authorizing `identity:get_user(user_id=65605145c2d84cce8746bb3e047916a8)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:12.742799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa85614-4fbe-4fb6-a44d-46b5d9658885 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:12.745805 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 82/173] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:12 2022] GET /identity/v3/users/65605145c2d84cce8746bb3e047916a8 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:12.747085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2b45f-f48e-4632-9a3e-38a61f489d84 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.747377 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2b45f-f48e-4632-9a3e-38a61f489d84 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.747658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2b45f-f48e-4632-9a3e-38a61f489d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.768406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-01f2b45f-f48e-4632-9a3e-38a61f489d84 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:12.769640 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-01f2b45f-f48e-4632-9a3e-38a61f489d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:12.831311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987c716-74b6-40bc-ac8d-b6a337d0ba05 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:12.840002 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 83/174] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:12.841313 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f70a-60f4-4754-829b-ed28881e0e15 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.841581 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f70a-60f4-4754-829b-ed28881e0e15 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.841892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f70a-60f4-4754-829b-ed28881e0e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.861264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-6bc6f70a-60f4-4754-829b-ed28881e0e15 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:12.862493 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-6bc6f70a-60f4-4754-829b-ed28881e0e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:12.869130 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a8a8a-9c7c-4809-8411-da984c636782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:12.869795 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8a8a-9c7c-4809-8411-da984c636782 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.870076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8a8a-9c7c-4809-8411-da984c636782 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.870630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8a8a-9c7c-4809-8411-da984c636782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.899266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:12.932537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 84/175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:12.933709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493bbc9-730b-4884-a168-aa78b14ae6eb None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:12.934008 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493bbc9-730b-4884-a168-aa78b14ae6eb None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:12.934340 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493bbc9-730b-4884-a168-aa78b14ae6eb None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:12.937948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 85/176] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:12.937948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493bbc9-730b-4884-a168-aa78b14ae6eb None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:12.950245 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a8a8a-9c7c-4809-8411-da984c636782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:12.951680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a8a8a-9c7c-4809-8411-da984c636782 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:13.015336 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.057330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493bbc9-730b-4884-a168-aa78b14ae6eb None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:13.060365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 92/177] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:13.061499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c5bd9-8978-44ed-a793-89e4ee7b4166 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.061793 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c5bd9-8978-44ed-a793-89e4ee7b4166 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.062073 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c5bd9-8978-44ed-a793-89e4ee7b4166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.080933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-9e8c5bd9-8978-44ed-a793-89e4ee7b4166 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:13.082187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-9e8c5bd9-8978-44ed-a793-89e4ee7b4166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:13.097699 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97489-bb04-45cb-ba63-46ba8fadd821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:13.098488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97489-bb04-45cb-ba63-46ba8fadd821 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.098772 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97489-bb04-45cb-ba63-46ba8fadd821 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.099071 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97489-bb04-45cb-ba63-46ba8fadd821 None admin] PATH_INFO: `/v3/users/a5086738a1284cb6a9c74c681b71749c` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.100255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b97489-bb04-45cb-ba63-46ba8fadd821 None admin] RBAC: Authorizing `identity:get_user(user_id=a5086738a1284cb6a9c74c681b71749c)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:13.101743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b97489-bb04-45cb-ba63-46ba8fadd821 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:13.103462 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 86/178] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:12 2022] GET /identity/v3/users/a5086738a1284cb6a9c74c681b71749c => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:13.108315 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97489-bb04-45cb-ba63-46ba8fadd821 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.141516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.158182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 93/179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:13.158182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c5bd9-8978-44ed-a793-89e4ee7b4166 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.222194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97489-bb04-45cb-ba63-46ba8fadd821 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:13.247998 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.256089 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23967f74-60ec-4db5-ae32-213a3d1eaef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:13.256734 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23967f74-60ec-4db5-ae32-213a3d1eaef9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.257024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23967f74-60ec-4db5-ae32-213a3d1eaef9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.257305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23967f74-60ec-4db5-ae32-213a3d1eaef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.257643 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c5bd9-8978-44ed-a793-89e4ee7b4166 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:13.291611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d7dbb-54a2-4d31-8e2d-9e56ac982680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:13.292281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d7dbb-54a2-4d31-8e2d-9e56ac982680 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.292567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d7dbb-54a2-4d31-8e2d-9e56ac982680 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.292844 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d7dbb-54a2-4d31-8e2d-9e56ac982680 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.303088 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d7dbb-54a2-4d31-8e2d-9e56ac982680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-286d7dbb-54a2-4d31-8e2d-9e56ac982680 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:13.308182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:13.309244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:13.310480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 94/180] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:13.311529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c99be-4754-4fd8-bc61-448371f1fa3d None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.311529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c99be-4754-4fd8-bc61-448371f1fa3d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.311529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c99be-4754-4fd8-bc61-448371f1fa3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.329105 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-7f8c99be-4754-4fd8-bc61-448371f1fa3d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:13.330356 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-7f8c99be-4754-4fd8-bc61-448371f1fa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:13.331061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23967f74-60ec-4db5-ae32-213a3d1eaef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:13.332546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23967f74-60ec-4db5-ae32-213a3d1eaef9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:13.376665 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.398032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 95/181] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:13.399089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c99be-4754-4fd8-bc61-448371f1fa3d None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.448143 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 87/182] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:13.448813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23967f74-60ec-4db5-ae32-213a3d1eaef9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.480452 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.508423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c99be-4754-4fd8-bc61-448371f1fa3d None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:13.542485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7d17-4a21-4f67-aad1-df6204be8fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:13.543150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7d17-4a21-4f67-aad1-df6204be8fcf None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.543424 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7d17-4a21-4f67-aad1-df6204be8fcf None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.543696 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7d17-4a21-4f67-aad1-df6204be8fcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.544865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b7d17-4a21-4f67-aad1-df6204be8fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:13.546366 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b7d17-4a21-4f67-aad1-df6204be8fcf None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:13.552604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 96/183] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:13.552604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7d17-4a21-4f67-aad1-df6204be8fcf None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.558788 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23967f74-60ec-4db5-ae32-213a3d1eaef9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:13.594190 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b320d-bd55-49cb-ac55-8206d64553e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:13.595037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b320d-bd55-49cb-ac55-8206d64553e0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.595222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b320d-bd55-49cb-ac55-8206d64553e0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.595508 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b320d-bd55-49cb-ac55-8206d64553e0 None admin] PATH_INFO: `/v3/projects/2608c6d855fa478e95db30be9aab107a` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.596713 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b320d-bd55-49cb-ac55-8206d64553e0 None admin] RBAC: Authorizing `identity:get_project(project_id=2608c6d855fa478e95db30be9aab107a)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:13.598229 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b320d-bd55-49cb-ac55-8206d64553e0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:13.599939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 88/184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/projects/2608c6d855fa478e95db30be9aab107a => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:13.600694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b320d-bd55-49cb-ac55-8206d64553e0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.602452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.655306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7d17-4a21-4f67-aad1-df6204be8fcf None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:13.688741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f816cfb-dda6-48e4-92bc-426dd51499ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:13.689387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816cfb-dda6-48e4-92bc-426dd51499ce None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.689672 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816cfb-dda6-48e4-92bc-426dd51499ce None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.689952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816cfb-dda6-48e4-92bc-426dd51499ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.703800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.710595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b320d-bd55-49cb-ac55-8206d64553e0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:13.748270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd6bcc-18d4-4fff-8301-05bcd75fcde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:13.748861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd6bcc-18d4-4fff-8301-05bcd75fcde5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.749140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd6bcc-18d4-4fff-8301-05bcd75fcde5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.749421 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd6bcc-18d4-4fff-8301-05bcd75fcde5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.761025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd6bcc-18d4-4fff-8301-05bcd75fcde5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:13.764777 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f816cfb-dda6-48e4-92bc-426dd51499ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-07dd6bcc-18d4-4fff-8301-05bcd75fcde5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:13.766050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:13.767046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:13.768113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:13.769047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f816cfb-dda6-48e4-92bc-426dd51499ce None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:13.769135 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 89/185] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:13.770971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd6bcc-18d4-4fff-8301-05bcd75fcde5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.816323 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.873258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 97/186] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:13.874186 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f816cfb-dda6-48e4-92bc-426dd51499ce None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.874961 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd6bcc-18d4-4fff-8301-05bcd75fcde5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:13.909923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:13.910590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:13.910872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:13.911151 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:13.911886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:13.912178 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:13.913590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:13.922026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 90/187] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:13.923136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:13.942391 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:13.985961 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f816cfb-dda6-48e4-92bc-426dd51499ce None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.023437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7c822-e358-4d2e-a05a-988fcb6ea119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.024099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c822-e358-4d2e-a05a-988fcb6ea119 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.024383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c822-e358-4d2e-a05a-988fcb6ea119 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.024664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c822-e358-4d2e-a05a-988fcb6ea119 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.028940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede8295-0528-44ea-a8f7-a915fac7a41f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.037207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7c822-e358-4d2e-a05a-988fcb6ea119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-01a7c822-e358-4d2e-a05a-988fcb6ea119 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:14.044402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:14.045435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:14.046339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 98/188] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:14.047068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7c822-e358-4d2e-a05a-988fcb6ea119 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.062651 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:14.069492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972b96-c785-4147-a30c-1fd555b3d231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.070203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972b96-c785-4147-a30c-1fd555b3d231 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.070492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972b96-c785-4147-a30c-1fd555b3d231 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.070798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972b96-c785-4147-a30c-1fd555b3d231 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.072070 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972b96-c785-4147-a30c-1fd555b3d231 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.073548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972b96-c785-4147-a30c-1fd555b3d231 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.075299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 91/189] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:13 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:14.075299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972b96-c785-4147-a30c-1fd555b3d231 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.153009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7c822-e358-4d2e-a05a-988fcb6ea119 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.161959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:14.178361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972b96-c785-4147-a30c-1fd555b3d231 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.186996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9f6dd-7ced-4908-98bd-6457f52247f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.187623 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f6dd-7ced-4908-98bd-6457f52247f0 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.187926 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f6dd-7ced-4908-98bd-6457f52247f0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.188201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f6dd-7ced-4908-98bd-6457f52247f0 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.188899 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9f6dd-7ced-4908-98bd-6457f52247f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.190405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9f6dd-7ced-4908-98bd-6457f52247f0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.216440 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.217081 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 99/190] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:14 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:14.218231 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.218231 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.218231 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.218502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9f6dd-7ced-4908-98bd-6457f52247f0 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.218639 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] RBAC: Adding query filter params (role.id=2935a7836c864df48e32fa734f020882, scope.project.id=2608c6d855fa478e95db30be9aab107a, user.id=a5086738a1284cb6a9c74c681b71749c) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:14.218981 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.220477 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.227422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 92/191] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/role_assignments?user.id=a5086738a1284cb6a9c74c681b71749c&scope.project.id=2608c6d855fa478e95db30be9aab107a&role.id=2935a7836c864df48e32fa734f020882 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:14.228388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.310277 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:14.364509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9f6dd-7ced-4908-98bd-6457f52247f0 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.411013 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.411990 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.412442 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.412871 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.413761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df610358-d193-4825-90fb-ce51661e4f3b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.414422 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:14.414950 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.416666 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.428783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 100/192] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:14.433678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.459970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991cfc-4871-4a2d-85a3-dc610788ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.460946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991cfc-4871-4a2d-85a3-dc610788ba42 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.461237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991cfc-4871-4a2d-85a3-dc610788ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.461600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991cfc-4871-4a2d-85a3-dc610788ba42 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.464167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3991cfc-4871-4a2d-85a3-dc610788ba42 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.465684 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3991cfc-4871-4a2d-85a3-dc610788ba42 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.467508 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 93/193] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:14.469463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991cfc-4871-4a2d-85a3-dc610788ba42 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.495287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:14.574552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc06aa8-44ff-4e99-a534-c76378a1c559 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.601343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991cfc-4871-4a2d-85a3-dc610788ba42 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.607278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:14.616973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fc01c-4ff1-449c-868b-ca74adda100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.617686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fc01c-4ff1-449c-868b-ca74adda100a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.617983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fc01c-4ff1-449c-868b-ca74adda100a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.618305 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fc01c-4ff1-449c-868b-ca74adda100a None admin] PATH_INFO: `/v3/users/11381ba0f93e4a2ab86903c5a02321a7` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.638009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fc01c-4ff1-449c-868b-ca74adda100a None admin] RBAC: Authorizing `identity:get_user(user_id=11381ba0f93e4a2ab86903c5a02321a7)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.639589 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fc01c-4ff1-449c-868b-ca74adda100a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.641615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 101/194] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/users/11381ba0f93e4a2ab86903c5a02321a7 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:14.642304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fc01c-4ff1-449c-868b-ca74adda100a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.655374 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.656932 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.657450 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.658426 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.659746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] RBAC: Adding query filter params (name=reader) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:14.660295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.662794 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.678523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 94/195] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:14.680104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.779724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fc01c-4ff1-449c-868b-ca74adda100a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.802941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:14.827103 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d17cd6-65a2-4864-9ae5-da4ad65deb03 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:14.828578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.829416 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.829770 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.830124 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.831430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=6a427cb96a904a66aca0187281bdad57, user.id=65605145c2d84cce8746bb3e047916a8) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:14.831552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.833037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.840313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 102/196] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/role_assignments?user.id=65605145c2d84cce8746bb3e047916a8&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:14.846811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.876036 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3b5d-e8d8-430b-b082-937617572ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:14.877123 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3b5d-e8d8-430b-b082-937617572ceb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:14.877600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3b5d-e8d8-430b-b082-937617572ceb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:14.878292 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3b5d-e8d8-430b-b082-937617572ceb None admin] PATH_INFO: `/v3/users/11381ba0f93e4a2ab86903c5a02321a7` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:14.879869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb3b5d-e8d8-430b-b082-937617572ceb None admin] RBAC: Authorizing `identity:get_user(user_id=11381ba0f93e4a2ab86903c5a02321a7)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:14.881652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb3b5d-e8d8-430b-b082-937617572ceb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:14.883669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 95/197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/users/11381ba0f93e4a2ab86903c5a02321a7 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:14.889272 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3b5d-e8d8-430b-b082-937617572ceb None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:14.917841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:15.011955 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7c0ce-94f8-4a54-8f9d-72d6bb2f7464 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:15.054002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:15.054858 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:15.055192 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:15.055678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:15.056852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] RBAC: Adding query filter params (role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=6a427cb96a904a66aca0187281bdad57, user.id=a5086738a1284cb6a9c74c681b71749c) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:15.057171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:15.062141 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:15.069588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 103/198] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/role_assignments?user.id=a5086738a1284cb6a9c74c681b71749c&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=601939cb90d641e08b97775c0c12d374 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:15.070388 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:15.115437 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3b5d-e8d8-430b-b082-937617572ceb None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:15.160745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94969713-fff7-49b7-aefa-db7cb2a444bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:15.161406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969713-fff7-49b7-aefa-db7cb2a444bf None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:15.161689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969713-fff7-49b7-aefa-db7cb2a444bf None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:15.161971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969713-fff7-49b7-aefa-db7cb2a444bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:15.190153 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:15.298187 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94969713-fff7-49b7-aefa-db7cb2a444bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:15.299874 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94969713-fff7-49b7-aefa-db7cb2a444bf None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:15.308474 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe5b82-d06b-42c5-bc42-26e017465219 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:15.332824 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:15.355561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e709-6352-4937-ab20-ef8374df75d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:15.356629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e709-6352-4937-ab20-ef8374df75d7 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:15.357132 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e709-6352-4937-ab20-ef8374df75d7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:15.357695 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e709-6352-4937-ab20-ef8374df75d7 None admin] PATH_INFO: `/v3/users/44969c9e86984ff38dbc5d664aa8ade2` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:15.359233 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264e709-6352-4937-ab20-ef8374df75d7 None admin] RBAC: Authorizing `identity:get_user(user_id=44969c9e86984ff38dbc5d664aa8ade2)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:15.360963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264e709-6352-4937-ab20-ef8374df75d7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:15.365642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 104/199] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:15 2022] GET /identity/v3/users/44969c9e86984ff38dbc5d664aa8ade2 => generated 289 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:15.366846 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e709-6352-4937-ab20-ef8374df75d7 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:15.414060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 96/200] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:15.415302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94969713-fff7-49b7-aefa-db7cb2a444bf None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:15.498686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e709-6352-4937-ab20-ef8374df75d7 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:15.506496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:15.544258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea996ef-9435-4151-a1f7-8f88c5ff4ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:15.545311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea996ef-9435-4151-a1f7-8f88c5ff4ba4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:15.545815 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea996ef-9435-4151-a1f7-8f88c5ff4ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:15.546394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea996ef-9435-4151-a1f7-8f88c5ff4ba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:15.580438 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94969713-fff7-49b7-aefa-db7cb2a444bf None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:15.610576 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:15.634932 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea996ef-9435-4151-a1f7-8f88c5ff4ba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:15.636943 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea996ef-9435-4151-a1f7-8f88c5ff4ba4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:15.657459 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1935-6f6e-4245-95ff-6a37a0777d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:15.658136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1935-6f6e-4245-95ff-6a37a0777d8f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:15.658415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1935-6f6e-4245-95ff-6a37a0777d8f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:15.658689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1935-6f6e-4245-95ff-6a37a0777d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:15.775403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e1935-6f6e-4245-95ff-6a37a0777d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:15.777674 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e1935-6f6e-4245-95ff-6a37a0777d8f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:15.803250 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 105/201] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:15.805559 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea996ef-9435-4151-a1f7-8f88c5ff4ba4 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:15.851764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:15.946329 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 97/202] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:15.951099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1935-6f6e-4245-95ff-6a37a0777d8f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:15.977921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:16.016306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea996ef-9435-4151-a1f7-8f88c5ff4ba4 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:16.078216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636a267-1483-477e-8ab7-6771f7acdabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:16.079165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636a267-1483-477e-8ab7-6771f7acdabf None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.079460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636a267-1483-477e-8ab7-6771f7acdabf None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.079736 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636a267-1483-477e-8ab7-6771f7acdabf None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.081960 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636a267-1483-477e-8ab7-6771f7acdabf None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:16.083524 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636a267-1483-477e-8ab7-6771f7acdabf None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:16.086195 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 106/203] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:15 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:16.087167 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636a267-1483-477e-8ab7-6771f7acdabf None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:16.125930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1935-6f6e-4245-95ff-6a37a0777d8f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:16.173319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db88ec-fa9b-4c89-b5d1-3ca77ee9b900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:16.174501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db88ec-fa9b-4c89-b5d1-3ca77ee9b900 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.174786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db88ec-fa9b-4c89-b5d1-3ca77ee9b900 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.175065 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db88ec-fa9b-4c89-b5d1-3ca77ee9b900 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.177250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db88ec-fa9b-4c89-b5d1-3ca77ee9b900 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:16.178869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db88ec-fa9b-4c89-b5d1-3ca77ee9b900 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:16.180647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 98/204] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:15 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:16.182050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db88ec-fa9b-4c89-b5d1-3ca77ee9b900 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:16.201420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:16.218706 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636a267-1483-477e-8ab7-6771f7acdabf None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:16.257492 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cc4f5-9dd5-487c-919b-2b5c80428a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:16.258675 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cc4f5-9dd5-487c-919b-2b5c80428a6f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.259179 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cc4f5-9dd5-487c-919b-2b5c80428a6f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.259697 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cc4f5-9dd5-487c-919b-2b5c80428a6f None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.261161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cc4f5-9dd5-487c-919b-2b5c80428a6f None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:16.262929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cc4f5-9dd5-487c-919b-2b5c80428a6f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:16.265341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 107/205] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:16 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:16.266516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cc4f5-9dd5-487c-919b-2b5c80428a6f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:16.286811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:16.294156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db88ec-fa9b-4c89-b5d1-3ca77ee9b900 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:16.334655 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:16.335202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.335487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.335762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.336749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] RBAC: Adding query filter params (role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=ff937f0574a548faac1d7abf71ec1698, user.id=11381ba0f93e4a2ab86903c5a02321a7) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:16.337034 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:16.338563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:16.344722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 99/206] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:16 2022] GET /identity/v3/role_assignments?user.id=11381ba0f93e4a2ab86903c5a02321a7&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=601939cb90d641e08b97775c0c12d374 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:16.345561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:16.384623 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cc4f5-9dd5-487c-919b-2b5c80428a6f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:16.424602 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec0ac0-811f-4198-89a1-747615474a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:16.425671 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec0ac0-811f-4198-89a1-747615474a4b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.426217 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec0ac0-811f-4198-89a1-747615474a4b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.426743 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec0ac0-811f-4198-89a1-747615474a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.427810 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec0ac0-811f-4198-89a1-747615474a4b None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=ff937f0574a548faac1d7abf71ec1698, user.id=11381ba0f93e4a2ab86903c5a02321a7) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:16.428360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec0ac0-811f-4198-89a1-747615474a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:16.430434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec0ac0-811f-4198-89a1-747615474a4b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:16.437330 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 108/207] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:16 2022] GET /identity/v3/role_assignments?user.id=11381ba0f93e4a2ab86903c5a02321a7&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:16.446971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:16.467476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256867-bc5d-455f-a520-0d75b2200241 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:16.508105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e82b6-2d37-4f76-a682-dba738c782c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:16.508768 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e82b6-2d37-4f76-a682-dba738c782c0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.509061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e82b6-2d37-4f76-a682-dba738c782c0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.509339 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e82b6-2d37-4f76-a682-dba738c782c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.510199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e82b6-2d37-4f76-a682-dba738c782c0 None admin] RBAC: Adding query filter params (role.id=73552b9e126d49b188a201b0e02d1011, scope.project.id=6a427cb96a904a66aca0187281bdad57, user.id=44969c9e86984ff38dbc5d664aa8ade2) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:16.510526 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e82b6-2d37-4f76-a682-dba738c782c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:16.512188 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e82b6-2d37-4f76-a682-dba738c782c0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:16.519848 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 100/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:16 2022] GET /identity/v3/role_assignments?user.id=44969c9e86984ff38dbc5d664aa8ade2&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=73552b9e126d49b188a201b0e02d1011 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:16.759980 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d395c-55ea-4ad1-aba6-1748e4c596cd None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.760344 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d395c-55ea-4ad1-aba6-1748e4c596cd None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.760463 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d395c-55ea-4ad1-aba6-1748e4c596cd None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.761996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 109/209] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:16.767912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a0ff7-e2a2-44ee-a58d-f4bbafdafb95 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.768563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a0ff7-e2a2-44ee-a58d-f4bbafdafb95 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.769044 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a0ff7-e2a2-44ee-a58d-f4bbafdafb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.792003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-3f2a0ff7-e2a2-44ee-a58d-f4bbafdafb95 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:16.793515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca1a5-53e7-4134-b18a-db6299bf5c92 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.793813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca1a5-53e7-4134-b18a-db6299bf5c92 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.794020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-3f2a0ff7-e2a2-44ee-a58d-f4bbafdafb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:16.794184 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca1a5-53e7-4134-b18a-db6299bf5c92 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.795934 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 110/210] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:16.806192 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c51e6-a3c0-433c-9add-cac574a05342 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:16.806192 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c51e6-a3c0-433c-9add-cac574a05342 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:16.806192 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c51e6-a3c0-433c-9add-cac574a05342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:16.826664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-af7c51e6-a3c0-433c-9add-cac574a05342 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:16.827905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-af7c51e6-a3c0-433c-9add-cac574a05342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:16.889356 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 101/211] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:16.917975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a0ff7-e2a2-44ee-a58d-f4bbafdafb95 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:16.927312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 111/212] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:16.957490 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c51e6-a3c0-433c-9add-cac574a05342 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:16.972248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:17.046222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:17.074891 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a0ff7-e2a2-44ee-a58d-f4bbafdafb95 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:17.083310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c51e6-a3c0-433c-9add-cac574a05342 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:17.115391 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32df23c-8a34-4311-9f93-9b8e78dcef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:17.116061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df23c-8a34-4311-9f93-9b8e78dcef54 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:17.116350 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df23c-8a34-4311-9f93-9b8e78dcef54 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:17.116630 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df23c-8a34-4311-9f93-9b8e78dcef54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:17.117851 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32df23c-8a34-4311-9f93-9b8e78dcef54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:17.119460 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32df23c-8a34-4311-9f93-9b8e78dcef54 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:17.121470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 102/213] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:17.127020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32df23c-8a34-4311-9f93-9b8e78dcef54 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:17.128023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ee1d-7265-4be0-b865-179f37c26738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:17.128700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ee1d-7265-4be0-b865-179f37c26738 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:17.128987 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ee1d-7265-4be0-b865-179f37c26738 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:17.129266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ee1d-7265-4be0-b865-179f37c26738 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:17.144418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338ee1d-7265-4be0-b865-179f37c26738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-f338ee1d-7265-4be0-b865-179f37c26738 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:17.151207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:17.152256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:17.153102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 112/214] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:17.160619 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ee1d-7265-4be0-b865-179f37c26738 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:17.247202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32df23c-8a34-4311-9f93-9b8e78dcef54 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:17.252017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:17.285426 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215ce88-2255-4d87-8e85-56b20c64a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:17.285710 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ee1d-7265-4be0-b865-179f37c26738 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:17.286689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215ce88-2255-4d87-8e85-56b20c64a2d4 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:17.287308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215ce88-2255-4d87-8e85-56b20c64a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:17.287783 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215ce88-2255-4d87-8e85-56b20c64a2d4 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:17.288752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8215ce88-2255-4d87-8e85-56b20c64a2d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:17.290545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8215ce88-2255-4d87-8e85-56b20c64a2d4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:17.310308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:17.325408 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:17.326213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 103/215] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:17 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:17.326327 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:17.326527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:17.327099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:17.327866 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:17.328176 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:17.329650 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:17.340580 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 113/216] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:17.346536 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215ce88-2255-4d87-8e85-56b20c64a2d4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:17.463820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215ce88-2255-4d87-8e85-56b20c64a2d4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:17.507801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e94ad-b267-468f-8db6-c2d645e08588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:17.508442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e94ad-b267-468f-8db6-c2d645e08588 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:17.508727 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e94ad-b267-468f-8db6-c2d645e08588 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:17.509003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e94ad-b267-468f-8db6-c2d645e08588 None admin] PATH_INFO: `/v3/users/9c197a97aa92420aa28209877ad1e86d` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:17.525986 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e94ad-b267-468f-8db6-c2d645e08588 None admin] RBAC: Authorizing `identity:get_user(user_id=9c197a97aa92420aa28209877ad1e86d)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:17.527564 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e94ad-b267-468f-8db6-c2d645e08588 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:17.529489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 104/217] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:17 2022] GET /identity/v3/users/9c197a97aa92420aa28209877ad1e86d => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:17.535266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:17.645014 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc0e45-32c8-48a8-894b-9a70276c7061 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:17.676403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:17.683563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108857e-5d77-4ea7-9594-ff699efc91df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:17.684259 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108857e-5d77-4ea7-9594-ff699efc91df None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:17.684546 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108857e-5d77-4ea7-9594-ff699efc91df None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:17.684825 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108857e-5d77-4ea7-9594-ff699efc91df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:17.762480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108857e-5d77-4ea7-9594-ff699efc91df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:17.763976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108857e-5d77-4ea7-9594-ff699efc91df None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:17.899877 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 114/218] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:17.905654 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e94ad-b267-468f-8db6-c2d645e08588 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:17.994594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:18.057346 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e94ad-b267-468f-8db6-c2d645e08588 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:18.099141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb73d2b-bef1-403d-bb2d-7736cf97826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:18.100223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73d2b-bef1-403d-bb2d-7736cf97826f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:18.100733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73d2b-bef1-403d-bb2d-7736cf97826f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:18.101241 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73d2b-bef1-403d-bb2d-7736cf97826f None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:18.102844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb73d2b-bef1-403d-bb2d-7736cf97826f None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:18.104563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb73d2b-bef1-403d-bb2d-7736cf97826f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:18.106973 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 105/219] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:17 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:18.113470 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108857e-5d77-4ea7-9594-ff699efc91df None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:18.155470 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:18.249053 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108857e-5d77-4ea7-9594-ff699efc91df None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:18.294348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d713d-36e0-4c3e-b45b-b147e3a9a13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:18.295000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d713d-36e0-4c3e-b45b-b147e3a9a13b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:18.295285 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d713d-36e0-4c3e-b45b-b147e3a9a13b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:18.295562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d713d-36e0-4c3e-b45b-b147e3a9a13b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:18.296379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d713d-36e0-4c3e-b45b-b147e3a9a13b None admin] RBAC: Adding query filter params (role.id=2935a7836c864df48e32fa734f020882, scope.project.id=ff937f0574a548faac1d7abf71ec1698, user.id=9c197a97aa92420aa28209877ad1e86d) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:18.296692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d713d-36e0-4c3e-b45b-b147e3a9a13b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:18.298215 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d713d-36e0-4c3e-b45b-b147e3a9a13b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:18.310354 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 115/220] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:18 2022] GET /identity/v3/role_assignments?user.id=9c197a97aa92420aa28209877ad1e86d&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=2935a7836c864df48e32fa734f020882 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:18.724430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ac7e2-b8fa-44e2-b1e5-ae9bb899950d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:18.725103 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ac7e2-b8fa-44e2-b1e5-ae9bb899950d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:18.725103 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ac7e2-b8fa-44e2-b1e5-ae9bb899950d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:18.726778 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 106/221] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:08:18.735776 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7eba2b-659d-419b-80b5-d4b7dab6278e None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:18.736086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7eba2b-659d-419b-80b5-d4b7dab6278e None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:18.736355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7eba2b-659d-419b-80b5-d4b7dab6278e None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:18.738465 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716e26f-6203-4163-a026-5f8488a7c7a0 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:18.738755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716e26f-6203-4163-a026-5f8488a7c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:18.739033 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716e26f-6203-4163-a026-5f8488a7c7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:18.739729 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 116/222] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:18.745716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973c0d7-fe4e-4692-9931-ff8b2a8efbdf None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:18.746306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973c0d7-fe4e-4692-9931-ff8b2a8efbdf None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:18.746816 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973c0d7-fe4e-4692-9931-ff8b2a8efbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:18.767432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-e973c0d7-fe4e-4692-9931-ff8b2a8efbdf None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:18.768802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-e716e26f-6203-4163-a026-5f8488a7c7a0 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:18.769286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-e973c0d7-fe4e-4692-9931-ff8b2a8efbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:18.770005 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-e716e26f-6203-4163-a026-5f8488a7c7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:18.853130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 107/223] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:18.875591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716e26f-6203-4163-a026-5f8488a7c7a0 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:18.939443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 117/224] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:18.946512 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:18.975075 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973c0d7-fe4e-4692-9931-ff8b2a8efbdf None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:18.998896 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716e26f-6203-4163-a026-5f8488a7c7a0 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:19.044080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a967e7a-42d7-44e3-adee-1a1a8e964b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:19.045177 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a967e7a-42d7-44e3-adee-1a1a8e964b21 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:19.045665 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a967e7a-42d7-44e3-adee-1a1a8e964b21 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:19.046177 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a967e7a-42d7-44e3-adee-1a1a8e964b21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:19.050231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:19.059907 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a967e7a-42d7-44e3-adee-1a1a8e964b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-4a967e7a-42d7-44e3-adee-1a1a8e964b21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:19.065518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:19.066918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:19.068303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:19.071182 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 108/225] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:08:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:19.077517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a967e7a-42d7-44e3-adee-1a1a8e964b21 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:19.181151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973c0d7-fe4e-4692-9931-ff8b2a8efbdf None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:19.209369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a967e7a-42d7-44e3-adee-1a1a8e964b21 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:19.239550 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:19.242286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f9210-eb5c-4476-810d-81cae4af2e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:19.243247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f9210-eb5c-4476-810d-81cae4af2e13 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:19.243586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f9210-eb5c-4476-810d-81cae4af2e13 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:19.243876 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f9210-eb5c-4476-810d-81cae4af2e13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:19.265703 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f9210-eb5c-4476-810d-81cae4af2e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-7f7f9210-eb5c-4476-810d-81cae4af2e13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:19.271971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:19.272952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:19.273869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:19.278095 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 118/226] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:19.279298 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:19.279951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:19.280235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:19.280508 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:19.281245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:19.281575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:19.283132 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:19.285359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f9210-eb5c-4476-810d-81cae4af2e13 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:19.301691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 109/227] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:19.308014 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:19.417221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:19.439785 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeccf97-996f-4b42-830a-b54bf7256cc3 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:19.464336 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f9210-eb5c-4476-810d-81cae4af2e13 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:19.476961 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46018c2-6003-4a22-a922-6a677c373045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:19.482224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46018c2-6003-4a22-a922-6a677c373045 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:19.482224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46018c2-6003-4a22-a922-6a677c373045 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:19.482224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46018c2-6003-4a22-a922-6a677c373045 None admin] PATH_INFO: `/v3/users/11381ba0f93e4a2ab86903c5a02321a7` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:19.484842 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46018c2-6003-4a22-a922-6a677c373045 None admin] RBAC: Authorizing `identity:get_user(user_id=11381ba0f93e4a2ab86903c5a02321a7)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:19.487594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46018c2-6003-4a22-a922-6a677c373045 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:19.490204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 110/228] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:19 2022] GET /identity/v3/users/11381ba0f93e4a2ab86903c5a02321a7 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:19.496531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46018c2-6003-4a22-a922-6a677c373045 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:19.540870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:19.541534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:19.541818 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:19.542226 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:19.542979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:19.543291 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:19.544707 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:19.566290 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 119/229] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:19.572735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:19.589986 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:19.656833 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46018c2-6003-4a22-a922-6a677c373045 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:19.703745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8a765-afa2-4d14-9ac9-671cbf51fbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:19.704415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8a765-afa2-4d14-9ac9-671cbf51fbe9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:19.704704 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8a765-afa2-4d14-9ac9-671cbf51fbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:19.704983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8a765-afa2-4d14-9ac9-671cbf51fbe9 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:19.706257 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8a765-afa2-4d14-9ac9-671cbf51fbe9 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:19.707754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8a765-afa2-4d14-9ac9-671cbf51fbe9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:19.718279 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 111/230] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:19 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:19.719242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8a765-afa2-4d14-9ac9-671cbf51fbe9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:19.784513 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b3c51-5550-48e1-8530-96e8985c5709 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:19.846004 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49200eb6-11af-4fce-9890-8bf86320db8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:19.846716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49200eb6-11af-4fce-9890-8bf86320db8e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:19.847005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49200eb6-11af-4fce-9890-8bf86320db8e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:19.847286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49200eb6-11af-4fce-9890-8bf86320db8e None admin] PATH_INFO: `/v3/users/11381ba0f93e4a2ab86903c5a02321a7` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:19.850246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49200eb6-11af-4fce-9890-8bf86320db8e None admin] RBAC: Authorizing `identity:get_user(user_id=11381ba0f93e4a2ab86903c5a02321a7)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:19.851752 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49200eb6-11af-4fce-9890-8bf86320db8e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:19.853787 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 120/231] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:19 2022] GET /identity/v3/users/11381ba0f93e4a2ab86903c5a02321a7 => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:19.859233 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49200eb6-11af-4fce-9890-8bf86320db8e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:19.894272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:19.912295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8a765-afa2-4d14-9ac9-671cbf51fbe9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:19.950458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:19.998519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677bd1c-51f8-4da7-a4c3-48040bf322ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:19.999255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677bd1c-51f8-4da7-a4c3-48040bf322ab None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:19.999540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677bd1c-51f8-4da7-a4c3-48040bf322ab None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:19.999826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677bd1c-51f8-4da7-a4c3-48040bf322ab None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698/users/11381ba0f93e4a2ab86903c5a02321a7/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.003078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677bd1c-51f8-4da7-a4c3-48040bf322ab None admin] RBAC: Authorizing `identity:create_grant(project_id=ff937f0574a548faac1d7abf71ec1698, user_id=11381ba0f93e4a2ab86903c5a02321a7, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:20.004601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677bd1c-51f8-4da7-a4c3-48040bf322ab None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:20.026431 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 112/232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:19 2022] PUT /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698/users/11381ba0f93e4a2ab86903c5a02321a7/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:20.028063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e0bd4-f9cf-4d22-911f-48b34fca876c None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.028437 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e0bd4-f9cf-4d22-911f-48b34fca876c None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.028876 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e0bd4-f9cf-4d22-911f-48b34fca876c None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.030369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 113/233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:20.036616 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87ff0-98ff-4a86-a12c-ec08cfb647b6 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.036908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87ff0-98ff-4a86-a12c-ec08cfb647b6 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.037087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87ff0-98ff-4a86-a12c-ec08cfb647b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.050211 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49200eb6-11af-4fce-9890-8bf86320db8e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:20.069318 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-76b87ff0-98ff-4a86-a12c-ec08cfb647b6 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:20.070608 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-76b87ff0-98ff-4a86-a12c-ec08cfb647b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:20.087757 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7a0e4-9e66-4fbd-b7db-7db89db1770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:20.088567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7a0e4-9e66-4fbd-b7db-7db89db1770a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.088862 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7a0e4-9e66-4fbd-b7db-7db89db1770a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.089147 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7a0e4-9e66-4fbd-b7db-7db89db1770a None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.090489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7a0e4-9e66-4fbd-b7db-7db89db1770a None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:20.092097 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7a0e4-9e66-4fbd-b7db-7db89db1770a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:20.094170 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 121/234] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:19 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:20.100344 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7a0e4-9e66-4fbd-b7db-7db89db1770a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:20.176583 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 114/235] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:20.178258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9ac35-daf8-4576-a809-1272672b01f2 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.178580 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9ac35-daf8-4576-a809-1272672b01f2 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.178942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9ac35-daf8-4576-a809-1272672b01f2 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.180324 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 115/236] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:20.186459 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c386b-73e7-4849-8ec9-a489f727b7d0 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.186905 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c386b-73e7-4849-8ec9-a489f727b7d0 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.187399 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c386b-73e7-4849-8ec9-a489f727b7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.195275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:20.209869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-bf9c386b-73e7-4849-8ec9-a489f727b7d0 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:20.211807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-bf9c386b-73e7-4849-8ec9-a489f727b7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:20.223851 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7a0e4-9e66-4fbd-b7db-7db89db1770a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:20.261702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b05c-5a25-49e1-9012-1a9dae325675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:20.262679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b05c-5a25-49e1-9012-1a9dae325675 None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.262825 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b05c-5a25-49e1-9012-1a9dae325675 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.263114 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b05c-5a25-49e1-9012-1a9dae325675 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698/users/11381ba0f93e4a2ab86903c5a02321a7/roles/601939cb90d641e08b97775c0c12d374` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.265577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b05c-5a25-49e1-9012-1a9dae325675 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff937f0574a548faac1d7abf71ec1698, user_id=11381ba0f93e4a2ab86903c5a02321a7, role_id=601939cb90d641e08b97775c0c12d374)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:20.267374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b05c-5a25-49e1-9012-1a9dae325675 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:20.287346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 122/237] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:20 2022] PUT /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698/users/11381ba0f93e4a2ab86903c5a02321a7/roles/601939cb90d641e08b97775c0c12d374 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:20.296174 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b05c-5a25-49e1-9012-1a9dae325675 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:20.314260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 116/238] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:20.337949 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c386b-73e7-4849-8ec9-a489f727b7d0 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:20.387429 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:20.453596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b05c-5a25-49e1-9012-1a9dae325675 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:20.471646 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c386b-73e7-4849-8ec9-a489f727b7d0 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:20.495821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d091-1ce8-4dc3-a373-79fd27327918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:20.496937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d091-1ce8-4dc3-a373-79fd27327918 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.497586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d091-1ce8-4dc3-a373-79fd27327918 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.498209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d091-1ce8-4dc3-a373-79fd27327918 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.499676 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0d091-1ce8-4dc3-a373-79fd27327918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:20.501595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0d091-1ce8-4dc3-a373-79fd27327918 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:20.504188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 123/239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:20.513873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d091-1ce8-4dc3-a373-79fd27327918 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:20.530906 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07dd31-61fd-46cd-9afc-60940fa4fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:20.531165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:20.531803 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07dd31-61fd-46cd-9afc-60940fa4fe9e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.531918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07dd31-61fd-46cd-9afc-60940fa4fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.532155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07dd31-61fd-46cd-9afc-60940fa4fe9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.555674 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07dd31-61fd-46cd-9afc-60940fa4fe9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-eb07dd31-61fd-46cd-9afc-60940fa4fe9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:20.560700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:20.561650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:20.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 117/240] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:20.571847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07dd31-61fd-46cd-9afc-60940fa4fe9e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:20.660077 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d091-1ce8-4dc3-a373-79fd27327918 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:20.697790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649dcde-a8ad-40b2-9090-39c7dec8d753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:20.698469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649dcde-a8ad-40b2-9090-39c7dec8d753 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.698759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649dcde-a8ad-40b2-9090-39c7dec8d753 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.699039 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649dcde-a8ad-40b2-9090-39c7dec8d753 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.699750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649dcde-a8ad-40b2-9090-39c7dec8d753 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:20.701238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649dcde-a8ad-40b2-9090-39c7dec8d753 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:20.725986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 124/241] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:20 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:20.825126 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:20.838616 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07dd31-61fd-46cd-9afc-60940fa4fe9e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:20.890877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100219c-e71c-4194-8230-aabd30178b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:20.891538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100219c-e71c-4194-8230-aabd30178b84 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:20.891827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100219c-e71c-4194-8230-aabd30178b84 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:20.892107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100219c-e71c-4194-8230-aabd30178b84 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:20.892877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100219c-e71c-4194-8230-aabd30178b84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:20.893183 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100219c-e71c-4194-8230-aabd30178b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:20.894669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100219c-e71c-4194-8230-aabd30178b84 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:20.904816 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 118/242] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:20.910301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649dcde-a8ad-40b2-9090-39c7dec8d753 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:20.992713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:21.014672 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068db8-dbec-4b30-95ce-9f2ccf6dd804 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.015238 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068db8-dbec-4b30-95ce-9f2ccf6dd804 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.015238 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068db8-dbec-4b30-95ce-9f2ccf6dd804 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.017133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 119/243] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:21.022560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c124e7-a420-4fea-b816-2624bf3e8a2a None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.022857 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c124e7-a420-4fea-b816-2624bf3e8a2a None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.023138 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c124e7-a420-4fea-b816-2624bf3e8a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.032593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649dcde-a8ad-40b2-9090-39c7dec8d753 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:21.047991 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-18c124e7-a420-4fea-b816-2624bf3e8a2a None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:21.049229 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-18c124e7-a420-4fea-b816-2624bf3e8a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:21.076725 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70398e07-91d9-4751-926c-2c5751608261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:21.077327 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70398e07-91d9-4751-926c-2c5751608261 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.077614 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70398e07-91d9-4751-926c-2c5751608261 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.077892 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70398e07-91d9-4751-926c-2c5751608261 None admin] PATH_INFO: `/v3/users/5722aee29e314de1b5aabf7c927fdc8e` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.094567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70398e07-91d9-4751-926c-2c5751608261 None admin] RBAC: Authorizing `identity:get_user(user_id=5722aee29e314de1b5aabf7c927fdc8e)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:21.096094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70398e07-91d9-4751-926c-2c5751608261 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:21.098062 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 125/244] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:20 2022] GET /identity/v3/users/5722aee29e314de1b5aabf7c927fdc8e => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:21.104073 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70398e07-91d9-4751-926c-2c5751608261 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:21.134086 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 120/245] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:21.158866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c124e7-a420-4fea-b816-2624bf3e8a2a None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:21.221170 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70398e07-91d9-4751-926c-2c5751608261 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:21.259854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:21.264495 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cf942-b1e9-4762-8651-3bac92a5b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:21.265382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cf942-b1e9-4762-8651-3bac92a5b72f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.265678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cf942-b1e9-4762-8651-3bac92a5b72f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.265968 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cf942-b1e9-4762-8651-3bac92a5b72f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.274853 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:21.294225 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c124e7-a420-4fea-b816-2624bf3e8a2a None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:21.332908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1e35b-43f3-4f56-a893-7dda7d04417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:21.333528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1e35b-43f3-4f56-a893-7dda7d04417b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.333813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1e35b-43f3-4f56-a893-7dda7d04417b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.334097 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1e35b-43f3-4f56-a893-7dda7d04417b None admin] PATH_INFO: `/v3/roles/member` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.346578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1e35b-43f3-4f56-a893-7dda7d04417b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-24c1e35b-43f3-4f56-a893-7dda7d04417b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:21.352752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:21.353799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:21.354795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 121/246] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:21.360344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1e35b-43f3-4f56-a893-7dda7d04417b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:21.384226 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cf942-b1e9-4762-8651-3bac92a5b72f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:21.387543 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cf942-b1e9-4762-8651-3bac92a5b72f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:21.475720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1e35b-43f3-4f56-a893-7dda7d04417b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:21.514615 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:21.515268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.515557 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.515843 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.516596 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] RBAC: Adding query filter params (name=member) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:21.516905 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:21.518357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:21.528098 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 122/247] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:21.533947 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:21.543935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:21.563219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 126/248] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 09:08:21.569507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cf942-b1e9-4762-8651-3bac92a5b72f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:21.610650 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:21.649578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c7961-eceb-4d8b-8791-dc2cf8d0851f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:21.684503 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cf942-b1e9-4762-8651-3bac92a5b72f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:21.686340 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9acd85-97cc-4673-b60d-bbfe82d2174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:21.687026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9acd85-97cc-4673-b60d-bbfe82d2174a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.687316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9acd85-97cc-4673-b60d-bbfe82d2174a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.687598 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9acd85-97cc-4673-b60d-bbfe82d2174a None admin] PATH_INFO: `/v3/users/9c197a97aa92420aa28209877ad1e86d` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.688857 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9acd85-97cc-4673-b60d-bbfe82d2174a None admin] RBAC: Authorizing `identity:get_user(user_id=9c197a97aa92420aa28209877ad1e86d)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:21.690388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9acd85-97cc-4673-b60d-bbfe82d2174a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:21.692787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 123/249] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:21 2022] GET /identity/v3/users/9c197a97aa92420aa28209877ad1e86d => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:21.698504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9acd85-97cc-4673-b60d-bbfe82d2174a None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:21.722630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283d36a-25fd-4754-a96d-a311e1de78a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:21.723414 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283d36a-25fd-4754-a96d-a311e1de78a2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.723705 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283d36a-25fd-4754-a96d-a311e1de78a2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.723984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283d36a-25fd-4754-a96d-a311e1de78a2 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.725240 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283d36a-25fd-4754-a96d-a311e1de78a2 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:21.727012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283d36a-25fd-4754-a96d-a311e1de78a2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:21.729167 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 127/250] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:21 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:21.735250 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283d36a-25fd-4754-a96d-a311e1de78a2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:21.806264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:21.867944 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9acd85-97cc-4673-b60d-bbfe82d2174a None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:21.945447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:21.949719 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e35de-4087-43c2-b6de-74241971d110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:21.950683 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e35de-4087-43c2-b6de-74241971d110 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:21.950817 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e35de-4087-43c2-b6de-74241971d110 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:21.951001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e35de-4087-43c2-b6de-74241971d110 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:21.954022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e35de-4087-43c2-b6de-74241971d110 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:21.956192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e35de-4087-43c2-b6de-74241971d110 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:21.958162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 124/251] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:21 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:21.958296 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283d36a-25fd-4754-a96d-a311e1de78a2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:21.966762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e35de-4087-43c2-b6de-74241971d110 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:22.046259 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80121f48-f8d5-41f3-ad05-8a27dedefe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:22.047120 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80121f48-f8d5-41f3-ad05-8a27dedefe99 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:22.047534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80121f48-f8d5-41f3-ad05-8a27dedefe99 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:22.047816 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80121f48-f8d5-41f3-ad05-8a27dedefe99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:22.048645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80121f48-f8d5-41f3-ad05-8a27dedefe99 None admin] RBAC: Adding query filter params (role.id=73552b9e126d49b188a201b0e02d1011, scope.project.id=ff937f0574a548faac1d7abf71ec1698, user.id=5722aee29e314de1b5aabf7c927fdc8e) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:22.048967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80121f48-f8d5-41f3-ad05-8a27dedefe99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:22.050684 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80121f48-f8d5-41f3-ad05-8a27dedefe99 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:22.064333 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 128/252] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:21 2022] GET /identity/v3/role_assignments?user.id=5722aee29e314de1b5aabf7c927fdc8e&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=73552b9e126d49b188a201b0e02d1011 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:22.176191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e35de-4087-43c2-b6de-74241971d110 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:22.253400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140a18c-bf58-42ab-8b6a-b4b943bd3f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:22.254144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140a18c-bf58-42ab-8b6a-b4b943bd3f85 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:22.254442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140a18c-bf58-42ab-8b6a-b4b943bd3f85 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:22.254723 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140a18c-bf58-42ab-8b6a-b4b943bd3f85 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698/users/9c197a97aa92420aa28209877ad1e86d/roles/2935a7836c864df48e32fa734f020882` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:22.259827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140a18c-bf58-42ab-8b6a-b4b943bd3f85 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff937f0574a548faac1d7abf71ec1698, user_id=9c197a97aa92420aa28209877ad1e86d, role_id=2935a7836c864df48e32fa734f020882)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:22.261406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140a18c-bf58-42ab-8b6a-b4b943bd3f85 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:22.285428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 125/253] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:21 2022] PUT /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698/users/9c197a97aa92420aa28209877ad1e86d/roles/2935a7836c864df48e32fa734f020882 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:08:22.955741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa09c1-fa4d-4c24-b1eb-1edbb0ebf63f None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:22.956334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa09c1-fa4d-4c24-b1eb-1edbb0ebf63f None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:22.956334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa09c1-fa4d-4c24-b1eb-1edbb0ebf63f None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:22.957714 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 129/254] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:22.960395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490371df-7691-45f7-9341-64a3033693c5 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:22.960677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490371df-7691-45f7-9341-64a3033693c5 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:22.960963 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490371df-7691-45f7-9341-64a3033693c5 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:22.962652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 126/255] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:22.964507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc1fc-74ae-4e21-a972-73098714c889 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:22.964947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc1fc-74ae-4e21-a972-73098714c889 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:22.965233 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc1fc-74ae-4e21-a972-73098714c889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:22.974081 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96655d6a-2d37-45cf-9173-bd9708dbb5eb None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:22.974442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96655d6a-2d37-45cf-9173-bd9708dbb5eb None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:22.974665 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96655d6a-2d37-45cf-9173-bd9708dbb5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:22.992518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:23.009307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-563cc1fc-74ae-4e21-a972-73098714c889 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:23.010591 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-563cc1fc-74ae-4e21-a972-73098714c889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:23.020627 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-96655d6a-2d37-45cf-9173-bd9708dbb5eb None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:23.021881 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-96655d6a-2d37-45cf-9173-bd9708dbb5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:23.168669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 127/256] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:23.176772 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 130/257] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:23.196106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96655d6a-2d37-45cf-9173-bd9708dbb5eb None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:23.210852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cc1fc-74ae-4e21-a972-73098714c889 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:23.372398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96655d6a-2d37-45cf-9173-bd9708dbb5eb None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:23.390959 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:23.409099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:23.419258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c0ed5-f3b7-472a-9b91-8052d8a3f288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:23.419941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c0ed5-f3b7-472a-9b91-8052d8a3f288 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:23.420224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c0ed5-f3b7-472a-9b91-8052d8a3f288 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:23.420502 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c0ed5-f3b7-472a-9b91-8052d8a3f288 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:23.422963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cc1fc-74ae-4e21-a972-73098714c889 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:23.446879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c0ed5-f3b7-472a-9b91-8052d8a3f288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-091c0ed5-f3b7-472a-9b91-8052d8a3f288 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:23.452140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:23.453114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:23.453964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 128/258] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:23.459784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c0ed5-f3b7-472a-9b91-8052d8a3f288 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:23.546448 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e228f-6779-4353-872c-934daf62365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:23.547058 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e228f-6779-4353-872c-934daf62365a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:23.547350 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e228f-6779-4353-872c-934daf62365a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:23.547633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e228f-6779-4353-872c-934daf62365a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:23.570631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c0ed5-f3b7-472a-9b91-8052d8a3f288 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:23.582022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e228f-6779-4353-872c-934daf62365a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-917e228f-6779-4353-872c-934daf62365a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:23.587447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:23.588453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:23.589307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 131/259] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:23.594895 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e228f-6779-4353-872c-934daf62365a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:23.608343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:23.609000 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:23.609291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:23.610076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:23.610893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:23.611206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:23.612639 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:23.623274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 129/260] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:23.629762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:23.698498 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:23.774589 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436fd8d-6042-4cec-9245-4310f900e1bd None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:23.851542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf6e91-6162-431a-a50a-307730caceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:23.852225 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf6e91-6162-431a-a50a-307730caceee None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:23.852517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf6e91-6162-431a-a50a-307730caceee None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:23.852804 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf6e91-6162-431a-a50a-307730caceee None admin] PATH_INFO: `/v3/users/11381ba0f93e4a2ab86903c5a02321a7` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:23.854079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf6e91-6162-431a-a50a-307730caceee None admin] RBAC: Authorizing `identity:get_user(user_id=11381ba0f93e4a2ab86903c5a02321a7)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:23.855600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf6e91-6162-431a-a50a-307730caceee None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:23.857646 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 130/261] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:23 2022] GET /identity/v3/users/11381ba0f93e4a2ab86903c5a02321a7 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:23.865119 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf6e91-6162-431a-a50a-307730caceee None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:23.908062 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e228f-6779-4353-872c-934daf62365a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:23.991504 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3bc75-8d2c-437f-a481-cc6a79597a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:23.992213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3bc75-8d2c-437f-a481-cc6a79597a10 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:23.992622 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3bc75-8d2c-437f-a481-cc6a79597a10 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:23.992804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3bc75-8d2c-437f-a481-cc6a79597a10 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:23.993520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3bc75-8d2c-437f-a481-cc6a79597a10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:23.993830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3bc75-8d2c-437f-a481-cc6a79597a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:23.996494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3bc75-8d2c-437f-a481-cc6a79597a10 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:24.017236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 132/262] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:24.018597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13695404-bc1d-4a0c-870a-9721fca74d22 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:24.018886 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13695404-bc1d-4a0c-870a-9721fca74d22 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:24.019166 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13695404-bc1d-4a0c-870a-9721fca74d22 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:24.021704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 133/263] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:24.028034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13695404-bc1d-4a0c-870a-9721fca74d22 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:24.058704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:24.113154 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf6e91-6162-431a-a50a-307730caceee None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:24.167662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:24.202763 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf3ce-9ff0-45d9-a159-01e4e8f0b9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:24.203466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf3ce-9ff0-45d9-a159-01e4e8f0b9eb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:24.203716 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf3ce-9ff0-45d9-a159-01e4e8f0b9eb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:24.203993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf3ce-9ff0-45d9-a159-01e4e8f0b9eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:24.269803 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13695404-bc1d-4a0c-870a-9721fca74d22 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:24.345334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebf3ce-9ff0-45d9-a159-01e4e8f0b9eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:24.346865 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebf3ce-9ff0-45d9-a159-01e4e8f0b9eb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:24.350912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185505f-9db1-44c4-96d8-4e6837d6d498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:24.351614 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185505f-9db1-44c4-96d8-4e6837d6d498 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:24.352080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185505f-9db1-44c4-96d8-4e6837d6d498 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:24.352398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185505f-9db1-44c4-96d8-4e6837d6d498 None admin] PATH_INFO: `/v3/users/11381ba0f93e4a2ab86903c5a02321a7` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:24.358997 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185505f-9db1-44c4-96d8-4e6837d6d498 None admin] RBAC: Authorizing `identity:get_user(user_id=11381ba0f93e4a2ab86903c5a02321a7)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:24.360503 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185505f-9db1-44c4-96d8-4e6837d6d498 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:24.362323 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 134/264] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:24 2022] GET /identity/v3/users/11381ba0f93e4a2ab86903c5a02321a7 => generated 283 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:24.363683 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cff29-ad64-4540-836d-42464dda2f7a None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:24.363920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cff29-ad64-4540-836d-42464dda2f7a None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:24.364202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cff29-ad64-4540-836d-42464dda2f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:24.403413 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-b18cff29-ad64-4540-836d-42464dda2f7a None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:24.404677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-b18cff29-ad64-4540-836d-42464dda2f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:24.488610 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 131/265] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:24.489924 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf3ce-9ff0-45d9-a159-01e4e8f0b9eb None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:24.518596 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:24.553398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 135/266] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:24.558138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb1224-d9d4-4078-8b1b-d08befb08021 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:24.558138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb1224-d9d4-4078-8b1b-d08befb08021 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:24.558138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb1224-d9d4-4078-8b1b-d08befb08021 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:24.560640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 136/267] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:24.560640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb1224-d9d4-4078-8b1b-d08befb08021 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:24.578766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:24.631791 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf3ce-9ff0-45d9-a159-01e4e8f0b9eb None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:24.675276 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314423c-b23a-48b2-95bc-3535de582e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:24.676550 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314423c-b23a-48b2-95bc-3535de582e33 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:24.677459 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314423c-b23a-48b2-95bc-3535de582e33 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:24.677944 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314423c-b23a-48b2-95bc-3535de582e33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:24.695069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb1224-d9d4-4078-8b1b-d08befb08021 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:24.735821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b4d79-cea5-478a-bbcb-214e7aa6d122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:24.736791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4d79-cea5-478a-bbcb-214e7aa6d122 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:24.736978 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4d79-cea5-478a-bbcb-214e7aa6d122 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:24.737215 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4d79-cea5-478a-bbcb-214e7aa6d122 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:24.738484 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4d79-cea5-478a-bbcb-214e7aa6d122 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:24.739937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4d79-cea5-478a-bbcb-214e7aa6d122 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:24.741948 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 137/268] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:24 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:24.748404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84790c2-f91b-43b3-b6e2-089935c60c97 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:24.748700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84790c2-f91b-43b3-b6e2-089935c60c97 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:24.748983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84790c2-f91b-43b3-b6e2-089935c60c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:24.767063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314423c-b23a-48b2-95bc-3535de582e33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:24.768917 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-f84790c2-f91b-43b3-b6e2-089935c60c97 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:24.769280 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314423c-b23a-48b2-95bc-3535de582e33 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:24.770259 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-f84790c2-f91b-43b3-b6e2-089935c60c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:24.800923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:24.847436 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 138/269] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:24.857145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84790c2-f91b-43b3-b6e2-089935c60c97 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:24.869538 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:24.899003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 132/270] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:24.900435 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314423c-b23a-48b2-95bc-3535de582e33 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:24.996310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84790c2-f91b-43b3-b6e2-089935c60c97 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.035437 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314423c-b23a-48b2-95bc-3535de582e33 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.044037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7ca84-2f3a-4ce5-b61b-b0395bebee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.044719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7ca84-2f3a-4ce5-b61b-b0395bebee5e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.045011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7ca84-2f3a-4ce5-b61b-b0395bebee5e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.045291 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7ca84-2f3a-4ce5-b61b-b0395bebee5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.057461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7ca84-2f3a-4ce5-b61b-b0395bebee5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:25.061453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-2ae7ca84-2f3a-4ce5-b61b-b0395bebee5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:25.062582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:25.063726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:25.064577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 139/271] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:25.065296 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7ca84-2f3a-4ce5-b61b-b0395bebee5e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.080345 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.080992 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.081287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.081569 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.082424 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=ff937f0574a548faac1d7abf71ec1698, user.id=11381ba0f93e4a2ab86903c5a02321a7) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:25.082742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:25.084223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:25.090709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 133/272] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:24 2022] GET /identity/v3/role_assignments?user.id=11381ba0f93e4a2ab86903c5a02321a7&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:25.091355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.163532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:25.215616 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7ca84-2f3a-4ce5-b61b-b0395bebee5e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.225810 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4149d-3644-49db-b8b6-8724d87b6c89 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.262035 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87721d81-556e-4856-b423-35bcb8e91d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.262758 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721d81-556e-4856-b423-35bcb8e91d0b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.263049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721d81-556e-4856-b423-35bcb8e91d0b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.263330 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721d81-556e-4856-b423-35bcb8e91d0b None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.264710 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87721d81-556e-4856-b423-35bcb8e91d0b None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:25.266359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87721d81-556e-4856-b423-35bcb8e91d0b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:25.268625 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 140/273] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:25 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:25.269980 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a8112-0dd3-4ef7-bc1e-d3b8549a9901 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.270334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a8112-0dd3-4ef7-bc1e-d3b8549a9901 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.270620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a8112-0dd3-4ef7-bc1e-d3b8549a9901 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.272184 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 141/274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:25.272932 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a8112-0dd3-4ef7-bc1e-d3b8549a9901 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.273571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f628e-97d3-4b95-a8b7-3aed7ae85143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.274317 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f628e-97d3-4b95-a8b7-3aed7ae85143 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.274699 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f628e-97d3-4b95-a8b7-3aed7ae85143 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.274979 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f628e-97d3-4b95-a8b7-3aed7ae85143 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.276461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f628e-97d3-4b95-a8b7-3aed7ae85143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:25.277978 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f628e-97d3-4b95-a8b7-3aed7ae85143 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:25.279676 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 134/275] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:25.280866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f628e-97d3-4b95-a8b7-3aed7ae85143 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.361935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:25.412097 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a8112-0dd3-4ef7-bc1e-d3b8549a9901 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.426037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f628e-97d3-4b95-a8b7-3aed7ae85143 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.449591 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330d57e-9a96-4205-bf0d-331754efb5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.450279 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330d57e-9a96-4205-bf0d-331754efb5b5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.450570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330d57e-9a96-4205-bf0d-331754efb5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.450855 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330d57e-9a96-4205-bf0d-331754efb5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.451599 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330d57e-9a96-4205-bf0d-331754efb5b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:25.451911 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330d57e-9a96-4205-bf0d-331754efb5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:25.453338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330d57e-9a96-4205-bf0d-331754efb5b5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:25.455302 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:25.464660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.465393 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.465747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.466028 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.466884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] RBAC: Adding query filter params (role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=ff937f0574a548faac1d7abf71ec1698, user.id=11381ba0f93e4a2ab86903c5a02321a7) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:25.467189 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:25.468628 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 142/276] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:25.468752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:25.470028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d60f1-d394-4bdd-90a7-478aa0eabe45 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.470349 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d60f1-d394-4bdd-90a7-478aa0eabe45 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.470764 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d60f1-d394-4bdd-90a7-478aa0eabe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.486689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 135/277] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:25 2022] GET /identity/v3/role_assignments?user.id=11381ba0f93e4a2ab86903c5a02321a7&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=601939cb90d641e08b97775c0c12d374 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:25.486689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.503497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-1c9d60f1-d394-4bdd-90a7-478aa0eabe45 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:25.505047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-1c9d60f1-d394-4bdd-90a7-478aa0eabe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:25.594247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 143/278] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:25.599182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d60f1-d394-4bdd-90a7-478aa0eabe45 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.620635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186bbc-4581-439b-b135-4468d77d4d2d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.641929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:25.661998 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46793273-451f-4daf-b9e7-819fa28e4d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.662702 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46793273-451f-4daf-b9e7-819fa28e4d20 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.662993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46793273-451f-4daf-b9e7-819fa28e4d20 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.663281 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46793273-451f-4daf-b9e7-819fa28e4d20 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.664172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46793273-451f-4daf-b9e7-819fa28e4d20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:25.665782 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46793273-451f-4daf-b9e7-819fa28e4d20 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:25.690519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 136/279] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:25 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:25.692615 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5611c23-80f9-42fc-a0f3-d4595f49b6e6 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.693119 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5611c23-80f9-42fc-a0f3-d4595f49b6e6 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.693513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5611c23-80f9-42fc-a0f3-d4595f49b6e6 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.695167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 137/280] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:25.696073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5611c23-80f9-42fc-a0f3-d4595f49b6e6 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.744601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d60f1-d394-4bdd-90a7-478aa0eabe45 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.785347 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:25.797891 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f59bf5-4fcd-49e8-9e41-0d0016d8679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.798899 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f59bf5-4fcd-49e8-9e41-0d0016d8679f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.799099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f59bf5-4fcd-49e8-9e41-0d0016d8679f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.799846 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f59bf5-4fcd-49e8-9e41-0d0016d8679f None admin] PATH_INFO: `/v3/users/1a423ebb26d74eb6ae9faa52be2a3505` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.822985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f59bf5-4fcd-49e8-9e41-0d0016d8679f None admin] RBAC: Authorizing `identity:get_user(user_id=1a423ebb26d74eb6ae9faa52be2a3505)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:25.825205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f59bf5-4fcd-49e8-9e41-0d0016d8679f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:25.827566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 144/281] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:25 2022] GET /identity/v3/users/1a423ebb26d74eb6ae9faa52be2a3505 => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:25.831651 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d535-2acb-4515-ae9c-78246c3d8424 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.831973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d535-2acb-4515-ae9c-78246c3d8424 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.832209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d535-2acb-4515-ae9c-78246c3d8424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.852954 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-2e50d535-2acb-4515-ae9c-78246c3d8424 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:25.854259 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-2e50d535-2acb-4515-ae9c-78246c3d8424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:25.861360 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5611c23-80f9-42fc-a0f3-d4595f49b6e6 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:25.909428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17361b-6357-4a2a-a667-4c71b1b1c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:25.910144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17361b-6357-4a2a-a667-4c71b1b1c744 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:25.910461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17361b-6357-4a2a-a667-4c71b1b1c744 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:25.910740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17361b-6357-4a2a-a667-4c71b1b1c744 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:25.923632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17361b-6357-4a2a-a667-4c71b1b1c744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-2d17361b-6357-4a2a-a667-4c71b1b1c744 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:25.929743 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:25.931146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 138/282] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:25.932196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17361b-6357-4a2a-a667-4c71b1b1c744 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.962710 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 145/283] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:25.963951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50d535-2acb-4515-ae9c-78246c3d8424 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:25.976818 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:26.076041 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17361b-6357-4a2a-a667-4c71b1b1c744 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:26.117519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50d535-2acb-4515-ae9c-78246c3d8424 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:26.127398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:26.130590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:26.130590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:26.130832 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:26.131648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] RBAC: Adding query filter params (role.id=2935a7836c864df48e32fa734f020882, user.id=1a423ebb26d74eb6ae9faa52be2a3505, scope.system=all) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:26.131765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:26.133327 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:26.139263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 139/284] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 09:08:25 2022] GET /identity/v3/role_assignments?user.id=1a423ebb26d74eb6ae9faa52be2a3505&scope.system=all&role.id=2935a7836c864df48e32fa734f020882 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:26.140107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:26.141091 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:26.170817 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:26.171924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:26.172446 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:26.172938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:26.173929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:26.174516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:26.176598 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:26.188882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 146/285] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:26.195138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:26.354317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197c2bb-711d-4cf1-ad63-8917a386eb77 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:26.403003 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0688c2-f014-4547-ae0d-7d16a2ff2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:26.404684 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0688c2-f014-4547-ae0d-7d16a2ff2690 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:26.405198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0688c2-f014-4547-ae0d-7d16a2ff2690 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:26.405722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0688c2-f014-4547-ae0d-7d16a2ff2690 None admin] PATH_INFO: `/v3/users/5722aee29e314de1b5aabf7c927fdc8e` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:26.407490 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0688c2-f014-4547-ae0d-7d16a2ff2690 None admin] RBAC: Authorizing `identity:get_user(user_id=5722aee29e314de1b5aabf7c927fdc8e)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:26.409308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0688c2-f014-4547-ae0d-7d16a2ff2690 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:26.412583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 147/286] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:26 2022] GET /identity/v3/users/5722aee29e314de1b5aabf7c927fdc8e => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:26.419271 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0688c2-f014-4547-ae0d-7d16a2ff2690 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:26.425028 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:26.429271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99547774-14d9-4a62-bdf4-eb3e74d04892 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:26.497282 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74fa95-74a0-4b5a-8bc1-06f9d1fe08e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:26.497956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fa95-74a0-4b5a-8bc1-06f9d1fe08e8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:26.498284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fa95-74a0-4b5a-8bc1-06f9d1fe08e8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:26.498563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fa95-74a0-4b5a-8bc1-06f9d1fe08e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:26.519920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74fa95-74a0-4b5a-8bc1-06f9d1fe08e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:26.525158 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-ed74fa95-74a0-4b5a-8bc1-06f9d1fe08e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:26.527024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:26.528268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:26.529355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:26.530119 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 140/287] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:26.543197 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74fa95-74a0-4b5a-8bc1-06f9d1fe08e8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:26.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0688c2-f014-4547-ae0d-7d16a2ff2690 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:26.722335 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913375f0-e1b7-48ea-ac8a-eaa98fc79d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:26.723449 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913375f0-e1b7-48ea-ac8a-eaa98fc79d54 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:26.723941 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913375f0-e1b7-48ea-ac8a-eaa98fc79d54 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:26.724431 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913375f0-e1b7-48ea-ac8a-eaa98fc79d54 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:26.725849 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74fa95-74a0-4b5a-8bc1-06f9d1fe08e8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:26.726173 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913375f0-e1b7-48ea-ac8a-eaa98fc79d54 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:26.728175 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913375f0-e1b7-48ea-ac8a-eaa98fc79d54 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:26.730568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 148/288] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:26 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:26.739115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913375f0-e1b7-48ea-ac8a-eaa98fc79d54 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:26.766137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:26.766137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:26.766137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:26.766137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:26.766599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] RBAC: Adding query filter params (name=member) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:26.766599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:26.768032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:26.782247 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 141/289] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:26.784538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:26.809442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:26.947628 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1877b6-08cf-4a2b-904c-d77320d46553 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:26.960681 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913375f0-e1b7-48ea-ac8a-eaa98fc79d54 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:26.993826 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:27.023517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364909d4-ef87-40b8-a795-06fb4056bcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:27.025486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364909d4-ef87-40b8-a795-06fb4056bcaa None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:27.026264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364909d4-ef87-40b8-a795-06fb4056bcaa None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:27.027558 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364909d4-ef87-40b8-a795-06fb4056bcaa None admin] PATH_INFO: `/v3/users/9c197a97aa92420aa28209877ad1e86d` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:27.029404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364909d4-ef87-40b8-a795-06fb4056bcaa None admin] RBAC: Authorizing `identity:get_user(user_id=9c197a97aa92420aa28209877ad1e86d)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:27.032559 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364909d4-ef87-40b8-a795-06fb4056bcaa None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:27.034981 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 142/290] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:26 2022] GET /identity/v3/users/9c197a97aa92420aa28209877ad1e86d => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:27.041255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364909d4-ef87-40b8-a795-06fb4056bcaa None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:27.044133 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0377d-97e8-424d-9fa6-6b3db3e68a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:27.047236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0377d-97e8-424d-9fa6-6b3db3e68a6a None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:27.047823 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0377d-97e8-424d-9fa6-6b3db3e68a6a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:27.048313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0377d-97e8-424d-9fa6-6b3db3e68a6a None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698/users/5722aee29e314de1b5aabf7c927fdc8e/roles/73552b9e126d49b188a201b0e02d1011` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:27.057562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d0377d-97e8-424d-9fa6-6b3db3e68a6a None admin] RBAC: Authorizing `identity:create_grant(project_id=ff937f0574a548faac1d7abf71ec1698, user_id=5722aee29e314de1b5aabf7c927fdc8e, role_id=73552b9e126d49b188a201b0e02d1011)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:27.059640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d0377d-97e8-424d-9fa6-6b3db3e68a6a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:27.086553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 149/291] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:08:26 2022] PUT /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698/users/5722aee29e314de1b5aabf7c927fdc8e/roles/73552b9e126d49b188a201b0e02d1011 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:27.200268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:27.255181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364909d4-ef87-40b8-a795-06fb4056bcaa None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:27.298233 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69b9c4-47ba-4752-ac44-ce5638cffa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:27.299362 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69b9c4-47ba-4752-ac44-ce5638cffa9b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:27.299876 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69b9c4-47ba-4752-ac44-ce5638cffa9b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:27.300353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69b9c4-47ba-4752-ac44-ce5638cffa9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:27.375506 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69b9c4-47ba-4752-ac44-ce5638cffa9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:27.377366 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69b9c4-47ba-4752-ac44-ce5638cffa9b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:27.474479 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:27.492024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 143/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:27.497873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0377d-97e8-424d-9fa6-6b3db3e68a6a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:27.609337 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0377d-97e8-424d-9fa6-6b3db3e68a6a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:27.645328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a46d-443b-49c7-90c8-48d2fda7d51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:27.646006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a46d-443b-49c7-90c8-48d2fda7d51e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:27.646318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a46d-443b-49c7-90c8-48d2fda7d51e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:27.646593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a46d-443b-49c7-90c8-48d2fda7d51e None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:27.647780 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145a46d-443b-49c7-90c8-48d2fda7d51e None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:27.649292 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145a46d-443b-49c7-90c8-48d2fda7d51e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:27.651373 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 150/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:27 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:27.659287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69b9c4-47ba-4752-ac44-ce5638cffa9b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:27.793738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69b9c4-47ba-4752-ac44-ce5638cffa9b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:27.832491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58813cda-5752-4e00-9724-8fdf7df5849b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:27.833560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58813cda-5752-4e00-9724-8fdf7df5849b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:27.834066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58813cda-5752-4e00-9724-8fdf7df5849b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:27.834630 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58813cda-5752-4e00-9724-8fdf7df5849b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:27.835659 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58813cda-5752-4e00-9724-8fdf7df5849b None admin] RBAC: Adding query filter params (role.id=2935a7836c864df48e32fa734f020882, scope.project.id=ff937f0574a548faac1d7abf71ec1698, user.id=9c197a97aa92420aa28209877ad1e86d) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:27.836265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58813cda-5752-4e00-9724-8fdf7df5849b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:27.838003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58813cda-5752-4e00-9724-8fdf7df5849b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:27.845632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 144/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:27 2022] GET /identity/v3/role_assignments?user.id=9c197a97aa92420aa28209877ad1e86d&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=2935a7836c864df48e32fa734f020882 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:28.238158 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b280-d390-4b83-aae8-250c27e64a3b None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.238529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b280-d390-4b83-aae8-250c27e64a3b None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.238748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b280-d390-4b83-aae8-250c27e64a3b None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.240474 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 151/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:28.246273 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741e74c-30eb-4278-a467-4683b206d2da None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.246576 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741e74c-30eb-4278-a467-4683b206d2da None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.246869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741e74c-30eb-4278-a467-4683b206d2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.269967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-e741e74c-30eb-4278-a467-4683b206d2da None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:28.271275 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-e741e74c-30eb-4278-a467-4683b206d2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:28.339565 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:28.363603 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 145/296] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:28.388538 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0b280-d390-4b83-aae8-250c27e64a3b None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:28.424597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968c9d9-46bb-4097-9cb6-0b917ae62a2d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.424954 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968c9d9-46bb-4097-9cb6-0b917ae62a2d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.425193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968c9d9-46bb-4097-9cb6-0b917ae62a2d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.426820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 146/297] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:28.432204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fac5d-7145-4306-8bbd-b2655652c9ae None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.432500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fac5d-7145-4306-8bbd-b2655652c9ae None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.432791 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fac5d-7145-4306-8bbd-b2655652c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.442649 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:28.453648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-524fac5d-7145-4306-8bbd-b2655652c9ae None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:28.455449 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-524fac5d-7145-4306-8bbd-b2655652c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:28.507844 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0b280-d390-4b83-aae8-250c27e64a3b None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:28.538238 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 147/298] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:28.544859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab5a59-3e0f-4927-9403-c0c39c3f428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:28.545860 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab5a59-3e0f-4927-9403-c0c39c3f428f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.546390 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab5a59-3e0f-4927-9403-c0c39c3f428f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.546901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab5a59-3e0f-4927-9403-c0c39c3f428f None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.548276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab5a59-3e0f-4927-9403-c0c39c3f428f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:28.549965 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab5a59-3e0f-4927-9403-c0c39c3f428f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:28.552278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 152/299] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:28.558476 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab5a59-3e0f-4927-9403-c0c39c3f428f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:28.558709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fac5d-7145-4306-8bbd-b2655652c9ae None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:28.688878 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fac5d-7145-4306-8bbd-b2655652c9ae None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:28.693783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab5a59-3e0f-4927-9403-c0c39c3f428f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:28.706063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:28.709578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:28.741548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a944f-b8b6-4dd0-aab9-7b3978337e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:28.742365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a944f-b8b6-4dd0-aab9-7b3978337e4b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.743237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a944f-b8b6-4dd0-aab9-7b3978337e4b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.743534 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a944f-b8b6-4dd0-aab9-7b3978337e4b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.749010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd271b59-b23f-40e1-90ed-ec1838de2a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:28.750040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271b59-b23f-40e1-90ed-ec1838de2a0c None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.750588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271b59-b23f-40e1-90ed-ec1838de2a0c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.750990 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271b59-b23f-40e1-90ed-ec1838de2a0c None admin] PATH_INFO: `/v3/groups` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.751897 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd271b59-b23f-40e1-90ed-ec1838de2a0c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:28.754467 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd271b59-b23f-40e1-90ed-ec1838de2a0c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:28.757479 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a944f-b8b6-4dd0-aab9-7b3978337e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-9f6a944f-b8b6-4dd0-aab9-7b3978337e4b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:28.764533 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:28.765757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:28.767689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 148/300] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:28.774709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 153/301] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 09:08:28 2022] POST /identity/v3/groups => generated 229 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:28.775928 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a944f-b8b6-4dd0-aab9-7b3978337e4b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:28.778246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2ca0-8f41-4653-ae95-6a211b282f9c None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.778852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2ca0-8f41-4653-ae95-6a211b282f9c None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.779351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2ca0-8f41-4653-ae95-6a211b282f9c None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.781915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 154/302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:08:28.788229 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8863d96-58be-4254-afd1-21ddda342f80 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.788229 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8863d96-58be-4254-afd1-21ddda342f80 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.788520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8863d96-58be-4254-afd1-21ddda342f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.809612 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-e8863d96-58be-4254-afd1-21ddda342f80 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:28.810996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-e8863d96-58be-4254-afd1-21ddda342f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:28.907605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 155/303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:28.936152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a944f-b8b6-4dd0-aab9-7b3978337e4b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:28.937988 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8863d96-58be-4254-afd1-21ddda342f80 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:28.994073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:28.994073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:28.994073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:28.994073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:28.994776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:28.995087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:28.996505 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:29.007486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 149/304] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:29.013158 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:29.059611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8863d96-58be-4254-afd1-21ddda342f80 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:29.062547 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:29.064871 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:29.103240 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e07595-a96c-47b9-9345-2bbb2a5ba4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:29.103925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e07595-a96c-47b9-9345-2bbb2a5ba4d3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.104218 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e07595-a96c-47b9-9345-2bbb2a5ba4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.104604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e07595-a96c-47b9-9345-2bbb2a5ba4d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.117202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e07595-a96c-47b9-9345-2bbb2a5ba4d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-13e07595-a96c-47b9-9345-2bbb2a5ba4d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:29.122811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:29.123801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:29.124680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 156/305] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:29.130317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e07595-a96c-47b9-9345-2bbb2a5ba4d3 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:29.138331 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2cf95-d0c3-442b-9bb9-197ed0ef89e4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:29.175005 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fcab8-2402-4722-b091-44cbc3a6adc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:29.175689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fcab8-2402-4722-b091-44cbc3a6adc9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.175964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fcab8-2402-4722-b091-44cbc3a6adc9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.176240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fcab8-2402-4722-b091-44cbc3a6adc9 None admin] PATH_INFO: `/v3/users/5302a6e1556b4ba9a2f986ed0938778b` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.191547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40fcab8-2402-4722-b091-44cbc3a6adc9 None admin] RBAC: Authorizing `identity:get_user(user_id=5302a6e1556b4ba9a2f986ed0938778b)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:29.193609 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40fcab8-2402-4722-b091-44cbc3a6adc9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:29.200832 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 150/306] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:29 2022] GET /identity/v3/users/5302a6e1556b4ba9a2f986ed0938778b => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:29.201751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fcab8-2402-4722-b091-44cbc3a6adc9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:29.242989 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e07595-a96c-47b9-9345-2bbb2a5ba4d3 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:29.283188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:29.283839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.284127 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.284407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.285145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:29.285456 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:29.287801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:29.302282 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 157/307] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:29.305520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:29.331045 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:29.334600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:29.344081 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fcab8-2402-4722-b091-44cbc3a6adc9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:29.385188 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:29.386355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.386827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.387306 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.388357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] RBAC: Adding query filter params (role.id=73552b9e126d49b188a201b0e02d1011, user.id=5302a6e1556b4ba9a2f986ed0938778b, scope.system=all) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:29.388878 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:29.390602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:29.396717 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 151/308] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 09:08:29 2022] GET /identity/v3/role_assignments?user.id=5302a6e1556b4ba9a2f986ed0938778b&scope.system=all&role.id=73552b9e126d49b188a201b0e02d1011 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:29.439376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb19ed5-6fb0-464d-aff8-3ce864a0b1a2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:29.479414 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1c0c2-d17c-473c-947d-f799e8dcfded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:29.480519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1c0c2-d17c-473c-947d-f799e8dcfded None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.481025 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1c0c2-d17c-473c-947d-f799e8dcfded None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.481533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1c0c2-d17c-473c-947d-f799e8dcfded None admin] PATH_INFO: `/v3/users/1a423ebb26d74eb6ae9faa52be2a3505` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.483081 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1c0c2-d17c-473c-947d-f799e8dcfded None admin] RBAC: Authorizing `identity:get_user(user_id=1a423ebb26d74eb6ae9faa52be2a3505)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:29.484883 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1c0c2-d17c-473c-947d-f799e8dcfded None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:29.487303 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 158/309] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:29 2022] GET /identity/v3/users/1a423ebb26d74eb6ae9faa52be2a3505 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:29.493618 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:29.626221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f045f6-d675-48cd-a2c4-9d5a1afa1a6d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:29.663282 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ea599-dee3-4690-a435-cbf397c36ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:29.664575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea599-dee3-4690-a435-cbf397c36ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.665089 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea599-dee3-4690-a435-cbf397c36ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.665574 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea599-dee3-4690-a435-cbf397c36ee9 None admin] PATH_INFO: `/v3/system/users/1a423ebb26d74eb6ae9faa52be2a3505/roles/2935a7836c864df48e32fa734f020882` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.667365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ea599-dee3-4690-a435-cbf397c36ee9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1a423ebb26d74eb6ae9faa52be2a3505, role_id=2935a7836c864df48e32fa734f020882)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:29.669341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94ad62-cd42-4af7-bfb9-a5a1f4ff3bc0 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.669844 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94ad62-cd42-4af7-bfb9-a5a1f4ff3bc0 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.669928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94ad62-cd42-4af7-bfb9-a5a1f4ff3bc0 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.671451 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 159/310] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:08:29.672694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ea599-dee3-4690-a435-cbf397c36ee9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:29.678571 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5742e-8339-47de-815b-61e65b7c11a0 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.679107 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5742e-8339-47de-815b-61e65b7c11a0 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.679560 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5742e-8339-47de-815b-61e65b7c11a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.685066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 152/311] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 09:08:29 2022] PUT /identity/v3/system/users/1a423ebb26d74eb6ae9faa52be2a3505/roles/2935a7836c864df48e32fa734f020882 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:29.701564 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-54c5742e-8339-47de-815b-61e65b7c11a0 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:29.703147 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-54c5742e-8339-47de-815b-61e65b7c11a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:29.790662 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 160/312] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:29.811203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ea599-dee3-4690-a435-cbf397c36ee9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:29.838184 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:29.935966 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ea599-dee3-4690-a435-cbf397c36ee9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:29.972512 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcec735-bc06-437f-9e7a-f6dbba00ef36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:29.973186 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec735-bc06-437f-9e7a-f6dbba00ef36 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:29.973528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec735-bc06-437f-9e7a-f6dbba00ef36 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:29.973813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec735-bc06-437f-9e7a-f6dbba00ef36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:29.985211 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcec735-bc06-437f-9e7a-f6dbba00ef36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-adcec735-bc06-437f-9e7a-f6dbba00ef36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:29.990685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:29.991635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:29.992648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 153/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:29.998521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5742e-8339-47de-815b-61e65b7c11a0 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:30.018127 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:30.108738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5742e-8339-47de-815b-61e65b7c11a0 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:30.145588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:30.146283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:30.146570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:30.146852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:30.147601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] RBAC: Adding query filter params (name=reader) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:30.147916 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:30.149404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:30.158637 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 161/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:30.164832 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcec735-bc06-437f-9e7a-f6dbba00ef36 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:30.274981 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:30.302209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcec735-bc06-437f-9e7a-f6dbba00ef36 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:30.341925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f7fcc-a812-40c2-9606-0d46657cc0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:30.342971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f7fcc-a812-40c2-9606-0d46657cc0a4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:30.343470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f7fcc-a812-40c2-9606-0d46657cc0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:30.343947 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f7fcc-a812-40c2-9606-0d46657cc0a4 None admin] PATH_INFO: `/v3/users/5722aee29e314de1b5aabf7c927fdc8e` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:30.345439 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f7fcc-a812-40c2-9606-0d46657cc0a4 None admin] RBAC: Authorizing `identity:get_user(user_id=5722aee29e314de1b5aabf7c927fdc8e)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:30.347209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f7fcc-a812-40c2-9606-0d46657cc0a4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:30.349547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 154/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:30 2022] GET /identity/v3/users/5722aee29e314de1b5aabf7c927fdc8e => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:30.355438 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:30.452856 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:30.474949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c172669-47a3-4aeb-9282-61ec80500309 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:30.518004 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622c4a3-e45c-4b1f-8c0b-3684de61b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:30.518714 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c4a3-e45c-4b1f-8c0b-3684de61b6c8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:30.519007 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c4a3-e45c-4b1f-8c0b-3684de61b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:30.519289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c4a3-e45c-4b1f-8c0b-3684de61b6c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:30.597011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622c4a3-e45c-4b1f-8c0b-3684de61b6c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:30.598563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622c4a3-e45c-4b1f-8c0b-3684de61b6c8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:30.682961 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:30.718237 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 162/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:30.720915 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f7fcc-a812-40c2-9606-0d46657cc0a4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:30.825793 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f7fcc-a812-40c2-9606-0d46657cc0a4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:30.860881 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4506f3c-9de7-4036-b4e4-7f95e89bed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:30.861660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506f3c-9de7-4036-b4e4-7f95e89bed65 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:30.861994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506f3c-9de7-4036-b4e4-7f95e89bed65 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:30.862299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506f3c-9de7-4036-b4e4-7f95e89bed65 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:30.863516 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4506f3c-9de7-4036-b4e4-7f95e89bed65 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:30.864991 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4506f3c-9de7-4036-b4e4-7f95e89bed65 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:30.866769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 155/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:30 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:30.873665 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622c4a3-e45c-4b1f-8c0b-3684de61b6c8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:30.981301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622c4a3-e45c-4b1f-8c0b-3684de61b6c8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:31.018531 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987a7ad-150a-43df-84b1-eef7180086b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:31.019183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987a7ad-150a-43df-84b1-eef7180086b6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.019472 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987a7ad-150a-43df-84b1-eef7180086b6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.019756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987a7ad-150a-43df-84b1-eef7180086b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.020575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987a7ad-150a-43df-84b1-eef7180086b6 None admin] RBAC: Adding query filter params (role.id=73552b9e126d49b188a201b0e02d1011, scope.project.id=ff937f0574a548faac1d7abf71ec1698, user.id=5722aee29e314de1b5aabf7c927fdc8e) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:31.020894 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987a7ad-150a-43df-84b1-eef7180086b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:31.022762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987a7ad-150a-43df-84b1-eef7180086b6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:31.029579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 163/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:30 2022] GET /identity/v3/role_assignments?user.id=5722aee29e314de1b5aabf7c927fdc8e&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=73552b9e126d49b188a201b0e02d1011 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:31.119446 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525a87a-bf98-40e6-b91e-b47dfc6bf880 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.119808 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525a87a-bf98-40e6-b91e-b47dfc6bf880 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.120200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525a87a-bf98-40e6-b91e-b47dfc6bf880 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.121800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 156/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:31.127901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11833-a103-412c-9b49-322efb4e201f None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.128349 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11833-a103-412c-9b49-322efb4e201f None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.128728 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11833-a103-412c-9b49-322efb4e201f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.148971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-54d11833-a103-412c-9b49-322efb4e201f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:31.150307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-54d11833-a103-412c-9b49-322efb4e201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:31.198202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:31.237840 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 164/320] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:31.260338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525a87a-bf98-40e6-b91e-b47dfc6bf880 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:31.281144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:31.372146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525a87a-bf98-40e6-b91e-b47dfc6bf880 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:31.409471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306b7cd-88f9-49bb-8075-4d252e98c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:31.410486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b7cd-88f9-49bb-8075-4d252e98c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.410945 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b7cd-88f9-49bb-8075-4d252e98c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.411383 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b7cd-88f9-49bb-8075-4d252e98c0c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.412737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306b7cd-88f9-49bb-8075-4d252e98c0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:31.414457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306b7cd-88f9-49bb-8075-4d252e98c0c1 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:31.416260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 157/321] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:31.422257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d11833-a103-412c-9b49-322efb4e201f None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:31.496680 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e61eee-da7c-4fcd-a80f-73e23d7d6c32 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.496981 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e61eee-da7c-4fcd-a80f-73e23d7d6c32 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.497240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e61eee-da7c-4fcd-a80f-73e23d7d6c32 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.498787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 158/322] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:08:31.503795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1be1ba-9391-4552-a67f-181d6ccd5577 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.504075 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1be1ba-9391-4552-a67f-181d6ccd5577 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.504346 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1be1ba-9391-4552-a67f-181d6ccd5577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.523821 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-4a1be1ba-9391-4552-a67f-181d6ccd5577 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:31.525066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-4a1be1ba-9391-4552-a67f-181d6ccd5577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:31.537632 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d11833-a103-412c-9b49-322efb4e201f None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:31.576930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b411c4d-b884-4c27-a9a7-90c1d2d2e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:31.577633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b411c4d-b884-4c27-a9a7-90c1d2d2e7e4 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.577921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b411c4d-b884-4c27-a9a7-90c1d2d2e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.578206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b411c4d-b884-4c27-a9a7-90c1d2d2e7e4 None admin] PATH_INFO: `/v3/groups` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.579072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b411c4d-b884-4c27-a9a7-90c1d2d2e7e4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:31.581248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b411c4d-b884-4c27-a9a7-90c1d2d2e7e4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:31.602230 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 159/323] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:31.614135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 165/324] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 09:08:31 2022] POST /identity/v3/groups => generated 226 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:08:31.625904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1be1ba-9391-4552-a67f-181d6ccd5577 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:31.642995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:31.742746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1be1ba-9391-4552-a67f-181d6ccd5577 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:31.783005 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32eb26-7fdc-4ac6-9cac-d564b4ff3ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:31.783679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32eb26-7fdc-4ac6-9cac-d564b4ff3ca2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.783968 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32eb26-7fdc-4ac6-9cac-d564b4ff3ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.784247 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32eb26-7fdc-4ac6-9cac-d564b4ff3ca2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.795700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32eb26-7fdc-4ac6-9cac-d564b4ff3ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-ac32eb26-7fdc-4ac6-9cac-d564b4ff3ca2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:31.800740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:31.801797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:31.802895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 160/325] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:31.808091 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b411c4d-b884-4c27-a9a7-90c1d2d2e7e4 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:31.846490 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:31.853893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7b71-94bd-43f1-8f21-b24359a99a76 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.853893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7b71-94bd-43f1-8f21-b24359a99a76 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.853893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7b71-94bd-43f1-8f21-b24359a99a76 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.855461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 161/326] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:08:31.862669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a76318-624a-4498-9738-53d6fef1505f None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.863019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a76318-624a-4498-9738-53d6fef1505f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.863243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a76318-624a-4498-9738-53d6fef1505f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.888758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-f7a76318-624a-4498-9738-53d6fef1505f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:31.890574 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-f7a76318-624a-4498-9738-53d6fef1505f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:31.937820 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b411c4d-b884-4c27-a9a7-90c1d2d2e7e4 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:31.993159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 162/327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:31.994919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:31.995582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:31.995864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:31.996137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:31.996882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] RBAC: Adding query filter params (name=reader) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:31.997189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:31.998620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:32.008239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 166/328] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:32.013959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a76318-624a-4498-9738-53d6fef1505f None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:32.016231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:32.019625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:32.149661 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:32.153513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a76318-624a-4498-9738-53d6fef1505f None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:32.157704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1b4aa-2c1f-49e8-9ee2-ac4f5fb61559 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:32.198624 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a69fe-5f57-479b-9755-907fe9507f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:32.199316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a69fe-5f57-479b-9755-907fe9507f25 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:32.199603 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a69fe-5f57-479b-9755-907fe9507f25 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:32.199886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a69fe-5f57-479b-9755-907fe9507f25 None admin] PATH_INFO: `/v3/users/5302a6e1556b4ba9a2f986ed0938778b` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:32.202247 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a69fe-5f57-479b-9755-907fe9507f25 None admin] RBAC: Authorizing `identity:get_user(user_id=5302a6e1556b4ba9a2f986ed0938778b)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:32.203747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a69fe-5f57-479b-9755-907fe9507f25 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:32.206053 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 163/329] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:32 2022] GET /identity/v3/users/5302a6e1556b4ba9a2f986ed0938778b => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:32.209612 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80747fb1-2914-4bda-8edb-d5d94c2cb81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:32.210335 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80747fb1-2914-4bda-8edb-d5d94c2cb81c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:32.210619 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80747fb1-2914-4bda-8edb-d5d94c2cb81c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:32.210896 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80747fb1-2914-4bda-8edb-d5d94c2cb81c None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:32.212879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a69fe-5f57-479b-9755-907fe9507f25 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:32.226407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80747fb1-2914-4bda-8edb-d5d94c2cb81c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-80747fb1-2914-4bda-8edb-d5d94c2cb81c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:32.233991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:32.235060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:32.238857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 167/330] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:32.246138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80747fb1-2914-4bda-8edb-d5d94c2cb81c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:32.313117 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:32.355774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a69fe-5f57-479b-9755-907fe9507f25 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:32.395078 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80747fb1-2914-4bda-8edb-d5d94c2cb81c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:32.436556 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:32.437107 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:32.437405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:32.437758 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:32.438536 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:32.438846 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:32.439492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c5113-641e-4d2b-b1ec-f64ff07a8aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:32.440256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:32.440622 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c5113-641e-4d2b-b1ec-f64ff07a8aea None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:32.441097 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c5113-641e-4d2b-b1ec-f64ff07a8aea None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:32.441567 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c5113-641e-4d2b-b1ec-f64ff07a8aea None admin] PATH_INFO: `/v3/system/users/5302a6e1556b4ba9a2f986ed0938778b/roles/73552b9e126d49b188a201b0e02d1011` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:32.446948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c5113-641e-4d2b-b1ec-f64ff07a8aea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5302a6e1556b4ba9a2f986ed0938778b, role_id=73552b9e126d49b188a201b0e02d1011)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:32.448678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c5113-641e-4d2b-b1ec-f64ff07a8aea None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:32.450367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 168/331] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:32.457045 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:32.464140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 164/332] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 09:08:32 2022] PUT /identity/v3/system/users/5302a6e1556b4ba9a2f986ed0938778b/roles/73552b9e126d49b188a201b0e02d1011 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:08:32.492050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:32.603940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a7eba-4ead-4de9-9ed3-87fdfa598514 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:32.652371 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ffeea-1e16-4927-86df-0439cf4d2e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:32.653506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ffeea-1e16-4927-86df-0439cf4d2e61 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:32.654032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ffeea-1e16-4927-86df-0439cf4d2e61 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:32.654533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ffeea-1e16-4927-86df-0439cf4d2e61 None admin] PATH_INFO: `/v3/users/1a423ebb26d74eb6ae9faa52be2a3505` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:32.659023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ffeea-1e16-4927-86df-0439cf4d2e61 None admin] RBAC: Authorizing `identity:get_user(user_id=1a423ebb26d74eb6ae9faa52be2a3505)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:32.660920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ffeea-1e16-4927-86df-0439cf4d2e61 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:32.662995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 169/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:32 2022] GET /identity/v3/users/1a423ebb26d74eb6ae9faa52be2a3505 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:32.669585 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c5113-641e-4d2b-b1ec-f64ff07a8aea None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:32.790265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c5113-641e-4d2b-b1ec-f64ff07a8aea None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:32.839491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b9f4b-808e-4fc4-afb0-adefa6dfc654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:32.840508 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b9f4b-808e-4fc4-afb0-adefa6dfc654 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:32.840991 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b9f4b-808e-4fc4-afb0-adefa6dfc654 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:32.841463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b9f4b-808e-4fc4-afb0-adefa6dfc654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:32.842523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b9f4b-808e-4fc4-afb0-adefa6dfc654 None admin] RBAC: Adding query filter params (role.id=2935a7836c864df48e32fa734f020882, user.id=1a423ebb26d74eb6ae9faa52be2a3505, scope.system=all) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:32.843075 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b9f4b-808e-4fc4-afb0-adefa6dfc654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:32.844773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b9f4b-808e-4fc4-afb0-adefa6dfc654 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:32.851404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:32.865259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 165/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 09:08:32 2022] GET /identity/v3/role_assignments?user.id=1a423ebb26d74eb6ae9faa52be2a3505&scope.system=all&role.id=2935a7836c864df48e32fa734f020882 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:34.567382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e79d-a154-49cc-b6b0-9f09205de5ee None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:34.567897 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e79d-a154-49cc-b6b0-9f09205de5ee None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:34.567897 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e79d-a154-49cc-b6b0-9f09205de5ee None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:34.569645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 170/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:34.588537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79719689-f9d8-4218-8011-3ef16c0b12b5 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:34.589095 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79719689-f9d8-4218-8011-3ef16c0b12b5 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:34.589095 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79719689-f9d8-4218-8011-3ef16c0b12b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:34.619266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-79719689-f9d8-4218-8011-3ef16c0b12b5 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:34.620506 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-79719689-f9d8-4218-8011-3ef16c0b12b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:34.701006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 166/336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:34.736293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e79d-a154-49cc-b6b0-9f09205de5ee None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:34.845800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:34.863939 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e79d-a154-49cc-b6b0-9f09205de5ee None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:34.906909 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883748cd-bacc-463b-9c67-ab6448ad2de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:34.907583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883748cd-bacc-463b-9c67-ab6448ad2de8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:34.907866 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883748cd-bacc-463b-9c67-ab6448ad2de8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:34.908151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883748cd-bacc-463b-9c67-ab6448ad2de8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:34.924686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883748cd-bacc-463b-9c67-ab6448ad2de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-883748cd-bacc-463b-9c67-ab6448ad2de8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:34.930836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:34.931841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:34.934553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 171/337] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:34.939504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79719689-f9d8-4218-8011-3ef16c0b12b5 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:35.057872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79719689-f9d8-4218-8011-3ef16c0b12b5 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:35.090229 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:35.103984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e6b4c-10c7-46fb-a665-99620c576dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:35.104556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e6b4c-10c7-46fb-a665-99620c576dbb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.104840 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e6b4c-10c7-46fb-a665-99620c576dbb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.105116 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e6b4c-10c7-46fb-a665-99620c576dbb None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.105877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e6b4c-10c7-46fb-a665-99620c576dbb None admin] RBAC: Adding query filter params (name=member) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:35.106206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e6b4c-10c7-46fb-a665-99620c576dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:35.107636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e6b4c-10c7-46fb-a665-99620c576dbb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:35.120515 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 167/338] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:35.126554 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883748cd-bacc-463b-9c67-ab6448ad2de8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:35.238700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883748cd-bacc-463b-9c67-ab6448ad2de8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:35.278003 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40402ba9-9b9a-40bd-8d79-47b2514007df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:35.278936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402ba9-9b9a-40bd-8d79-47b2514007df None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.279230 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402ba9-9b9a-40bd-8d79-47b2514007df None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.279717 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402ba9-9b9a-40bd-8d79-47b2514007df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.305177 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:35.315020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6ca3-cc4f-40a6-b0ff-7e3bceb0925e None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.315690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6ca3-cc4f-40a6-b0ff-7e3bceb0925e None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.316194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6ca3-cc4f-40a6-b0ff-7e3bceb0925e None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.320101 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 168/339] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:35.325041 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317f25b-dd11-4a95-89fb-a8c9d548be38 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.325664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317f25b-dd11-4a95-89fb-a8c9d548be38 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.326212 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317f25b-dd11-4a95-89fb-a8c9d548be38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.356200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-5317f25b-dd11-4a95-89fb-a8c9d548be38 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:35.360376 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-5317f25b-dd11-4a95-89fb-a8c9d548be38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:35.401431 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40402ba9-9b9a-40bd-8d79-47b2514007df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:35.403373 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40402ba9-9b9a-40bd-8d79-47b2514007df None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:35.502258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 169/340] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:35.526621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317f25b-dd11-4a95-89fb-a8c9d548be38 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:35.615339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 172/341] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:35.620881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40402ba9-9b9a-40bd-8d79-47b2514007df None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:35.703575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317f25b-dd11-4a95-89fb-a8c9d548be38 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:35.705017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:35.729185 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:35.749621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c26fc9-3909-4749-b039-58f09bdf7e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:35.750342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c26fc9-3909-4749-b039-58f09bdf7e72 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.750632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c26fc9-3909-4749-b039-58f09bdf7e72 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.750912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c26fc9-3909-4749-b039-58f09bdf7e72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.765552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c26fc9-3909-4749-b039-58f09bdf7e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-76c26fc9-3909-4749-b039-58f09bdf7e72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:35.772237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:35.773199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:35.774142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 170/342] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:35.775378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea25ef-a500-43aa-984d-6ac741e23d16 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.775561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea25ef-a500-43aa-984d-6ac741e23d16 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.775820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea25ef-a500-43aa-984d-6ac741e23d16 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.778545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 171/343] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:35.778545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d212-4a3f-479f-ab08-1be70d8dbfc5 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.778761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d212-4a3f-479f-ab08-1be70d8dbfc5 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.778952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d212-4a3f-479f-ab08-1be70d8dbfc5 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.781648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 172/344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:35.782576 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0d212-4a3f-479f-ab08-1be70d8dbfc5 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:35.800758 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40402ba9-9b9a-40bd-8d79-47b2514007df None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:35.851164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a64bdb-adec-40cc-8802-981271286172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:35.851926 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a64bdb-adec-40cc-8802-981271286172 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.852212 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a64bdb-adec-40cc-8802-981271286172 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.852507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a64bdb-adec-40cc-8802-981271286172 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.854135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a64bdb-adec-40cc-8802-981271286172 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:35.855438 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a64bdb-adec-40cc-8802-981271286172 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:35.857471 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 173/345] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:35 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:35.860131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf1013-211c-4950-8d94-2f1029687edc None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.860131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf1013-211c-4950-8d94-2f1029687edc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.860131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf1013-211c-4950-8d94-2f1029687edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.879920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-48cf1013-211c-4950-8d94-2f1029687edc None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:35.881346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-48cf1013-211c-4950-8d94-2f1029687edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:35.920405 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0d212-4a3f-479f-ab08-1be70d8dbfc5 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:35.959786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 174/346] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:35.961155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79895763-02d4-4cee-bc01-73a0c9bd9bba None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.961452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79895763-02d4-4cee-bc01-73a0c9bd9bba None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.961732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79895763-02d4-4cee-bc01-73a0c9bd9bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.962525 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:35.963433 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:35.963937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:35.964422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:35.965402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:35.966195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:35.968082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:35.979372 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 173/347] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:35.980732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:35.982994 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-79895763-02d4-4cee-bc01-73a0c9bd9bba None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:35.984642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-79895763-02d4-4cee-bc01-73a0c9bd9bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:36.030139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:36.035946 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:36.073898 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 175/348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:36.074452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79895763-02d4-4cee-bc01-73a0c9bd9bba None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:36.107527 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d687812-698c-4076-8448-d58cb9def8a8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:36.146966 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9600b97-7616-47ab-b7bd-a20bedd077d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:36.148085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9600b97-7616-47ab-b7bd-a20bedd077d9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.148722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9600b97-7616-47ab-b7bd-a20bedd077d9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.149220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9600b97-7616-47ab-b7bd-a20bedd077d9 None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.156569 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9600b97-7616-47ab-b7bd-a20bedd077d9 None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:36.158572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9600b97-7616-47ab-b7bd-a20bedd077d9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:36.160694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 174/349] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:35 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:36.161768 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9600b97-7616-47ab-b7bd-a20bedd077d9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:36.186686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79895763-02d4-4cee-bc01-73a0c9bd9bba None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:36.228238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be464e-cc76-4748-9758-a6e94bb5d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:36.228875 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be464e-cc76-4748-9758-a6e94bb5d398 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.229164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be464e-cc76-4748-9758-a6e94bb5d398 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.229497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be464e-cc76-4748-9758-a6e94bb5d398 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.279169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:36.282518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9600b97-7616-47ab-b7bd-a20bedd077d9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:36.284711 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:36.312699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be464e-cc76-4748-9758-a6e94bb5d398 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:36.314240 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be464e-cc76-4748-9758-a6e94bb5d398 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:36.324572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95061a-90cd-409b-9908-6e5b709667ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:36.325250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95061a-90cd-409b-9908-6e5b709667ea None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.325539 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95061a-90cd-409b-9908-6e5b709667ea None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95061a-90cd-409b-9908-6e5b709667ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.338376 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95061a-90cd-409b-9908-6e5b709667ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-5b95061a-90cd-409b-9908-6e5b709667ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:36.345499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:36.346796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 175/350] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:36.347703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95061a-90cd-409b-9908-6e5b709667ea None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:36.445668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 176/351] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:36.446379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be464e-cc76-4748-9758-a6e94bb5d398 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:36.482537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95061a-90cd-409b-9908-6e5b709667ea None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:36.528568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbb900-a366-4e35-84cb-1466468b7422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:36.529892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbb900-a366-4e35-84cb-1466468b7422 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.530426 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbb900-a366-4e35-84cb-1466468b7422 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.530895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbb900-a366-4e35-84cb-1466468b7422 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.543816 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:36.546450 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbb900-a366-4e35-84cb-1466468b7422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-c0dbb900-a366-4e35-84cb-1466468b7422 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:36.552284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:36.553297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:36.554205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 176/352] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:36.556918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744d297-ae87-4364-a8f3-ce92fc9429af None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.557215 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744d297-ae87-4364-a8f3-ce92fc9429af None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.557494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744d297-ae87-4364-a8f3-ce92fc9429af None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.559020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 177/353] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:36.563510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744d297-ae87-4364-a8f3-ce92fc9429af None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:36.587139 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be464e-cc76-4748-9758-a6e94bb5d398 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:36.588162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:36.626180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:36.626855 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.627094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.627374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.628186 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] RBAC: Adding query filter params (group.id=1e8110403a944b5c802e3878e5ff5d12, role.id=2935a7836c864df48e32fa734f020882, scope.project.id=6a427cb96a904a66aca0187281bdad57) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:36.628510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:36.629989 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:36.640432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 177/354] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/role_assignments?group.id=1e8110403a944b5c802e3878e5ff5d12&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=2935a7836c864df48e32fa734f020882 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:36.641721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:36.701371 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744d297-ae87-4364-a8f3-ce92fc9429af None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:36.758139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:36.758875 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.759138 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.759381 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.760146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] RBAC: Adding query filter params (name=member) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:36.760461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:36.761982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:36.777179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 178/355] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:36.777853 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:36.794295 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8b67-a8fb-4e97-8298-1bd3a85b9b5e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:36.852238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0d2d6-e623-4709-8b9d-df7c5a12b12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:36.853672 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0d2d6-e623-4709-8b9d-df7c5a12b12d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.853963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0d2d6-e623-4709-8b9d-df7c5a12b12d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.854394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0d2d6-e623-4709-8b9d-df7c5a12b12d None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.856388 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0d2d6-e623-4709-8b9d-df7c5a12b12d None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:36.857920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0d2d6-e623-4709-8b9d-df7c5a12b12d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:36.860679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 178/356] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:36.866730 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b217fa-7a1b-4108-8c67-28eb5a689c1f None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.867080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b217fa-7a1b-4108-8c67-28eb5a689c1f None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.867380 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b217fa-7a1b-4108-8c67-28eb5a689c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.884495 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:36.893755 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-d6b217fa-7a1b-4108-8c67-28eb5a689c1f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:36.895120 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-d6b217fa-7a1b-4108-8c67-28eb5a689c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:36.910798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:36.928387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa3883-362c-4dcf-b1b4-2caa22c94ea3 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:36.975569 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:36.976148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.976433 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.976762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.977537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] RBAC: Adding query filter params (name=reader) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:36.977840 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:36.980183 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:36.989834 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 179/357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:36.991757 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330ae8-c63e-4d4b-9612-e4461aec334b None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:36.992362 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330ae8-c63e-4d4b-9612-e4461aec334b None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:36.992800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 179/358] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:36.992952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330ae8-c63e-4d4b-9612-e4461aec334b None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:36.993787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.002136 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 180/359] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:37.002136 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86330ae8-c63e-4d4b-9612-e4461aec334b None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.142331 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e272-fe7d-4bcf-9dea-bf4791b33eed None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:37.150469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86330ae8-c63e-4d4b-9612-e4461aec334b None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:37.181843 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13906f47-afd4-44eb-be22-8eb820d01c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:37.182527 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13906f47-afd4-44eb-be22-8eb820d01c06 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:37.182813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13906f47-afd4-44eb-be22-8eb820d01c06 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:37.183088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13906f47-afd4-44eb-be22-8eb820d01c06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:37.190283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f817c47-71cf-4ec6-b6f2-aeb21e8ed193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:37.191405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f817c47-71cf-4ec6-b6f2-aeb21e8ed193 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:37.191405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f817c47-71cf-4ec6-b6f2-aeb21e8ed193 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:37.191584 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f817c47-71cf-4ec6-b6f2-aeb21e8ed193 None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:37.193051 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f817c47-71cf-4ec6-b6f2-aeb21e8ed193 None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:37.194569 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f817c47-71cf-4ec6-b6f2-aeb21e8ed193 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:37.196684 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 181/360] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:37.197811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b2b80-b8b1-49a5-94b5-6995d95c48dd None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:37.198083 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b2b80-b8b1-49a5-94b5-6995d95c48dd None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:37.199501 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b2b80-b8b1-49a5-94b5-6995d95c48dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:37.208457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:37.212005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:37.225423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-d93b2b80-b8b1-49a5-94b5-6995d95c48dd None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:37.228170 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-d93b2b80-b8b1-49a5-94b5-6995d95c48dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:37.283867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13906f47-afd4-44eb-be22-8eb820d01c06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:37.285668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13906f47-afd4-44eb-be22-8eb820d01c06 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:37.319561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 182/361] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:37.320535 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b2b80-b8b1-49a5-94b5-6995d95c48dd None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.418192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 180/362] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:37.419594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13906f47-afd4-44eb-be22-8eb820d01c06 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.460364 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b2b80-b8b1-49a5-94b5-6995d95c48dd None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:37.483056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:37.509249 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486d908-ae32-40a9-ad08-f4c853f56aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:37.509942 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486d908-ae32-40a9-ad08-f4c853f56aa9 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:37.510279 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486d908-ae32-40a9-ad08-f4c853f56aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:37.510565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486d908-ae32-40a9-ad08-f4c853f56aa9 None admin] PATH_INFO: `/v3/users/5302a6e1556b4ba9a2f986ed0938778b` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:37.512268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486d908-ae32-40a9-ad08-f4c853f56aa9 None admin] RBAC: Authorizing `identity:get_user(user_id=5302a6e1556b4ba9a2f986ed0938778b)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:37.513734 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486d908-ae32-40a9-ad08-f4c853f56aa9 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:37.516909 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 183/363] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:08:37 2022] GET /identity/v3/users/5302a6e1556b4ba9a2f986ed0938778b => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:37.516909 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486d908-ae32-40a9-ad08-f4c853f56aa9 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.524734 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:37.551567 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13906f47-afd4-44eb-be22-8eb820d01c06 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:37.601219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86686dd-0f19-4884-b84e-72c94f809f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:37.602038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86686dd-0f19-4884-b84e-72c94f809f64 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:37.602423 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86686dd-0f19-4884-b84e-72c94f809f64 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:37.602700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86686dd-0f19-4884-b84e-72c94f809f64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:37.615491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86686dd-0f19-4884-b84e-72c94f809f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a86686dd-0f19-4884-b84e-72c94f809f64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:37.620593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:37.621605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:37.622568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 181/364] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:37.623300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86686dd-0f19-4884-b84e-72c94f809f64 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.658642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486d908-ae32-40a9-ad08-f4c853f56aa9 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:37.703781 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:37.704703 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:37.707087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:37.707087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:37.708016 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] RBAC: Adding query filter params (group.id=1e8110403a944b5c802e3878e5ff5d12, role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=6a427cb96a904a66aca0187281bdad57) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:37.708345 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:37.710359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:37.726709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 184/365] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:37 2022] GET /identity/v3/role_assignments?group.id=1e8110403a944b5c802e3878e5ff5d12&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=601939cb90d641e08b97775c0c12d374 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:37.727749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.746689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86686dd-0f19-4884-b84e-72c94f809f64 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:37.758595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:37.787916 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc2e14-7281-40e3-9ee8-350710f95320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:37.788767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2e14-7281-40e3-9ee8-350710f95320 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:37.789173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2e14-7281-40e3-9ee8-350710f95320 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:37.789518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2e14-7281-40e3-9ee8-350710f95320 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:37.802706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc2e14-7281-40e3-9ee8-350710f95320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:37.806536 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-dccc2e14-7281-40e3-9ee8-350710f95320 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:37.808892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:37.810162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 182/366] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:08:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:37.811088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc2e14-7281-40e3-9ee8-350710f95320 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.864775 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731af7a-1a8e-4fdc-9095-ef57d72e9f34 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:37.916320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cf6bb-3747-4bf6-b3c0-5ed43ed2a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:37.917050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cf6bb-3747-4bf6-b3c0-5ed43ed2a6da None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:37.917338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cf6bb-3747-4bf6-b3c0-5ed43ed2a6da None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:37.918365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cf6bb-3747-4bf6-b3c0-5ed43ed2a6da None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:37.919713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cf6bb-3747-4bf6-b3c0-5ed43ed2a6da None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:37.921832 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cf6bb-3747-4bf6-b3c0-5ed43ed2a6da None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:37.924886 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 185/367] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:37 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:37.925969 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cf6bb-3747-4bf6-b3c0-5ed43ed2a6da None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:37.957858 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc2e14-7281-40e3-9ee8-350710f95320 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:38.003576 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:38.004147 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:38.004434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:38.004717 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:38.005541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] RBAC: Adding query filter params (role.id=73552b9e126d49b188a201b0e02d1011, user.id=5302a6e1556b4ba9a2f986ed0938778b, scope.system=all) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:38.005859 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:38.007386 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:38.020667 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 183/368] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 09:08:37 2022] GET /identity/v3/role_assignments?user.id=5302a6e1556b4ba9a2f986ed0938778b&scope.system=all&role.id=73552b9e126d49b188a201b0e02d1011 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:38.021552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:38.046776 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cf6bb-3747-4bf6-b3c0-5ed43ed2a6da None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:38.052092 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:38.089464 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:38.090369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:38.090664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:38.091287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:38.092052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:38.092372 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:38.094358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:38.096592 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:38.107687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 186/369] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:38.108953 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:38.162408 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb61b4-946b-4fa4-8392-76c203032ffc None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:38.245596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f70ab5-22c7-473f-bcb3-422b0cb2f155 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:38.247862 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:38.248630 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:38.248921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:38.249199 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:38.249939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:38.250274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:38.251705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:38.269818 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 184/370] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:38.270645 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:38.291177 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346be9cf-a55c-44c1-bd82-058e451e0069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:38.291882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be9cf-a55c-44c1-bd82-058e451e0069 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:38.292175 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be9cf-a55c-44c1-bd82-058e451e0069 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:38.292459 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be9cf-a55c-44c1-bd82-058e451e0069 None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:38.294945 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346be9cf-a55c-44c1-bd82-058e451e0069 None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:38.296644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346be9cf-a55c-44c1-bd82-058e451e0069 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:38.298782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 187/371] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:38 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:38.302426 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346be9cf-a55c-44c1-bd82-058e451e0069 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:38.406403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:38.412270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:38.434513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d9b38-3c2c-45d8-aea3-69e4ea968d88 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:38.466514 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346be9cf-a55c-44c1-bd82-058e451e0069 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:38.484137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c35e5-42a0-4a71-97a0-500af5cef0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:38.484764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c35e5-42a0-4a71-97a0-500af5cef0d0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:38.485041 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c35e5-42a0-4a71-97a0-500af5cef0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:38.485322 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c35e5-42a0-4a71-97a0-500af5cef0d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:38.514712 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a301a-db39-4174-b6c1-a97e8d9e53d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:38.515369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a301a-db39-4174-b6c1-a97e8d9e53d0 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:38.515721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a301a-db39-4174-b6c1-a97e8d9e53d0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:38.516002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a301a-db39-4174-b6c1-a97e8d9e53d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:38.567780 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c35e5-42a0-4a71-97a0-500af5cef0d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:38.569203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c35e5-42a0-4a71-97a0-500af5cef0d0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:38.598595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a301a-db39-4174-b6c1-a97e8d9e53d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:38.600503 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a301a-db39-4174-b6c1-a97e8d9e53d0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:38.686322 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:38.687910 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:38.709172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 185/372] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:38.710191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c35e5-42a0-4a71-97a0-500af5cef0d0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:38.738637 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 188/373] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:38.739418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a301a-db39-4174-b6c1-a97e8d9e53d0 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:38.846927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c35e5-42a0-4a71-97a0-500af5cef0d0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:38.861940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a301a-db39-4174-b6c1-a97e8d9e53d0 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:38.895411 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:38.896079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:38.896368 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:38.896650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:38.897757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] RBAC: Adding query filter params (group.id=1e8110403a944b5c802e3878e5ff5d12, role.id=2935a7836c864df48e32fa734f020882, scope.project.id=ff937f0574a548faac1d7abf71ec1698) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:38.898076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:38.900891 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:38.910151 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 186/374] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:38 2022] GET /identity/v3/role_assignments?group.id=1e8110403a944b5c802e3878e5ff5d12&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=2935a7836c864df48e32fa734f020882 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:38.910151 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:38.918971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16e68-ac81-4da9-aaa6-3f6fcf292b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:38.919691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16e68-ac81-4da9-aaa6-3f6fcf292b15 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:38.919985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16e68-ac81-4da9-aaa6-3f6fcf292b15 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:38.920268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16e68-ac81-4da9-aaa6-3f6fcf292b15 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:38.921619 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c16e68-ac81-4da9-aaa6-3f6fcf292b15 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:38.923139 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c16e68-ac81-4da9-aaa6-3f6fcf292b15 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:38.925193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 189/375] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:38 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:38.930777 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16e68-ac81-4da9-aaa6-3f6fcf292b15 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:38.974723 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:39.017603 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:39.077756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16e68-ac81-4da9-aaa6-3f6fcf292b15 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:39.090753 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139f35-b302-4dd1-a7bc-1f580525450f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:39.133492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d888b-b9ff-4c58-a61a-ab50baef960b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:39.134649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d888b-b9ff-4c58-a61a-ab50baef960b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.135168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d888b-b9ff-4c58-a61a-ab50baef960b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.135281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec293a-a27d-4603-881f-3ea467212602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:39.135792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d888b-b9ff-4c58-a61a-ab50baef960b None admin] PATH_INFO: `/v3/projects/b2203339adc34a0499886c3bb35876aa` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.135982 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec293a-a27d-4603-881f-3ea467212602 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.136256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec293a-a27d-4603-881f-3ea467212602 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.136545 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec293a-a27d-4603-881f-3ea467212602 None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.137895 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec293a-a27d-4603-881f-3ea467212602 None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:39.138488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d888b-b9ff-4c58-a61a-ab50baef960b None admin] RBAC: Authorizing `identity:get_project(project_id=b2203339adc34a0499886c3bb35876aa)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:39.139506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec293a-a27d-4603-881f-3ea467212602 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:39.140555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d888b-b9ff-4c58-a61a-ab50baef960b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:39.141707 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 190/376] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:38 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:39.143760 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ab838-68ab-497e-85e4-26258f729b55 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.143760 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ab838-68ab-497e-85e4-26258f729b55 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.143760 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ab838-68ab-497e-85e4-26258f729b55 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.145361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 187/377] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:38 2022] GET /identity/v3/projects/b2203339adc34a0499886c3bb35876aa => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:39.150202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 191/378] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:39.151897 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d888b-b9ff-4c58-a61a-ab50baef960b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:39.152537 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3bdb9-fbaf-47f7-82cc-779ff4cb40a5 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.153033 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3bdb9-fbaf-47f7-82cc-779ff4cb40a5 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.153389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3bdb9-fbaf-47f7-82cc-779ff4cb40a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.183951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-66e3bdb9-fbaf-47f7-82cc-779ff4cb40a5 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:39.185403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-66e3bdb9-fbaf-47f7-82cc-779ff4cb40a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:39.279881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 192/379] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:39.281177 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3bdb9-fbaf-47f7-82cc-779ff4cb40a5 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:39.291911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d888b-b9ff-4c58-a61a-ab50baef960b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:39.319200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:39.326244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:39.356164 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:39.356896 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.357180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.357453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.358286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] RBAC: Adding query filter params (group.id=1e8110403a944b5c802e3878e5ff5d12, role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=ff937f0574a548faac1d7abf71ec1698) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:39.358940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:39.360204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:39.366627 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 188/380] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:39 2022] GET /identity/v3/role_assignments?group.id=1e8110403a944b5c802e3878e5ff5d12&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=601939cb90d641e08b97775c0c12d374 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:39.367738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:39.428542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3bdb9-fbaf-47f7-82cc-779ff4cb40a5 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:39.511383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b419b-b7b2-4ca2-9a9c-5698e1e7ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:39.512493 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b419b-b7b2-4ca2-9a9c-5698e1e7ee17 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.512997 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b419b-b7b2-4ca2-9a9c-5698e1e7ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.513498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b419b-b7b2-4ca2-9a9c-5698e1e7ee17 None admin] PATH_INFO: `/v3/groups/f9977230f6434693a7e40797b554623b` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.523790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b419b-b7b2-4ca2-9a9c-5698e1e7ee17 None admin] RBAC: Authorizing `identity:get_group(group_id=f9977230f6434693a7e40797b554623b)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:39.525430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b419b-b7b2-4ca2-9a9c-5698e1e7ee17 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:39.527831 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 193/381] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:39 2022] GET /identity/v3/groups/f9977230f6434693a7e40797b554623b => generated 229 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:39.533508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b419b-b7b2-4ca2-9a9c-5698e1e7ee17 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:39.556366 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc44cb-9d85-4b2b-a616-7ce164cf24c0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:39.606999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553521fb-e7f9-4412-a73f-4f352ba47068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:39.608004 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553521fb-e7f9-4412-a73f-4f352ba47068 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.608475 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553521fb-e7f9-4412-a73f-4f352ba47068 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.608908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553521fb-e7f9-4412-a73f-4f352ba47068 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.627945 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553521fb-e7f9-4412-a73f-4f352ba47068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-553521fb-e7f9-4412-a73f-4f352ba47068 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:39.634159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:39.635168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:39.636035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:39.643883 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 189/382] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:39.649702 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553521fb-e7f9-4412-a73f-4f352ba47068 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:39.651694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:39.666503 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b419b-b7b2-4ca2-9a9c-5698e1e7ee17 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:39.707278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:39.708053 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.708241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.708408 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.709204 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] RBAC: Adding query filter params (group.id=f9977230f6434693a7e40797b554623b, role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=b2203339adc34a0499886c3bb35876aa) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:39.709526 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:39.711214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:39.716893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:39.719841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 194/383] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:39 2022] GET /identity/v3/role_assignments?group.id=f9977230f6434693a7e40797b554623b&scope.project.id=b2203339adc34a0499886c3bb35876aa&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:39.822459 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553521fb-e7f9-4412-a73f-4f352ba47068 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:39.888142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:39.888955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:39.889139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:39.889300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:39.890047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] RBAC: Adding query filter params (name=member) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:39.890375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:39.891776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:39.902672 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 190/384] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:39.908556 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:40.053312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9ef6e-ef1a-4a31-aec4-952c563ef07a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:40.101162 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e684-24e0-4f23-be6a-5d9df9c4b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:40.101928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e684-24e0-4f23-be6a-5d9df9c4b064 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:40.102299 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e684-24e0-4f23-be6a-5d9df9c4b064 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:40.102584 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e684-24e0-4f23-be6a-5d9df9c4b064 None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:40.103840 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4e684-24e0-4f23-be6a-5d9df9c4b064 None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:40.105353 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4e684-24e0-4f23-be6a-5d9df9c4b064 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:40.107605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 195/385] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:39 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:40.112914 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:40.292525 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a35938-3299-400e-9efd-22de8c544765 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:40.295735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:40.340887 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e57f6-c6ba-454e-971e-5b7b833989d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:40.341479 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e57f6-c6ba-454e-971e-5b7b833989d9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:40.341760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e57f6-c6ba-454e-971e-5b7b833989d9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:40.342032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e57f6-c6ba-454e-971e-5b7b833989d9 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:40.343254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e57f6-c6ba-454e-971e-5b7b833989d9 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:40.344730 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e57f6-c6ba-454e-971e-5b7b833989d9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:40.346859 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 191/386] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:40 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:40.352735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e684-24e0-4f23-be6a-5d9df9c4b064 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:40.400730 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:40.455800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c3ea2-952a-4cc5-ab8a-3a0411d3447a None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:40.456250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c3ea2-952a-4cc5-ab8a-3a0411d3447a None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:40.456659 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c3ea2-952a-4cc5-ab8a-3a0411d3447a None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:40.459679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 192/387] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:40.466266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01315c4-569b-4045-8aa6-d9aa5df3047b None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:40.466565 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01315c4-569b-4045-8aa6-d9aa5df3047b None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:40.466840 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01315c4-569b-4045-8aa6-d9aa5df3047b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:40.501736 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-d01315c4-569b-4045-8aa6-d9aa5df3047b None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:40.503004 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-d01315c4-569b-4045-8aa6-d9aa5df3047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:40.519465 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e684-24e0-4f23-be6a-5d9df9c4b064 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:40.571965 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e7956-bb63-4470-afcf-5e7c0f402524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:40.573107 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7956-bb63-4470-afcf-5e7c0f402524 None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:40.573640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7956-bb63-4470-afcf-5e7c0f402524 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:40.574199 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7956-bb63-4470-afcf-5e7c0f402524 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57/groups/1e8110403a944b5c802e3878e5ff5d12/roles/2935a7836c864df48e32fa734f020882` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:40.576709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e7956-bb63-4470-afcf-5e7c0f402524 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a427cb96a904a66aca0187281bdad57, group_id=1e8110403a944b5c802e3878e5ff5d12, role_id=2935a7836c864df48e32fa734f020882)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:40.578467 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e7956-bb63-4470-afcf-5e7c0f402524 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:40.598131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 196/388] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:08:40 2022] PUT /identity/v3/projects/6a427cb96a904a66aca0187281bdad57/groups/1e8110403a944b5c802e3878e5ff5d12/roles/2935a7836c864df48e32fa734f020882 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:40.641693 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 193/389] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:40.673033 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e7956-bb63-4470-afcf-5e7c0f402524 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:40.793106 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:40.812248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e7956-bb63-4470-afcf-5e7c0f402524 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:40.849387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a886461-880b-42b0-bb73-d8ce09804576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:40.850343 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a886461-880b-42b0-bb73-d8ce09804576 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:40.850818 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a886461-880b-42b0-bb73-d8ce09804576 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:40.851252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a886461-880b-42b0-bb73-d8ce09804576 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:40.862748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a886461-880b-42b0-bb73-d8ce09804576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-8a886461-880b-42b0-bb73-d8ce09804576 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:40.868827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:40.870205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:40.871341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:40.873976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 197/390] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:40.879834 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01315c4-569b-4045-8aa6-d9aa5df3047b None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:40.996604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01315c4-569b-4045-8aa6-d9aa5df3047b None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:40.999049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:41.034713 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:41.035663 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:41.036121 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:41.036566 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:41.037466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:41.037930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:41.039589 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:41.049137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 194/391] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:41.054459 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a886461-880b-42b0-bb73-d8ce09804576 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:41.213564 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a886461-880b-42b0-bb73-d8ce09804576 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:41.261252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648c3e6-30b6-48ef-bc04-e0e06b3b5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:41.261928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c3e6-30b6-48ef-bc04-e0e06b3b5b73 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:41.262248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c3e6-30b6-48ef-bc04-e0e06b3b5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:41.262539 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c3e6-30b6-48ef-bc04-e0e06b3b5b73 None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:41.263742 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648c3e6-30b6-48ef-bc04-e0e06b3b5b73 None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:41.265225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648c3e6-30b6-48ef-bc04-e0e06b3b5b73 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:41.266919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 198/392] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:41 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:41.272818 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:41.385140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0ad71-f8ad-4c9b-8238-4a41c820b08b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:41.424613 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b7dfb-e975-4cc7-969f-6f446feb9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:41.425340 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7dfb-e975-4cc7-969f-6f446feb9a6b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:41.425586 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7dfb-e975-4cc7-969f-6f446feb9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:41.425869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7dfb-e975-4cc7-969f-6f446feb9a6b None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:41.427112 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b7dfb-e975-4cc7-969f-6f446feb9a6b None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:41.428585 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b7dfb-e975-4cc7-969f-6f446feb9a6b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:41.430831 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 195/393] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:41 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:41.436299 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648c3e6-30b6-48ef-bc04-e0e06b3b5b73 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:41.470610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:41.565807 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648c3e6-30b6-48ef-bc04-e0e06b3b5b73 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:41.617414 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91034bd-10e9-429d-87e6-a6c957888a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:41.618146 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91034bd-10e9-429d-87e6-a6c957888a29 None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:41.618446 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91034bd-10e9-429d-87e6-a6c957888a29 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:41.618727 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91034bd-10e9-429d-87e6-a6c957888a29 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57/groups/1e8110403a944b5c802e3878e5ff5d12/roles/601939cb90d641e08b97775c0c12d374` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:41.623376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91034bd-10e9-429d-87e6-a6c957888a29 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a427cb96a904a66aca0187281bdad57, group_id=1e8110403a944b5c802e3878e5ff5d12, role_id=601939cb90d641e08b97775c0c12d374)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:41.624899 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91034bd-10e9-429d-87e6-a6c957888a29 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:41.648931 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 199/394] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:08:41 2022] PUT /identity/v3/projects/6a427cb96a904a66aca0187281bdad57/groups/1e8110403a944b5c802e3878e5ff5d12/roles/601939cb90d641e08b97775c0c12d374 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:41.721793 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4909c-f67f-4091-b70a-ec11a32c8aff None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:41.721982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4909c-f67f-4091-b70a-ec11a32c8aff None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:41.722293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4909c-f67f-4091-b70a-ec11a32c8aff None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:41.724020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 196/395] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:41.729957 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803161a3-44f1-4eda-8726-8facb86e03cc None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:41.730284 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803161a3-44f1-4eda-8726-8facb86e03cc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:41.730571 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803161a3-44f1-4eda-8726-8facb86e03cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:41.759404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-803161a3-44f1-4eda-8726-8facb86e03cc None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:41.760740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-803161a3-44f1-4eda-8726-8facb86e03cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:41.852005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 200/396] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:41.896797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4909c-f67f-4091-b70a-ec11a32c8aff None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:41.941361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:42.031774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4909c-f67f-4091-b70a-ec11a32c8aff None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:42.051382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972615e-61b1-42ec-ab58-7451935c8254 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.051709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972615e-61b1-42ec-ab58-7451935c8254 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.051926 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972615e-61b1-42ec-ab58-7451935c8254 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.053678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 201/397] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:42.061052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6beca-88ca-468e-907c-5a115bf09d1c None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.061379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6beca-88ca-468e-907c-5a115bf09d1c None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.061613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6beca-88ca-468e-907c-5a115bf09d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.080816 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:42.103106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fec87-a90e-4d05-98d1-a65d61902df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:42.103787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fec87-a90e-4d05-98d1-a65d61902df2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.104071 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fec87-a90e-4d05-98d1-a65d61902df2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.104380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fec87-a90e-4d05-98d1-a65d61902df2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.106423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-8dd6beca-88ca-468e-907c-5a115bf09d1c None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:42.107882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-8dd6beca-88ca-468e-907c-5a115bf09d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:42.131292 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fec87-a90e-4d05-98d1-a65d61902df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-cb9fec87-a90e-4d05-98d1-a65d61902df2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:42.142504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:42.144852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:42.146965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 197/398] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:42.152352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fec87-a90e-4d05-98d1-a65d61902df2 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:42.269600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:42.281700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fec87-a90e-4d05-98d1-a65d61902df2 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:42.295919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 202/399] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:42.319675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16eab8-8f37-42da-99fb-3f9bc3fe7668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:42.320351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16eab8-8f37-42da-99fb-3f9bc3fe7668 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.320630 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16eab8-8f37-42da-99fb-3f9bc3fe7668 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.320904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16eab8-8f37-42da-99fb-3f9bc3fe7668 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.321638 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16eab8-8f37-42da-99fb-3f9bc3fe7668 None admin] RBAC: Adding query filter params (name=member) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:42.321942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16eab8-8f37-42da-99fb-3f9bc3fe7668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:42.322443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dde228-7ff9-4aed-b902-25c61541d4cf None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.322743 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dde228-7ff9-4aed-b902-25c61541d4cf None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.323030 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dde228-7ff9-4aed-b902-25c61541d4cf None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.323370 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16eab8-8f37-42da-99fb-3f9bc3fe7668 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:42.324406 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 203/400] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:42.331055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dde228-7ff9-4aed-b902-25c61541d4cf None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:42.332745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 198/401] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:42.334191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acc255-eb83-4daa-957a-ffe25d613fe5 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.334482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acc255-eb83-4daa-957a-ffe25d613fe5 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.334769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acc255-eb83-4daa-957a-ffe25d613fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.356789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-27acc255-eb83-4daa-957a-ffe25d613fe5 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:42.357993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-27acc255-eb83-4daa-957a-ffe25d613fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:42.432347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 199/402] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:42.439054 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27acc255-eb83-4daa-957a-ffe25d613fe5 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:42.456849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dde228-7ff9-4aed-b902-25c61541d4cf None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:42.504581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b390fa-56c9-473c-aa2b-2eea5222d8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:42.506369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b390fa-56c9-473c-aa2b-2eea5222d8b3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.506990 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b390fa-56c9-473c-aa2b-2eea5222d8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.507273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b390fa-56c9-473c-aa2b-2eea5222d8b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.510204 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:42.525134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b390fa-56c9-473c-aa2b-2eea5222d8b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-45b390fa-56c9-473c-aa2b-2eea5222d8b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:42.532610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:42.534316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 204/403] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:42.535287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b390fa-56c9-473c-aa2b-2eea5222d8b3 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:42.552095 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:42.572077 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27acc255-eb83-4daa-957a-ffe25d613fe5 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:42.613862 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fdbcd-7da6-4431-ab1e-60b37475d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:42.614590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fdbcd-7da6-4431-ab1e-60b37475d861 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.614867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fdbcd-7da6-4431-ab1e-60b37475d861 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.615142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fdbcd-7da6-4431-ab1e-60b37475d861 None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.616414 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fdbcd-7da6-4431-ab1e-60b37475d861 None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:42.618224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fdbcd-7da6-4431-ab1e-60b37475d861 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:42.624946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 200/404] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:42 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:42.624946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fdbcd-7da6-4431-ab1e-60b37475d861 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:42.669117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b390fa-56c9-473c-aa2b-2eea5222d8b3 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:42.708734 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07bc8-9f4f-4fb2-98a5-da25606e6109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:42.709419 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07bc8-9f4f-4fb2-98a5-da25606e6109 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.709695 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07bc8-9f4f-4fb2-98a5-da25606e6109 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.709966 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07bc8-9f4f-4fb2-98a5-da25606e6109 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.723351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d07bc8-9f4f-4fb2-98a5-da25606e6109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-f8d07bc8-9f4f-4fb2-98a5-da25606e6109 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:42.728731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:42.729719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:42.731092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 205/405] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:08:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:42.732148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07bc8-9f4f-4fb2-98a5-da25606e6109 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:42.735258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fdbcd-7da6-4431-ab1e-60b37475d861 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:42.773198 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:42.774286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.774570 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.774842 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.775986 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:42.776298 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:42.777868 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:42.788220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 201/406] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:42.789205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:42.790606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:42.808761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:42.844233 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07bc8-9f4f-4fb2-98a5-da25606e6109 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:42.881580 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b789e-a4ff-4659-b3bc-3e0f2d89e298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:42.882537 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b789e-a4ff-4659-b3bc-3e0f2d89e298 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.882891 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b789e-a4ff-4659-b3bc-3e0f2d89e298 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.883642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b789e-a4ff-4659-b3bc-3e0f2d89e298 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.885063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b789e-a4ff-4659-b3bc-3e0f2d89e298 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:42.887211 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b789e-a4ff-4659-b3bc-3e0f2d89e298 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:42.888946 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 206/407] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:42 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:42.889731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b789e-a4ff-4659-b3bc-3e0f2d89e298 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:42.904053 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c984ad-c5f1-4622-8214-7f00d6c73e93 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:42.944204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:42.944943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:42.945315 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:42.945591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:42.946473 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:42.946902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:42.948320 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:42.958450 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 202/408] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:42.959417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:43.012579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b789e-a4ff-4659-b3bc-3e0f2d89e298 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.036377 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:43.048717 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393d38d-bc82-4ed7-8a41-0f7ef4df71a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.049388 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393d38d-bc82-4ed7-8a41-0f7ef4df71a0 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.049674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393d38d-bc82-4ed7-8a41-0f7ef4df71a0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.049958 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393d38d-bc82-4ed7-8a41-0f7ef4df71a0 None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.051173 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393d38d-bc82-4ed7-8a41-0f7ef4df71a0 None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:43.052646 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393d38d-bc82-4ed7-8a41-0f7ef4df71a0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:43.054772 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 207/409] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:42 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:43.055189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393d38d-bc82-4ed7-8a41-0f7ef4df71a0 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:43.066669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:43.092006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63186ac-10cc-4bb8-b8a3-0649a22744d0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.132950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e0c0c-fb00-4503-8d08-aee4ca9da6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.133597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e0c0c-fb00-4503-8d08-aee4ca9da6d7 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.133885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e0c0c-fb00-4503-8d08-aee4ca9da6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.134219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e0c0c-fb00-4503-8d08-aee4ca9da6d7 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698/groups/1e8110403a944b5c802e3878e5ff5d12/roles/2935a7836c864df48e32fa734f020882` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.136170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e0c0c-fb00-4503-8d08-aee4ca9da6d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff937f0574a548faac1d7abf71ec1698, group_id=1e8110403a944b5c802e3878e5ff5d12, role_id=2935a7836c864df48e32fa734f020882)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:43.137658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e0c0c-fb00-4503-8d08-aee4ca9da6d7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:43.152277 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 203/410] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:08:42 2022] PUT /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698/groups/1e8110403a944b5c802e3878e5ff5d12/roles/2935a7836c864df48e32fa734f020882 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:43.153539 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a302b-5186-47d2-a89d-91a0b9e0ae49 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.153827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a302b-5186-47d2-a89d-91a0b9e0ae49 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.154127 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a302b-5186-47d2-a89d-91a0b9e0ae49 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.155554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 204/411] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:43.156113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a302b-5186-47d2-a89d-91a0b9e0ae49 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:43.181787 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393d38d-bc82-4ed7-8a41-0f7ef4df71a0 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.221109 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8660d-def8-4eb7-8a60-28c1784a3196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.222308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8660d-def8-4eb7-8a60-28c1784a3196 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.222831 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8660d-def8-4eb7-8a60-28c1784a3196 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.223434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8660d-def8-4eb7-8a60-28c1784a3196 None admin] PATH_INFO: `/v3/groups/f9977230f6434693a7e40797b554623b` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.225104 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8660d-def8-4eb7-8a60-28c1784a3196 None admin] RBAC: Authorizing `identity:get_group(group_id=f9977230f6434693a7e40797b554623b)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:43.227094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8660d-def8-4eb7-8a60-28c1784a3196 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:43.229115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 208/412] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:43 2022] GET /identity/v3/groups/f9977230f6434693a7e40797b554623b => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:43.230631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423143-05fe-4a99-b5d9-e31fe481f364 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.231159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423143-05fe-4a99-b5d9-e31fe481f364 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.231663 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423143-05fe-4a99-b5d9-e31fe481f364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.252145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-ea423143-05fe-4a99-b5d9-e31fe481f364 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:43.253928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-ea423143-05fe-4a99-b5d9-e31fe481f364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:43.295796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a302b-5186-47d2-a89d-91a0b9e0ae49 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.322939 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:43.343845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002e936-f536-4a06-aa74-eebf04448f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.344234 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 209/413] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:43.348195 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423143-05fe-4a99-b5d9-e31fe481f364 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:43.349514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002e936-f536-4a06-aa74-eebf04448f92 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.349805 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002e936-f536-4a06-aa74-eebf04448f92 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.350087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002e936-f536-4a06-aa74-eebf04448f92 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.355757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002e936-f536-4a06-aa74-eebf04448f92 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:43.357464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002e936-f536-4a06-aa74-eebf04448f92 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:43.361425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 205/414] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:43 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:43.367282 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002e936-f536-4a06-aa74-eebf04448f92 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:43.373932 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:43.481895 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423143-05fe-4a99-b5d9-e31fe481f364 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.517366 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002e936-f536-4a06-aa74-eebf04448f92 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.534460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5a575-8940-42ce-8e71-ff56bba23b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.535726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5a575-8940-42ce-8e71-ff56bba23b13 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.536197 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5a575-8940-42ce-8e71-ff56bba23b13 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.536872 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5a575-8940-42ce-8e71-ff56bba23b13 None admin] PATH_INFO: `/v3/projects/b2203339adc34a0499886c3bb35876aa` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.538902 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5a575-8940-42ce-8e71-ff56bba23b13 None admin] RBAC: Authorizing `identity:get_project(project_id=b2203339adc34a0499886c3bb35876aa)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:43.540569 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5a575-8940-42ce-8e71-ff56bba23b13 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:43.543611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 210/415] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:43 2022] GET /identity/v3/projects/b2203339adc34a0499886c3bb35876aa => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:43.545057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5a575-8940-42ce-8e71-ff56bba23b13 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:43.558795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a599b8-6d41-4ba6-bdc5-53ad847ea2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.559613 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a599b8-6d41-4ba6-bdc5-53ad847ea2f5 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.559964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a599b8-6d41-4ba6-bdc5-53ad847ea2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.560299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a599b8-6d41-4ba6-bdc5-53ad847ea2f5 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698/groups/1e8110403a944b5c802e3878e5ff5d12/roles/601939cb90d641e08b97775c0c12d374` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.563727 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a599b8-6d41-4ba6-bdc5-53ad847ea2f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff937f0574a548faac1d7abf71ec1698, group_id=1e8110403a944b5c802e3878e5ff5d12, role_id=601939cb90d641e08b97775c0c12d374)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:43.565441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a599b8-6d41-4ba6-bdc5-53ad847ea2f5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:43.586276 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 206/416] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:08:43 2022] PUT /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698/groups/1e8110403a944b5c802e3878e5ff5d12/roles/601939cb90d641e08b97775c0c12d374 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:43.587449 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a599b8-6d41-4ba6-bdc5-53ad847ea2f5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:43.635057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:43.723575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5a575-8940-42ce-8e71-ff56bba23b13 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.728104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:43.766328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b7d0-1a21-4141-919c-7b18f0250176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.767003 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b7d0-1a21-4141-919c-7b18f0250176 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.767289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b7d0-1a21-4141-919c-7b18f0250176 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.767567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b7d0-1a21-4141-919c-7b18f0250176 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.772220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a599b8-6d41-4ba6-bdc5-53ad847ea2f5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.779721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1b7d0-1a21-4141-919c-7b18f0250176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-f9e1b7d0-1a21-4141-919c-7b18f0250176 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:43.785244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:43.786363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:43.787222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 211/417] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:43.792961 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b7d0-1a21-4141-919c-7b18f0250176 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:43.815391 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d148e0d-7996-4df8-a14c-c67d343228d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.816522 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d148e0d-7996-4df8-a14c-c67d343228d7 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.817390 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d148e0d-7996-4df8-a14c-c67d343228d7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.817929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d148e0d-7996-4df8-a14c-c67d343228d7 None admin] PATH_INFO: `/v3/projects/b2203339adc34a0499886c3bb35876aa/groups/f9977230f6434693a7e40797b554623b/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.820284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d148e0d-7996-4df8-a14c-c67d343228d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2203339adc34a0499886c3bb35876aa, group_id=f9977230f6434693a7e40797b554623b, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:43.822339 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d148e0d-7996-4df8-a14c-c67d343228d7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:43.841774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 207/418] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:08:43 2022] PUT /identity/v3/projects/b2203339adc34a0499886c3bb35876aa/groups/f9977230f6434693a7e40797b554623b/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:08:43.928974 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b7d0-1a21-4141-919c-7b18f0250176 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:43.949796 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:43.976493 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b7b32-c431-41e3-9708-c5cf3beea1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:43.977172 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b7b32-c431-41e3-9708-c5cf3beea1b1 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:43.977463 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b7b32-c431-41e3-9708-c5cf3beea1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:43.977746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b7b32-c431-41e3-9708-c5cf3beea1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:43.978596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b7b32-c431-41e3-9708-c5cf3beea1b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:43.978915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b7b32-c431-41e3-9708-c5cf3beea1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:43.980387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b7b32-c431-41e3-9708-c5cf3beea1b1 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:43.993916 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 212/419] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:43.999935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d148e0d-7996-4df8-a14c-c67d343228d7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:44.094069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095210d-6a5e-4c27-a8ea-7aa3fb30ca28 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:44.094306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095210d-6a5e-4c27-a8ea-7aa3fb30ca28 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:44.094575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095210d-6a5e-4c27-a8ea-7aa3fb30ca28 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:44.096116 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 213/420] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:44.103304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc86c2-d0d6-4431-bc31-a580b002230c None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:44.103749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc86c2-d0d6-4431-bc31-a580b002230c None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:44.104138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc86c2-d0d6-4431-bc31-a580b002230c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:44.124715 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-e7dc86c2-d0d6-4431-bc31-a580b002230c None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:44.126252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-e7dc86c2-d0d6-4431-bc31-a580b002230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:44.126429 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d148e0d-7996-4df8-a14c-c67d343228d7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:44.153403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:44.172557 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773aed7d-4e30-4cc2-b085-1eb1fa8ba806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:44.173212 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aed7d-4e30-4cc2-b085-1eb1fa8ba806 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:44.173498 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aed7d-4e30-4cc2-b085-1eb1fa8ba806 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:44.173776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aed7d-4e30-4cc2-b085-1eb1fa8ba806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:44.212266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 214/421] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:44.235995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc86c2-d0d6-4431-bc31-a580b002230c None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:44.340412 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773aed7d-4e30-4cc2-b085-1eb1fa8ba806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:44.342329 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773aed7d-4e30-4cc2-b085-1eb1fa8ba806 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:44.348000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc86c2-d0d6-4431-bc31-a580b002230c None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:44.358400 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:44.387677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bf9a3-865f-4aae-aed8-e7528ace8251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:44.388364 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bf9a3-865f-4aae-aed8-e7528ace8251 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:44.388654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bf9a3-865f-4aae-aed8-e7528ace8251 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:44.388937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bf9a3-865f-4aae-aed8-e7528ace8251 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:44.400822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bf9a3-865f-4aae-aed8-e7528ace8251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-e50bf9a3-865f-4aae-aed8-e7528ace8251 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:44.406352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:44.407328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:44.408202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 215/422] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:44.413979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bf9a3-865f-4aae-aed8-e7528ace8251 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:44.471626 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 208/423] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:44.478513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773aed7d-4e30-4cc2-b085-1eb1fa8ba806 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:44.523267 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:44.534195 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bf9a3-865f-4aae-aed8-e7528ace8251 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:44.576150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:44.576913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:44.577209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:44.577581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:44.578387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:44.578705 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:44.580123 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:44.589956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 216/424] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:44.596015 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:44.633405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:44.650412 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773aed7d-4e30-4cc2-b085-1eb1fa8ba806 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:44.693561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5986ae9-42da-49e8-b658-549d3be5edf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:44.694253 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5986ae9-42da-49e8-b658-549d3be5edf0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:44.694535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5986ae9-42da-49e8-b658-549d3be5edf0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:44.694923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5986ae9-42da-49e8-b658-549d3be5edf0 None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:44.698844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5986ae9-42da-49e8-b658-549d3be5edf0 None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:44.700368 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5986ae9-42da-49e8-b658-549d3be5edf0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:44.702412 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 209/425] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:44 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:44.710796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5986ae9-42da-49e8-b658-549d3be5edf0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:44.719889 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f59c79-7271-443c-bd22-29fdcccacf15 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:44.760470 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b422c-d77a-49f9-9e6e-accd82780c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:44.761223 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b422c-d77a-49f9-9e6e-accd82780c65 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:44.761519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b422c-d77a-49f9-9e6e-accd82780c65 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:44.761832 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b422c-d77a-49f9-9e6e-accd82780c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:44.835271 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b422c-d77a-49f9-9e6e-accd82780c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:44.836803 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b422c-d77a-49f9-9e6e-accd82780c65 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:44.870015 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:44.943617 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:44.955340 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 217/426] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 09:08:44.961427 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b422c-d77a-49f9-9e6e-accd82780c65 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:44.972479 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5986ae9-42da-49e8-b658-549d3be5edf0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:45.009385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d551364-edb2-4dd8-bb99-5584721d3ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:45.010061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d551364-edb2-4dd8-bb99-5584721d3ace None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:45.013162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d551364-edb2-4dd8-bb99-5584721d3ace None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:45.013162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d551364-edb2-4dd8-bb99-5584721d3ace None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:45.013802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d551364-edb2-4dd8-bb99-5584721d3ace None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:45.015339 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d551364-edb2-4dd8-bb99-5584721d3ace None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:45.017372 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 210/427] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:44 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:45.027232 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d551364-edb2-4dd8-bb99-5584721d3ace None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:45.071588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b422c-d77a-49f9-9e6e-accd82780c65 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:45.106602 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:45.110746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddfb7c-3426-4e17-a024-b7ad1b069bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:45.111444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddfb7c-3426-4e17-a024-b7ad1b069bae None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:45.111732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddfb7c-3426-4e17-a024-b7ad1b069bae None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:45.112015 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddfb7c-3426-4e17-a024-b7ad1b069bae None admin] PATH_INFO: `/v3/projects/6a427cb96a904a66aca0187281bdad57` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:45.113290 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddfb7c-3426-4e17-a024-b7ad1b069bae None admin] RBAC: Authorizing `identity:get_project(project_id=6a427cb96a904a66aca0187281bdad57)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:45.117114 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddfb7c-3426-4e17-a024-b7ad1b069bae None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:45.119278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 218/428] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:44 2022] GET /identity/v3/projects/6a427cb96a904a66aca0187281bdad57 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:45.124975 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddfb7c-3426-4e17-a024-b7ad1b069bae None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:45.190255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d551364-edb2-4dd8-bb99-5584721d3ace None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:45.234858 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:45.235512 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:45.235799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:45.236078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:45.236895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] RBAC: Adding query filter params (group.id=1e8110403a944b5c802e3878e5ff5d12, role.id=2935a7836c864df48e32fa734f020882, scope.project.id=6a427cb96a904a66aca0187281bdad57) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:45.237207 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:45.238732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:45.254895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 211/429] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:45 2022] GET /identity/v3/role_assignments?group.id=1e8110403a944b5c802e3878e5ff5d12&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=2935a7836c864df48e32fa734f020882 => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:45.324622 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddfb7c-3426-4e17-a024-b7ad1b069bae None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:45.373228 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280871dc-638e-4202-8d99-153a21c23f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:45.373918 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280871dc-638e-4202-8d99-153a21c23f1d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:45.374229 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280871dc-638e-4202-8d99-153a21c23f1d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:45.374508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280871dc-638e-4202-8d99-153a21c23f1d None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:45.375777 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280871dc-638e-4202-8d99-153a21c23f1d None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:45.377297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280871dc-638e-4202-8d99-153a21c23f1d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:45.379394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 219/430] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:45 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:45.385177 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:45.433994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:45.543824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b19fe-9a11-456c-b4c8-9bcd1a0d3022 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:45.589576 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec880de-edb3-408f-b8e1-c2db805e3636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:45.590265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec880de-edb3-408f-b8e1-c2db805e3636 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:45.590555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec880de-edb3-408f-b8e1-c2db805e3636 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:45.590831 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec880de-edb3-408f-b8e1-c2db805e3636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:45.591640 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec880de-edb3-408f-b8e1-c2db805e3636 None admin] RBAC: Adding query filter params (group.id=1e8110403a944b5c802e3878e5ff5d12, role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=6a427cb96a904a66aca0187281bdad57) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:45.591947 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec880de-edb3-408f-b8e1-c2db805e3636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:45.593396 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec880de-edb3-408f-b8e1-c2db805e3636 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:45.603240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 212/431] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:45 2022] GET /identity/v3/role_assignments?group.id=1e8110403a944b5c802e3878e5ff5d12&scope.project.id=6a427cb96a904a66aca0187281bdad57&role.id=601939cb90d641e08b97775c0c12d374 => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:45.768627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed765ea-85d5-439e-8f6c-bc0a0f26ad06 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:45.769072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed765ea-85d5-439e-8f6c-bc0a0f26ad06 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:45.769181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed765ea-85d5-439e-8f6c-bc0a0f26ad06 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:45.770610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 220/432] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:45.777683 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614f34-722f-4902-ab4b-b770f7f8c6ef None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:45.778000 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614f34-722f-4902-ab4b-b770f7f8c6ef None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:45.778289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614f34-722f-4902-ab4b-b770f7f8c6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:45.798482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-7e614f34-722f-4902-ab4b-b770f7f8c6ef None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:45.799660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-7e614f34-722f-4902-ab4b-b770f7f8c6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:45.888380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 213/433] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:45.913154 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed765ea-85d5-439e-8f6c-bc0a0f26ad06 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:45.998461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:46.003624 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d323-be98-4d55-bc41-1a531dc35e90 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.003908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d323-be98-4d55-bc41-1a531dc35e90 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.004183 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d323-be98-4d55-bc41-1a531dc35e90 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.005868 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 214/434] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:08:46.015915 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb02dee-3a8b-4ad0-b97c-b3bde7c6c176 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.016205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb02dee-3a8b-4ad0-b97c-b3bde7c6c176 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.016489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb02dee-3a8b-4ad0-b97c-b3bde7c6c176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.043238 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-7fb02dee-3a8b-4ad0-b97c-b3bde7c6c176 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:46.044072 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-7fb02dee-3a8b-4ad0-b97c-b3bde7c6c176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:46.044271 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed765ea-85d5-439e-8f6c-bc0a0f26ad06 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:46.061094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:46.089437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad333ea-054c-4941-96d0-a55f60e23474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:46.090151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad333ea-054c-4941-96d0-a55f60e23474 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.090657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad333ea-054c-4941-96d0-a55f60e23474 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.090939 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad333ea-054c-4941-96d0-a55f60e23474 None admin] PATH_INFO: `/v3/roles/member` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.104603 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad333ea-054c-4941-96d0-a55f60e23474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-4ad333ea-054c-4941-96d0-a55f60e23474 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:46.110145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:46.111148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:46.112028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 221/435] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:46.119565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad333ea-054c-4941-96d0-a55f60e23474 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:46.130330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 215/436] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:46.153243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb02dee-3a8b-4ad0-b97c-b3bde7c6c176 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:46.259117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad333ea-054c-4941-96d0-a55f60e23474 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:46.294469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb02dee-3a8b-4ad0-b97c-b3bde7c6c176 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:46.296655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:46.310398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:46.311056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.311404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.311625 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.312377 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] RBAC: Adding query filter params (name=member) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:46.312686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:46.314091 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:46.326321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 222/437] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:08:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:46.334686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:46.346460 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff7422-53d8-4934-80a3-7c8b692905bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:46.347554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7422-53d8-4934-80a3-7c8b692905bf None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.348054 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7422-53d8-4934-80a3-7c8b692905bf None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.348561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7422-53d8-4934-80a3-7c8b692905bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.362771 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff7422-53d8-4934-80a3-7c8b692905bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-aaff7422-53d8-4934-80a3-7c8b692905bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:46.370775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:46.371820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:46.372740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:46.375546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 216/438] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:46.381874 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff7422-53d8-4934-80a3-7c8b692905bf None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:46.396091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:46.453434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f482db-8cd9-462b-8750-a2f1c4660c9b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:46.490691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f76afa-7175-4fff-a4cf-755a2f70bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:46.491504 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f76afa-7175-4fff-a4cf-755a2f70bc61 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.491899 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f76afa-7175-4fff-a4cf-755a2f70bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.492244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f76afa-7175-4fff-a4cf-755a2f70bc61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.494803 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff7422-53d8-4934-80a3-7c8b692905bf None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:46.532965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d6c7f-df01-44c8-9fde-3711336afd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:46.533637 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d6c7f-df01-44c8-9fde-3711336afd67 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.533919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d6c7f-df01-44c8-9fde-3711336afd67 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.534237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d6c7f-df01-44c8-9fde-3711336afd67 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.535006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d6c7f-df01-44c8-9fde-3711336afd67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:46.535321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d6c7f-df01-44c8-9fde-3711336afd67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:46.536785 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d6c7f-df01-44c8-9fde-3711336afd67 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:46.546075 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 217/439] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:46.547528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c708e-5c64-4221-b2c6-fa8a8393d5af None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.547828 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c708e-5c64-4221-b2c6-fa8a8393d5af None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.548111 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c708e-5c64-4221-b2c6-fa8a8393d5af None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.549468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 218/440] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:46.551986 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c708e-5c64-4221-b2c6-fa8a8393d5af None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:46.563511 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:46.578393 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f76afa-7175-4fff-a4cf-755a2f70bc61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:46.579905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f76afa-7175-4fff-a4cf-755a2f70bc61 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:46.628607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:46.664525 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c708e-5c64-4221-b2c6-fa8a8393d5af None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:46.687617 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 223/441] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 09:08:46.689017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b3e21-99b5-4de0-b760-a2d71413e3a0 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.689285 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b3e21-99b5-4de0-b760-a2d71413e3a0 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.689602 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b3e21-99b5-4de0-b760-a2d71413e3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.706024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aae7f8-3eef-45b5-bea1-43760e7fac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:46.706765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aae7f8-3eef-45b5-bea1-43760e7fac7a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.707015 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aae7f8-3eef-45b5-bea1-43760e7fac7a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.707302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aae7f8-3eef-45b5-bea1-43760e7fac7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.708800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-9f2b3e21-99b5-4de0-b760-a2d71413e3a0 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:46.710052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-9f2b3e21-99b5-4de0-b760-a2d71413e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:46.780190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 224/442] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:46.781102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b3e21-99b5-4de0-b760-a2d71413e3a0 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:46.794430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aae7f8-3eef-45b5-bea1-43760e7fac7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:46.796090 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aae7f8-3eef-45b5-bea1-43760e7fac7a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:46.803712 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:46.868517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:46.888494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b3e21-99b5-4de0-b760-a2d71413e3a0 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:46.900751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 219/443] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:46.901992 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aae7f8-3eef-45b5-bea1-43760e7fac7a None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:46.927501 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ada7a-421f-4ab9-a884-5dc214bb9ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:46.928174 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ada7a-421f-4ab9-a884-5dc214bb9ae0 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:46.928457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ada7a-421f-4ab9-a884-5dc214bb9ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:46.928757 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ada7a-421f-4ab9-a884-5dc214bb9ae0 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:46.929936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ada7a-421f-4ab9-a884-5dc214bb9ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:46.931435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ada7a-421f-4ab9-a884-5dc214bb9ae0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:46.933376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 225/444] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:46 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:46.933950 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ada7a-421f-4ab9-a884-5dc214bb9ae0 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.012224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aae7f8-3eef-45b5-bea1-43760e7fac7a None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.020155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:47.038327 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ada7a-421f-4ab9-a884-5dc214bb9ae0 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.052184 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cfc1b-0814-47d2-bfa9-59e1aafe7ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.052862 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfc1b-0814-47d2-bfa9-59e1aafe7ad2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.053145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfc1b-0814-47d2-bfa9-59e1aafe7ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.053422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfc1b-0814-47d2-bfa9-59e1aafe7ad2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.068339 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cfc1b-0814-47d2-bfa9-59e1aafe7ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-c71cfc1b-0814-47d2-bfa9-59e1aafe7ad2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:47.073510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:47.074547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:47.075456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 220/445] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:08:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:47.076177 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cfc1b-0814-47d2-bfa9-59e1aafe7ad2 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.082457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb29bc-da68-4fcc-928a-6b454477ba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.083145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb29bc-da68-4fcc-928a-6b454477ba60 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.083433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb29bc-da68-4fcc-928a-6b454477ba60 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.083711 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb29bc-da68-4fcc-928a-6b454477ba60 None admin] PATH_INFO: `/v3/projects/ff937f0574a548faac1d7abf71ec1698` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.085456 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb29bc-da68-4fcc-928a-6b454477ba60 None admin] RBAC: Authorizing `identity:get_project(project_id=ff937f0574a548faac1d7abf71ec1698)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:47.087396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb29bc-da68-4fcc-928a-6b454477ba60 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:47.089526 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 226/446] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:46 2022] GET /identity/v3/projects/ff937f0574a548faac1d7abf71ec1698 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:47.090278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb29bc-da68-4fcc-928a-6b454477ba60 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.111117 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:47.184846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cfc1b-0814-47d2-bfa9-59e1aafe7ad2 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.193397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb29bc-da68-4fcc-928a-6b454477ba60 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.222341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bb736-4d86-44d2-b77e-d3e6f8913f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.223035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bb736-4d86-44d2-b77e-d3e6f8913f3a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.223286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bb736-4d86-44d2-b77e-d3e6f8913f3a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.223562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bb736-4d86-44d2-b77e-d3e6f8913f3a None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.224733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bb736-4d86-44d2-b77e-d3e6f8913f3a None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:47.226237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bb736-4d86-44d2-b77e-d3e6f8913f3a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:47.227884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 221/447] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:47 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:47.228650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bb736-4d86-44d2-b77e-d3e6f8913f3a None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.231322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.231977 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.232578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.232838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.233599 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:47.233898 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:47.235351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:47.244494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 227/448] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:47.245361 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.267608 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:47.327864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:47.331669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bb736-4d86-44d2-b77e-d3e6f8913f3a None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.352284 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab638fb-9dc5-45fa-9a1d-146d2f616221 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.367232 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9d79b-1407-417c-a36f-cc78e865d74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.368420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9d79b-1407-417c-a36f-cc78e865d74d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.369000 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9d79b-1407-417c-a36f-cc78e865d74d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.369494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9d79b-1407-417c-a36f-cc78e865d74d None admin] PATH_INFO: `/v3/groups/1e8110403a944b5c802e3878e5ff5d12` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.371421 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9d79b-1407-417c-a36f-cc78e865d74d None admin] RBAC: Authorizing `identity:get_group(group_id=1e8110403a944b5c802e3878e5ff5d12)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:47.373055 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9d79b-1407-417c-a36f-cc78e865d74d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:47.380535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 222/449] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:47 2022] GET /identity/v3/groups/1e8110403a944b5c802e3878e5ff5d12 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:47.380535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9d79b-1407-417c-a36f-cc78e865d74d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.390631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.391290 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.391581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.392154 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.392683 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] RBAC: Adding query filter params (group.id=1e8110403a944b5c802e3878e5ff5d12, role.id=2935a7836c864df48e32fa734f020882, scope.project.id=ff937f0574a548faac1d7abf71ec1698) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:47.393007 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:47.394523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:47.400825 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 228/450] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:47 2022] GET /identity/v3/role_assignments?group.id=1e8110403a944b5c802e3878e5ff5d12&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=2935a7836c864df48e32fa734f020882 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:47.401662 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.491669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9d79b-1407-417c-a36f-cc78e865d74d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.500513 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:47.522967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69997c4-1a54-4b4a-aafd-bb70d761f8d2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.529470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7dd68-0460-476c-af42-e915b61c7bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.530140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7dd68-0460-476c-af42-e915b61c7bb8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.530429 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7dd68-0460-476c-af42-e915b61c7bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.530706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7dd68-0460-476c-af42-e915b61c7bb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.561772 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.562457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.562753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.563030 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.563845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] RBAC: Adding query filter params (group.id=1e8110403a944b5c802e3878e5ff5d12, role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=ff937f0574a548faac1d7abf71ec1698) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:47.564165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:47.565629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:47.571616 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:47.572887 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 229/451] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:47 2022] GET /identity/v3/role_assignments?group.id=1e8110403a944b5c802e3878e5ff5d12&scope.project.id=ff937f0574a548faac1d7abf71ec1698&role.id=601939cb90d641e08b97775c0c12d374 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:47.617074 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7dd68-0460-476c-af42-e915b61c7bb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:47.618641 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7dd68-0460-476c-af42-e915b61c7bb8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:47.736172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 223/452] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:47.742379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.852234 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f41c5-d4b1-47ac-95d0-65d899404c91 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:47.886488 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7110cce-5c76-4672-911b-f6d2f6af1c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:47.887162 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7110cce-5c76-4672-911b-f6d2f6af1c79 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:47.887454 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7110cce-5c76-4672-911b-f6d2f6af1c79 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:47.887744 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7110cce-5c76-4672-911b-f6d2f6af1c79 None admin] PATH_INFO: `/v3/projects/b2203339adc34a0499886c3bb35876aa` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:47.888939 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7110cce-5c76-4672-911b-f6d2f6af1c79 None admin] RBAC: Authorizing `identity:get_project(project_id=b2203339adc34a0499886c3bb35876aa)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:47.890457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7110cce-5c76-4672-911b-f6d2f6af1c79 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:47.892209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 230/453] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:08:47 2022] GET /identity/v3/projects/b2203339adc34a0499886c3bb35876aa => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:47.898347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7dd68-0460-476c-af42-e915b61c7bb8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:47.966984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:48.005100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7dd68-0460-476c-af42-e915b61c7bb8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:48.039890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827e493-2e4f-400a-b03a-1095dfbebeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:48.040569 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e493-2e4f-400a-b03a-1095dfbebeed None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:48.040849 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e493-2e4f-400a-b03a-1095dfbebeed None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:48.041125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e493-2e4f-400a-b03a-1095dfbebeed None admin] PATH_INFO: `/v3/groups/f9977230f6434693a7e40797b554623b` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:48.042385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827e493-2e4f-400a-b03a-1095dfbebeed None admin] RBAC: Authorizing `identity:get_group(group_id=f9977230f6434693a7e40797b554623b)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:48.043849 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827e493-2e4f-400a-b03a-1095dfbebeed None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:48.045465 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 224/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:08:47 2022] GET /identity/v3/groups/f9977230f6434693a7e40797b554623b => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:48.051847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7110cce-5c76-4672-911b-f6d2f6af1c79 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:48.069181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:48.157846 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7110cce-5c76-4672-911b-f6d2f6af1c79 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:48.193124 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428838-c7bb-43a2-8c19-730aab6aad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:48.193752 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428838-c7bb-43a2-8c19-730aab6aad76 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:48.194033 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428838-c7bb-43a2-8c19-730aab6aad76 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:48.194343 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428838-c7bb-43a2-8c19-730aab6aad76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:48.195172 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27428838-c7bb-43a2-8c19-730aab6aad76 None admin] RBAC: Adding query filter params (group.id=f9977230f6434693a7e40797b554623b, role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=b2203339adc34a0499886c3bb35876aa) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:48.195489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27428838-c7bb-43a2-8c19-730aab6aad76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:48.196935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27428838-c7bb-43a2-8c19-730aab6aad76 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:48.203127 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 231/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:08:48 2022] GET /identity/v3/role_assignments?group.id=f9977230f6434693a7e40797b554623b&scope.project.id=b2203339adc34a0499886c3bb35876aa&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:50.776849 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea02aa-a25a-4dd2-97cf-e7dc8d4fca35 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:50.777944 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea02aa-a25a-4dd2-97cf-e7dc8d4fca35 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:50.778487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea02aa-a25a-4dd2-97cf-e7dc8d4fca35 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:50.780371 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 225/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:50.785866 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ed401-54ad-4688-9718-70b5f7a08b89 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:50.786786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ed401-54ad-4688-9718-70b5f7a08b89 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:50.787309 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ed401-54ad-4688-9718-70b5f7a08b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:50.810497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-5d1ed401-54ad-4688-9718-70b5f7a08b89 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:50.812074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-5d1ed401-54ad-4688-9718-70b5f7a08b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:50.836536 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305837a9-efe4-45ea-9809-631e7410c7a7 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:50.837170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305837a9-efe4-45ea-9809-631e7410c7a7 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:50.837675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305837a9-efe4-45ea-9809-631e7410c7a7 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:50.839544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 226/457] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:50.844373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d55db-d322-4dea-95fc-19ae8bbff697 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:50.844938 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d55db-d322-4dea-95fc-19ae8bbff697 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:50.845427 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d55db-d322-4dea-95fc-19ae8bbff697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:50.873821 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-fd7d55db-d322-4dea-95fc-19ae8bbff697 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:50.875109 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-fd7d55db-d322-4dea-95fc-19ae8bbff697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:50.931666 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 232/458] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:50.955840 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ed401-54ad-4688-9718-70b5f7a08b89 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:50.977172 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:50.993784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 227/459] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:51.017464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d55db-d322-4dea-95fc-19ae8bbff697 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:51.098658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:51.139464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d55db-d322-4dea-95fc-19ae8bbff697 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:51.180626 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:51.181613 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.182202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.182648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.199093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ed401-54ad-4688-9718-70b5f7a08b89 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:51.211025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:51.212657 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:51.243474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:51.244677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:51.245840 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:51.245840 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:51.245840 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:51.245840 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:51.248331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250eab5-a149-4766-bd0b-bbbbbb8dd3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:51.249214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250eab5-a149-4766-bd0b-bbbbbb8dd3b3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.249646 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250eab5-a149-4766-bd0b-bbbbbb8dd3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.250067 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250eab5-a149-4766-bd0b-bbbbbb8dd3b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.255742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 228/460] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:51.261584 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:51.262303 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250eab5-a149-4766-bd0b-bbbbbb8dd3b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:51.264027 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250eab5-a149-4766-bd0b-bbbbbb8dd3b3 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-d250eab5-a149-4766-bd0b-bbbbbb8dd3b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:51.301143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:51.302187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:51.303126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:51.303126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:51.303126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:51.303126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:51.304278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 233/461] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:51.307453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1ce1e-481f-43ed-9598-957186536578 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.308010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1ce1e-481f-43ed-9598-957186536578 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.308536 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1ce1e-481f-43ed-9598-957186536578 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.316027 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 234/462] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:51.317593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1ce1e-481f-43ed-9598-957186536578 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:51.383165 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35302a70-cd14-4d02-babf-075bc21ea6b9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:51.402473 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:51.406313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:51.425125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56026fff-68fe-4e19-b46d-79ae22b33a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:51.426038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56026fff-68fe-4e19-b46d-79ae22b33a32 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.426543 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56026fff-68fe-4e19-b46d-79ae22b33a32 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.426983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56026fff-68fe-4e19-b46d-79ae22b33a32 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.427877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56026fff-68fe-4e19-b46d-79ae22b33a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:51.428337 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56026fff-68fe-4e19-b46d-79ae22b33a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:51.430083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56026fff-68fe-4e19-b46d-79ae22b33a32 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:51.444771 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 229/463] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:51.446518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b935f3-142d-421b-8de0-f18163337ca1 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.446950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b935f3-142d-421b-8de0-f18163337ca1 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.447365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b935f3-142d-421b-8de0-f18163337ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.467663 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1ce1e-481f-43ed-9598-957186536578 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:51.467994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-67b935f3-142d-421b-8de0-f18163337ca1 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:51.469549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-67b935f3-142d-421b-8de0-f18163337ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:51.510699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:51.512373 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.512924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.513437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.514402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:51.514931 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:51.516592 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:51.533825 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 235/464] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:51.534803 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:51.553854 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 230/465] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:51.555275 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b935f3-142d-421b-8de0-f18163337ca1 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:51.647675 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7e406-ca3d-4dc2-bdd2-b6b04d9061d6 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:51.667358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:51.667939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b935f3-142d-421b-8de0-f18163337ca1 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:51.677172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:51.687502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1bb3a-1c2c-4266-a81a-1f193f81d7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:51.688154 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bb3a-1c2c-4266-a81a-1f193f81d7fa None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.688441 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bb3a-1c2c-4266-a81a-1f193f81d7fa None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.688720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bb3a-1c2c-4266-a81a-1f193f81d7fa None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.689466 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1bb3a-1c2c-4266-a81a-1f193f81d7fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:51.690954 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1bb3a-1c2c-4266-a81a-1f193f81d7fa None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:51.713091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3306e3b-8c7d-43f2-af93-f16033c0506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:51.713848 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3306e3b-8c7d-43f2-af93-f16033c0506e None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.714163 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3306e3b-8c7d-43f2-af93-f16033c0506e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.714445 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3306e3b-8c7d-43f2-af93-f16033c0506e None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.719732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 236/466] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:51 2022] POST /identity/v3/users => generated 248 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:51.719884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3306e3b-8c7d-43f2-af93-f16033c0506e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:51.719966 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20119146-36b3-4831-8755-674260006598 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.719966 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20119146-36b3-4831-8755-674260006598 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.719966 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20119146-36b3-4831-8755-674260006598 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.720209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3306e3b-8c7d-43f2-af93-f16033c0506e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:51.720293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 237/467] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:51.723589 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20119146-36b3-4831-8755-674260006598 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:51.746330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 231/468] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:51 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:51.748920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36bfd8-ba56-4ac0-9937-797416236db2 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.749291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36bfd8-ba56-4ac0-9937-797416236db2 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.749485 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36bfd8-ba56-4ac0-9937-797416236db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.805829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-bd36bfd8-ba56-4ac0-9937-797416236db2 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:51.807951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-bd36bfd8-ba56-4ac0-9937-797416236db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:51.938283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20119146-36b3-4831-8755-674260006598 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:51.941209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 232/469] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:51.943078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572e3f1-4a6b-4929-b41f-e15cf2361031 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.943438 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572e3f1-4a6b-4929-b41f-e15cf2361031 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.943733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572e3f1-4a6b-4929-b41f-e15cf2361031 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.946247 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 233/470] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:51.953555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38edce7-461e-40ec-836d-eebf4a010e2a None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.953849 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38edce7-461e-40ec-836d-eebf4a010e2a None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.954197 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38edce7-461e-40ec-836d-eebf4a010e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:51.974375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-e38edce7-461e-40ec-836d-eebf4a010e2a None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:51.976399 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-e38edce7-461e-40ec-836d-eebf4a010e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:51.991984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:51.993243 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:51.993360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:51.996919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.014310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:52.015799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:52.031993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:52.032991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:52.033922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:52.033922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:52.033922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:52.033922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:52.033922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 238/471] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:52.034909 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:52.070197 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:52.070588 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 234/472] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:52.071485 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38edce7-461e-40ec-836d-eebf4a010e2a None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:52.118880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:52.163980 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e0b27-fa94-476c-a9bf-527fce6e1f98 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:52.204778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:52.205401 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:52.205762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:52.205955 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.211744 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38edce7-461e-40ec-836d-eebf4a010e2a None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:52.217715 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:52.219260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:52.236316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:52.237325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:52.238287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:52.238287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:52.238287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:52.238287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:52.238287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 239/473] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:52.239021 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:52.259694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:52.260713 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:52.261226 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:52.262157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.263351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:52.263867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:52.266190 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:52.285386 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 235/474] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:52.285386 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:52.335120 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:52.367201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2b28-97e1-4042-900f-b5e28756bb4a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:52.384799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:52.410915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:52.411297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b440-bdd3-4b76-afec-e0090ee00d54 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:52.412171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:52.413041 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:52.414490 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.415001 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:52.416428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:52.427940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.sql.core [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 09:08:52.427940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 09:08:52.427940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [parameters: {'id': 'af81941a9e004564a4120f9352b3faea', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 09:08:52.427940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=411108) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application result.read() Jul 28 09:08:52.450878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:08:52.451912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application yield resource Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise self.value Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application session.commit() Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 09:08:52.452935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.session.flush() Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application compat.raise_( Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 09:08:52.453906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:08:52.455640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application result.read() Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:08:52.456550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [parameters: {'id': 'af81941a9e004564a4120f9352b3faea', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:52.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 09:08:52.458403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:52.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 240/475] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:08:52 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:52.460943 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:52.466627 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:52.467689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:52.468191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:52.468828 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.469862 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:52.470532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:52.472388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:52.489701 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 236/476] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:52.491219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:52.587360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6f299-2eb1-48bd-b350-c37245a08de2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:52.618006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd41ef3-a53c-4662-a282-7cae87c46991 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:52.624856 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe708bb-d852-4da9-bd45-f7c3777d4596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:52.625518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe708bb-d852-4da9-bd45-f7c3777d4596 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:52.625809 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe708bb-d852-4da9-bd45-f7c3777d4596 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:52.626092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe708bb-d852-4da9-bd45-f7c3777d4596 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.626835 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe708bb-d852-4da9-bd45-f7c3777d4596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:52.628429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe708bb-d852-4da9-bd45-f7c3777d4596 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:52.648438 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:52.658768 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 241/477] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:52 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:52.658768 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe708bb-d852-4da9-bd45-f7c3777d4596 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:52.677879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc072c50-2497-4da0-805a-77c9678359e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:52.679115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc072c50-2497-4da0-805a-77c9678359e4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:52.679740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc072c50-2497-4da0-805a-77c9678359e4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:52.680240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc072c50-2497-4da0-805a-77c9678359e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.701742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc072c50-2497-4da0-805a-77c9678359e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-cc072c50-2497-4da0-805a-77c9678359e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:52.707545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:52.708513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:08:52.709415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:52.712419 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 237/478] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:08:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:52.727742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc072c50-2497-4da0-805a-77c9678359e4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:52.732203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:52.800055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe708bb-d852-4da9-bd45-f7c3777d4596 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:52.876572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014bae7-529e-44c2-8bf7-8f88f386f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:52.877352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014bae7-529e-44c2-8bf7-8f88f386f446 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:52.877779 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014bae7-529e-44c2-8bf7-8f88f386f446 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:52.878063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014bae7-529e-44c2-8bf7-8f88f386f446 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.878799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014bae7-529e-44c2-8bf7-8f88f386f446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:52.881594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014bae7-529e-44c2-8bf7-8f88f386f446 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:52.907879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 242/479] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:52 2022] POST /identity/v3/users => generated 249 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:52.937595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc072c50-2497-4da0-805a-77c9678359e4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:52.991895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a204f51-a280-4bd2-89fd-e0d913696042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:52.992735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204f51-a280-4bd2-89fd-e0d913696042 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:52.993025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204f51-a280-4bd2-89fd-e0d913696042 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:52.993305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204f51-a280-4bd2-89fd-e0d913696042 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:52.994063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a204f51-a280-4bd2-89fd-e0d913696042 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:52.994397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a204f51-a280-4bd2-89fd-e0d913696042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:52.995847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a204f51-a280-4bd2-89fd-e0d913696042 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:53.006494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 238/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:54.149921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47544091-b577-40f1-8fcf-95f35dc1ae7e None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:54.150543 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47544091-b577-40f1-8fcf-95f35dc1ae7e None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:54.150543 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47544091-b577-40f1-8fcf-95f35dc1ae7e None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:54.152278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 243/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:54.157884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3b92-e559-4d90-962f-bd7b24fa5dfd None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:54.158587 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3b92-e559-4d90-962f-bd7b24fa5dfd None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:54.158587 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3b92-e559-4d90-962f-bd7b24fa5dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:54.173050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197642c-afe2-471d-81d5-e33646b4e293 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:54.173405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197642c-afe2-471d-81d5-e33646b4e293 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:54.173610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197642c-afe2-471d-81d5-e33646b4e293 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:54.175351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 244/482] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:54.182913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84997f-cdcd-4d59-b150-895f1b6a2e02 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:54.183256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84997f-cdcd-4d59-b150-895f1b6a2e02 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:54.183471 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84997f-cdcd-4d59-b150-895f1b6a2e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:54.201879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-f9de3b92-e559-4d90-962f-bd7b24fa5dfd None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:54.203609 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-f9de3b92-e559-4d90-962f-bd7b24fa5dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:54.212143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-0d84997f-cdcd-4d59-b150-895f1b6a2e02 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:54.213440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-0d84997f-cdcd-4d59-b150-895f1b6a2e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:54.294503 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:54.304124 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 245/483] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:08:54.328765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84997f-cdcd-4d59-b150-895f1b6a2e02 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:54.356284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:54.384017 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 239/484] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:54.424767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de3b92-e559-4d90-962f-bd7b24fa5dfd None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:54.471699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84997f-cdcd-4d59-b150-895f1b6a2e02 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:54.541990 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436f334-b5bf-44ac-ae86-392c7e1a131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:54.542696 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f334-b5bf-44ac-ae86-392c7e1a131b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:54.542981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f334-b5bf-44ac-ae86-392c7e1a131b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:54.543260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f334-b5bf-44ac-ae86-392c7e1a131b None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:54.563583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436f334-b5bf-44ac-ae86-392c7e1a131b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-e436f334-b5bf-44ac-ae86-392c7e1a131b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:54.571301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:54.572268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:08:54.573131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:54.573791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 246/485] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:08:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:54.581721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436f334-b5bf-44ac-ae86-392c7e1a131b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:54.588368 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de3b92-e559-4d90-962f-bd7b24fa5dfd None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:54.658557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:54.665521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac6c2a-7f1e-4c85-add2-f47a1024fe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:54.666466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6c2a-7f1e-4c85-add2-f47a1024fe1f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:54.666751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6c2a-7f1e-4c85-add2-f47a1024fe1f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:54.667025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6c2a-7f1e-4c85-add2-f47a1024fe1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:54.692535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac6c2a-7f1e-4c85-add2-f47a1024fe1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a1ac6c2a-7f1e-4c85-add2-f47a1024fe1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:54.697547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:54.698556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:54.699488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 240/486] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:08:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:54.707569 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac6c2a-7f1e-4c85-add2-f47a1024fe1f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:54.742738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436f334-b5bf-44ac-ae86-392c7e1a131b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:54.816040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:54.816696 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:54.817339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:54.817624 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:54.818397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:54.818709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:54.824633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:54.826496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:54.849771 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 247/487] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:08:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:54.850794 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:54.921935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac6c2a-7f1e-4c85-add2-f47a1024fe1f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:54.972633 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:54.973066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:54.973359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:54.973638 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:54.974457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:54.974763 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:54.982257 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:54.985754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 241/488] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:08:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:55.008147 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.036552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046f05-de8d-4803-bfc2-5fc735117d9f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.082618 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:55.087786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.089179 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.089179 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.089412 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.103546 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:55.105059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:55.121221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:55.122276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:55.123520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:55.123520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:55.123520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:55.123520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:55.123520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 248/489] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:55.128300 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.201706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061422-b6d2-48c5-8da7-f0004e95761d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.237253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ae99-146a-4cad-a672-fe817d72cd89 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.252868 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:55.276935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b51de5-e616-4a72-a23f-e1e7bc04af0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.277593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b51de5-e616-4a72-a23f-e1e7bc04af0d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.277883 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b51de5-e616-4a72-a23f-e1e7bc04af0d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.278196 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b51de5-e616-4a72-a23f-e1e7bc04af0d None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.278920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b51de5-e616-4a72-a23f-e1e7bc04af0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:55.279233 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b51de5-e616-4a72-a23f-e1e7bc04af0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:55.280648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b51de5-e616-4a72-a23f-e1e7bc04af0d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:55.287321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c2c37-5f69-4fbc-8843-c8461364dd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.287998 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c2c37-5f69-4fbc-8843-c8461364dd8e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.288290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c2c37-5f69-4fbc-8843-c8461364dd8e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.288562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c2c37-5f69-4fbc-8843-c8461364dd8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.295562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 249/490] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:55.301255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c2c37-5f69-4fbc-8843-c8461364dd8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:55.301419 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1118cb-36ac-4c3c-922c-2ae972fbdf20 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.301823 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1118cb-36ac-4c3c-922c-2ae972fbdf20 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.302140 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1118cb-36ac-4c3c-922c-2ae972fbdf20 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.302718 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c2c37-5f69-4fbc-8843-c8461364dd8e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:55.304185 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 250/491] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:55.305128 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1118cb-36ac-4c3c-922c-2ae972fbdf20 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-777c2c37-5f69-4fbc-8843-c8461364dd8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:55.325193 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:55.327461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:55.329725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:55.329725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:55.329725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:55.329725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:55.330289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:55.334138 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 242/492] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:55.334138 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a12d1-9c41-491b-8b89-75f1301a9006 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.334138 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a12d1-9c41-491b-8b89-75f1301a9006 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.334138 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a12d1-9c41-491b-8b89-75f1301a9006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.352058 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-e54a12d1-9c41-491b-8b89-75f1301a9006 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:55.353373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-e54a12d1-9c41-491b-8b89-75f1301a9006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:55.424334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1118cb-36ac-4c3c-922c-2ae972fbdf20 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.426264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 243/493] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:55.427139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a12d1-9c41-491b-8b89-75f1301a9006 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.463720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304271c6-d5e3-47ed-b90b-79ce4407b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.464082 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304271c6-d5e3-47ed-b90b-79ce4407b133 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.464082 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304271c6-d5e3-47ed-b90b-79ce4407b133 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.464410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304271c6-d5e3-47ed-b90b-79ce4407b133 None admin] PATH_INFO: `/v3/users/nova` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.472648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304271c6-d5e3-47ed-b90b-79ce4407b133 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:55.474182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304271c6-d5e3-47ed-b90b-79ce4407b133 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-304271c6-d5e3-47ed-b90b-79ce4407b133 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:08:55.487691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:55.488708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:08:55.489677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:55.493818 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 251/494] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:55.495468 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acbad7-6f8f-438c-9414-66e4c03cdebf None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.495612 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acbad7-6f8f-438c-9414-66e4c03cdebf None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.495826 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acbad7-6f8f-438c-9414-66e4c03cdebf None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.498281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 252/495] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:55.498281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acbad7-6f8f-438c-9414-66e4c03cdebf None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.551760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:55.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a12d1-9c41-491b-8b89-75f1301a9006 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.588377 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:55.594379 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.595021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.595306 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.595583 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.596285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:55.596598 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:55.597983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:55.610579 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 244/496] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:55.611610 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.617668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acbad7-6f8f-438c-9414-66e4c03cdebf None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.652136 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31328b6c-c469-4cd6-bc76-2136f02a6bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.652804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328b6c-c469-4cd6-bc76-2136f02a6bd6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.653089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328b6c-c469-4cd6-bc76-2136f02a6bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.653370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328b6c-c469-4cd6-bc76-2136f02a6bd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.663884 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31328b6c-c469-4cd6-bc76-2136f02a6bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-31328b6c-c469-4cd6-bc76-2136f02a6bd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:55.669352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:55.670567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:55.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 253/497] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:55.672210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164d12-fd63-40b9-819f-b6b05954d0c2 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.672210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164d12-fd63-40b9-819f-b6b05954d0c2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.672336 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164d12-fd63-40b9-819f-b6b05954d0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.691605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-a9164d12-fd63-40b9-819f-b6b05954d0c2 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:55.694067 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-a9164d12-fd63-40b9-819f-b6b05954d0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:55.725331 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53128a30-6ff2-47dc-9f3e-70ac9163bd1d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.762994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.763535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.763824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.764090 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.764895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:55.765204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:55.766693 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:55.769050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 254/498] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:55.770390 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56beb5c-b66f-4439-97a4-34e285bdbf9b None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.770684 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56beb5c-b66f-4439-97a4-34e285bdbf9b None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.770965 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56beb5c-b66f-4439-97a4-34e285bdbf9b None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.772388 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 255/499] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:55.772939 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56beb5c-b66f-4439-97a4-34e285bdbf9b None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.805432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 245/500] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:55.806334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.825043 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:55.852393 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:55.888801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56beb5c-b66f-4439-97a4-34e285bdbf9b None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.909094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad884a-6301-4916-86ca-7386fca1cf04 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:55.923927 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005f119-273e-4f51-a13e-815d272331e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.924605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005f119-273e-4f51-a13e-815d272331e6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.924925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005f119-273e-4f51-a13e-815d272331e6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.925156 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005f119-273e-4f51-a13e-815d272331e6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.933115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005f119-273e-4f51-a13e-815d272331e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:55.934656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005f119-273e-4f51-a13e-815d272331e6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:55.944769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:55.945460 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.945744 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.946021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.946774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:55.947080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-8005f119-273e-4f51-a13e-815d272331e6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:08:55.947183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:55.948213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:08:55.954421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:55.955112 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:55.955201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 256/501] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:55.955201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56267cee-2c25-4661-8b8e-a20dda2288bb None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:55.955201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56267cee-2c25-4661-8b8e-a20dda2288bb None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:55.955201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56267cee-2c25-4661-8b8e-a20dda2288bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:55.960670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 246/502] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:55.960670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:55.976215 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-56267cee-2c25-4661-8b8e-a20dda2288bb None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:55.977441 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-56267cee-2c25-4661-8b8e-a20dda2288bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:56.048505 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:56.050078 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 257/503] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:56.051857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56267cee-2c25-4661-8b8e-a20dda2288bb None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.072413 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0afe6f-5085-4cda-baad-a38329ccf7e2 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.110210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47c7f3-149c-40a7-a2dc-85d2da70de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.110894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47c7f3-149c-40a7-a2dc-85d2da70de5e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.111164 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47c7f3-149c-40a7-a2dc-85d2da70de5e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.111443 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47c7f3-149c-40a7-a2dc-85d2da70de5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.117731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:56.124051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47c7f3-149c-40a7-a2dc-85d2da70de5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-4a47c7f3-149c-40a7-a2dc-85d2da70de5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:08:56.129762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:56.131076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 247/504] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:08:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:56.132179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47c7f3-149c-40a7-a2dc-85d2da70de5e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.167383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56267cee-2c25-4661-8b8e-a20dda2288bb None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.204898 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b55e-7dc6-44c2-a066-d39cfa495cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.205563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b55e-7dc6-44c2-a066-d39cfa495cbf None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.205855 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b55e-7dc6-44c2-a066-d39cfa495cbf None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.206158 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b55e-7dc6-44c2-a066-d39cfa495cbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.244941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47c7f3-149c-40a7-a2dc-85d2da70de5e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.281542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.282019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167b55e-7dc6-44c2-a066-d39cfa495cbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:56.282517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.282517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.282740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.283549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:56.283658 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167b55e-7dc6-44c2-a066-d39cfa495cbf None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:56.283864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:56.285341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:56.295453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:56.306694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 248/505] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:56.307476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.344417 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:56.391009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 258/506] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:56.392091 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b55e-7dc6-44c2-a066-d39cfa495cbf None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.426269 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7def48-3b0b-41e2-b1b1-e62f7d39e3ea None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.461964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.462658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.462948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.463229 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.473486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:56.475017 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:56.491442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:56.492653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:56.493572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:56.493572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:56.493572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:56.493572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:56.493572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 249/507] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:56.493572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.503205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b55e-7dc6-44c2-a066-d39cfa495cbf None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.541595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.542394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.542687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.542950 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.544637 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:56.553886 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:56.555434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:56.571600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:56.572731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:56.573761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:56.573761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:56.573761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:56.573761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:56.573761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 259/508] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:56.573761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.587957 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:56.599887 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922eeb36-a28a-4282-8fa2-6300f2871fba None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.634442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.635085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.635369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.635646 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.636374 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:56.636688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:56.638122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:56.646996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 250/509] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:56.647834 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.677719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ad9ea-9816-4d72-b264-0881f09467a6 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.712357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a997b89-9b14-4039-8022-adc4e383f44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.713351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a997b89-9b14-4039-8022-adc4e383f44d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.713740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a997b89-9b14-4039-8022-adc4e383f44d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.714146 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a997b89-9b14-4039-8022-adc4e383f44d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.751969 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce673a-f43a-4000-98ad-3054f048a8ed None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.762823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:56.784991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a997b89-9b14-4039-8022-adc4e383f44d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:56.786535 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a997b89-9b14-4039-8022-adc4e383f44d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:56.787441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.788110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.788400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.788776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.798239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:56.800802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:56.802334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:56.816869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:56.818407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:56.819505 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:56.819505 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:56.819505 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:56.819505 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:56.819505 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 251/510] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:56.819836 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.891371 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 260/511] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:56.892367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a997b89-9b14-4039-8022-adc4e383f44d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.922899 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8f186-7e84-478d-8940-823bd0e44bc0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:56.959383 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:56.960025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:56.960311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:56.960634 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:56.961357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:56.961666 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:56.963157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:56.977362 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 252/512] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:56.978385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:56.998789 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a997b89-9b14-4039-8022-adc4e383f44d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.004290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:57.010736 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:57.035594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.036272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.036753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.037236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.038342 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:57.038688 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.040092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.054539 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 261/513] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:57.055396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.084202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef780542-41e9-4fed-87b0-7837946cc96b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.119486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.120162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.120444 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.120722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.131056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.132584 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:57.148084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:57.149258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:57.150394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:57.150394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:57.150394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:57.150394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:57.150394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 253/514] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:57.151184 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.163522 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21aeda-f048-473e-96af-f37f28250b85 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.198753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.199394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.199681 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.199962 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.200674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:57.200985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.202851 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.216920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 262/515] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:57.217798 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.237182 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:57.249547 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:57.264785 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b38e2-26f9-47cb-8667-506d7e202fe1 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.299646 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.300304 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.300587 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.300978 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.312077 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.313550 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.323293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99b7a4-74ec-4c32-9670-3964d238f54d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:57.328300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:57.329330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:57.330556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:57.330556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:57.330556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:57.330556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:57.330556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 254/516] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:57.331078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.364499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.365180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.365477 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.365785 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.373798 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.375327 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:08:57.387914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:57.389034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 263/517] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:57.390006 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.439690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bd81d-ceff-43e1-9807-eb989071b425 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.471300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:57.481836 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d961e-bad6-49d6-98c5-bec6e3a0c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.482552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d961e-bad6-49d6-98c5-bec6e3a0c598 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.482802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d961e-bad6-49d6-98c5-bec6e3a0c598 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.483105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d961e-bad6-49d6-98c5-bec6e3a0c598 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.483797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d961e-bad6-49d6-98c5-bec6e3a0c598 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.484646 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:57.485252 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d961e-bad6-49d6-98c5-bec6e3a0c598 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.495265 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b585f-e219-4b3f-b6e9-b5dd8428df33 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.509531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 255/518] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:08:57 2022] POST /identity/v3/users => generated 247 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:08:57.510664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d961e-bad6-49d6-98c5-bec6e3a0c598 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.538997 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.540362 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.540838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.541276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.542229 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:57.543415 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.545326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.564403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 264/519] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:57.565544 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.642138 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d961e-bad6-49d6-98c5-bec6e3a0c598 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.683549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.684200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.684462 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.684738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.692791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beefc5-ad53-467f-ad19-b2f3cd72ddb5 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.693978 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.696385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:08:57.709931 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:57.711068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:57.712001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 256/520] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:57.712537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.738621 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.739382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.739677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.739957 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.740680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:57.740991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.742654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.757088 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 265/521] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:57.758016 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.772766 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:57.774860 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:57.848359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc571d-25d4-4128-8a5c-f45e886b1cb8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.879858 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc55443-942b-4687-8ae1-6f73b58e832a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:57.886136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.886794 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.887079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.887359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.888186 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:57.888498 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.889967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.909454 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 257/522] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:57.910824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:57.923553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:57.924656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:57.925154 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:57.925622 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:57.934911 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:57.936788 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:08:57.950620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:57.951615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:08:57.952595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:57.955188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 266/523] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:57.956333 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:58.025451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:58.029016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487b8b0-02fd-4b16-a68b-d07c8c415ced None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:58.037633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:58.069288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:58.069982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:58.070299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:58.070580 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:58.071355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:58.071664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:58.073148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:58.074365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3e794-bcff-4262-8815-363da0437cf4 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:58.097311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 258/524] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:58.097311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:58.119085 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:58.120135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:58.120605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:58.121066 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:58.128594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:58.130084 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:08:58.143246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:58.144434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 267/525] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:58.150287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:58.224033 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312feecd-1d70-4419-bedf-43c88a2b8659 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:58.253669 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d22a63-42c3-438d-88aa-32098fb6d018 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:58.262905 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a127e-3b1c-476c-b533-53b68f21d1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:58.263555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a127e-3b1c-476c-b533-53b68f21d1f7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:58.264086 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a127e-3b1c-476c-b533-53b68f21d1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:58.264086 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a127e-3b1c-476c-b533-53b68f21d1f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:58.285430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:58.292594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:58.293228 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:58.293531 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:58.293805 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:58.294655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:58.294964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:58.296403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:58.296687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:58.335108 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 268/526] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 09:08:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:58.336030 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:58.344022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a127e-3b1c-476c-b533-53b68f21d1f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:58.345820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a127e-3b1c-476c-b533-53b68f21d1f7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:58.446629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767bda2-b6de-4ae8-a81b-fd548f1833db None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:58.456464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 259/527] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:58.457330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a127e-3b1c-476c-b533-53b68f21d1f7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:58.492051 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:58.492998 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:58.493466 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:58.493940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:58.494953 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=1e579a84d5e34a828c5981f34b458a0d) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:58.495465 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:58.496926 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:58.502859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 269/528] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:58 2022] GET /identity/v3/role_assignments?user.id=1e579a84d5e34a828c5981f34b458a0d&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:58.503645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:58.538365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:58.572357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:58.604480 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a127e-3b1c-476c-b533-53b68f21d1f7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:58.632498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510761d-fe91-4551-a7be-691d00536c50 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:58.656367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:58.658250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:58.658946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:58.659497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:58.660453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:58.661203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:58.664458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:58.678673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 260/529] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:58.679665 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:58.681366 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5aa54-50b3-46cd-bfcb-6e9a1b684060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:58.682397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5aa54-50b3-46cd-bfcb-6e9a1b684060 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:58.682900 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5aa54-50b3-46cd-bfcb-6e9a1b684060 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:58.683367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5aa54-50b3-46cd-bfcb-6e9a1b684060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:58.761206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5aa54-50b3-46cd-bfcb-6e9a1b684060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:58.763125 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5aa54-50b3-46cd-bfcb-6e9a1b684060 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:58.813216 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7fa27-4fbe-41e3-a408-9c44e9d47f6c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:58.824803 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:58.852027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:58.852888 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:58.853097 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:58.853383 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:58.854271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:58.870401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:58.872089 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:58.882641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 270/530] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:08:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:08:58.883817 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5aa54-50b3-46cd-bfcb-6e9a1b684060 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:58.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:58.889661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:58.891365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:58.891365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:58.891365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:58.891365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:58.891365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 261/531] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:58.891365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:59.004871 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b28627-78c1-4d3a-a2e3-46cd9c8e8928 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:59.025908 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5aa54-50b3-46cd-bfcb-6e9a1b684060 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:59.047010 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:59.047941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.048235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.048913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.064725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:59.066409 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:59.067487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:59.068138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.068422 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.068705 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.069710 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=045e62a8ad8e4c4f916ea448f03cc438) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:59.070121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:59.071977 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:59.079570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 271/532] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:58 2022] GET /identity/v3/role_assignments?user.id=045e62a8ad8e4c4f916ea448f03cc438&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:59.080436 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:08:59.092290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:59.093648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:59.094621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:08:59.094621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:08:59.094621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:08:59.094621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:59.099434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 262/533] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:59.103693 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:59.125995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:59.166556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:59.300254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2b94-5e2d-4ed4-a564-27f338940836 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:59.314347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abc800-e896-45e5-b822-4ec11df918ad None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:59.345387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:59.346064 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.346400 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.346687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.347439 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:59.347759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:59.349257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:59.362874 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:59.363979 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.364478 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.364966 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.365939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:59.366535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:59.368390 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:59.369278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 272/534] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:59.370924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:59.384014 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 263/535] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:59.390157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:59.465465 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:59.494753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675f509-be02-4211-b0b2-0b919ab463c8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:59.536642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:59.537332 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.537624 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.537908 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.546239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:59.547770 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:08:59.562069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:59.563322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:59.564297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:08:59.564297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:08:59.564297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:08:59.564297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:08:59.564297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:08:59.564297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:08:59.564297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:08:59.564297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:08:59.568968 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:59.569439 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 273/536] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:08:59.570910 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:59.591674 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555ba60-e713-41a3-b1ce-feaefa2bfa0d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:59.666781 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:59.667464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.667751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.668032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.680322 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:59.683190 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:08:59.700739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:08:59.702761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:08:59.704912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 264/537] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:08:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:08:59.708803 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:59.752725 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2501-8541-4976-96d1-a0cbc6d2988a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:59.809152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de2d4c-4c56-4e8f-9bf7-1b5f4c28a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:59.810475 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de2d4c-4c56-4e8f-9bf7-1b5f4c28a770 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.810934 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de2d4c-4c56-4e8f-9bf7-1b5f4c28a770 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.811448 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de2d4c-4c56-4e8f-9bf7-1b5f4c28a770 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.812520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de2d4c-4c56-4e8f-9bf7-1b5f4c28a770 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:59.813041 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de2d4c-4c56-4e8f-9bf7-1b5f4c28a770 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:59.815041 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de2d4c-4c56-4e8f-9bf7-1b5f4c28a770 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:59.820747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:59.839582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 274/538] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:08:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:08:59.841265 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4023-dd4d-4d02-a48f-e9a2b972dac3 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.841797 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4023-dd4d-4d02-a48f-e9a2b972dac3 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.842313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4023-dd4d-4d02-a48f-e9a2b972dac3 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.848454 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 275/539] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:08:59.850368 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b544-5336-46c3-97d7-f711e1bf7857 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.850930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b544-5336-46c3-97d7-f711e1bf7857 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.851544 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b544-5336-46c3-97d7-f711e1bf7857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.872014 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-a027b544-5336-46c3-97d7-f711e1bf7857 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:08:59.873893 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-a027b544-5336-46c3-97d7-f711e1bf7857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:08:59.880470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fc9a5-5d3b-490c-8f4d-bd2b0222222d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:08:59.932006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:08:59.932806 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:08:59.933131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:08:59.933547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:08:59.934422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:08:59.934739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:08:59.937056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:08:59.955889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 265/540] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:08:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:08:59.956877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:08:59.966128 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 276/541] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:08:59.966337 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:08:59.967805 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027b544-5336-46c3-97d7-f711e1bf7857 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:00.090385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75700e49-79b4-4070-bf38-7f262bceb293 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:00.102049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027b544-5336-46c3-97d7-f711e1bf7857 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:00.131524 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:00.145042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:00.145685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:00.145968 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:00.146274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:00.147106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=d4dcfcbdf5df4c55a640573828c9e910) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:00.147416 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:00.148946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:00.151173 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:00.152557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:00.153333 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:00.153820 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:00.154868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=b3b1a6c31cc64159847f7ca39efe8105) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:00.156149 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 266/542] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:59 2022] GET /identity/v3/role_assignments?user.id=d4dcfcbdf5df4c55a640573828c9e910&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:00.156149 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:00.156912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:00.166042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:00.180750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 277/543] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:08:59 2022] GET /identity/v3/role_assignments?user.id=b3b1a6c31cc64159847f7ca39efe8105&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:00.315663 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:00.349074 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34b34d-3753-4d61-b2c1-2db5f52fbf7d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:00.390237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b42e50-7162-4a24-b660-8c3794a4bda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:00.390932 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b42e50-7162-4a24-b660-8c3794a4bda7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:00.391215 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b42e50-7162-4a24-b660-8c3794a4bda7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:00.391489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b42e50-7162-4a24-b660-8c3794a4bda7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:00.405273 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b42e50-7162-4a24-b660-8c3794a4bda7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-10b42e50-7162-4a24-b660-8c3794a4bda7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:00.410997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:00.412110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:00.413020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 267/544] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:00.418757 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:00.552304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29497a14-7181-43bc-a2f6-a558037ecaca None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:00.590209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8a619-6bfa-498e-98c3-4febbdc84a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:00.590868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8a619-6bfa-498e-98c3-4febbdc84a89 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:00.591144 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8a619-6bfa-498e-98c3-4febbdc84a89 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:00.591417 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8a619-6bfa-498e-98c3-4febbdc84a89 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:00.592158 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8a619-6bfa-498e-98c3-4febbdc84a89 None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:00.592463 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8a619-6bfa-498e-98c3-4febbdc84a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:00.593878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8a619-6bfa-498e-98c3-4febbdc84a89 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:00.603674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 278/545] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:00.608933 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b42e50-7162-4a24-b660-8c3794a4bda7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:00.755164 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b42e50-7162-4a24-b660-8c3794a4bda7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:00.795237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:00.803996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26becf44-557d-4778-9241-177dd579a891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:00.804782 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26becf44-557d-4778-9241-177dd579a891 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:00.805082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26becf44-557d-4778-9241-177dd579a891 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:00.805418 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26becf44-557d-4778-9241-177dd579a891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:00.826086 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26becf44-557d-4778-9241-177dd579a891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:00.827612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26becf44-557d-4778-9241-177dd579a891 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:00.841264 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95828a-0106-4a9a-a7ab-cc26621b7020 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:00.841597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95828a-0106-4a9a-a7ab-cc26621b7020 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:00.841831 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95828a-0106-4a9a-a7ab-cc26621b7020 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:00.843398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 279/546] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:00.849157 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6ae7-763e-4e16-bfc3-ea0a0579261f None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:00.849445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6ae7-763e-4e16-bfc3-ea0a0579261f None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:00.849730 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6ae7-763e-4e16-bfc3-ea0a0579261f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-26becf44-557d-4778-9241-177dd579a891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:00.859950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:00.860936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:00.861943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:00.861943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:00.861943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:00.861943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:00.861943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 268/547] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:00.867678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26becf44-557d-4778-9241-177dd579a891 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:00.896626 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-820e6ae7-763e-4e16-bfc3-ea0a0579261f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:00.897882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-820e6ae7-763e-4e16-bfc3-ea0a0579261f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:00.994506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:01.002128 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 280/548] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:09:01.022500 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6ae7-763e-4e16-bfc3-ea0a0579261f None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:01.068696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26becf44-557d-4778-9241-177dd579a891 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:01.156496 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6ae7-763e-4e16-bfc3-ea0a0579261f None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:01.179410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:01.180056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.180395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.180676 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.181414 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:01.181692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:01.183155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:01.197735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccd468-7640-4392-98ec-4b85fbb51c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:01.198849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccd468-7640-4392-98ec-4b85fbb51c60 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.199359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccd468-7640-4392-98ec-4b85fbb51c60 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.199884 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccd468-7640-4392-98ec-4b85fbb51c60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.213434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccd468-7640-4392-98ec-4b85fbb51c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-d0ccd468-7640-4392-98ec-4b85fbb51c60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:01.218985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:01.220020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:01.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:01.221670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 269/549] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:01.223933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 281/550] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:01.231179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:01.235645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccd468-7640-4392-98ec-4b85fbb51c60 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:01.308489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:01.327502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:01.428132 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a082d-b226-433f-a578-687956adf8a4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:01.453628 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccd468-7640-4392-98ec-4b85fbb51c60 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:01.495272 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d76d5-847a-41d8-8d70-5ca4d88004f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:01.496385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d76d5-847a-41d8-8d70-5ca4d88004f2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.496897 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d76d5-847a-41d8-8d70-5ca4d88004f2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.497394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d76d5-847a-41d8-8d70-5ca4d88004f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.505947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1241f-d869-4499-830b-6459f634ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:01.506639 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1241f-d869-4499-830b-6459f634ede2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.507433 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d76d5-847a-41d8-8d70-5ca4d88004f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:01.508930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d76d5-847a-41d8-8d70-5ca4d88004f2 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:01.511839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1241f-d869-4499-830b-6459f634ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.511839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1241f-d869-4499-830b-6459f634ede2 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.511839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1241f-d869-4499-830b-6459f634ede2 None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:01.511839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1241f-d869-4499-830b-6459f634ede2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:01.512403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1241f-d869-4499-830b-6459f634ede2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:01.523748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 282/551] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:01.525071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39042796-f224-4c6c-acd2-bd8a53fff0dc None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.525371 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39042796-f224-4c6c-acd2-bd8a53fff0dc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.525640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39042796-f224-4c6c-acd2-bd8a53fff0dc None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.526911 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 283/552] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-640d76d5-847a-41d8-8d70-5ca4d88004f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:01.531749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:01.532879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:01.533980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:01.534685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39042796-f224-4c6c-acd2-bd8a53fff0dc None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:01.535892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 270/553] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:01.537006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d18efa-fe74-458f-8aae-b9fcf458082b None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.537295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d18efa-fe74-458f-8aae-b9fcf458082b None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.537570 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d18efa-fe74-458f-8aae-b9fcf458082b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.557720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-e5d18efa-fe74-458f-8aae-b9fcf458082b None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:01.558990 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-e5d18efa-fe74-458f-8aae-b9fcf458082b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:01.654224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 271/554] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:01.655685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d18efa-fe74-458f-8aae-b9fcf458082b None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:01.690464 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39042796-f224-4c6c-acd2-bd8a53fff0dc None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:01.694206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:01.708384 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:01.735561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:01.736669 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.737217 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.737693 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.755309 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:01.757173 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:01.780258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:01.781348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:01.782375 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:01.782375 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:01.782375 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:01.782375 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:01.782375 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 284/555] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:01.782856 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:01.809838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d18efa-fe74-458f-8aae-b9fcf458082b None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:01.856117 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:01.856877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.857231 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.857580 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.858497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:01.858876 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:01.860395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:01.882786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 272/556] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:09:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:01.883789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:01.903828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d074-525d-46aa-8917-d38c4fba28e5 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:01.941351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7c199-43ce-46dc-84c0-2cb02a1ab800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:01.942372 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7c199-43ce-46dc-84c0-2cb02a1ab800 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:01.942699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7c199-43ce-46dc-84c0-2cb02a1ab800 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:01.942981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7c199-43ce-46dc-84c0-2cb02a1ab800 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:01.955191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7c199-43ce-46dc-84c0-2cb02a1ab800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-18d7c199-43ce-46dc-84c0-2cb02a1ab800 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:01.961352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:01.962499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:01.963519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 285/557] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:01.964318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7c199-43ce-46dc-84c0-2cb02a1ab800 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:01.984351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:01.986225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:02.000509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefc08f-408c-4f0c-8cf1-f5702da540f2 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.043265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.044299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.044787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.045265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.046320 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:02.046895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.048662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.066884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 273/558] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:02.068642 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:02.082134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7c199-43ce-46dc-84c0-2cb02a1ab800 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.121379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ac171-4067-4279-9251-9b907fcc9e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.122037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac171-4067-4279-9251-9b907fcc9e65 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.122394 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac171-4067-4279-9251-9b907fcc9e65 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.122685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac171-4067-4279-9251-9b907fcc9e65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.184148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4f586-2308-471d-815a-3c991a31036d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.200825 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ac171-4067-4279-9251-9b907fcc9e65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.202163 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ac171-4067-4279-9251-9b907fcc9e65 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.221914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:02.232180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.233207 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.233717 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.234227 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.235403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:02.235968 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.237684 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.252922 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 274/559] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:02.254256 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:02.269125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:02.326224 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 286/560] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:09:02.327354 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ac171-4067-4279-9251-9b907fcc9e65 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:02.379920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3490db-446b-48d6-a8fd-3d237ac44d61 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.429913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.431765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.432521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.432964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.443479 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.445970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.462888 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ac171-4067-4279-9251-9b907fcc9e65 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:02.463560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:02.464795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:02.466080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:02.475080 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 275/561] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:09:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:02.476601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:02.486661 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:02.508668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.510252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.510642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.511033 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.522958 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.524631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:02.541209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:02.542527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:02.543756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:02.543756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:02.543756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:02.543756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:02.549762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 287/562] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:02.549762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:02.587872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:02.640865 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b702a0-de87-41cc-a314-50580fb8fb35 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.663666 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbf110-0222-450c-bcaf-5c122979b24b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.679738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.680413 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.680702 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.680982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.692387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.693863 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.700019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531e5a1-8d41-4936-820a-4dd8acac3197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.700967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531e5a1-8d41-4936-820a-4dd8acac3197 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.700967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531e5a1-8d41-4936-820a-4dd8acac3197 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.701192 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531e5a1-8d41-4936-820a-4dd8acac3197 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.701991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531e5a1-8d41-4936-820a-4dd8acac3197 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:02.702351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531e5a1-8d41-4936-820a-4dd8acac3197 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.703789 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531e5a1-8d41-4936-820a-4dd8acac3197 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:02.708838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:02.709937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:02.710958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:02.710958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:02.710958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:02.710958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:02.710958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 276/563] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:02.710958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:02.723900 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 288/564] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:09:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:02.725172 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94e593-d8a3-4eee-b724-124b31a64b02 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.725488 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94e593-d8a3-4eee-b724-124b31a64b02 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.725829 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94e593-d8a3-4eee-b724-124b31a64b02 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.727183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 289/565] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:02.731341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4606b-1628-4a11-adb8-e68d74a53bdc None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.731624 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4606b-1628-4a11-adb8-e68d74a53bdc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.731890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4606b-1628-4a11-adb8-e68d74a53bdc None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.733134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 290/566] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:02.733826 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4606b-1628-4a11-adb8-e68d74a53bdc None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:02.765639 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:02.822412 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df9b10-91e7-4afa-97eb-39e0dd3a3b59 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.847463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:02.850712 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4606b-1628-4a11-adb8-e68d74a53bdc None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:02.860590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.861323 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.861614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.861893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.862638 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:02.862975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.864376 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.877287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 277/567] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:02.878436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:02.892856 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec3c70-c012-485e-8817-44850e8c4ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:02.893552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec3c70-c012-485e-8817-44850e8c4ca1 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.893842 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec3c70-c012-485e-8817-44850e8c4ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.894140 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec3c70-c012-485e-8817-44850e8c4ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.894860 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec3c70-c012-485e-8817-44850e8c4ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:02.895302 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec3c70-c012-485e-8817-44850e8c4ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:02.896749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec3c70-c012-485e-8817-44850e8c4ca1 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:02.909009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 291/568] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:02.910327 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3915b5-844f-41c2-b9ba-4272003b9351 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:02.910622 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3915b5-844f-41c2-b9ba-4272003b9351 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:02.910908 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3915b5-844f-41c2-b9ba-4272003b9351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:02.930131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-8a3915b5-844f-41c2-b9ba-4272003b9351 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:02.931415 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-8a3915b5-844f-41c2-b9ba-4272003b9351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:02.988363 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de7b19-5f2a-44af-88e7-f152e60881bb None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.001258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 292/569] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:03.002586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865868c-a688-4253-b5d2-c8fb0547f2df None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.002874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865868c-a688-4253-b5d2-c8fb0547f2df None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.003151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865868c-a688-4253-b5d2-c8fb0547f2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.025037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-d865868c-a688-4253-b5d2-c8fb0547f2df None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:03.026277 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-d865868c-a688-4253-b5d2-c8fb0547f2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:03.030594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.031262 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.031547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.031826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.037599 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:03.044283 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.045760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:03.061800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:03.062867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:03.063839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:03.063839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:03.063839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:03.063839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:03.063839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 278/570] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:03.064166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.098063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 293/571] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:03.098980 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865868c-a688-4253-b5d2-c8fb0547f2df None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.105343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:03.178543 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ca499-e568-4f53-b60e-1b80b309928d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.204932 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865868c-a688-4253-b5d2-c8fb0547f2df None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.219812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.220490 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.220770 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.221051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] PATH_INFO: `/v3/users/glance` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.228746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.230270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.241999 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:03.242631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:03.243644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:09:03.244725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:03.245402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.245402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.245402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.245632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 279/572] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:03.245632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.254247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.256102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:03.268019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:03.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 294/573] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:03.270194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.280223 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:03.343477 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:03.350670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6934a-3058-4767-8d72-164ab3a3b67c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.373105 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540f716-36c2-461f-aebd-72ccadbfd6e3 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.384689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05546a-7d09-44e4-95ad-9df830e90693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.385354 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05546a-7d09-44e4-95ad-9df830e90693 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.385642 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05546a-7d09-44e4-95ad-9df830e90693 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.385921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05546a-7d09-44e4-95ad-9df830e90693 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.396938 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05546a-7d09-44e4-95ad-9df830e90693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-1c05546a-7d09-44e4-95ad-9df830e90693 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:03.402528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:03.403651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:03.404687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:03.407866 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.408489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.408633 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 280/574] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:03.408916 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.409037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.409491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05546a-7d09-44e4-95ad-9df830e90693 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.409738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:03.410045 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.411518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.423573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 295/575] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:03.424423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.492514 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:03.518301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05546a-7d09-44e4-95ad-9df830e90693 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.528641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36dafe-e850-4062-ae21-4f592243776c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.552054 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ac194-4952-4f07-9527-d10ddadf866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.552721 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ac194-4952-4f07-9527-d10ddadf866b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.553007 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ac194-4952-4f07-9527-d10ddadf866b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.553306 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ac194-4952-4f07-9527-d10ddadf866b None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.563039 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.563677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.563967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.564073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ac194-4952-4f07-9527-d10ddadf866b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:03.564251 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.565066 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:03.565383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.566879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-419ac194-4952-4f07-9527-d10ddadf866b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:03.569802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:03.570861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:03.571820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 281/576] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:03.573084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ac194-4952-4f07-9527-d10ddadf866b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.587345 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 296/577] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:03.588221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.589430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:03.679439 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ac194-4952-4f07-9527-d10ddadf866b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.692674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4199c-b803-426c-b3fb-d7db7e5382cb None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.716231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:03.717961 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.718717 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.719082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.719404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.720090 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:03.720406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.721888 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.728162 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.730605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.730605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.730605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.730605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:03.730605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.731815 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.735789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 282/578] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:03.736767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.741259 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 297/579] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:03.742437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.815003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:03.841177 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83ad38-aac5-46e5-992a-110de367e4ef None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.848474 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace5a92-2c8e-4bdc-893d-4cd866d310e2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:03.879601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.880280 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.880556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.880854 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.886447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:03.887090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:03.887385 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:03.887660 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:03.888392 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:03.888501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.888702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:03.890005 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.890130 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:03.898629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 298/580] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:03.899436 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:03.901401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:03.902514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 283/581] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:03.903496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:03.955998 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:04.005773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f39f5-30a5-45ca-9ee8-b18fbc3828b1 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.008318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c04a6e-55fa-4cc4-89a9-b5015c804a62 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.040731 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.041456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.041810 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.042205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.042988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=7f85536ad53f474797f19311e1f19810) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:04.043297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.043650 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.044313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.044603 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.044740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.044882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.046674 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:04.056859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.058400 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.062601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 284/582] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/role_assignments?user.id=7f85536ad53f474797f19311e1f19810&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:04.063629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:04.083515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:04.084694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:04.085991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:04.085991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:04.085991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:04.085991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:04.088023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 299/583] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:04.089160 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.184343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e423664-1be3-4814-85cc-552e755ad67b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.209424 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a8bdd-f57a-4e19-9b50-ed3550fd8658 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.213103 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:04.231124 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.231899 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.232259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.232613 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.244843 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.246377 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.251685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.252437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.253123 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.253123 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:04.261632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:04.262873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:04.263951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:04.263951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:04.263951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:04.263951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:04.263951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 285/584] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:04.264830 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.269810 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.271339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:04.287050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:04.288293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:04.289586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:04.289586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:04.289586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:04.289586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:04.293096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 300/585] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:04.293974 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.321855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:04.377668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376631-9850-46a5-9160-5b475dce6cf6 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.405258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971ebc7-aae5-479b-8ce8-575541638b04 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.416215 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.416863 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.417151 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.417432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.418227 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:04.418541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.419996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.432416 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 286/586] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:04.433261 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.449999 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.451040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.451537 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.452009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.452960 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:04.453494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.455167 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.475172 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 301/587] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:04.476353 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.493305 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:04.554114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a79da-da12-4284-a1ff-5d114107498c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.568681 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:04.595404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a1db-03a7-4a2b-b983-ecc4bc0164c2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.597694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.598376 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.598665 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.598943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.599648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:04.599955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.601407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.614038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 287/588] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:04.614723 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.637692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.638283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.638573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.638867 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.639563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:04.639871 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.641274 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.653358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 302/589] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:04.654297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.728353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92bb27-9627-4507-8fc6-506a95d2187e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.730632 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:04.764321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6369d-487b-4e86-b544-6e26e053aa4a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.766290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a363-deab-4efc-a01b-2bbfc865274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.767599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a363-deab-4efc-a01b-2bbfc865274e None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.767599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a363-deab-4efc-a01b-2bbfc865274e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.767599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a363-deab-4efc-a01b-2bbfc865274e None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/1e579a84d5e34a828c5981f34b458a0d/roles/5d22bf96fee74e9eaf2b9cdba5b26ead` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.782906 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8a363-deab-4efc-a01b-2bbfc865274e None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=1e579a84d5e34a828c5981f34b458a0d, role_id=5d22bf96fee74e9eaf2b9cdba5b26ead)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.784424 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8a363-deab-4efc-a01b-2bbfc865274e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.798949 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 288/590] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:04 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/1e579a84d5e34a828c5981f34b458a0d/roles/5d22bf96fee74e9eaf2b9cdba5b26ead => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:04.800842 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:04.801549 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:04.801839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:04.802135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:04.810323 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:04.811351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a363-deab-4efc-a01b-2bbfc865274e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.811818 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:04.825339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:04.826483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:04.827487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:04.827487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:04.827487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:04.827487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:04.827487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:04.827487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:04.827487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:04.827487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:04.831751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 303/591] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:04.832773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:04.847670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:04.958166 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae848c5-fe8d-442d-80bd-72d310f5654a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.974070 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a363-deab-4efc-a01b-2bbfc865274e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:04.999615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:05.006136 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:05.007079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:05.007280 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:05.007533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:05.016641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:05.022328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:05.022904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:05.024087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:05.024087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:05.024295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:05.032546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:05.034069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:05.034324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:05.035523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:09:05.036718 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:05.038690 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 304/592] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:05.039646 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:05.051708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:05.053073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 289/593] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:05.054378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:05.150008 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:05.170628 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c177f-7eec-4068-a8dc-86b1047e4b19 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:05.177878 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b40ec-57a3-47a3-8d1a-d418c49c6855 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:05.216996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:05.217763 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:05.218202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:05.218572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:05.219658 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:05.219834 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:05.221498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:05.225724 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:05.226711 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:05.227173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:05.227619 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:05.228602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:05.229073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:05.231425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:05.236500 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 305/594] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:05.237542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:05.259974 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 290/595] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 09:09:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:05.261094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:05.309127 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:05.367421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb95a5-dbff-42a8-a08b-ce04c988be3d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:05.385914 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e4ec2-9735-4e15-844e-28554b7e1a09 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:05.416018 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:05.417245 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:05.418227 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:05.418863 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:05.419995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:05.420498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:05.422201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:05.423156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e979e-2467-4e8c-b3e5-dfb6441f1176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:05.423872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e979e-2467-4e8c-b3e5-dfb6441f1176 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:05.424158 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e979e-2467-4e8c-b3e5-dfb6441f1176 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:05.424438 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e979e-2467-4e8c-b3e5-dfb6441f1176 None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/045e62a8ad8e4c4f916ea448f03cc438/roles/5d22bf96fee74e9eaf2b9cdba5b26ead` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:05.434882 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:05.445878 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e979e-2467-4e8c-b3e5-dfb6441f1176 None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=045e62a8ad8e4c4f916ea448f03cc438, role_id=5d22bf96fee74e9eaf2b9cdba5b26ead)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:05.447773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e979e-2467-4e8c-b3e5-dfb6441f1176 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:05.452787 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 306/596] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:09:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:05.454430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:05.473462 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 291/597] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:05 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/045e62a8ad8e4c4f916ea448f03cc438/roles/5d22bf96fee74e9eaf2b9cdba5b26ead => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:05.474982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e979e-2467-4e8c-b3e5-dfb6441f1176 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:05.659904 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49edd31-24a6-4c64-a81b-17d7eab34695 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:05.675908 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:05.711319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:05.713194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:05.714019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:05.714606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:05.729894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e979e-2467-4e8c-b3e5-dfb6441f1176 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:05.732265 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:05.734928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:05.765201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:05.766397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:05.767491 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:05.767491 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:05.767491 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:05.767491 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:05.768351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 307/598] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:05.775262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:05.823035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:05.823916 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:05.824204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:05.824482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:05.860516 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:05.862688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:05.896295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:05.897382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:05.898466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:05.898466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:05.898466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:05.898466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:05.898466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 292/599] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:05.903316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:05.949954 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4256de-aa85-4139-8f2e-4c4208192200 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:05.997571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:06.021359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.022644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.024177 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.027611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.028666 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:06.029206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:06.030898 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:06.061633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 308/600] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:06.067831 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:06.138505 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:06.182636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd224ca-f83b-4638-aab0-fc0d582e7db7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:06.196376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1c00a-3b53-404f-8c71-b54d6317f44a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:06.238442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0bc19-dd06-46d4-bda4-9e8d892bee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.240037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0bc19-dd06-46d4-bda4-9e8d892bee37 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.240673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0bc19-dd06-46d4-bda4-9e8d892bee37 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.241067 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0bc19-dd06-46d4-bda4-9e8d892bee37 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.241973 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0bc19-dd06-46d4-bda4-9e8d892bee37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:06.242305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0bc19-dd06-46d4-bda4-9e8d892bee37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:06.245160 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0bc19-dd06-46d4-bda4-9e8d892bee37 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:06.249713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b5206-d1b1-47fd-8b36-36b2448ff519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.250721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5206-d1b1-47fd-8b36-36b2448ff519 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.251230 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5206-d1b1-47fd-8b36-36b2448ff519 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.251725 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5206-d1b1-47fd-8b36-36b2448ff519 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.264696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 293/601] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:06.264696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316033b-cf66-4503-bcdc-7fbc04712084 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.264696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316033b-cf66-4503-bcdc-7fbc04712084 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.264696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316033b-cf66-4503-bcdc-7fbc04712084 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.266294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b5206-d1b1-47fd-8b36-36b2448ff519 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:06.268087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b5206-d1b1-47fd-8b36-36b2448ff519 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:06.268355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 294/602] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:06.269261 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316033b-cf66-4503-bcdc-7fbc04712084 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-b80b5206-d1b1-47fd-8b36-36b2448ff519 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:06.284210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:06.285293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:06.286294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:06.286294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:06.286294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:06.286294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:06.286294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:06.286294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:06.286294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:06.286294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:06.294376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 309/603] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:06.294376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728c29e-7a00-415b-82de-4f270a0ecd4e None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.294376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728c29e-7a00-415b-82de-4f270a0ecd4e None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.294376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728c29e-7a00-415b-82de-4f270a0ecd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.320646 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-6728c29e-7a00-415b-82de-4f270a0ecd4e None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:06.322785 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-6728c29e-7a00-415b-82de-4f270a0ecd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:06.417305 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 310/604] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:06.418745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728c29e-7a00-415b-82de-4f270a0ecd4e None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:06.440496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:06.446781 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:06.448826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316033b-cf66-4503-bcdc-7fbc04712084 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:06.497687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.498874 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.499164 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.499445 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.510540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:06.511965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:06.524175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:06.525223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:06.526844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:06.526844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:06.526844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:06.526844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:06.526844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:06.526844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:06.526844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:06.526844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:06.530651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 295/605] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:06.531422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:06.566134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728c29e-7a00-415b-82de-4f270a0ecd4e None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:06.606135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.607302 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.607827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.608389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.609386 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:06.609926 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:06.611662 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:06.626151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 311/606] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:06.627499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:06.646263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065875e1-8cc2-4ea5-a522-871de5ce065a None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:06.683228 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8d19d-4bbf-4b71-bbc5-f9ce8d54a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.683904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8d19d-4bbf-4b71-bbc5-f9ce8d54a85c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.684200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8d19d-4bbf-4b71-bbc5-f9ce8d54a85c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.684492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8d19d-4bbf-4b71-bbc5-f9ce8d54a85c None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.697149 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8d19d-4bbf-4b71-bbc5-f9ce8d54a85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-e1b8d19d-4bbf-4b71-bbc5-f9ce8d54a85c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:06.703746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:06.704811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:06.705729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:06.709981 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 296/607] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:06.711550 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8d19d-4bbf-4b71-bbc5-f9ce8d54a85c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:06.715762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:06.734622 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:06.754496 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7ce8c-dda4-40fc-bacb-19f85ea5ad3b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:06.796075 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.797142 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.797706 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.798242 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.799233 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:06.799798 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:06.801792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:06.817798 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 312/608] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:06.819113 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:06.825974 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8d19d-4bbf-4b71-bbc5-f9ce8d54a85c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:06.861889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da30a7f-9438-4df5-9bbd-e0cbd7a0dc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.862621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da30a7f-9438-4df5-9bbd-e0cbd7a0dc36 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.862908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da30a7f-9438-4df5-9bbd-e0cbd7a0dc36 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.863187 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da30a7f-9438-4df5-9bbd-e0cbd7a0dc36 None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/b3b1a6c31cc64159847f7ca39efe8105/roles/5d22bf96fee74e9eaf2b9cdba5b26ead` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.879290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da30a7f-9438-4df5-9bbd-e0cbd7a0dc36 None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=b3b1a6c31cc64159847f7ca39efe8105, role_id=5d22bf96fee74e9eaf2b9cdba5b26ead)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:06.880787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da30a7f-9438-4df5-9bbd-e0cbd7a0dc36 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:06.906139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 297/609] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:06 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/b3b1a6c31cc64159847f7ca39efe8105/roles/5d22bf96fee74e9eaf2b9cdba5b26ead => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:06.907167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da30a7f-9438-4df5-9bbd-e0cbd7a0dc36 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:06.938063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36249d09-1102-42da-b625-4cb0bd912840 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:06.974557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:06.982656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:06.983775 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:06.984293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:06.984788 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:06.985851 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:06.986451 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:06.988203 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:06.999320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 313/610] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:07.009489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:07.030487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:07.069456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da30a7f-9438-4df5-9bbd-e0cbd7a0dc36 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:07.111925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be0f6a-9dd0-4de5-9304-57a474852f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:07.112838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be0f6a-9dd0-4de5-9304-57a474852f7f None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.113226 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be0f6a-9dd0-4de5-9304-57a474852f7f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.113607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be0f6a-9dd0-4de5-9304-57a474852f7f None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/d4dcfcbdf5df4c55a640573828c9e910/roles/5d22bf96fee74e9eaf2b9cdba5b26ead` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.130950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be0f6a-9dd0-4de5-9304-57a474852f7f None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=d4dcfcbdf5df4c55a640573828c9e910, role_id=5d22bf96fee74e9eaf2b9cdba5b26ead)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:07.132712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be0f6a-9dd0-4de5-9304-57a474852f7f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:07.150829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 298/611] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:06 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/d4dcfcbdf5df4c55a640573828c9e910/roles/5d22bf96fee74e9eaf2b9cdba5b26ead => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:07.152108 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965def93-0d19-4bc4-9af5-03f43d6a7966 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.152330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965def93-0d19-4bc4-9af5-03f43d6a7966 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.152787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965def93-0d19-4bc4-9af5-03f43d6a7966 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.157910 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 299/612] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:07.160995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280a0a-fee4-4b7c-a972-3017e89dfd3d None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.161362 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280a0a-fee4-4b7c-a972-3017e89dfd3d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.162406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280a0a-fee4-4b7c-a972-3017e89dfd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.172052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f7a3e-5de0-4350-9ab7-4629fdd85a4b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:07.184078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-0e280a0a-fee4-4b7c-a972-3017e89dfd3d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:07.186496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-0e280a0a-fee4-4b7c-a972-3017e89dfd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:07.230323 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:07.231473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.232022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.232546 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.246639 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:07.248567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:07.266370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:07.267521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:07.268865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:07.268865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:07.268865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:07.268865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:07.272605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 314/613] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:07.276705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 300/614] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:09:07.282256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:07.307756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e280a0a-fee4-4b7c-a972-3017e89dfd3d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:07.365580 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:07.395509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:07.432092 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e280a0a-fee4-4b7c-a972-3017e89dfd3d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:07.472713 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b95ed-4152-40f0-a2e3-e71457e5ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:07.473872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b95ed-4152-40f0-a2e3-e71457e5ce53 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.474396 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b95ed-4152-40f0-a2e3-e71457e5ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.474857 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b95ed-4152-40f0-a2e3-e71457e5ce53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.487102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfcd0f-650a-4ed7-89ca-65edbdde1e07 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:07.487745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b95ed-4152-40f0-a2e3-e71457e5ce53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-c10b95ed-4152-40f0-a2e3-e71457e5ce53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:07.493422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:07.494523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:07.495428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:07.497500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 301/615] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:07.507424 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b95ed-4152-40f0-a2e3-e71457e5ce53 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:07.539806 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:07.540560 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.540848 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.541224 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.541950 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:07.542304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:07.543721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:07.559538 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 315/616] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:07.566339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:07.683905 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b95ed-4152-40f0-a2e3-e71457e5ce53 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:07.712300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:07.717709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f501-35cb-424b-8db8-16d541f429aa None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:07.726042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:07.763368 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f3bf0-168b-4750-a12d-845a777ac986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:07.764047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f3bf0-168b-4750-a12d-845a777ac986 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.764338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f3bf0-168b-4750-a12d-845a777ac986 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.764620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f3bf0-168b-4750-a12d-845a777ac986 None admin] PATH_INFO: `/v3/users/swift` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.764737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d5c1e-5ac0-4499-9649-9d17ae8d1128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:07.765601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5c1e-5ac0-4499-9649-9d17ae8d1128 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.765997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5c1e-5ac0-4499-9649-9d17ae8d1128 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.766392 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5c1e-5ac0-4499-9649-9d17ae8d1128 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.767237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d5c1e-5ac0-4499-9649-9d17ae8d1128 None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:07.767648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d5c1e-5ac0-4499-9649-9d17ae8d1128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:07.769144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d5c1e-5ac0-4499-9649-9d17ae8d1128 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:07.775081 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f3bf0-168b-4750-a12d-845a777ac986 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:07.776649 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f3bf0-168b-4750-a12d-845a777ac986 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:07.779130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 302/617] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:07.780522 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8da09-9804-4809-9b69-d1df896a0417 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.780893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8da09-9804-4809-9b69-d1df896a0417 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.781244 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8da09-9804-4809-9b69-d1df896a0417 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.786894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 303/618] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:07.787857 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8da09-9804-4809-9b69-d1df896a0417 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-042f3bf0-168b-4750-a12d-845a777ac986 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:07.790694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:07.791930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:07.793267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:07.793929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 316/619] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:07.795275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1376bf-9078-40be-971f-62e620c987f7 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.795570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1376bf-9078-40be-971f-62e620c987f7 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.795852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1376bf-9078-40be-971f-62e620c987f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.814790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-6b1376bf-9078-40be-971f-62e620c987f7 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:07.816113 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-6b1376bf-9078-40be-971f-62e620c987f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:07.892850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 317/620] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:07.893828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1376bf-9078-40be-971f-62e620c987f7 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:07.911830 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8da09-9804-4809-9b69-d1df896a0417 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:07.966755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:07.967679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:07.968042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:07.968563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:07.984495 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:07.986295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.002217 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1376bf-9078-40be-971f-62e620c987f7 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:08.006259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:08.007544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:08.008751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:08.008751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:08.008751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:08.008751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:08.009786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 304/621] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:08.010794 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.034531 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:08.048407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.049755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:08.051412 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.051940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.052833 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.054485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:08.055458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:08.058599 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.084565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 318/622] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:09:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:08.086413 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.136739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ad6b-e51e-42bd-8a24-f7cb8c1dd9e0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.173213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184e82d-51df-46a6-866d-8f58988baa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.173992 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184e82d-51df-46a6-866d-8f58988baa56 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.174371 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184e82d-51df-46a6-866d-8f58988baa56 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.174723 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184e82d-51df-46a6-866d-8f58988baa56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.186924 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184e82d-51df-46a6-866d-8f58988baa56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-7184e82d-51df-46a6-866d-8f58988baa56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:08.192880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:08.193921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:08.194967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:08.199441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 305/623] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:08.200456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184e82d-51df-46a6-866d-8f58988baa56 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.229594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074c978-68bc-4a67-9401-222294aadbee None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.266004 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.266593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.266879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.267150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.267869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:08.268174 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:08.269576 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.282582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 319/624] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:08.283648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.301709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:08.328196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:08.353601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184e82d-51df-46a6-866d-8f58988baa56 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.394995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371f122-b9bb-4610-acff-cf0a9489f6c7 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.401486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.402211 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.402488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.402764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.414874 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:08.416361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:08.432779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:08.433815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:08.434826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:08.434826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:08.434826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:08.434826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:08.434826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 306/625] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:08.435572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.444320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.445229 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.446238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.446688 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.447608 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:08.448115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:08.450584 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.463620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 320/626] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:08.464574 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.545992 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:08.573424 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6bc0-84db-4eca-aa03-9a9b9c8f2723 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.574323 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b90709-0645-42e7-8003-808404959b1f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.613008 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.613875 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.614394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.614703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.618524 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:08.620763 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.621600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.622047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.622783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.623716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:08.624234 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:08.626687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.628110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:08.629945 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.643859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 321/627] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:08.645072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:08.649169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:08.650491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:08.651807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:08.653638 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 307/628] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:08.658240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.768421 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151b1e-2622-49d6-8b0b-1b49b0b23be5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.789708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb63dd-b587-4059-a450-250232dd365f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:08.806196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.806883 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.807158 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.807486 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.815166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:08.816652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:08.830206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:08.831987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 308/629] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:08.833219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.838850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:08.838850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:08.838850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:08.838850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:08.840773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:08.851547 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:08.853894 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:08.875482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:08.876583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:08.877570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:08.877570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:08.877570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:08.877570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:08.877570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 322/630] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:08.877570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:08.904375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:08.958866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815551f-6ac4-424d-8f3d-f9e640e7a370 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.001646 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.002619 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.003117 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.003843 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.005027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:09.005552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.008084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.019838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51318f15-6bb9-428d-ab79-4876407caca8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.032789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 309/631] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:09.034151 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:09.082121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.082720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.083317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.083610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.084444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:09.084759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.086618 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.106204 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 323/632] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:09.107367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:09.166353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553caad-d6bb-4170-9cdd-f4d217888a6d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.176400 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:09.185426 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:09.217732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.218703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.219150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.219619 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.220489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:09.220970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.222583 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.236418 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 310/633] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:09.237518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:09.249960 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93132911-f484-4b0e-8843-ea857d370703 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.296078 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7a7e1-82ac-4d77-a660-ce82993e358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.296727 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7a7e1-82ac-4d77-a660-ce82993e358c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.297010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7a7e1-82ac-4d77-a660-ce82993e358c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.297283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7a7e1-82ac-4d77-a660-ce82993e358c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.351245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a3757-f92d-4393-b2e3-eaf503350ae7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.368867 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7a7e1-82ac-4d77-a660-ce82993e358c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.370408 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7a7e1-82ac-4d77-a660-ce82993e358c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.387273 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9175e-f32c-43e7-841d-2b56833ec2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.387970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9175e-f32c-43e7-841d-2b56833ec2a9 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.388256 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9175e-f32c-43e7-841d-2b56833ec2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.388534 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9175e-f32c-43e7-841d-2b56833ec2a9 None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/7f85536ad53f474797f19311e1f19810/roles/5d22bf96fee74e9eaf2b9cdba5b26ead` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.405371 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9175e-f32c-43e7-841d-2b56833ec2a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=7f85536ad53f474797f19311e1f19810, role_id=5d22bf96fee74e9eaf2b9cdba5b26ead)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.406902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9175e-f32c-43e7-841d-2b56833ec2a9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.421948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 311/634] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:09 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/7f85536ad53f474797f19311e1f19810/roles/5d22bf96fee74e9eaf2b9cdba5b26ead => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:09.422198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:09.423303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9175e-f32c-43e7-841d-2b56833ec2a9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:09.469776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:09.509060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 324/635] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:09:09.510679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b033d-58c4-405a-8af5-ae1e1eb59b2e None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.511047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b033d-58c4-405a-8af5-ae1e1eb59b2e None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.511440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b033d-58c4-405a-8af5-ae1e1eb59b2e None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.518232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 325/636] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:09.519678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b033d-58c4-405a-8af5-ae1e1eb59b2e None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:09.576474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9175e-f32c-43e7-841d-2b56833ec2a9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.625130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f18328-58d8-4341-82bb-8ea0808ef3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.630147 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f18328-58d8-4341-82bb-8ea0808ef3ea None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.630147 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f18328-58d8-4341-82bb-8ea0808ef3ea None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.630147 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f18328-58d8-4341-82bb-8ea0808ef3ea None admin] PATH_INFO: `/v3/users/glance` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.637346 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f18328-58d8-4341-82bb-8ea0808ef3ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.638882 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f18328-58d8-4341-82bb-8ea0808ef3ea None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.650130 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b033d-58c4-405a-8af5-ae1e1eb59b2e None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-b3f18328-58d8-4341-82bb-8ea0808ef3ea None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:09.652106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:09.653335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:09.654760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 312/637] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:09.655575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105983-3fcd-41d0-a7e0-8294dd8db052 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.655575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105983-3fcd-41d0-a7e0-8294dd8db052 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.655575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105983-3fcd-41d0-a7e0-8294dd8db052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.676006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-b5105983-3fcd-41d0-a7e0-8294dd8db052 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:09.677356 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-b5105983-3fcd-41d0-a7e0-8294dd8db052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:09.704837 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf6105-fa62-49b6-a7e2-951728c17c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.705536 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf6105-fa62-49b6-a7e2-951728c17c9d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.705910 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf6105-fa62-49b6-a7e2-951728c17c9d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.706130 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf6105-fa62-49b6-a7e2-951728c17c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.719759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf6105-fa62-49b6-a7e2-951728c17c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.721308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf6105-fa62-49b6-a7e2-951728c17c9d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.733007 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-80cf6105-fa62-49b6-a7e2-951728c17c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:09.740938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:09.742150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:09.743150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:09.743150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:09.743150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:09.743150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:09.750189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 326/638] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:09.751778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a84c58-96bb-4a61-aa5c-527675a7a0a5 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.752078 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a84c58-96bb-4a61-aa5c-527675a7a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.752347 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a84c58-96bb-4a61-aa5c-527675a7a0a5 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.754010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 327/639] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:09.754643 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a84c58-96bb-4a61-aa5c-527675a7a0a5 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:09.769564 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 313/640] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:09.772318 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105983-3fcd-41d0-a7e0-8294dd8db052 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:09.796650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:09.885996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a84c58-96bb-4a61-aa5c-527675a7a0a5 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.895548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105983-3fcd-41d0-a7e0-8294dd8db052 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:09.930984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.931645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.931930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.932215 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.933049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:09.933367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.934787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83d996-f5af-4f67-a322-b126043a8c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:09.934949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.935464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83d996-f5af-4f67-a322-b126043a8c65 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.935748 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83d996-f5af-4f67-a322-b126043a8c65 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.936022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83d996-f5af-4f67-a322-b126043a8c65 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.936741 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83d996-f5af-4f67-a322-b126043a8c65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:09.937044 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83d996-f5af-4f67-a322-b126043a8c65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:09.938467 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83d996-f5af-4f67-a322-b126043a8c65 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:09.956311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 314/641] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:09.956311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459225bc-9a6d-4c4f-b908-66f7a31441fd None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:09.956311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459225bc-9a6d-4c4f-b908-66f7a31441fd None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:09.956311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459225bc-9a6d-4c4f-b908-66f7a31441fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:09.960902 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 328/642] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:09:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:09.961944 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:09.974821 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-459225bc-9a6d-4c4f-b908-66f7a31441fd None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:09.976097 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-459225bc-9a6d-4c4f-b908-66f7a31441fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:10.032312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:10.061001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 315/643] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:10.062338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459225bc-9a6d-4c4f-b908-66f7a31441fd None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.067833 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:10.091817 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1480fa-9ae9-4d86-bc66-70101168e1e6 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.135914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b71650-a26d-4b38-b191-8fbbffe3a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.136631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b71650-a26d-4b38-b191-8fbbffe3a187 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.136923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b71650-a26d-4b38-b191-8fbbffe3a187 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.137155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b71650-a26d-4b38-b191-8fbbffe3a187 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.149209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b71650-a26d-4b38-b191-8fbbffe3a187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-45b71650-a26d-4b38-b191-8fbbffe3a187 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:10.154444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:10.155630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:10.156568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:10.158445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 329/644] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:10.159633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b71650-a26d-4b38-b191-8fbbffe3a187 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.196141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459225bc-9a6d-4c4f-b908-66f7a31441fd None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.245355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.246021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.246354 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.246626 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.254133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:10.255637 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:10.268747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:10.269824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 316/645] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:10.271016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.287403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b71650-a26d-4b38-b191-8fbbffe3a187 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.306599 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:10.326772 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c89e-0a83-4a0b-8a5f-b0d7bcc70b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.327767 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c89e-0a83-4a0b-8a5f-b0d7bcc70b7c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.328389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c89e-0a83-4a0b-8a5f-b0d7bcc70b7c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.328996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c89e-0a83-4a0b-8a5f-b0d7bcc70b7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.331987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:10.369599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2779bc-3ded-46f4-a472-e5387580f76f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.405232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c89e-0a83-4a0b-8a5f-b0d7bcc70b7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:10.406800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c89e-0a83-4a0b-8a5f-b0d7bcc70b7c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:10.407639 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcfc5e-7245-4adb-885e-b1bf85ccd2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.408302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcfc5e-7245-4adb-885e-b1bf85ccd2a3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.408696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcfc5e-7245-4adb-885e-b1bf85ccd2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.408979 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcfc5e-7245-4adb-885e-b1bf85ccd2a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.422917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcfc5e-7245-4adb-885e-b1bf85ccd2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-c6fcfc5e-7245-4adb-885e-b1bf85ccd2a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:10.427829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:10.428898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:10.429838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 317/646] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:10.431231 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcfc5e-7245-4adb-885e-b1bf85ccd2a3 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.515717 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 330/647] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:09:10.516892 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c89e-0a83-4a0b-8a5f-b0d7bcc70b7c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.535793 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:10.547657 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcfc5e-7245-4adb-885e-b1bf85ccd2a3 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.576256 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:10.590015 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.590971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.591301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.591524 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.592610 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:10.592610 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:10.594020 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:10.603814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 318/648] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:10.604523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.633030 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c89e-0a83-4a0b-8a5f-b0d7bcc70b7c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.670868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.671489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.671713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.671993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.672760 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:10.673083 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:10.674558 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:10.686712 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 331/649] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:10.692910 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.709654 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7aff3-eae7-40c0-860e-f6ecb7e4dadd None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.747263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.749013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.749307 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.750043 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:10.750389 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:10.751800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:10.760457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 319/650] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:10.761268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.776676 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:10.800904 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180599-7a45-4430-84aa-ade40911e50b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.820052 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:10.835936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.836600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.836893 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.837170 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.848617 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:10.850079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:10.865267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:10.866331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:10.867392 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:10.867392 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:10.867392 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:10.867392 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:10.871425 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 332/651] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:10.872398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.876999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e4854-2206-45cc-ad08-d7371c1a48eb None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:10.916077 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:10.916744 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:10.917032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:10.917334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:10.928970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:10.930401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:10.945388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:10.946542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:10.947563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:10.947563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:10.947563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:10.947563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:10.947563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 320/652] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:10.947563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:10.993592 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143258f9-11eb-46d1-8f5f-d7f8d9d7183a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.035500 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:11.043776 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.044426 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.044691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.045001 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.045816 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=1e579a84d5e34a828c5981f34b458a0d) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:11.046137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.047615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.054422 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 333/653] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/role_assignments?user.id=1e579a84d5e34a828c5981f34b458a0d&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:11.055070 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.081062 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:11.084599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9f228-bbcb-49c1-831e-21a89778a09c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.130421 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.131497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.132098 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.132562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.145398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.147290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:11.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:11.165361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:11.166398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:11.166398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:11.166398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:11.166398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:11.168822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 321/654] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:11.169946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.200258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113524c-bed3-4fd7-954a-14aad084d1fd None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.242417 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.243078 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.243370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.243652 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.244376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:11.244691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.246129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.260138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 334/655] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:11.261032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.288187 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a6d5-b28c-4e0a-b3d5-d917393ae338 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.334556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.335673 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:11.335826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.336136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.336514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.337323 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:11.337807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.339309 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.353654 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 322/656] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:11.354680 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.369956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:11.406161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e2e4-97df-46db-ae3f-6002a0fca459 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.462695 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.463368 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.463729 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.464080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.464871 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:11.465257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.467319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.479736 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525473d2-dc2c-41d7-b0fb-37e4bbd8ca68 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.488655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 335/657] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:11.488655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.520851 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.521512 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.521867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.522131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.529195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.530690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:11.542254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:11.543299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 323/658] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:11.544310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.599421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3601961-5a95-49e0-be23-b7de7336d19c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.613607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:11.620382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:11.635917 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dfecf-9e98-4c62-9f8f-1f7922ac62d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.636583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfecf-9e98-4c62-9f8f-1f7922ac62d6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.636938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfecf-9e98-4c62-9f8f-1f7922ac62d6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.637145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfecf-9e98-4c62-9f8f-1f7922ac62d6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.645238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dfecf-9e98-4c62-9f8f-1f7922ac62d6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.646728 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dfecf-9e98-4c62-9f8f-1f7922ac62d6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.653317 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afcadd-9d70-4938-bf42-e0970de3706d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-134dfecf-9e98-4c62-9f8f-1f7922ac62d6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:11.658398 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:11.659410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:11.660450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 336/659] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:11.661113 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e7b4-0552-49c5-b4f2-ace1705fa59a None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.661685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e7b4-0552-49c5-b4f2-ace1705fa59a None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.661685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e7b4-0552-49c5-b4f2-ace1705fa59a None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.666497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 337/660] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:11.666497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9e7b4-0552-49c5-b4f2-ace1705fa59a None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.697690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93714fda-dc30-4b8c-9227-22561c03ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.698377 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93714fda-dc30-4b8c-9227-22561c03ec23 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.698661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93714fda-dc30-4b8c-9227-22561c03ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.698937 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93714fda-dc30-4b8c-9227-22561c03ec23 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.707202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93714fda-dc30-4b8c-9227-22561c03ec23 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.708656 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93714fda-dc30-4b8c-9227-22561c03ec23 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-93714fda-dc30-4b8c-9227-22561c03ec23 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:11.720856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:11.721913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:11.723016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 324/661] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:11.723762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8810e-6cda-4cc3-9580-419f2c09852e None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.723762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8810e-6cda-4cc3-9580-419f2c09852e None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.723762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8810e-6cda-4cc3-9580-419f2c09852e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.747897 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-9ea8810e-6cda-4cc3-9580-419f2c09852e None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:11.749104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-9ea8810e-6cda-4cc3-9580-419f2c09852e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:11.782545 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9e7b4-0552-49c5-b4f2-ace1705fa59a None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.819614 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.820286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.820575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.820857 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.826323 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 325/662] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:11.826809 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:11.826809 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.826957 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8810e-6cda-4cc3-9580-419f2c09852e None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.827042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.836988 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 338/663] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:11.837846 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:11.875250 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:11.906972 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:11.948425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8810e-6cda-4cc3-9580-419f2c09852e None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.950876 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b430ef-ad62-4087-86c8-fad0ce4bc5a3 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:11.985612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.986184 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.986471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.986741 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.987538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:11.987657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:11.987864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.988287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:11.988576 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:11.988852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:11.989276 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:11.989658 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:11.989968 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:11.991498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:12.011729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 326/664] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:12.011729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:12.013342 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 339/665] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:09:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:12.014253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:12.110537 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:12.121121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cce5e-cdb6-4275-b6cd-ce31db951c55 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:12.122869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3a98f-29cb-4eb9-b516-150f71bc17d2 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:12.146435 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:12.158936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549b9af-319e-4d32-a2c3-d734b8e42878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:12.159593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549b9af-319e-4d32-a2c3-d734b8e42878 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:12.159879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549b9af-319e-4d32-a2c3-d734b8e42878 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:12.160152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549b9af-319e-4d32-a2c3-d734b8e42878 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:12.162392 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:12.163356 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:12.163643 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:12.163921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:12.164716 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=045e62a8ad8e4c4f916ea448f03cc438) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:12.165179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:12.166657 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:12.170756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549b9af-319e-4d32-a2c3-d734b8e42878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:12.182495 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 327/666] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:12 2022] GET /identity/v3/role_assignments?user.id=045e62a8ad8e4c4f916ea448f03cc438&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:12.182495 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-e549b9af-319e-4d32-a2c3-d734b8e42878 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:12.182828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:12.183812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 340/667] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:12.184765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549b9af-319e-4d32-a2c3-d734b8e42878 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:12.313765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68e496-454e-4fc9-a99c-7d3117d169b5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:12.324958 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549b9af-319e-4d32-a2c3-d734b8e42878 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:12.358593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d56ce3-da66-4755-a276-c3b75cd262e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:12.359253 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d56ce3-da66-4755-a276-c3b75cd262e9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:12.359535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d56ce3-da66-4755-a276-c3b75cd262e9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:12.360651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d56ce3-da66-4755-a276-c3b75cd262e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:12.372931 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a4177-248f-46a6-9b07-ca4a433a2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:12.373637 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4177-248f-46a6-9b07-ca4a433a2738 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:12.373929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4177-248f-46a6-9b07-ca4a433a2738 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:12.374236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4177-248f-46a6-9b07-ca4a433a2738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:12.406721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:12.445435 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:12.467947 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d56ce3-da66-4755-a276-c3b75cd262e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:12.471125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d56ce3-da66-4755-a276-c3b75cd262e9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:12.480674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a4177-248f-46a6-9b07-ca4a433a2738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:12.482264 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a4177-248f-46a6-9b07-ca4a433a2738 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:12.622737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 328/668] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:09:12.623500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d56ce3-da66-4755-a276-c3b75cd262e9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:12.641731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 341/669] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:09:12.646828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a4177-248f-46a6-9b07-ca4a433a2738 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:12.721721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:12.744896 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:12.767518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d56ce3-da66-4755-a276-c3b75cd262e9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:12.790483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a4177-248f-46a6-9b07-ca4a433a2738 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:12.809767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:12.810667 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:12.811214 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:12.811636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:12.813106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:12.813474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:12.815546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:12.825845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 329/670] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:12.826888 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:12.837102 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:12.837802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:12.838094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:12.838407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:12.850860 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:12.852485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:12.871181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:12.872297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:12.873352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:12.873352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:12.873352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:12.873352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:12.877788 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 342/671] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:12.879100 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:12.942829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387bc0b-dc5c-42dd-805c-c965612904fb None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:12.979884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:12.980585 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:12.980866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:12.981140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:12.993838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:12.995487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.000386 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075df44f-368a-49d2-8875-142115c9ece4 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.002915 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:13.004502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:13.012338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:13.013709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:13.015065 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:13.015065 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:13.015065 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:13.015065 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:13.020736 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 330/672] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:13.021856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.047359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:13.048712 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.049236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.049681 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.062664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:13.064505 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:13.081252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:13.082610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:13.083588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:13.083588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:13.083588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:13.083588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:13.085787 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 343/673] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:13.086996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.142022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386ea0f-659f-4543-8dc1-15aacf67a262 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.182856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:13.183675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.184161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.184594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.185541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:13.186039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:13.187591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.206196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 331/674] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:13.206196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.223512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbfb0f-e25b-42b3-8d2f-649b45e15ac6 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.262145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:13.262991 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.263286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.263566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.264297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:13.264609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:13.266049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.275804 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:13.282475 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 344/675] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:13.283479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.310549 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:13.334618 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36b10e-8097-478f-b183-5768313681f6 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.373572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b91427-6810-4c84-9a27-3941c7bd096c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:13.374544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b91427-6810-4c84-9a27-3941c7bd096c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.375185 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b91427-6810-4c84-9a27-3941c7bd096c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.377956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b91427-6810-4c84-9a27-3941c7bd096c None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.377956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b91427-6810-4c84-9a27-3941c7bd096c None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:13.377956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b91427-6810-4c84-9a27-3941c7bd096c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:13.378568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b91427-6810-4c84-9a27-3941c7bd096c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.395598 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 332/676] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:13.397018 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55f700-4c07-40ce-85fb-e09f1c6bea08 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.397357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55f700-4c07-40ce-85fb-e09f1c6bea08 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.397754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55f700-4c07-40ce-85fb-e09f1c6bea08 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.404014 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 333/677] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:13.404014 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55f700-4c07-40ce-85fb-e09f1c6bea08 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.430794 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eca500-1fc3-4fbe-bcf7-57b269c3c405 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.497285 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:13.498173 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.500723 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.500723 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.511159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:13.512529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:13.528420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:13.530473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:13.532443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:13.532443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:13.532443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:13.532443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:13.532443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:13.532443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:13.532443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:13.532443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:13.537134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 345/678] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:13.537134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.596176 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55f700-4c07-40ce-85fb-e09f1c6bea08 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.624104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:13.647448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066d22-7c7b-4978-aeef-f4d20af5ed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:13.648136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066d22-7c7b-4978-aeef-f4d20af5ed74 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.648450 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066d22-7c7b-4978-aeef-f4d20af5ed74 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.648711 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066d22-7c7b-4978-aeef-f4d20af5ed74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.656913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066d22-7c7b-4978-aeef-f4d20af5ed74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:13.658506 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066d22-7c7b-4978-aeef-f4d20af5ed74 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-d9066d22-7c7b-4978-aeef-f4d20af5ed74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:13.673523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:13.674677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 334/679] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb1dd-20af-412c-8bf5-d024b9b3e3d1 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.681537 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb1dd-20af-412c-8bf5-d024b9b3e3d1 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.682275 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb1dd-20af-412c-8bf5-d024b9b3e3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.689629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:13.700225 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-a99fb1dd-20af-412c-8bf5-d024b9b3e3d1 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:13.701557 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-a99fb1dd-20af-412c-8bf5-d024b9b3e3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:13.728482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2a0e7-cf7c-4d3e-9edd-8e25ea623208 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.778034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:13.778694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.778975 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.782194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] PATH_INFO: `/v3/users/swift` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.788019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:13.789739 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.802178 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 335/680] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:13.802178 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb1dd-20af-412c-8bf5-d024b9b3e3d1 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:13.816276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:13.818043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:13.819749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:13.820535 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 346/681] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:13.820742 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.952468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb1dd-20af-412c-8bf5-d024b9b3e3d1 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.957251 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e7f66-6f50-4145-93f8-d1e0f8aa757d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.966043 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:13.999597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.000339 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.000636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.000802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.000935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.001379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.001671 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.001781 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:14.001951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.002061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:14.002866 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:14.003203 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:14.003473 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:14.004734 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:14.008326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:14.028004 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 347/682] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:14.028298 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 336/683] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:14.028298 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:14.028611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:14.162492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2266ab-a80e-48fe-816e-c41c81a082b5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:14.188691 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb56b37-4eb3-46d0-b9e4-4300b5f4feec None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:14.206894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.207384 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.207720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.207905 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.210275 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:14.210275 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:14.210710 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:14.244545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 337/684] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:09:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:14.245869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:14.265714 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ef89d-6cb7-4f86-b41f-a09cc23dd68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.267009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef89d-6cb7-4f86-b41f-a09cc23dd68a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.267559 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef89d-6cb7-4f86-b41f-a09cc23dd68a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.267984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef89d-6cb7-4f86-b41f-a09cc23dd68a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.282835 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:14.289121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ef89d-6cb7-4f86-b41f-a09cc23dd68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-a63ef89d-6cb7-4f86-b41f-a09cc23dd68a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:14.298462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:14.300281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:14.301627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 348/685] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:14.302647 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ef89d-6cb7-4f86-b41f-a09cc23dd68a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:14.392254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:14.398663 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d258b-1375-4301-a27e-aa6a6201d755 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:14.450835 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ef89d-6cb7-4f86-b41f-a09cc23dd68a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:14.463131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.464565 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.465073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.465527 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.467102 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=b3b1a6c31cc64159847f7ca39efe8105) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:14.467658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:14.470436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:14.490144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 338/686] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:14 2022] GET /identity/v3/role_assignments?user.id=b3b1a6c31cc64159847f7ca39efe8105&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:14.490144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:14.503195 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.504124 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.504518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.504899 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.505851 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=d4dcfcbdf5df4c55a640573828c9e910) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:14.506621 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:14.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:14.526152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 349/687] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:14 2022] GET /identity/v3/role_assignments?user.id=d4dcfcbdf5df4c55a640573828c9e910&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:14.526152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:14.701779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:14.719945 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81ed07-e88e-4b39-8a31-1cae5e1a12e5 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:14.721173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370999d-1526-418d-a1f5-d6b269093cef None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:14.759702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:14.787355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290383d-7ab1-47a4-b9b0-eb692918645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.787355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290383d-7ab1-47a4-b9b0-eb692918645a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.787355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290383d-7ab1-47a4-b9b0-eb692918645a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.787355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290383d-7ab1-47a4-b9b0-eb692918645a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.788049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422e7bb-bb9f-4893-959e-949a22295dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.788049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422e7bb-bb9f-4893-959e-949a22295dbf None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.788049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422e7bb-bb9f-4893-959e-949a22295dbf None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.788049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422e7bb-bb9f-4893-959e-949a22295dbf None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.788719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422e7bb-bb9f-4893-959e-949a22295dbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:14.789412 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422e7bb-bb9f-4893-959e-949a22295dbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:14.792199 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422e7bb-bb9f-4893-959e-949a22295dbf None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:14.802309 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 350/688] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:14.805256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ac37f-3c18-45b9-9e8d-11a4c08e0702 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.805769 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ac37f-3c18-45b9-9e8d-11a4c08e0702 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.806384 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ac37f-3c18-45b9-9e8d-11a4c08e0702 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.809178 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 351/689] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:14.812223 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ac37f-3c18-45b9-9e8d-11a4c08e0702 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:14.918803 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290383d-7ab1-47a4-b9b0-eb692918645a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:14.921576 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290383d-7ab1-47a4-b9b0-eb692918645a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:15.040022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ac37f-3c18-45b9-9e8d-11a4c08e0702 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:15.095700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:15.115656 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 339/690] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:09:15.118162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d70c18-4052-4fc3-8b39-938e55c060ea None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:15.118621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d70c18-4052-4fc3-8b39-938e55c060ea None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:15.119095 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d70c18-4052-4fc3-8b39-938e55c060ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:15.120194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:15.121699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:15.122548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:15.123357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:15.147712 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:15.150453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-e7d70c18-4052-4fc3-8b39-938e55c060ea None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:15.151570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:15.152554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-e7d70c18-4052-4fc3-8b39-938e55c060ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:15.186421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:15.188791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:15.194161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:15.194161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:15.194161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:15.194161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:15.198606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 352/691] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:15.212598 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:15.291123 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:15.309833 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 340/692] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:15.312209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d70c18-4052-4fc3-8b39-938e55c060ea None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:15.445764 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbd181-c77d-4596-bd86-dad54cc5c8d5 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:15.494025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d70c18-4052-4fc3-8b39-938e55c060ea None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:15.515082 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:15.516220 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:15.516679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:15.517135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:15.531390 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:15.541606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:15.544567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:15.560363 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:15.561689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:15.562372 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:15.563349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:15.564378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:15.565062 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:15.567938 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:15.577148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:15.579241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:15.581117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:15.581117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:15.581117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:15.581117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:15.584764 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 353/693] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:15.586873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:15.598311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 341/694] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:15.599787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:15.757701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:15.765956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf6d32-4a2f-473c-93fb-222ad5d86018 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:15.788612 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bff6b-3606-4e39-9c57-f077c1a932d3 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:15.829762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:15.830721 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:15.831173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:15.831551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:15.833205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:15.833205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:15.834980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:15.859734 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 342/695] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:15.861489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:15.862010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:15.863529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:15.864397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:15.865103 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:15.887839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:15.891365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:15.923096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:15.925055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:15.927301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:15.927301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:15.927301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:15.927301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:15.931395 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 354/696] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:15.933293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:15.934083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:16.036042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90e0e9-c410-450b-b4b8-2ef932cb671c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:16.118150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:16.119370 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:16.119745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:16.120239 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] PATH_INFO: `/v3/users/nova` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:16.139303 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58b0b4-0989-4f35-a9fd-030971d91ae6 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:16.151407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:16.154211 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:16.187755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:16.189798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:16.191866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:16.201391 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 343/697] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:09:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:16.202687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:16.261541 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:16.264027 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:16.264852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:16.265679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:16.280313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:16.283729 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:16.287469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:16.309864 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:16.312012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:16.314092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:16.314092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:16.314092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:16.314092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:16.314092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:16.314092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:16.314092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:16.314092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:16.319137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 355/698] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:16.321247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:16.494636 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:16.503708 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0cb48-4a98-41a7-9f89-b13ad4d9b139 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:16.579230 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:16.583207 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:16.584309 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:16.586278 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:16.587977 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:16.590535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:16.595076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6a847-97fa-45cc-a8ca-7983be29fbf0 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:16.595705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:16.625975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 344/699] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:16.628787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:16.700938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:16.704793 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:16.705360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:16.705879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:16.707505 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:16.708056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:16.713727 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:16.774542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 356/700] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:09:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:16.776966 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:16.800657 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f0454-f5d0-4d52-a77f-f8a9a9c3ba76 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:16.854895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3294037-cf48-43d2-aaf0-91237f7c27cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:16.855591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3294037-cf48-43d2-aaf0-91237f7c27cf None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:16.855877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3294037-cf48-43d2-aaf0-91237f7c27cf None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:16.856158 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3294037-cf48-43d2-aaf0-91237f7c27cf None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:16.857001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3294037-cf48-43d2-aaf0-91237f7c27cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:16.857342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3294037-cf48-43d2-aaf0-91237f7c27cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:16.859012 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3294037-cf48-43d2-aaf0-91237f7c27cf None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:16.877605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 345/701] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:16.877605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6e65-65f0-4af2-8c0b-9da118c2b27f None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:16.877605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6e65-65f0-4af2-8c0b-9da118c2b27f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:16.877605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6e65-65f0-4af2-8c0b-9da118c2b27f None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:16.877605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 346/702] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:16.877605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea6e65-65f0-4af2-8c0b-9da118c2b27f None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:16.903305 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:16.908331 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:16.981057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58bf08-914d-4f72-bd21-fd13eac36571 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:17.020681 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea6e65-65f0-4af2-8c0b-9da118c2b27f None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:17.022781 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9bf35-08ae-4c90-99cf-b69e9202b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:17.023559 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9bf35-08ae-4c90-99cf-b69e9202b56b None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:17.023894 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9bf35-08ae-4c90-99cf-b69e9202b56b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:17.024169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9bf35-08ae-4c90-99cf-b69e9202b56b None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:17.024977 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9bf35-08ae-4c90-99cf-b69e9202b56b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:17.026557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9bf35-08ae-4c90-99cf-b69e9202b56b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:17.062139 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 357/703] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:09:16 2022] POST /identity/v3/users => generated 254 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:17.062139 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9bf35-08ae-4c90-99cf-b69e9202b56b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:17.079662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c68b8b-10b5-479c-a5d1-20632ad3d700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:17.080332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c68b8b-10b5-479c-a5d1-20632ad3d700 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:17.080622 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c68b8b-10b5-479c-a5d1-20632ad3d700 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:17.080910 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c68b8b-10b5-479c-a5d1-20632ad3d700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:17.201993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c68b8b-10b5-479c-a5d1-20632ad3d700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:17.205830 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c68b8b-10b5-479c-a5d1-20632ad3d700 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:17.241195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:17.248527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9bf35-08ae-4c90-99cf-b69e9202b56b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:17.271745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:17.307943 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cf700-6f0e-42ca-b67c-b84c8c3c29f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:17.309117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cf700-6f0e-42ca-b67c-b84c8c3c29f6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:17.309618 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cf700-6f0e-42ca-b67c-b84c8c3c29f6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:17.310618 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cf700-6f0e-42ca-b67c-b84c8c3c29f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:17.312037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cf700-6f0e-42ca-b67c-b84c8c3c29f6 None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=7f85536ad53f474797f19311e1f19810) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:17.312850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cf700-6f0e-42ca-b67c-b84c8c3c29f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:17.314639 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cf700-6f0e-42ca-b67c-b84c8c3c29f6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:17.322889 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 358/704] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:17 2022] GET /identity/v3/role_assignments?user.id=7f85536ad53f474797f19311e1f19810&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:17.324241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53255d02-9759-44ed-9483-5cf87d65d3ad None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:17.324530 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53255d02-9759-44ed-9483-5cf87d65d3ad None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:17.324813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53255d02-9759-44ed-9483-5cf87d65d3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:17.350625 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-53255d02-9759-44ed-9483-5cf87d65d3ad None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:17.352005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-53255d02-9759-44ed-9483-5cf87d65d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:17.374369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 347/705] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:09:17.376513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d323-b2f3-4d09-ac9f-4e7848c6be0f None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:17.377146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d323-b2f3-4d09-ac9f-4e7848c6be0f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:17.377862 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d323-b2f3-4d09-ac9f-4e7848c6be0f None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:17.380948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 348/706] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:17.390950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0890b9-50ac-455a-900b-efb0c7ed4fba None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:17.391771 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0890b9-50ac-455a-900b-efb0c7ed4fba None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:17.392376 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0890b9-50ac-455a-900b-efb0c7ed4fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:17.436762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-fc0890b9-50ac-455a-900b-efb0c7ed4fba None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:17.439408 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-fc0890b9-50ac-455a-900b-efb0c7ed4fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:17.517037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 359/707] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:17.518879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53255d02-9759-44ed-9483-5cf87d65d3ad None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:17.584360 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 349/708] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:17.585797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0890b9-50ac-455a-900b-efb0c7ed4fba None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:17.680787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:17.698220 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53255d02-9759-44ed-9483-5cf87d65d3ad None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:17.708122 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:17.757722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:17.759222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:17.759980 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:17.760723 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:17.782482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:17.784604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0890b9-50ac-455a-900b-efb0c7ed4fba None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:17.786343 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:17.815318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:17.816802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:17.818719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:17.818719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:17.818719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:17.818719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:17.823064 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 360/709] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:17.825009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:17.846980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:17.847801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:17.848149 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:17.848529 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] PATH_INFO: `/v3/services/network` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:17.849727 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:17.853776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:17.873819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:17.875499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:17.876872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 350/710] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:17 2022] GET /identity/v3/services/network => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:17.882367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:18.002020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587769-8b11-49aa-a754-ea47bb37514b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:18.054608 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a0f9d-2944-484e-b349-2635673a5590 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:18.054905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:18.055695 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:18.056193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:18.056675 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:18.057607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:18.059094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:18.074523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:18.076225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:18.077677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:18.082399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 361/711] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 09:09:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:18.083487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:18.088778 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:18.120570 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:18.122270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:18.123192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:18.124026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:18.125717 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:18.126142 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:18.127043 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:18.130723 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:18.151638 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 351/712] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:18.153443 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:18.266234 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698acd0-94ef-4018-8284-830eda80b976 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:18.321241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:18.321981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:18.322347 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:18.322519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:18.323156 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] RBAC: Adding query filter params (name=network) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:18.323461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:18.325079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:18.336505 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 362/713] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:18.337404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1586a0-f191-4729-ba1e-d6e7020ebf2d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:18.337919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:18.396788 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:18.398520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:18.399490 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:18.400385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:18.402249 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:18.403204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:18.407324 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:18.419165 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 352/714] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:09:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:18.421258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:18.474867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:18.489217 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:18.520424 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb29456-4ae2-44e3-9d6d-22d9d1ed7833 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:18.548859 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1d6a-d2c0-447e-81e2-95da775ff53e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:18.573873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:18.574766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:18.575109 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:18.575436 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:18.586227 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:18.588518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:18.605822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:18.607101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:18.608129 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 363/715] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:18.609268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:18.616019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:18.617315 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:18.617913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:18.618512 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:18.619560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] RBAC: Adding query filter params (type=network) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:18.620104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:18.621959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:18.630053 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 353/716] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:18.631446 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:18.881823 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9255f-fa29-4276-ab29-72601b2d0a44 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:18.937456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e883d3c-6725-4d7b-91d8-1577de44c251 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:18.939901 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:18.951359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:18.958152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:18.958152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:18.958152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:18.958152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:18.958152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:18.958152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:18.964053 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 364/717] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:09:18 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:19.031874 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a643f8f-23e6-4e71-8c18-64a15ed8e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:19.032628 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a643f8f-23e6-4e71-8c18-64a15ed8e7e6 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:19.032913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a643f8f-23e6-4e71-8c18-64a15ed8e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:19.033194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a643f8f-23e6-4e71-8c18-64a15ed8e7e6 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:19.034040 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a643f8f-23e6-4e71-8c18-64a15ed8e7e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:19.034397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a643f8f-23e6-4e71-8c18-64a15ed8e7e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:19.036060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a643f8f-23e6-4e71-8c18-64a15ed8e7e6 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:19.060922 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 354/718] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:19.067687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:19.091191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:19.192664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4db7ff-9661-45f1-8be8-cabddfaf5d1f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:19.233684 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b02911-a2bb-4c84-b773-88fced4729b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:19.234381 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b02911-a2bb-4c84-b773-88fced4729b6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:19.234707 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b02911-a2bb-4c84-b773-88fced4729b6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:19.234988 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b02911-a2bb-4c84-b773-88fced4729b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:19.235752 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b02911-a2bb-4c84-b773-88fced4729b6 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=1e579a84d5e34a828c5981f34b458a0d) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:19.236068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b02911-a2bb-4c84-b773-88fced4729b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:19.237574 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b02911-a2bb-4c84-b773-88fced4729b6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:19.244435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 365/719] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:19 2022] GET /identity/v3/role_assignments?user.id=1e579a84d5e34a828c5981f34b458a0d&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:19.557654 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdb0e1-e54d-40b8-bdfb-07754e2e7131 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:19.558500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdb0e1-e54d-40b8-bdfb-07754e2e7131 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:19.558500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdb0e1-e54d-40b8-bdfb-07754e2e7131 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:19.559814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 355/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:19.565759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e30b0-3fff-4fb0-ae54-99dc47643f72 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:19.566552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e30b0-3fff-4fb0-ae54-99dc47643f72 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:19.566552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e30b0-3fff-4fb0-ae54-99dc47643f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:19.587081 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-871e30b0-3fff-4fb0-ae54-99dc47643f72 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:19.588317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-871e30b0-3fff-4fb0-ae54-99dc47643f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:19.688761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 366/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:09:19.720740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdb0e1-e54d-40b8-bdfb-07754e2e7131 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:19.871005 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdb0e1-e54d-40b8-bdfb-07754e2e7131 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:20.059085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b82354-30ce-4eb8-8603-1c6b84d34ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:20.059828 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b82354-30ce-4eb8-8603-1c6b84d34ca8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:20.060116 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b82354-30ce-4eb8-8603-1c6b84d34ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:20.060389 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b82354-30ce-4eb8-8603-1c6b84d34ca8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:20.076754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b82354-30ce-4eb8-8603-1c6b84d34ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-36b82354-30ce-4eb8-8603-1c6b84d34ca8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:20.082284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:20.083425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:20.084352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 356/722] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:20.090847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e30b0-3fff-4fb0-ae54-99dc47643f72 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:20.100275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:20.367449 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e30b0-3fff-4fb0-ae54-99dc47643f72 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:20.447384 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11803ee2-8e57-47d6-9bf4-0c7c2526f648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:20.448455 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11803ee2-8e57-47d6-9bf4-0c7c2526f648 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:20.448949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11803ee2-8e57-47d6-9bf4-0c7c2526f648 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:20.449434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11803ee2-8e57-47d6-9bf4-0c7c2526f648 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:20.450782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11803ee2-8e57-47d6-9bf4-0c7c2526f648 None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:20.451311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11803ee2-8e57-47d6-9bf4-0c7c2526f648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:20.453823 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11803ee2-8e57-47d6-9bf4-0c7c2526f648 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:20.478606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 367/723] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:20.483647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b82354-30ce-4eb8-8603-1c6b84d34ca8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:20.489443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ccdf2-08d8-4778-be1c-2440e3446096 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:20.490062 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ccdf2-08d8-4778-be1c-2440e3446096 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:20.490605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ccdf2-08d8-4778-be1c-2440e3446096 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:20.492664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 368/724] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:20.499342 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e219e2f-48fb-4a80-bdd6-3241108e98d2 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:20.499861 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e219e2f-48fb-4a80-bdd6-3241108e98d2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:20.500362 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e219e2f-48fb-4a80-bdd6-3241108e98d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:20.536931 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-5e219e2f-48fb-4a80-bdd6-3241108e98d2 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:20.539833 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-5e219e2f-48fb-4a80-bdd6-3241108e98d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:20.543941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:20.674942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b82354-30ce-4eb8-8603-1c6b84d34ca8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:20.705699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:20.720258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:20.721406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:20.721906 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:20.722475 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:20.732090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 369/725] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:20.743309 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:20.746084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:20.772720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:20.774403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:20.775417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:20.775417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:20.775417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:20.775417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:20.778612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 357/726] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:20.780647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:20.781150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e219e2f-48fb-4a80-bdd6-3241108e98d2 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:20.905191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:20.979308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80454160-f67d-4092-ac56-3d00666f292c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:21.089391 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0581368-f913-4c5f-972b-dcc738123ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:21.089973 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581368-f913-4c5f-972b-dcc738123ab1 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.090358 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581368-f913-4c5f-972b-dcc738123ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.090694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581368-f913-4c5f-972b-dcc738123ab1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.109481 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e219e2f-48fb-4a80-bdd6-3241108e98d2 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:21.111307 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0581368-f913-4c5f-972b-dcc738123ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a0581368-f913-4c5f-972b-dcc738123ab1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:21.122607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:21.125314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:21.126551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:21.129322 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 358/727] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:21.136975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0581368-f913-4c5f-972b-dcc738123ab1 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:21.210932 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:21.211942 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.212529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.212992 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.213892 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:21.214397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:21.218654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:21.247446 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 370/728] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:21.256992 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:21.358169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0581368-f913-4c5f-972b-dcc738123ab1 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:21.370545 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:21.419946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72c931-7202-4bfc-9a28-726f1efbb1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:21.420795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c931-7202-4bfc-9a28-726f1efbb1b3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.421270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c931-7202-4bfc-9a28-726f1efbb1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.421658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c931-7202-4bfc-9a28-726f1efbb1b3 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.423417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72c931-7202-4bfc-9a28-726f1efbb1b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:21.423929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72c931-7202-4bfc-9a28-726f1efbb1b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:21.426180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72c931-7202-4bfc-9a28-726f1efbb1b3 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:21.429176 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:21.438480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a82442-45a9-4674-ac45-83ceab02fb91 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:21.460063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 359/729] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:21.462895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632fa4-b4ec-4a91-8717-1ed739195f83 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.466746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632fa4-b4ec-4a91-8717-1ed739195f83 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.467652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632fa4-b4ec-4a91-8717-1ed739195f83 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.471160 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 360/730] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:21.473089 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e632fa4-b4ec-4a91-8717-1ed739195f83 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:21.518457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc17e6-9016-42ef-8de5-286f33dbc4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:21.519321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc17e6-9016-42ef-8de5-286f33dbc4d0 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.519916 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc17e6-9016-42ef-8de5-286f33dbc4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.520391 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc17e6-9016-42ef-8de5-286f33dbc4d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.532846 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc17e6-9016-42ef-8de5-286f33dbc4d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:21.534624 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc17e6-9016-42ef-8de5-286f33dbc4d0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-55fc17e6-9016-42ef-8de5-286f33dbc4d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:21.552782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:21.555322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:21.557983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 371/731] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:09:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:21.560347 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07670-7ab7-4305-b00a-93bf9ca0ba34 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.561163 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07670-7ab7-4305-b00a-93bf9ca0ba34 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.561713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07670-7ab7-4305-b00a-93bf9ca0ba34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.584892 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-6ca07670-7ab7-4305-b00a-93bf9ca0ba34 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:21.586429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-6ca07670-7ab7-4305-b00a-93bf9ca0ba34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:21.641239 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e632fa4-b4ec-4a91-8717-1ed739195f83 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:21.682333 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:21.683949 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.684504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.684993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.686532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 372/732] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:21.687892 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca07670-7ab7-4305-b00a-93bf9ca0ba34 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:21.698795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:21.700653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:21.717293 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:21.718375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:21.719398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:21.719398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:21.719398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:21.719398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:21.721918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 361/733] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:21.735094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:21.763562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:21.804604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:21.829780 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca07670-7ab7-4305-b00a-93bf9ca0ba34 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:21.871971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:21.872671 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.872979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.873264 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.874162 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:21.874562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:21.876063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:21.891378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce26d2a-a573-4c5d-b57e-af3f9d6dac01 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:21.899209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 373/734] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:09:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:21.900564 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:21.951673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53eb4b1-00cf-4648-9332-6dfa9f6d1334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:21.953460 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53eb4b1-00cf-4648-9332-6dfa9f6d1334 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:21.954294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53eb4b1-00cf-4648-9332-6dfa9f6d1334 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:21.955107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53eb4b1-00cf-4648-9332-6dfa9f6d1334 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:21.956757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53eb4b1-00cf-4648-9332-6dfa9f6d1334 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:21.959832 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53eb4b1-00cf-4648-9332-6dfa9f6d1334 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:21.994060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 362/735] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:09:21 2022] POST /identity/v3/services => generated 240 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:09:21.996582 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53eb4b1-00cf-4648-9332-6dfa9f6d1334 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:22.036715 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cd49c-b977-4700-8f04-acfe30e1aba5 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:22.042583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:22.116612 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480fdec5-6ab8-4542-a01f-2407a22d7306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:22.117287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fdec5-6ab8-4542-a01f-2407a22d7306 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.117600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fdec5-6ab8-4542-a01f-2407a22d7306 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.118021 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fdec5-6ab8-4542-a01f-2407a22d7306 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.118688 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fdec5-6ab8-4542-a01f-2407a22d7306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:22.119025 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fdec5-6ab8-4542-a01f-2407a22d7306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:22.120626 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fdec5-6ab8-4542-a01f-2407a22d7306 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:22.136838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 374/736] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:22.138355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894535-86e4-480f-90ad-5f61eb23fa1e None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.138662 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894535-86e4-480f-90ad-5f61eb23fa1e None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.138958 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894535-86e4-480f-90ad-5f61eb23fa1e None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.140635 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 375/737] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:22.144439 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00894535-86e4-480f-90ad-5f61eb23fa1e None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:22.199980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53eb4b1-00cf-4648-9332-6dfa9f6d1334 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:22.216110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:22.257189 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18d58d-34de-434e-b377-279bfb92e6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:22.257488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18d58d-34de-434e-b377-279bfb92e6db None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.257671 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18d58d-34de-434e-b377-279bfb92e6db None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.258006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18d58d-34de-434e-b377-279bfb92e6db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.292868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00894535-86e4-480f-90ad-5f61eb23fa1e None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:22.339635 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af6fb5-6aa9-4a33-b495-1e11061354b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:22.340312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af6fb5-6aa9-4a33-b495-1e11061354b7 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.340594 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af6fb5-6aa9-4a33-b495-1e11061354b7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.340865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af6fb5-6aa9-4a33-b495-1e11061354b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.353742 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af6fb5-6aa9-4a33-b495-1e11061354b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:22.355605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af6fb5-6aa9-4a33-b495-1e11061354b7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:22.365756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18d58d-34de-434e-b377-279bfb92e6db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:22.367516 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18d58d-34de-434e-b377-279bfb92e6db None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-a1af6fb5-6aa9-4a33-b495-1e11061354b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:22.369740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:22.371272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 376/738] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf9b7-c5c6-4085-98b5-04c77d0ad9ce None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.372986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf9b7-c5c6-4085-98b5-04c77d0ad9ce None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.374952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf9b7-c5c6-4085-98b5-04c77d0ad9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.394055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-1a8cf9b7-c5c6-4085-98b5-04c77d0ad9ce None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:22.395451 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-1a8cf9b7-c5c6-4085-98b5-04c77d0ad9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:22.418746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:22.506464 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 377/739] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:22 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:22.507870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067011b-231b-47e7-a832-401a8fc75901 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.508166 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067011b-231b-47e7-a832-401a8fc75901 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.508447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067011b-231b-47e7-a832-401a8fc75901 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.510175 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 378/740] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:22.511109 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067011b-231b-47e7-a832-401a8fc75901 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:22.530683 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 363/741] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:21 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:09:22.531901 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af450096-bcba-45a6-9d0b-533872dbc50f None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.532130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af450096-bcba-45a6-9d0b-533872dbc50f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.532408 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af450096-bcba-45a6-9d0b-533872dbc50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.552275 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-af450096-bcba-45a6-9d0b-533872dbc50f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:22.553943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-af450096-bcba-45a6-9d0b-533872dbc50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:22.557406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:22.650048 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 364/742] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:22 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:09:22.650902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af450096-bcba-45a6-9d0b-533872dbc50f None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:22.671672 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067011b-231b-47e7-a832-401a8fc75901 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:22.739254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:22.740036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.740209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.740515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.741907 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:22.741907 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:22.743481 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:22.826594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af450096-bcba-45a6-9d0b-533872dbc50f None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:22.873073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bc651-65fe-415b-a29f-145ae04c0ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:22.873756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bc651-65fe-415b-a29f-145ae04c0ffa None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:22.874042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bc651-65fe-415b-a29f-145ae04c0ffa None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:22.874356 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bc651-65fe-415b-a29f-145ae04c0ffa None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:22.875133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bc651-65fe-415b-a29f-145ae04c0ffa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:22.876950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bc651-65fe-415b-a29f-145ae04c0ffa None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:22.891731 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:22.905691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 365/743] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:09:22 2022] POST /identity/v3/services => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:22.907361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bc651-65fe-415b-a29f-145ae04c0ffa None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:22.922137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 379/744] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:09:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:22.922137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:22.937722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:23.088063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9f0f0-44c8-43f7-976b-e86727b8d3ea None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:23.124222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bc651-65fe-415b-a29f-145ae04c0ffa None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:23.170234 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d79fe8-c172-485f-aeaf-01a94066c642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:23.171735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79fe8-c172-485f-aeaf-01a94066c642 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:23.172944 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79fe8-c172-485f-aeaf-01a94066c642 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:23.174859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79fe8-c172-485f-aeaf-01a94066c642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:23.181939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:23.182454 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:23.183678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:23.183678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:23.195743 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d79fe8-c172-485f-aeaf-01a94066c642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-38d79fe8-c172-485f-aeaf-01a94066c642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:23.200845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:23.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 380/745] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:23.204708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d79fe8-c172-485f-aeaf-01a94066c642 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:23.205775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:23.206877 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:23.238202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:23.239266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:23.240262 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:23.240262 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:23.240262 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:23.240262 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:23.240513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 366/746] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:23.246263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:23.318507 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:23.320014 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:23.364762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d79fe8-c172-485f-aeaf-01a94066c642 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:23.409144 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e245dce-c6ce-49d3-8cbd-ecc6f45a27fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:23.409798 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e245dce-c6ce-49d3-8cbd-ecc6f45a27fb None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:23.410085 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e245dce-c6ce-49d3-8cbd-ecc6f45a27fb None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:23.410390 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e245dce-c6ce-49d3-8cbd-ecc6f45a27fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:23.411674 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4c348-2d96-4d06-bb4c-234562fabec3 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:23.484536 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:23.486148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:23.487153 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:23.487982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:23.489786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:23.490902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:23.494032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:23.494377 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e245dce-c6ce-49d3-8cbd-ecc6f45a27fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:23.495873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e245dce-c6ce-49d3-8cbd-ecc6f45a27fb None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:23.513627 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 367/747] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:23.516022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:23.574958 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:23.637889 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 381/748] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:09:23.639892 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e245dce-c6ce-49d3-8cbd-ecc6f45a27fb None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:23.686045 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd8283-ef65-4c0a-b6aa-bb8be468b73b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:23.700696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:23.735796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:23.736344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:23.736627 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:23.736900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:23.737623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:23.737929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:23.739381 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:23.760201 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 368/749] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:23.764562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:23.856696 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e245dce-c6ce-49d3-8cbd-ecc6f45a27fb None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:23.905391 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:23.907376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:23.907935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:23.908432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:23.924149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:23.926754 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:23.929107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd3f10-c782-4d62-b5eb-31d895c4dbd8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:23.951979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:23.953334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:23.954611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:23.954611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:23.954611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:23.954611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:23.957100 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 382/750] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:23.958738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:23.996475 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:23.998201 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:23.998686 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:23.999156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.003916 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:24.014079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.015922 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:24.032590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:24.033668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:24.034688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:24.034688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:24.034688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:24.034688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:24.042405 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 369/751] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:24.044139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:24.071395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:24.156814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345ccbb-d000-43ee-ac6d-f837b06897c9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:24.170061 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229866d3-00b0-446d-b00d-0f49a4eb29a7 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:24.195726 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:24.196727 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.197374 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.197873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.199321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:24.199864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.201802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.217863 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 370/752] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:24.219148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:24.242299 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:24.243700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.244280 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.244854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.259399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.262469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:24.286320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:24.287458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:24.288509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:24.288509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:24.288509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:24.288509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:24.288509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:24.288509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:24.288509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:24.288509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:24.295145 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 383/753] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:24.296773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:24.324338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:24.348747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fb528-73a3-4a0e-a0ef-7b7b1aa90c53 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:24.391187 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:24.392403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.392906 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.393819 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.394775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:24.395622 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.397347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.417446 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 371/754] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:24.417446 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:24.470393 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:24.494027 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db0e43-a71a-4827-b6ca-1ae82214a63e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:24.553804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:24.554399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.554694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.554976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.564226 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.565710 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:24.580949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:24.583054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 384/755] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:24.585262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:24.620316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74843f79-e0e1-4bfc-a78d-ccf08808f197 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:24.665447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:24.666415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.666720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.667241 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.667987 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:24.668306 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.670145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.687372 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:24.698152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 372/756] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:24.698152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:24.794728 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d411f7-5013-458a-9b59-6f44fd5f53e6 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:24.845726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:24.846641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.847111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.847600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.854223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3add5-410b-4bc9-bd4c-08434ce32d1c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:24.860834 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.863572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.869186 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:24.887542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:24.888964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:24.890256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:24.890256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:24.890256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:24.890256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:24.890256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:24.890256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:24.890256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:24.890256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:24.894296 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 385/757] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:24.896396 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:24.908336 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f111800-8041-4644-9837-14a4599d0537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:24.909254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f111800-8041-4644-9837-14a4599d0537 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.909563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f111800-8041-4644-9837-14a4599d0537 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.909909 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f111800-8041-4644-9837-14a4599d0537 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.911670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f111800-8041-4644-9837-14a4599d0537 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:24.911816 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f111800-8041-4644-9837-14a4599d0537 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.914504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f111800-8041-4644-9837-14a4599d0537 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.952083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 373/758] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:24.954609 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629f956-ba7c-41a4-8bb5-7ae841885d96 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.954935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629f956-ba7c-41a4-8bb5-7ae841885d96 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.955239 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629f956-ba7c-41a4-8bb5-7ae841885d96 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.957237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 374/759] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:24.958551 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629f956-ba7c-41a4-8bb5-7ae841885d96 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:25.036893 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dea69f-9911-4ecc-831c-1c5d866f2f13 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:25.051447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:25.084005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:25.084701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.085066 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.085352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.086335 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=41037802b8d9420f825cfda7247a52ad) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:25.086756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:25.088208 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:25.094661 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 386/760] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/role_assignments?user.id=41037802b8d9420f825cfda7247a52ad&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:25.099897 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:25.120444 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629f956-ba7c-41a4-8bb5-7ae841885d96 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:25.180789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8388e6-388f-4c7e-b6a2-ac892edd9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:25.182632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8388e6-388f-4c7e-b6a2-ac892edd9f2e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.183250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8388e6-388f-4c7e-b6a2-ac892edd9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.183752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8388e6-388f-4c7e-b6a2-ac892edd9f2e None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.185483 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8388e6-388f-4c7e-b6a2-ac892edd9f2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:25.186737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8388e6-388f-4c7e-b6a2-ac892edd9f2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:25.189316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8388e6-388f-4c7e-b6a2-ac892edd9f2e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:25.216010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:25.221821 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 375/761] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:25.224835 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d7a8f-d7f7-4382-9bd0-006ba915624d None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.226046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d7a8f-d7f7-4382-9bd0-006ba915624d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.226307 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d7a8f-d7f7-4382-9bd0-006ba915624d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.260272 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-1d9d7a8f-d7f7-4382-9bd0-006ba915624d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:25.262344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-1d9d7a8f-d7f7-4382-9bd0-006ba915624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:25.282680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c78ebe-2ce2-4028-a0bd-ac5497193b3e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:25.340298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:25.341414 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.342010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.342329 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.358197 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:25.360573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:25.376706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 376/762] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:25.378162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d7a8f-d7f7-4382-9bd0-006ba915624d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:25.380750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:25.382363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:25.384028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:25.384028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:25.384028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:25.384028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:25.386581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 387/763] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:25.390696 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:25.509896 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:25.543551 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d944d-f0a8-4a6f-829c-c5b750ffb0ed None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:25.563279 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:25.588609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:25.589250 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.589571 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.590069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.590734 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:25.591050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:25.592466 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:25.608843 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 388/764] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:25.609526 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:25.621502 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d7a8f-d7f7-4382-9bd0-006ba915624d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:25.669407 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea934e6-da15-442e-be74-6776b827ab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:25.670149 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea934e6-da15-442e-be74-6776b827ab07 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.670485 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea934e6-da15-442e-be74-6776b827ab07 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.670798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea934e6-da15-442e-be74-6776b827ab07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.671584 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea934e6-da15-442e-be74-6776b827ab07 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=7f85536ad53f474797f19311e1f19810) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:25.671925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea934e6-da15-442e-be74-6776b827ab07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:25.673571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea934e6-da15-442e-be74-6776b827ab07 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:25.683329 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 377/765] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:25 2022] GET /identity/v3/role_assignments?user.id=7f85536ad53f474797f19311e1f19810&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:25.683853 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf283a4-5223-41e1-9afd-0e2faf02c58d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.685135 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf283a4-5223-41e1-9afd-0e2faf02c58d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.685135 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf283a4-5223-41e1-9afd-0e2faf02c58d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.692332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 378/766] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:25.692332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf283a4-5223-41e1-9afd-0e2faf02c58d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:25.777697 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432783aa-d705-454c-9b43-9addafe7d9c9 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:25.848998 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9678f-76a8-4116-8209-d5e8b6f7f99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:25.850579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9678f-76a8-4116-8209-d5e8b6f7f99f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.851277 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9678f-76a8-4116-8209-d5e8b6f7f99f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.852114 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9678f-76a8-4116-8209-d5e8b6f7f99f None admin] PATH_INFO: `/v3/services/network` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.854068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9678f-76a8-4116-8209-d5e8b6f7f99f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:25.857868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9678f-76a8-4116-8209-d5e8b6f7f99f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:25.876081 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf283a4-5223-41e1-9afd-0e2faf02c58d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-64e9678f-76a8-4116-8209-d5e8b6f7f99f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:25.877410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:25.878615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:25.879702 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 389/767] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:25 2022] GET /identity/v3/services/network => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:25.881697 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f0da5-2844-4e74-b4dd-785401d5e647 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.886273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f0da5-2844-4e74-b4dd-785401d5e647 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.886273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f0da5-2844-4e74-b4dd-785401d5e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.900532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:25.911097 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-604f0da5-2844-4e74-b4dd-785401d5e647 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:25.912732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-604f0da5-2844-4e74-b4dd-785401d5e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:25.928398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:25.929395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:25.929879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:25.930366 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:25.939761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:25.941870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:25.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:25.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:25.959623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:25.959623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:25.959623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:25.959623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:25.959623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:25.959623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:25.959623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:25.959623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:25.962039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 379/768] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:25.963244 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:25.978054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:26.026453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 390/769] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:26.027739 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f0da5-2844-4e74-b4dd-785401d5e647 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.131578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53050a91-60db-4c78-b20b-8c440c5d5088 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.163451 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f0da5-2844-4e74-b4dd-785401d5e647 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.182788 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.184498 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.185507 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.185822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.186601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] RBAC: Adding query filter params (name=network) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:26.186936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.188680 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.195476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 380/770] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:26.196676 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.214880 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.215855 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.216324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.216766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.218463 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:26.218969 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.220636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.235209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 391/771] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:26.236536 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.266889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:26.273726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:26.355544 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581ea8e-e25c-4b13-a313-ebbd80959d01 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.356653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b81b3e-9535-4b62-82e4-6b79d2b66f9d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.399472 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.400036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.400542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.400680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.401308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] RBAC: Adding query filter params (type=network) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:26.401631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.403089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.406538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.407477 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.408045 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.408484 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.412094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 392/772] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:26.412094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.412813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.414933 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:26.426395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:26.427870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:26.429046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:26.438049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 381/773] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 09:09:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:26.439710 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.535856 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:26.543663 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aeb5c-37d6-4cf1-8df1-e1a2442ae95f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.585118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ef6c8-e30f-4f17-bbe8-2f37a8927ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.585118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef6c8-e30f-4f17-bbe8-2f37a8927ecf None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.585544 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef6c8-e30f-4f17-bbe8-2f37a8927ecf None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.585661 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef6c8-e30f-4f17-bbe8-2f37a8927ecf None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/1e579a84d5e34a828c5981f34b458a0d/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.587936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ef6c8-e30f-4f17-bbe8-2f37a8927ecf None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=1e579a84d5e34a828c5981f34b458a0d, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.589558 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ef6c8-e30f-4f17-bbe8-2f37a8927ecf None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.607237 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 393/774] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:26 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/1e579a84d5e34a828c5981f34b458a0d/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:26.614359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58166bee-a0e2-47ba-bbe4-9033c69a3ff9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.615411 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ef6c8-e30f-4f17-bbe8-2f37a8927ecf None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.617340 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:26.672140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.672866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.673182 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.673489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.674372 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] RBAC: Adding query filter params (interface=public, service_id=8bdfe4b76159443ab6a4cb1928a0b53d, region_id=RegionOne) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:26.674709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.676504 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.696478 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 382/775] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:09:26 2022] GET /identity/v3/endpoints?service_id=8bdfe4b76159443ab6a4cb1928a0b53d&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:26.702934 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.835385 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ef6c8-e30f-4f17-bbe8-2f37a8927ecf None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.898794 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a0dc-3426-4f8b-b92a-cc0934c34f94 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.907800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.908460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.908813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.909944 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:26.910375 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.911879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.918351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 394/776] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:09:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:26.927921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.954497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:26.955430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222c3bd-1de2-4ea8-9978-62ec6ceb55ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.956852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c3bd-1de2-4ea8-9978-62ec6ceb55ee None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.957457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c3bd-1de2-4ea8-9978-62ec6ceb55ee None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.957591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c3bd-1de2-4ea8-9978-62ec6ceb55ee None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.958424 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222c3bd-1de2-4ea8-9978-62ec6ceb55ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.960025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222c3bd-1de2-4ea8-9978-62ec6ceb55ee None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.968400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 383/777] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:09:26 2022] GET /identity/v3/services => generated 772 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:27.120568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae4693-9ef5-4d9d-8568-55d7dc747a14 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:27.170591 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:27.171553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:27.171553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:27.171860 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:27.172507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:27.172823 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:27.174265 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:27.181027 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 395/778] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:09:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:27.188027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222c3bd-1de2-4ea8-9978-62ec6ceb55ee None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:27.238925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:27.339758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222c3bd-1de2-4ea8-9978-62ec6ceb55ee None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:27.385669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb32bbe-3cdc-4426-83c6-c4f157f25bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:27.386445 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb32bbe-3cdc-4426-83c6-c4f157f25bce None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:27.386794 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb32bbe-3cdc-4426-83c6-c4f157f25bce None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:27.387104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb32bbe-3cdc-4426-83c6-c4f157f25bce None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:27.388016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb32bbe-3cdc-4426-83c6-c4f157f25bce None admin] RBAC: Adding query filter params (interface=public, service_id=bcc8ea93c08545ca888a7a01c7f6c4d9, region_id=RegionOne) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:27.388364 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb32bbe-3cdc-4426-83c6-c4f157f25bce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:27.389973 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb32bbe-3cdc-4426-83c6-c4f157f25bce None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:27.397392 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 384/779] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:09:27 2022] GET /identity/v3/endpoints?service_id=bcc8ea93c08545ca888a7a01c7f6c4d9&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:27.407317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:27.554923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:27.581838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb57df7-a94a-41be-9018-fd901ec50dbd None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:27.587954 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de31770-2507-4947-a44c-96475c921a6b None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:27.588242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de31770-2507-4947-a44c-96475c921a6b None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:27.588527 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de31770-2507-4947-a44c-96475c921a6b None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:27.590329 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 385/780] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:27.597255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57991985-3ad5-4c99-9d26-8f9e5ec12dcd None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:27.597590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57991985-3ad5-4c99-9d26-8f9e5ec12dcd None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:27.597812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57991985-3ad5-4c99-9d26-8f9e5ec12dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:27.620896 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-57991985-3ad5-4c99-9d26-8f9e5ec12dcd None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:27.622221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-57991985-3ad5-4c99-9d26-8f9e5ec12dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:27.648813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486b9d1-6b1c-43a6-a939-e3947e78a684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:27.649541 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b9d1-6b1c-43a6-a939-e3947e78a684 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:27.649837 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b9d1-6b1c-43a6-a939-e3947e78a684 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:27.650133 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b9d1-6b1c-43a6-a939-e3947e78a684 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:27.650832 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486b9d1-6b1c-43a6-a939-e3947e78a684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:27.652277 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486b9d1-6b1c-43a6-a939-e3947e78a684 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:27.666618 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 396/781] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:09:27 2022] GET /identity/v3/services => generated 772 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:27.769572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 386/782] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:27.827979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486b9d1-6b1c-43a6-a939-e3947e78a684 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:27.971990 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486b9d1-6b1c-43a6-a939-e3947e78a684 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:28.033894 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca02b8-192a-4d5f-956b-7aa22e9a753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:28.035236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca02b8-192a-4d5f-956b-7aa22e9a753c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:28.035750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca02b8-192a-4d5f-956b-7aa22e9a753c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:28.036275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca02b8-192a-4d5f-956b-7aa22e9a753c None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:28.056434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca02b8-192a-4d5f-956b-7aa22e9a753c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-97ca02b8-192a-4d5f-956b-7aa22e9a753c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:28.067981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:28.069423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:28.070778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 397/783] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:28.078315 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57991985-3ad5-4c99-9d26-8f9e5ec12dcd None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:28.125978 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:28.234630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a6ef1-89d9-4a9f-acd8-e9fe95ae1c91 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:28.235237 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a6ef1-89d9-4a9f-acd8-e9fe95ae1c91 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:28.235322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a6ef1-89d9-4a9f-acd8-e9fe95ae1c91 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:28.242240 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 398/784] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:28.251833 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4152-e32b-494c-a8c7-ff9c9a364bc4 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:28.252170 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4152-e32b-494c-a8c7-ff9c9a364bc4 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:28.252534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4152-e32b-494c-a8c7-ff9c9a364bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:28.260942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57991985-3ad5-4c99-9d26-8f9e5ec12dcd None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:28.296765 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:28.309810 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:28.310526 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:28.310855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:28.311150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:28.311953 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:28.312317 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:28.314832 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:28.319444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-90ca4152-e32b-494c-a8c7-ff9c9a364bc4 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:28.320751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-90ca4152-e32b-494c-a8c7-ff9c9a364bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:28.329416 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 387/785] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:28.339903 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:28.477573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 399/786] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:28.505133 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4152-e32b-494c-a8c7-ff9c9a364bc4 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:28.549998 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:28.590027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b6dad-760f-4b65-9b3d-400a0dded5fc None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:28.647599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:28.648601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:28.649030 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:28.649350 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:28.679822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:28.682016 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:28.719958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:28.721085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:28.722274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:28.722274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:28.722274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:28.722274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:28.722274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 388/787] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:28.730334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:28.761617 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4152-e32b-494c-a8c7-ff9c9a364bc4 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:28.823404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395008df-fe93-4e5c-be68-2da35096db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:28.824639 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395008df-fe93-4e5c-be68-2da35096db6b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:28.825198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395008df-fe93-4e5c-be68-2da35096db6b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:28.825741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395008df-fe93-4e5c-be68-2da35096db6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:28.835736 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:28.847501 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395008df-fe93-4e5c-be68-2da35096db6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-395008df-fe93-4e5c-be68-2da35096db6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:28.856874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:28.859502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:28.861792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 400/788] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:28.871318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395008df-fe93-4e5c-be68-2da35096db6b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:28.976131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497a22f-65c8-4a18-b990-79ca2b5d744b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:29.035542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:29.036419 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.036752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.037107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.038130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:29.038544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:29.040228 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:29.059788 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:29.084424 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 389/789] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:29.086917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:29.144520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395008df-fe93-4e5c-be68-2da35096db6b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:29.231532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:29.232359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.232648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.232956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.233687 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:29.233995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:29.242357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed7c6b-2562-4d5c-ad12-4a9ceba8bdc1 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:29.244019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:29.269644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 401/790] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:29.275729 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:29.297813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:29.298616 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.301031 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.301327 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.320373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:29.321885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:29.347737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:29.349091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:29.350733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:29.358888 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 390/791] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:09:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:29.364513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:29.400581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:29.487761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:29.505066 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7523af7-0849-4991-9d89-9c51dcec65f0 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:29.537287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455481ba-e9ac-42c7-abca-16728a74df4d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:29.573954 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:29.574638 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.574929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.575210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.576036 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:29.576649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:29.578217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:29.605056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 391/792] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:09:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:29.606458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:29.619790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297ae87-c651-41a9-8cbf-23028c2eb432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:29.620439 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297ae87-c651-41a9-8cbf-23028c2eb432 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.620757 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297ae87-c651-41a9-8cbf-23028c2eb432 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.621069 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297ae87-c651-41a9-8cbf-23028c2eb432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.659171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297ae87-c651-41a9-8cbf-23028c2eb432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:29.660585 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297ae87-c651-41a9-8cbf-23028c2eb432 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-e297ae87-c651-41a9-8cbf-23028c2eb432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:29.696572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:29.699134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:29.701462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:29.701462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:29.701462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:29.701462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:29.701462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 402/793] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:29.701462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7168ce-606d-4421-a9b6-bc5c560853f8 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.701462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7168ce-606d-4421-a9b6-bc5c560853f8 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.701462 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7168ce-606d-4421-a9b6-bc5c560853f8 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.707155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 403/794] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:29.708183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7168ce-606d-4421-a9b6-bc5c560853f8 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:29.728882 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7fd3-a810-497d-abd9-13189eddf0c7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:29.747234 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:29.769841 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6444cf-dce9-45b2-bfe1-1282522b35b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:29.770681 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6444cf-dce9-45b2-bfe1-1282522b35b5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.771036 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6444cf-dce9-45b2-bfe1-1282522b35b5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.771385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6444cf-dce9-45b2-bfe1-1282522b35b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.785087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6444cf-dce9-45b2-bfe1-1282522b35b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:29.787118 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6444cf-dce9-45b2-bfe1-1282522b35b5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-6f6444cf-dce9-45b2-bfe1-1282522b35b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:29.805501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:29.807649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:29.809939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:29.809939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:29.809939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:29.809939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:29.815964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 392/795] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:29.815964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc57ae3-b0d5-4e51-b384-9fe6104b9c7a None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.815964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc57ae3-b0d5-4e51-b384-9fe6104b9c7a None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.815964 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc57ae3-b0d5-4e51-b384-9fe6104b9c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.833496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-5fc57ae3-b0d5-4e51-b384-9fe6104b9c7a None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:29.834880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-5fc57ae3-b0d5-4e51-b384-9fe6104b9c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:29.846047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7168ce-606d-4421-a9b6-bc5c560853f8 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:29.858543 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:29.889331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccc280-95eb-4891-9e80-487e9903656e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:29.890509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc280-95eb-4891-9e80-487e9903656e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.890509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc280-95eb-4891-9e80-487e9903656e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.890935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc280-95eb-4891-9e80-487e9903656e None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.891899 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccc280-95eb-4891-9e80-487e9903656e None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:29.892753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccc280-95eb-4891-9e80-487e9903656e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:29.894743 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccc280-95eb-4891-9e80-487e9903656e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:29.912566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 404/796] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:29.914263 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ab5d-4767-49bc-99ff-9397c7d0d7d0 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:29.914682 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ab5d-4767-49bc-99ff-9397c7d0d7d0 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:29.914979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ab5d-4767-49bc-99ff-9397c7d0d7d0 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:29.916876 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 405/797] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:29.917873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ab5d-4767-49bc-99ff-9397c7d0d7d0 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:29.934175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 393/798] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:29.934175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc57ae3-b0d5-4e51-b384-9fe6104b9c7a None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.043908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:30.051689 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc57ae3-b0d5-4e51-b384-9fe6104b9c7a None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.095517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0ff18-1ec0-4d81-97c8-4156e9f5d4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.096385 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ab5d-4767-49bc-99ff-9397c7d0d7d0 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.096524 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0ff18-1ec0-4d81-97c8-4156e9f5d4db None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.096524 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0ff18-1ec0-4d81-97c8-4156e9f5d4db None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.096934 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0ff18-1ec0-4d81-97c8-4156e9f5d4db None admin] PATH_INFO: `/v3/users/swift` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.105821 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0ff18-1ec0-4d81-97c8-4156e9f5d4db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:30.107437 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0ff18-1ec0-4d81-97c8-4156e9f5d4db None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-7ea0ff18-1ec0-4d81-97c8-4156e9f5d4db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:30.120802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:30.121927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:30.123456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 394/799] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:30.124454 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77784f-49d3-45e8-bae8-2df9bb31db36 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.124454 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77784f-49d3-45e8-bae8-2df9bb31db36 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.124454 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77784f-49d3-45e8-bae8-2df9bb31db36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.143656 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-5e77784f-49d3-45e8-bae8-2df9bb31db36 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:30.145059 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-5e77784f-49d3-45e8-bae8-2df9bb31db36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:30.145598 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.146735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.147517 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.148157 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.149151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:30.149600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:30.151111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:30.170481 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 406/800] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:30.170481 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.208282 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:30.236236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 395/801] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:30.237787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77784f-49d3-45e8-bae8-2df9bb31db36 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.316670 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4624e1-1f2e-43e2-ab3d-176816d10b15 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.351536 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:30.354384 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034a14-0c28-4feb-9b7a-787ec485244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.355052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034a14-0c28-4feb-9b7a-787ec485244e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.355342 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034a14-0c28-4feb-9b7a-787ec485244e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.355611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034a14-0c28-4feb-9b7a-787ec485244e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.366426 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3034a14-0c28-4feb-9b7a-787ec485244e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:30.370418 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77784f-49d3-45e8-bae8-2df9bb31db36 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-a3034a14-0c28-4feb-9b7a-787ec485244e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:30.372890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:30.374188 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 407/802] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:30.375138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034a14-0c28-4feb-9b7a-787ec485244e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.413427 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.414150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.414499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.414825 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.415733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:30.416083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:30.418191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:30.444128 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 396/803] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:30.446363 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.487235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:30.498621 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034a14-0c28-4feb-9b7a-787ec485244e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.535042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50816fa-100d-48f5-8235-6f798ba44088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.535703 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50816fa-100d-48f5-8235-6f798ba44088 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.536008 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50816fa-100d-48f5-8235-6f798ba44088 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.536260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50816fa-100d-48f5-8235-6f798ba44088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.543326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50816fa-100d-48f5-8235-6f798ba44088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:30.544838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50816fa-100d-48f5-8235-6f798ba44088 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-d50816fa-100d-48f5-8235-6f798ba44088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:30.558700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:30.560138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 408/804] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:30.561461 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b5f8-4b7b-41f7-a865-29b5359f25fa None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.562338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b5f8-4b7b-41f7-a865-29b5359f25fa None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.562338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b5f8-4b7b-41f7-a865-29b5359f25fa None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.566287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 409/805] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:30.567862 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480b5f8-4b7b-41f7-a865-29b5359f25fa None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.570543 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd1d92-4d29-4119-a9f5-ff3f7a3a77a7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.614928 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.616198 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.616786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.617270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] PATH_INFO: `/v3/services/network` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.618387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:30.620319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:30.632250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:30.633545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:30.634531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 397/806] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:30.635942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.650601 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:30.700353 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480b5f8-4b7b-41f7-a865-29b5359f25fa None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.748376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.749074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.749361 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.749653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.750497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:30.750814 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:30.752341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:30.761814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325ba35-1414-416e-8a51-752ffa3b483e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.764167 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 410/807] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:30.765197 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.783491 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:30.814583 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdaeaa-d7f6-4f19-a62f-c6ad4ec38eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.815359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdaeaa-d7f6-4f19-a62f-c6ad4ec38eaa None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.815678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdaeaa-d7f6-4f19-a62f-c6ad4ec38eaa None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.816004 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdaeaa-d7f6-4f19-a62f-c6ad4ec38eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.831476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdaeaa-d7f6-4f19-a62f-c6ad4ec38eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:30.833001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdaeaa-d7f6-4f19-a62f-c6ad4ec38eaa None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-cafdaeaa-d7f6-4f19-a62f-c6ad4ec38eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:30.851047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:30.852288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:30.853410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:30.853410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:30.853410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:30.853410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:30.853410 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 398/808] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:30.854301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392799e-7737-492f-8315-b8519cc90bcb None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.854301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392799e-7737-492f-8315-b8519cc90bcb None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.854582 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392799e-7737-492f-8315-b8519cc90bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.874681 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-9392799e-7737-492f-8315-b8519cc90bcb None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:30.875963 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-9392799e-7737-492f-8315-b8519cc90bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:30.891334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bede5-f998-411c-9819-da0e5e031dae None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:30.928903 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:30.929572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:30.929858 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:30.930186 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:30.930966 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:30.931283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:30.932748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:30.946649 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 411/809] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:30.947576 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:30.955938 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:30.974013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 399/810] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:30.974945 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392799e-7737-492f-8315-b8519cc90bcb None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.059121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:31.094616 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6938cbe-c2fe-418f-aa72-ded91df8d44b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.114029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392799e-7737-492f-8315-b8519cc90bcb None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.134326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.135109 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.135423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.135726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.136488 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:31.136803 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.138292 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.143942 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 412/811] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:31.144865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.163751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.164752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.165246 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.165588 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.180972 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.182727 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:31.199715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:31.201523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:31.203441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:31.203441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:31.203441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:31.203441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:31.203441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 400/812] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:31.203441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.268894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:31.275067 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279b98c-0d38-48c0-b24b-8e344831c0e2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.316532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.317995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.318988 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.318988 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.320005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:31.320447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.322546 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.325703 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:31.339322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 413/813] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:31.340141 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.346332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7858a16-df2b-42a7-a922-2b3653fe0b15 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.388879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f28c9-3bb4-4d72-802f-25ae7bf6dbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.389687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f28c9-3bb4-4d72-802f-25ae7bf6dbb4 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.390018 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f28c9-3bb4-4d72-802f-25ae7bf6dbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.390369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f28c9-3bb4-4d72-802f-25ae7bf6dbb4 None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/41037802b8d9420f825cfda7247a52ad/roles/5d22bf96fee74e9eaf2b9cdba5b26ead` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.409187 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f28c9-3bb4-4d72-802f-25ae7bf6dbb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=41037802b8d9420f825cfda7247a52ad, role_id=5d22bf96fee74e9eaf2b9cdba5b26ead)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.411101 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f28c9-3bb4-4d72-802f-25ae7bf6dbb4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.426379 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 401/814] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:31 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/41037802b8d9420f825cfda7247a52ad/roles/5d22bf96fee74e9eaf2b9cdba5b26ead => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:31.427782 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f28c9-3bb4-4d72-802f-25ae7bf6dbb4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.494215 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abb47f-ef51-4437-ac88-a5890f5df455 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.544849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.545604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.545943 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.546340 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.547042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.548709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:31.570821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:31.571986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:31.572933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 414/815] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 09:09:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:31.573881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.590480 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:31.608162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f28c9-3bb4-4d72-802f-25ae7bf6dbb4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.659324 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.660586 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.661383 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.662008 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.663685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:31.664578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] RBAC: Adding query filter params (type=network) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:31.665234 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.668017 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.678422 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 402/816] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:31.683595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.726136 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d7325-2b29-4b91-8efb-91ef80a02aeb None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.793919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.794967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.795481 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.795995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.797290 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:31.797850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.799862 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.819467 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860b70b-7862-402c-a24a-f45ecd0f5695 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.825309 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 415/817] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:31.825309 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.881772 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.882713 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.882858 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.883092 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.892600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.894345 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:31.907611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:31.908757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 403/818] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:31.909880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.940534 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:31.983852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d730787-f35c-4d88-a75f-aa9046b44e28 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:32.021655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:32.044674 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a46b7-d72d-4ef1-9c0f-028901002dd8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:32.048342 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:32.049505 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:32.050015 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:32.050558 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:32.051845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:32.052402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:32.055255 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:32.061360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 416/819] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:09:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:32.062374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:32.091469 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b36d78-5a86-4c6c-849e-ba7bcee80028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:32.092300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b36d78-5a86-4c6c-849e-ba7bcee80028 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:32.092589 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b36d78-5a86-4c6c-849e-ba7bcee80028 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:32.092864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b36d78-5a86-4c6c-849e-ba7bcee80028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:32.093605 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b36d78-5a86-4c6c-849e-ba7bcee80028 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:32.095062 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b36d78-5a86-4c6c-849e-ba7bcee80028 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:32.137392 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 404/820] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:09:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:32.138634 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b36d78-5a86-4c6c-849e-ba7bcee80028 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:32.262535 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55545c-d0d0-4a10-9668-3b9d6ff87f2b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:32.265270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:32.323804 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b36d78-5a86-4c6c-849e-ba7bcee80028 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:32.338530 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:32.340788 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:32.341785 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:32.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] PATH_INFO: `/v3/users/nova` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:32.354951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:32.356505 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:32.372839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:32.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 417/821] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:09:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:32.375761 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:32.383101 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:32.383834 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:32.384202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:32.384571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:32.385494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:32.385862 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:32.387523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:32.406764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 405/822] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:32.406764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:32.464158 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:32.534083 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890542f-96b0-44a0-bb7a-67a0aa568195 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:32.580803 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:32.581501 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:32.581920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:32.582196 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:32.582909 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:32.583544 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:32.584783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:32.597211 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 418/823] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:09:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:32.597468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b8df2-494b-4e98-9d81-c86e75f13b5f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:32.597596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:32.633950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:32.646415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:32.647060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:32.647403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:32.647707 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:32.648564 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:32.648910 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:32.650372 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:32.677597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 406/824] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:09:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:32.678630 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:32.749887 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cc4ae-f840-4a1d-b5bb-6b8fc6b8edae None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:32.763678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:32.801127 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd1631-b2ab-49e5-9aae-ca84c44e2795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:32.802351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd1631-b2ab-49e5-9aae-ca84c44e2795 None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:32.802862 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd1631-b2ab-49e5-9aae-ca84c44e2795 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:32.803397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd1631-b2ab-49e5-9aae-ca84c44e2795 None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/7f85536ad53f474797f19311e1f19810/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:32.806962 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd1631-b2ab-49e5-9aae-ca84c44e2795 None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=7f85536ad53f474797f19311e1f19810, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:32.809667 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd1631-b2ab-49e5-9aae-ca84c44e2795 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:32.833647 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 419/825] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:32 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/7f85536ad53f474797f19311e1f19810/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:32.834676 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd1631-b2ab-49e5-9aae-ca84c44e2795 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:32.843905 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069d5be-f584-4e63-84f8-4c80f0e31914 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:32.917377 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49445-22b4-4413-90c2-432f9105cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:32.919136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49445-22b4-4413-90c2-432f9105cab5 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:32.919911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49445-22b4-4413-90c2-432f9105cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:32.921527 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49445-22b4-4413-90c2-432f9105cab5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:32.923722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f49445-22b4-4413-90c2-432f9105cab5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:32.928125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f49445-22b4-4413-90c2-432f9105cab5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:32.973853 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 407/826] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:09:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:32.994762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd1631-b2ab-49e5-9aae-ca84c44e2795 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:33.054231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa688-a589-4402-9ef3-84a2007c38c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:33.054899 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa688-a589-4402-9ef3-84a2007c38c8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.055185 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa688-a589-4402-9ef3-84a2007c38c8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.055519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa688-a589-4402-9ef3-84a2007c38c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.106125 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:33.166869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa688-a589-4402-9ef3-84a2007c38c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:33.168743 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa688-a589-4402-9ef3-84a2007c38c8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:33.303563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 420/827] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:09:33.309425 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49445-22b4-4413-90c2-432f9105cab5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:33.364660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:33.456184 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49445-22b4-4413-90c2-432f9105cab5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:33.495958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2700dea-eba0-4be3-8846-d74e97cd3cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:33.496626 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2700dea-eba0-4be3-8846-d74e97cd3cc3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.496906 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2700dea-eba0-4be3-8846-d74e97cd3cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.497172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2700dea-eba0-4be3-8846-d74e97cd3cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.508949 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2700dea-eba0-4be3-8846-d74e97cd3cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:33.510460 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2700dea-eba0-4be3-8846-d74e97cd3cc3 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-c2700dea-eba0-4be3-8846-d74e97cd3cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:33.527733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:33.529728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:33.531632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:33.531632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:33.531632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:33.531632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:33.531632 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 408/828] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:33.535562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa688-a589-4402-9ef3-84a2007c38c8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:33.565028 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44d34-13ab-4d41-8518-c2e92b002027 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.565499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44d34-13ab-4d41-8518-c2e92b002027 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.565686 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44d34-13ab-4d41-8518-c2e92b002027 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.567306 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 409/829] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:33.573081 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33257-6729-4f95-abe0-26249473d838 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.573476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33257-6729-4f95-abe0-26249473d838 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.573746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33257-6729-4f95-abe0-26249473d838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.595505 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-b3d33257-6729-4f95-abe0-26249473d838 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:33.596900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-b3d33257-6729-4f95-abe0-26249473d838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:33.604870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:33.654189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa688-a589-4402-9ef3-84a2007c38c8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:33.701978 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:33.702575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 410/830] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:33.702781 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.702982 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.703349 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.704191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:33.704507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:33.706132 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:33.719990 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 421/831] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:33.729600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d33257-6729-4f95-abe0-26249473d838 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:33.733589 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:33.786896 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:33.853020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f225733-f518-4c58-9d17-9b5bd833f811 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:33.879533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:33.889936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d33257-6729-4f95-abe0-26249473d838 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:33.895146 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d078a-9b81-4b57-ada2-ae66c51d4562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:33.896239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d078a-9b81-4b57-ada2-ae66c51d4562 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.896754 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d078a-9b81-4b57-ada2-ae66c51d4562 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.897315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d078a-9b81-4b57-ada2-ae66c51d4562 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.910661 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d078a-9b81-4b57-ada2-ae66c51d4562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-7b5d078a-9b81-4b57-ada2-ae66c51d4562 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:33.917111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:33.918276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:09:33.919238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:33.927089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 422/832] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:09:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:33.928419 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d078a-9b81-4b57-ada2-ae66c51d4562 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:33.932521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:33.934286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.934286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.934286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.942716 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:33.944623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:33.958099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:33.959218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:33.960319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:33.960319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:33.960319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:33.960319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:33.960319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:33.960319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:33.960319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:33.960319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:33.965015 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 411/833] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:33.966166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:34.052315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d078a-9b81-4b57-ada2-ae66c51d4562 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:34.068805 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:34.087563 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37328cec-2259-4d84-8130-bbb338fb3532 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:34.094236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:34.095490 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:34.096011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:34.096622 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:34.097593 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:34.098182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:34.099971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:34.114566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 423/834] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:09:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:34.116839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:34.130191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:34.130949 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:34.131255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:34.132107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:34.132653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:34.133022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:34.134742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:34.154319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 412/835] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:34.156282 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:34.177924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:34.248579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b04a0d-859b-4bb2-b3dc-e81cc2153f24 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:34.288292 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3da696-d492-496a-a79a-17501eb04321 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:34.290140 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:34.291239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:34.291821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:34.292328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:34.305878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:34.307809 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:34.330307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:34.332086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:34.334032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:34.334032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:34.334032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:34.334032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:34.341647 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 424/836] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:34.343473 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:34.345180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:34.346352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:34.346904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:34.347362 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:34.348920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=1e579a84d5e34a828c5981f34b458a0d) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:34.349455 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:34.352996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:34.370542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 413/837] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:34 2022] GET /identity/v3/role_assignments?user.id=1e579a84d5e34a828c5981f34b458a0d&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:34.467326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:34.473307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005c060-47f2-4144-9fb4-259618b7d881 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:34.514468 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:34.515563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:34.516080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:34.516623 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:34.517580 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:34.518151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:34.519811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:34.535015 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 425/838] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:34.541435 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:34.560599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:34.675756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49ad42-ad07-4472-94d5-3e1b72a1f082 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:34.717745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:34.718477 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:34.718839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:34.719217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:34.729970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:34.731586 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:34.745612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:34.746827 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:34.747890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 414/839] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:09:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:34.755163 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:34.868661 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca8dd3-d05f-4a79-981b-95ce112a13a0 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:34.909128 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e8892-4fde-44fc-b27b-6320b36e9242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:34.910224 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e8892-4fde-44fc-b27b-6320b36e9242 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:34.910760 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e8892-4fde-44fc-b27b-6320b36e9242 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:34.911260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e8892-4fde-44fc-b27b-6320b36e9242 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:34.912298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e8892-4fde-44fc-b27b-6320b36e9242 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:34.912896 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e8892-4fde-44fc-b27b-6320b36e9242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:34.914811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e8892-4fde-44fc-b27b-6320b36e9242 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:34.935273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 426/840] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:09:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:34.941774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:35.026278 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:35.077612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2b1af-d977-4040-8126-4a74462a495b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:35.116530 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13039e-c2d2-4874-b8f0-470104bd40cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:35.117616 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13039e-c2d2-4874-b8f0-470104bd40cd None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.118156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13039e-c2d2-4874-b8f0-470104bd40cd None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.118661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13039e-c2d2-4874-b8f0-470104bd40cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.160903 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1631-3afb-4a14-8523-51cc6513060b None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.161270 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1631-3afb-4a14-8523-51cc6513060b None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.161512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1631-3afb-4a14-8523-51cc6513060b None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.166252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 427/841] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:35.169851 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2307e-56d3-45f9-9926-982e3c921879 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.170097 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2307e-56d3-45f9-9926-982e3c921879 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.170411 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2307e-56d3-45f9-9926-982e3c921879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.194196 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-80d2307e-56d3-45f9-9926-982e3c921879 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:35.195852 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-80d2307e-56d3-45f9-9926-982e3c921879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:35.201647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13039e-c2d2-4874-b8f0-470104bd40cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:35.204026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13039e-c2d2-4874-b8f0-470104bd40cd None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:35.225901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:35.290524 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:35.290855 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 428/842] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:35.319142 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2307e-56d3-45f9-9926-982e3c921879 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:35.332089 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 415/843] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:09:35.338286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13039e-c2d2-4874-b8f0-470104bd40cd None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:35.457301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13039e-c2d2-4874-b8f0-470104bd40cd None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:35.485896 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2307e-56d3-45f9-9926-982e3c921879 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:35.507335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41949-64de-4fb6-8169-641450901290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:35.508042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41949-64de-4fb6-8169-641450901290 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.508472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41949-64de-4fb6-8169-641450901290 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.508759 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41949-64de-4fb6-8169-641450901290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.522503 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41949-64de-4fb6-8169-641450901290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:35.524125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41949-64de-4fb6-8169-641450901290 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:35.528977 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001704c9-2010-43a1-b271-ee88b4180342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:35.529764 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001704c9-2010-43a1-b271-ee88b4180342 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.530059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001704c9-2010-43a1-b271-ee88b4180342 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.530371 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001704c9-2010-43a1-b271-ee88b4180342 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a3f41949-64de-4fb6-8169-641450901290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:35.541357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:35.542517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:35.544009 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:35.544009 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:35.544009 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:35.544009 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:35.544009 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 416/844] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:35.544642 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffcdf9-5933-405d-8394-1f82bb3836ab None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.544642 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffcdf9-5933-405d-8394-1f82bb3836ab None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.544909 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffcdf9-5933-405d-8394-1f82bb3836ab None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.546487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 417/845] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:35.548573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001704c9-2010-43a1-b271-ee88b4180342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:35.551946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffcdf9-5933-405d-8394-1f82bb3836ab None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-001704c9-2010-43a1-b271-ee88b4180342 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:35.554315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:35.556374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:35.557686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 429/846] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:35.558913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497391-8fbb-4488-a8a7-5a12e2ad080d None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.558913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497391-8fbb-4488-a8a7-5a12e2ad080d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.558913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497391-8fbb-4488-a8a7-5a12e2ad080d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.574182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:35.581438 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-35497391-8fbb-4488-a8a7-5a12e2ad080d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:35.582879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-35497391-8fbb-4488-a8a7-5a12e2ad080d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:35.587034 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:35.686040 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffcdf9-5933-405d-8394-1f82bb3836ab None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:35.691171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 430/847] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:35.692812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35497391-8fbb-4488-a8a7-5a12e2ad080d None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:35.741910 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:35.742603 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.742890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.743162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.743889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:35.744291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:35.746192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:35.764602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 418/848] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:35.764602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:35.843748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35497391-8fbb-4488-a8a7-5a12e2ad080d None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:35.867038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:35.890049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:35.890653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f57325-42b1-49d4-9364-6e6258e68e72 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:35.891349 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.891700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.892221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.894009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:35.894009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:35.895425 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:35.899563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:35.915875 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 431/849] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:35.918439 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:35.939890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:35.940621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:35.940917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:35.941203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:35.941865 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:35.943658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:35.953823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:35.955880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:35.957492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:35.960352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 419/850] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:35.961626 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:36.067786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82109fe9-99a9-4c16-93b9-a672fbb3d148 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:36.089393 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d99a9-45f0-4526-b42c-68e29db8bb82 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:36.121405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:36.122495 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.123090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.123579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.131974 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:36.133486 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:36.140681 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:36.141769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.142472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.143026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:36.148189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:36.149164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:36.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:36.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:36.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:36.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:36.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:36.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:36.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:36.150190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:36.154492 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 432/851] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:36.155250 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:36.170173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:36.172935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:36.177702 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:36.196260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:36.197367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:36.199692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:36.199692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:36.199692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:36.199692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:36.203634 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 420/852] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:36.203634 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:36.242248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:36.305805 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc39848-4450-4d10-baaf-c80c92e5320b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:36.316041 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce14206-b7d5-47ac-92e5-428ed7cb791c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:36.357815 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:36.358849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.359368 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.359865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.361201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:36.361762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:36.363509 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:36.374074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 433/853] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:36.374074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:36.378614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:36.379313 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.379595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.379868 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.380649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:36.380956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:36.382700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:36.401057 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 421/854] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:36.402577 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:36.493939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:36.509933 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42c968-9fcb-481c-9421-c1941a193423 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:36.549216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:36.551103 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ccff3-2489-44d8-8a51-c94e6b9b069d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:36.575830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:36.577045 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.577652 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.578294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.579648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:36.580223 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:36.583062 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:36.590386 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:36.591040 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.591328 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.591603 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.592304 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:36.592612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:36.594017 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:36.610140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 422/855] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:36.610140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:36.610434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 434/856] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:36.614275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:36.780059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3991cc-beb1-4b4a-bfe2-09a903727c51 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:36.800290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c54358-5700-4972-ad04-7ae07a7b641c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:36.824336 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618a292-5a02-4840-9dc7-c79cab0819fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:36.825015 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618a292-5a02-4840-9dc7-c79cab0819fb None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.825382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618a292-5a02-4840-9dc7-c79cab0819fb None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.825566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618a292-5a02-4840-9dc7-c79cab0819fb None admin] PATH_INFO: `/v3/users/swift` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.828729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:36.835135 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618a292-5a02-4840-9dc7-c79cab0819fb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:36.836625 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618a292-5a02-4840-9dc7-c79cab0819fb None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:36.847826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:36.848473 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.848761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.849066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.850055 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=41037802b8d9420f825cfda7247a52ad) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:36.850656 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-d618a292-5a02-4840-9dc7-c79cab0819fb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:36.851750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:36.860010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:09:36.861766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:36.862771 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:36.862913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 435/857] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:09:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:36.862913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea1622-7d55-43aa-8026-3265e9041e5f None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.862913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea1622-7d55-43aa-8026-3265e9041e5f None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.862913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea1622-7d55-43aa-8026-3265e9041e5f None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.862913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 436/858] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:36.864333 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36ea6f-d979-477b-9ee1-40371b17c73f None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:36.864487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36ea6f-d979-477b-9ee1-40371b17c73f None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:36.866846 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36ea6f-d979-477b-9ee1-40371b17c73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:36.869523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 423/859] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:36 2022] GET /identity/v3/role_assignments?user.id=41037802b8d9420f825cfda7247a52ad&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:36.870311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:36.893184 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-fe36ea6f-d979-477b-9ee1-40371b17c73f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:36.895940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-fe36ea6f-d979-477b-9ee1-40371b17c73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:36.955023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:37.006732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47406f9-6857-44f8-ab52-a547a054b699 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:37.009664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 437/860] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:37.055906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36ea6f-d979-477b-9ee1-40371b17c73f None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:37.057075 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:37.057747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.058066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.058397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.059227 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:37.059539 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:37.061058 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:37.099201 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 424/861] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:09:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:37.111458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:37.201902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:37.259876 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36ea6f-d979-477b-9ee1-40371b17c73f None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:37.298956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b5952-7f8c-4244-97bc-6050bab262e5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:37.311931 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:37.313000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.313508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.314001 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.315417 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:37.318279 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:37.335232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:37.336600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:37.337800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:37.338782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 438/862] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 09:09:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:37.344766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:37.352728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92109d3b-3030-4320-bebf-30cf20bf3eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:37.353421 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109d3b-3030-4320-bebf-30cf20bf3eae None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.353730 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109d3b-3030-4320-bebf-30cf20bf3eae None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.354226 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109d3b-3030-4320-bebf-30cf20bf3eae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.396918 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:37.443941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92109d3b-3030-4320-bebf-30cf20bf3eae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:37.445725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92109d3b-3030-4320-bebf-30cf20bf3eae None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:37.498682 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:37.509819 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d8a33-4e69-4469-8ad3-73a1bc08b8bf None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:37.574389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:37.575683 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.576432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.577150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.578801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:37.579608 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:37.582533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:37.592189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 439/863] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:37.597700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 425/864] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:09:37.599142 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:37.606144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92109d3b-3030-4320-bebf-30cf20bf3eae None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:37.757998 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92109d3b-3030-4320-bebf-30cf20bf3eae None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:37.785905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:37.791366 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb868a86-a3e9-48da-9f6a-5f2568986ab1 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:37.806959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:37.807949 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.808946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.809443 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.827358 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:37.830894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:37.833662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:37.844357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:37.845657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.846353 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.847009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.848511 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:37.849190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:37.852108 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:37.853519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:37.854705 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:37.855852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:37.855852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:37.855852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:37.855852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:37.858241 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 426/865] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:37.867074 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:37.870601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 440/866] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:37 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:38.020010 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b3fe6-8f9b-4901-a901-62d289db5801 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:38.065621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:38.066312 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:38.066590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:38.066866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:38.067903 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:38.068216 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:38.069681 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:38.084224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 427/867] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:38.091562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:38.209061 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8d21-132e-4733-8a8a-f94c311018f8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:38.246884 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:38.247569 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:38.247900 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:38.248184 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:38.256152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:38.257741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:38.271644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:38.272821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:38.274510 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 441/868] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:38.279374 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:38.342335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:38.389889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8886d-785d-4b4f-a981-8dad74737be9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:38.434751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a955c-d3f2-4cef-ba33-be89784a12d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:38.435846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a955c-d3f2-4cef-ba33-be89784a12d5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:38.436359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a955c-d3f2-4cef-ba33-be89784a12d5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:38.436866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a955c-d3f2-4cef-ba33-be89784a12d5 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:38.438387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a955c-d3f2-4cef-ba33-be89784a12d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:38.438969 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a955c-d3f2-4cef-ba33-be89784a12d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:38.441943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a955c-d3f2-4cef-ba33-be89784a12d5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:38.467202 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 428/869] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:38.477119 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:38.519034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:38.609268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f509b-5a5b-4c49-b116-8355227f6a1d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:38.649016 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcb85c-d242-443e-99f9-19bc225b8752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:38.649689 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb85c-d242-443e-99f9-19bc225b8752 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:38.649980 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb85c-d242-443e-99f9-19bc225b8752 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:38.650288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb85c-d242-443e-99f9-19bc225b8752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:38.651106 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcb85c-d242-443e-99f9-19bc225b8752 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=7f85536ad53f474797f19311e1f19810) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:38.651421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcb85c-d242-443e-99f9-19bc225b8752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:38.652890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcb85c-d242-443e-99f9-19bc225b8752 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:38.660562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 442/870] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:38 2022] GET /identity/v3/role_assignments?user.id=7f85536ad53f474797f19311e1f19810&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:38.901027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620dd33-60da-487d-a0a0-88ab46fd8423 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:38.901697 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620dd33-60da-487d-a0a0-88ab46fd8423 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:38.901697 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620dd33-60da-487d-a0a0-88ab46fd8423 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:38.903561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 429/871] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:38.909392 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1066a-7db6-4b76-aba7-fae77f126b21 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:38.909952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1066a-7db6-4b76-aba7-fae77f126b21 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:38.909952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1066a-7db6-4b76-aba7-fae77f126b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:38.931090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-78a1066a-7db6-4b76-aba7-fae77f126b21 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:38.932346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-78a1066a-7db6-4b76-aba7-fae77f126b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:39.014056 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 443/872] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:39.037172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620dd33-60da-487d-a0a0-88ab46fd8423 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:39.147721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ae3ae-ebca-486d-9d66-39dffada2af8 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:39.148043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ae3ae-ebca-486d-9d66-39dffada2af8 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:39.148453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ae3ae-ebca-486d-9d66-39dffada2af8 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:39.150003 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 444/873] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:39.155868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fa4c9-61ea-44fa-8ec2-3530c51d4717 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:39.156075 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fa4c9-61ea-44fa-8ec2-3530c51d4717 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:39.156348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fa4c9-61ea-44fa-8ec2-3530c51d4717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:39.177898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620dd33-60da-487d-a0a0-88ab46fd8423 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:39.178567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-8b7fa4c9-61ea-44fa-8ec2-3530c51d4717 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:39.179791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-8b7fa4c9-61ea-44fa-8ec2-3530c51d4717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:39.198951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:39.221233 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:39.242538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4796-e5e6-4eed-a8d8-ac2b5955b83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:39.243618 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4796-e5e6-4eed-a8d8-ac2b5955b83e None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:39.244133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4796-e5e6-4eed-a8d8-ac2b5955b83e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:39.244737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4796-e5e6-4eed-a8d8-ac2b5955b83e None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:39.245834 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd4796-e5e6-4eed-a8d8-ac2b5955b83e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:39.247619 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd4796-e5e6-4eed-a8d8-ac2b5955b83e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:39.265664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 430/874] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:09:39 2022] POST /identity/v3/services => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:39.316132 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 445/875] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:39.339955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4796-e5e6-4eed-a8d8-ac2b5955b83e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:39.477624 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4796-e5e6-4eed-a8d8-ac2b5955b83e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:39.523155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df4003-4745-4bbe-bd72-b12090153ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:39.523835 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df4003-4745-4bbe-bd72-b12090153ab5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:39.524124 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df4003-4745-4bbe-bd72-b12090153ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:39.524404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df4003-4745-4bbe-bd72-b12090153ab5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:39.537509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df4003-4745-4bbe-bd72-b12090153ab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-92df4003-4745-4bbe-bd72-b12090153ab5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:39.543669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:39.544742 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:39.545664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 431/876] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:09:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:39.554705 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fa4c9-61ea-44fa-8ec2-3530c51d4717 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:39.682032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fa4c9-61ea-44fa-8ec2-3530c51d4717 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:39.736973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:39.737608 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:39.737962 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:39.738312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:39.739324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:39.739760 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:39.741780 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:39.756307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 446/877] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:09:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:39.762328 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df4003-4745-4bbe-bd72-b12090153ab5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:39.805322 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:39.896411 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df4003-4745-4bbe-bd72-b12090153ab5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:39.937213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:39.937936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:39.938304 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:39.938585 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:39.952304 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:39.953851 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:39.971217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:39.972453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:39.973470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:39.973470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:39.973470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:39.973470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:39.973470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 432/878] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:39.980359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:39.996157 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:40.102581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b05e8f-a6d9-40ea-9011-69b6cf08cd78 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:40.140949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:40.141633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:40.141920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:40.142219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:40.142997 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:40.143309 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:40.144709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:40.159405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 447/879] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:40.165255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:40.283132 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:40.299351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3676b-5537-4a3e-adbd-6ac62e60bb61 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:40.336071 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64273794-9994-4205-8711-0d559194a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:40.336740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273794-9994-4205-8711-0d559194a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:40.337024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273794-9994-4205-8711-0d559194a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:40.337301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273794-9994-4205-8711-0d559194a3c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:40.345925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64273794-9994-4205-8711-0d559194a3c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:40.347436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64273794-9994-4205-8711-0d559194a3c8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-64273794-9994-4205-8711-0d559194a3c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:40.359855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:40.361930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:40.363474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 433/880] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:09:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:40.371108 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:40.413161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcfb05-4ded-4119-a67b-a70e9885533e None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:40.413642 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcfb05-4ded-4119-a67b-a70e9885533e None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:40.413758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcfb05-4ded-4119-a67b-a70e9885533e None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:40.415415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 434/881] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:40.421078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5370-ac77-4b16-8aa3-983bcfc0f890 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:40.421472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5370-ac77-4b16-8aa3-983bcfc0f890 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:40.421647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5370-ac77-4b16-8aa3-983bcfc0f890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:40.441332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-b8ad5370-ac77-4b16-8aa3-983bcfc0f890 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:40.442930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-b8ad5370-ac77-4b16-8aa3-983bcfc0f890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:40.461954 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:40.507685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154201e4-9aac-457c-aa7d-20b87fdd0811 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:40.526626 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 435/882] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:40.554290 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:40.555176 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5370-ac77-4b16-8aa3-983bcfc0f890 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:40.555610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:40.556261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:40.556751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:40.558169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:40.558713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:40.561370 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:40.587769 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 448/883] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:09:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:40.594274 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:40.661095 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:40.691688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5370-ac77-4b16-8aa3-983bcfc0f890 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:40.738738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7f4c2-fcaf-4f46-a512-52ce4923ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:40.739697 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7f4c2-fcaf-4f46-a512-52ce4923ce1e None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:40.739897 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63bef6-ea1e-4a86-bf8e-56285ff62fd3 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:40.740326 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7f4c2-fcaf-4f46-a512-52ce4923ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:40.740812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7f4c2-fcaf-4f46-a512-52ce4923ce1e None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:40.741983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7f4c2-fcaf-4f46-a512-52ce4923ce1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:40.744452 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7f4c2-fcaf-4f46-a512-52ce4923ce1e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:40.763268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 436/884] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:09:40 2022] POST /identity/v3/services => generated 269 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:09:40.772715 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:40.795023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82331cf-03cb-4de2-8f3d-d0a140cb2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:40.795967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82331cf-03cb-4de2-8f3d-d0a140cb2c63 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:40.796453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82331cf-03cb-4de2-8f3d-d0a140cb2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:40.796954 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82331cf-03cb-4de2-8f3d-d0a140cb2c63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:40.883012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82331cf-03cb-4de2-8f3d-d0a140cb2c63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:40.884910 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82331cf-03cb-4de2-8f3d-d0a140cb2c63 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:41.032543 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 449/885] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:40 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:09:41.039519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7f4c2-fcaf-4f46-a512-52ce4923ce1e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:41.101045 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d77a5-a1da-430e-919d-f9e5329b7466 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.101745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d77a5-a1da-430e-919d-f9e5329b7466 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.102313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d77a5-a1da-430e-919d-f9e5329b7466 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.104189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 450/886] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:41.110070 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd525891-c262-4fa7-80a8-d69090c06478 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.110467 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd525891-c262-4fa7-80a8-d69090c06478 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.110755 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd525891-c262-4fa7-80a8-d69090c06478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.130867 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-bd525891-c262-4fa7-80a8-d69090c06478 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:41.132570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-bd525891-c262-4fa7-80a8-d69090c06478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:41.170679 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:41.190941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7f4c2-fcaf-4f46-a512-52ce4923ce1e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:41.234160 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 451/887] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:41.244815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:41.246220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.247015 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.247789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.252428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:41.256023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd525891-c262-4fa7-80a8-d69090c06478 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:41.267535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:41.269828 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:41.293353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:41.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:41.295921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:41.295921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:41.295921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:41.295921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:41.300038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 437/888] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:41.306940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:41.377696 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd525891-c262-4fa7-80a8-d69090c06478 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:41.416023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:41.417178 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.417681 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.418217 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.419082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae2307-932e-420c-a861-74cbd4d6f889 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:41.419321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:41.421009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:41.431433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:41.432698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:41.433720 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:41.441581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 452/889] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:41.444791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:41.463791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:41.470208 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:41.471059 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.471344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.471623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.472602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:41.472911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:41.474882 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:41.496004 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 438/890] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:41.498084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:41.574870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:41.590529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95cac5-7b58-4f84-bf1c-ab01ee1735fe None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:41.636219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f3a3f-2ecf-4193-b912-cd5b3691d113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:41.636859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f3a3f-2ecf-4193-b912-cd5b3691d113 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.637149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f3a3f-2ecf-4193-b912-cd5b3691d113 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.637429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f3a3f-2ecf-4193-b912-cd5b3691d113 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.638144 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f3a3f-2ecf-4193-b912-cd5b3691d113 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:41.638458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f3a3f-2ecf-4193-b912-cd5b3691d113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:41.639865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f3a3f-2ecf-4193-b912-cd5b3691d113 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:41.645312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 453/891] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:41.646733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965baf7-0a01-4bd9-a20c-bf023e53216b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:41.652194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139bf08-a419-40e1-b7fe-d68e80900077 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.652534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139bf08-a419-40e1-b7fe-d68e80900077 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.652778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139bf08-a419-40e1-b7fe-d68e80900077 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.658667 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 454/892] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:41.658667 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139bf08-a419-40e1-b7fe-d68e80900077 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:41.695046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f3b3e-382f-42e8-8508-d371ce0949aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:41.695862 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f3b3e-382f-42e8-8508-d371ce0949aa None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.696166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f3b3e-382f-42e8-8508-d371ce0949aa None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.696503 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f3b3e-382f-42e8-8508-d371ce0949aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.704887 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f3b3e-382f-42e8-8508-d371ce0949aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:41.706432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f3b3e-382f-42e8-8508-d371ce0949aa None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-2b0f3b3e-382f-42e8-8508-d371ce0949aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:41.720510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:41.722019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:41.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:41.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:41.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:41.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:41.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:41.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:41.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:41.723296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:41.726602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 439/893] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:41.729241 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149033a0-68d6-47d1-843b-e4eb894b4737 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.729584 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149033a0-68d6-47d1-843b-e4eb894b4737 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.729970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149033a0-68d6-47d1-843b-e4eb894b4737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.753167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-149033a0-68d6-47d1-843b-e4eb894b4737 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:41.754453 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-149033a0-68d6-47d1-843b-e4eb894b4737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:41.766313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:41.800654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139bf08-a419-40e1-b7fe-d68e80900077 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:41.837225 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 440/894] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:41.838137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149033a0-68d6-47d1-843b-e4eb894b4737 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:41.851632 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:41.852749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:41.853275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:41.853795 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:41.854842 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:41.855550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:41.857182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:41.863693 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 455/895] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:41.876992 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:41.906658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:42.006775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149033a0-68d6-47d1-843b-e4eb894b4737 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:42.060549 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea8f8a-94db-40c8-b071-648508387c44 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:42.065691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:42.066287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:42.066609 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:42.066816 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:42.067997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:42.068312 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:42.069965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:42.085188 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 441/896] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:42.093158 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:42.126981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:42.128349 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:42.128862 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:42.129578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:42.130638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:42.132607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:42.145783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:42.147357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:42.148320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:42.151533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 456/897] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:42.162652 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:42.185817 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:42.250427 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:42.254343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a82f3a-7dab-4b30-84c9-459ead736e0f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:42.300645 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:42.301298 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:42.301651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:42.301993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:42.302885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] RBAC: Adding query filter params (role.id=7e05de6606ba48c984ba5aa6420aadbc, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=41037802b8d9420f825cfda7247a52ad) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:42.303390 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:42.305041 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:42.312273 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 442/898] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:42 2022] GET /identity/v3/role_assignments?user.id=41037802b8d9420f825cfda7247a52ad&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=7e05de6606ba48c984ba5aa6420aadbc => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:42.322171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3ed2d-bf34-47d8-864d-e04dd554320e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:42.365132 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:42.365903 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:42.366214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:42.366491 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:42.367288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:42.367600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:42.369119 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:42.375611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 457/899] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:42.381959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:42.513424 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfcb45-db9d-4486-a1b2-4b4c2e347658 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:42.560375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:42.561314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:42.561823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:42.562360 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:42.563693 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:42.564258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:42.567359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:42.578136 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 443/900] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:42.587240 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:42.671929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:42.709719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a9828-c0a2-40fe-92fc-3d7e91d0b08f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:42.747815 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bbbb3-044d-46c2-80d7-c69a83acee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:42.748505 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bbbb3-044d-46c2-80d7-c69a83acee46 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:42.748796 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bbbb3-044d-46c2-80d7-c69a83acee46 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:42.749077 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bbbb3-044d-46c2-80d7-c69a83acee46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:42.749859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bbbb3-044d-46c2-80d7-c69a83acee46 None admin] RBAC: Adding query filter params (interface=public, service_id=604c1dc3c1e1430b8b37118395bdd68a, region_id=RegionOne) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:42.750209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bbbb3-044d-46c2-80d7-c69a83acee46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:42.751605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bbbb3-044d-46c2-80d7-c69a83acee46 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:42.760376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 458/901] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:09:42 2022] GET /identity/v3/endpoints?service_id=604c1dc3c1e1430b8b37118395bdd68a&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:42.766092 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:42.796808 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:42.898952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb4e8c-c5a2-44c1-8d66-e04017f2d043 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:42.956826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe80bfc-3604-4f2a-8f86-94fdfabecf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:42.957984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80bfc-3604-4f2a-8f86-94fdfabecf12 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:42.958523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80bfc-3604-4f2a-8f86-94fdfabecf12 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:42.959168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80bfc-3604-4f2a-8f86-94fdfabecf12 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:42.960092 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe80bfc-3604-4f2a-8f86-94fdfabecf12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:42.962921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe80bfc-3604-4f2a-8f86-94fdfabecf12 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:42.975730 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 444/902] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:09:42 2022] GET /identity/v3/services => generated 1268 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:09:43.045325 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e72ca-4ca4-4e1e-b6fd-65d941bd3abc None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:43.045834 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e72ca-4ca4-4e1e-b6fd-65d941bd3abc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:43.046347 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e72ca-4ca4-4e1e-b6fd-65d941bd3abc None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:43.049116 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 459/903] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:43.055745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83df8e-e814-4833-a5c8-7cb0b11ed371 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:43.056287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83df8e-e814-4833-a5c8-7cb0b11ed371 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:43.056741 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83df8e-e814-4833-a5c8-7cb0b11ed371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:43.078546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-cd83df8e-e814-4833-a5c8-7cb0b11ed371 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:43.080660 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-cd83df8e-e814-4833-a5c8-7cb0b11ed371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:43.169918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 445/904] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:43.194408 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e72ca-4ca4-4e1e-b6fd-65d941bd3abc None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:43.345196 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e72ca-4ca4-4e1e-b6fd-65d941bd3abc None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:43.389716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:43.390445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:43.390801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:43.391090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:43.391849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:43.393317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:43.396068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:43.408247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:43.409836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:43.411214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 460/905] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 09:09:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:43.417253 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83df8e-e814-4833-a5c8-7cb0b11ed371 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:43.449590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:43.572357 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83df8e-e814-4833-a5c8-7cb0b11ed371 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:43.627591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:43.628663 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:43.629155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:43.629516 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:43.630547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:43.631127 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:43.633380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:43.642191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 446/906] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:43.649116 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:43.796200 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90c988-7cff-4239-be20-c064d69f10e6 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:43.833819 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:43.834965 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:43.835471 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:43.835979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:43.836907 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:43.837431 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:43.839070 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:43.845182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 461/907] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:43.852414 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:43.992461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:44.024177 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e87db1-041d-489e-8bb1-3b31cc83fc93 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:44.076673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0b727-7269-4cd1-8f84-a165ed6259a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:44.077438 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0b727-7269-4cd1-8f84-a165ed6259a6 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:44.077952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0b727-7269-4cd1-8f84-a165ed6259a6 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:44.078494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0b727-7269-4cd1-8f84-a165ed6259a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:44.079998 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0b727-7269-4cd1-8f84-a165ed6259a6 None admin] RBAC: Adding query filter params (interface=public, service_id=fcdce78f75b640d2ac78112b8deb3064, region_id=RegionOne) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:44.080575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0b727-7269-4cd1-8f84-a165ed6259a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:44.082598 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0b727-7269-4cd1-8f84-a165ed6259a6 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:44.090970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 447/908] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:09:43 2022] GET /identity/v3/endpoints?service_id=fcdce78f75b640d2ac78112b8deb3064&interface=public®ion_id=RegionOne => generated 194 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:44.096990 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:44.135516 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa479c0c-b58f-412e-8602-c7dd10335227 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:44.136133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa479c0c-b58f-412e-8602-c7dd10335227 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:44.136511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa479c0c-b58f-412e-8602-c7dd10335227 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:44.138758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 448/909] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:44.144740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea25a9-6e84-4940-865c-63d2e139ffca None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:44.145033 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea25a9-6e84-4940-865c-63d2e139ffca None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:44.145313 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea25a9-6e84-4940-865c-63d2e139ffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:44.167102 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-92ea25a9-6e84-4940-865c-63d2e139ffca None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:44.169056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-92ea25a9-6e84-4940-865c-63d2e139ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:44.184943 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:44.262245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 449/910] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:44.268708 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf25d8-b6b4-4b94-9ef4-315b193e8500 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:44.284452 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea25a9-6e84-4940-865c-63d2e139ffca None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:44.316120 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5ccb1-7fcc-4efb-9c2c-137df5f573e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:44.317643 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5ccb1-7fcc-4efb-9c2c-137df5f573e2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:44.318310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5ccb1-7fcc-4efb-9c2c-137df5f573e2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:44.318888 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5ccb1-7fcc-4efb-9c2c-137df5f573e2 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:44.319795 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5ccb1-7fcc-4efb-9c2c-137df5f573e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:44.321663 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5ccb1-7fcc-4efb-9c2c-137df5f573e2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:44.331037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 462/911] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:09:44 2022] GET /identity/v3/services => generated 1268 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:09:44.397588 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:44.429629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea25a9-6e84-4940-865c-63d2e139ffca None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:44.491085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765c2ca-17c4-43b5-b90c-78f430e74de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:44.492003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765c2ca-17c4-43b5-b90c-78f430e74de6 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:44.492323 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765c2ca-17c4-43b5-b90c-78f430e74de6 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:44.492607 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765c2ca-17c4-43b5-b90c-78f430e74de6 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:44.493488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765c2ca-17c4-43b5-b90c-78f430e74de6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:44.495147 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765c2ca-17c4-43b5-b90c-78f430e74de6 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:44.518283 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 450/912] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:09:44 2022] POST /identity/v3/services => generated 241 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:09:44.719732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1483-3a73-47f7-9cc3-23c7f7249b95 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:44.720084 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1483-3a73-47f7-9cc3-23c7f7249b95 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:44.720350 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1483-3a73-47f7-9cc3-23c7f7249b95 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:44.722204 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 463/913] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:44.729179 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984123dd-6b79-4850-8743-1bafeb757840 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:44.729680 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984123dd-6b79-4850-8743-1bafeb757840 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:44.730209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984123dd-6b79-4850-8743-1bafeb757840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:44.754864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-984123dd-6b79-4850-8743-1bafeb757840 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:44.756082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-984123dd-6b79-4850-8743-1bafeb757840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:44.840178 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 451/914] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:44.864395 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c1483-3a73-47f7-9cc3-23c7f7249b95 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:44.986706 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c1483-3a73-47f7-9cc3-23c7f7249b95 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:45.018322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:45.030047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09da1b-326c-4dbf-9f05-7defb9a8f524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:45.031318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09da1b-326c-4dbf-9f05-7defb9a8f524 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:45.031832 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09da1b-326c-4dbf-9f05-7defb9a8f524 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:45.032337 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09da1b-326c-4dbf-9f05-7defb9a8f524 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:45.047039 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09da1b-326c-4dbf-9f05-7defb9a8f524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-2a09da1b-326c-4dbf-9f05-7defb9a8f524 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:45.053288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:45.054657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:09:45.055923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:45.059921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 464/915] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:09:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:45.067803 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984123dd-6b79-4850-8743-1bafeb757840 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:45.149913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:45.204156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984123dd-6b79-4850-8743-1bafeb757840 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:45.255932 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:45.256777 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:45.257229 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:45.257521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:45.258406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:45.258922 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:45.261524 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:45.276723 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 452/916] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:09:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:45.282651 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09da1b-326c-4dbf-9f05-7defb9a8f524 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:45.400669 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09da1b-326c-4dbf-9f05-7defb9a8f524 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:45.445762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca962259-9b12-4ab1-9522-dc6a609f2539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:45.446582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca962259-9b12-4ab1-9522-dc6a609f2539 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:45.446969 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca962259-9b12-4ab1-9522-dc6a609f2539 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:45.447289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca962259-9b12-4ab1-9522-dc6a609f2539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:45.461096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca962259-9b12-4ab1-9522-dc6a609f2539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:45.462844 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca962259-9b12-4ab1-9522-dc6a609f2539 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-ca962259-9b12-4ab1-9522-dc6a609f2539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:45.482582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:45.483747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:45.486150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:45.486150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:45.486150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:45.486150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:45.486150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 465/917] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:45.490980 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:45.632132 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba032626-b4d0-49ad-9366-bae68b151bca None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:45.634790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464e6dc-07f5-409b-b05c-8454d7ac1f17 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:45.635094 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464e6dc-07f5-409b-b05c-8454d7ac1f17 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:45.635420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464e6dc-07f5-409b-b05c-8454d7ac1f17 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:45.637114 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 466/918] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:45.642919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ef6a-10ce-419d-921d-56cd8b7f7e85 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:45.643216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ef6a-10ce-419d-921d-56cd8b7f7e85 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:45.643476 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ef6a-10ce-419d-921d-56cd8b7f7e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:45.663223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:45.664592 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-b5e7ef6a-10ce-419d-921d-56cd8b7f7e85 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:45.666319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-b5e7ef6a-10ce-419d-921d-56cd8b7f7e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:45.673620 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:45.674432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:45.674737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:45.675077 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:45.675786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:45.676122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:45.677911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:45.696462 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 453/919] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:45.697699 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:45.710514 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:45.769785 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 467/920] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:45.792581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7ef6a-10ce-419d-921d-56cd8b7f7e85 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:45.845684 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce5774-8d5b-481e-96be-660b110815c7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:45.902735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510167b4-d397-4322-8288-2c425761a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:45.904337 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510167b4-d397-4322-8288-2c425761a995 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:45.904337 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510167b4-d397-4322-8288-2c425761a995 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:45.904337 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510167b4-d397-4322-8288-2c425761a995 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:45.914909 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510167b4-d397-4322-8288-2c425761a995 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:45.916927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510167b4-d397-4322-8288-2c425761a995 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-510167b4-d397-4322-8288-2c425761a995 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:45.935614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:45.936769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:45.939221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:45.941896 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 454/921] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:09:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:45.943712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510167b4-d397-4322-8288-2c425761a995 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:45.961187 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7ef6a-10ce-419d-921d-56cd8b7f7e85 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:46.009390 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:46.011121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62107732-078f-499b-8329-98a1d656f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:46.012054 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62107732-078f-499b-8329-98a1d656f086 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.012521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62107732-078f-499b-8329-98a1d656f086 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.012982 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62107732-078f-499b-8329-98a1d656f086 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.014631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62107732-078f-499b-8329-98a1d656f086 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:46.016535 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62107732-078f-499b-8329-98a1d656f086 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-62107732-078f-499b-8329-98a1d656f086 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:46.027369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:46.028713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:09:46.029634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:46.038038 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 468/922] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:46.039350 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62107732-078f-499b-8329-98a1d656f086 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:46.075326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:46.081305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510167b4-d397-4322-8288-2c425761a995 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:46.120223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:46.121571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.121571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.121571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.122591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:46.122972 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:46.124522 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:46.150771 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 455/923] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:09:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:46.151954 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:46.164239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62107732-078f-499b-8329-98a1d656f086 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:46.215655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:46.216437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.216827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.217178 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.217947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:46.218371 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:46.219836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:46.231374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 469/924] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:46.232525 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:46.299823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd34ba-235a-45e8-9137-2226fb026e36 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:46.302403 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:46.341545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:46.342288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.342595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.342882 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.357221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:46.359446 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:46.374511 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:46.381873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3f041-cf1b-4407-9139-cdfd26fe49fa None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:46.387289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:46.388597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:46.389895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:46.389895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:46.389895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:46.389895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:46.392368 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 456/925] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:46.399226 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:46.432635 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df57113-536c-445d-b70b-2278be262edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:46.433970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df57113-536c-445d-b70b-2278be262edc None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.434534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df57113-536c-445d-b70b-2278be262edc None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.435024 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df57113-536c-445d-b70b-2278be262edc None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.435964 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df57113-536c-445d-b70b-2278be262edc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:46.436497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df57113-536c-445d-b70b-2278be262edc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:46.438197 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df57113-536c-445d-b70b-2278be262edc None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:46.452469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 470/926] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:09:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:46.458515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df57113-536c-445d-b70b-2278be262edc None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:46.565775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62f9f-cfd5-4cc0-995d-78cc52953d77 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:46.583384 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df57113-536c-445d-b70b-2278be262edc None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:46.616656 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6bf03-1516-42bd-b616-06126d435488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:46.617466 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6bf03-1516-42bd-b616-06126d435488 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.617751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6bf03-1516-42bd-b616-06126d435488 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.618030 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6bf03-1516-42bd-b616-06126d435488 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.618783 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6bf03-1516-42bd-b616-06126d435488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:46.619071 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6bf03-1516-42bd-b616-06126d435488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:46.620464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6bf03-1516-42bd-b616-06126d435488 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:46.632144 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb731b-6269-41e5-bfe1-3003ffa9c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:46.632821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb731b-6269-41e5-bfe1-3003ffa9c7b4 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.633085 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb731b-6269-41e5-bfe1-3003ffa9c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.633343 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb731b-6269-41e5-bfe1-3003ffa9c7b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.634089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb731b-6269-41e5-bfe1-3003ffa9c7b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:46.635533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb731b-6269-41e5-bfe1-3003ffa9c7b4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:46.639617 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 457/927] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:46.641083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc00a8e-4aab-40af-9155-493d59f33356 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.641514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc00a8e-4aab-40af-9155-493d59f33356 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.641646 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc00a8e-4aab-40af-9155-493d59f33356 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.646400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 458/928] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:46.646400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc00a8e-4aab-40af-9155-493d59f33356 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:46.667228 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:46.668792 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 471/929] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:09:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:09:46.671031 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3ed3e-c1c7-405a-8d3b-d86e9c4dd981 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.671448 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3ed3e-c1c7-405a-8d3b-d86e9c4dd981 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.671740 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3ed3e-c1c7-405a-8d3b-d86e9c4dd981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.704471 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-8bc3ed3e-c1c7-405a-8d3b-d86e9c4dd981 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:46.705908 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-8bc3ed3e-c1c7-405a-8d3b-d86e9c4dd981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:46.721349 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:46.822487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 472/930] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:46.833262 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc00a8e-4aab-40af-9155-493d59f33356 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:46.851531 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3ed3e-c1c7-405a-8d3b-d86e9c4dd981 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:46.881552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:46.882379 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:46.882676 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:46.882958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:46.891864 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:46.893417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:46.908125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:46.909175 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:46.910236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 459/931] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:46.915356 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:46.973743 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3ed3e-c1c7-405a-8d3b-d86e9c4dd981 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:47.010272 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:47.013289 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f8b2d-f9af-47b5-9a9a-0301e5e6e13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:47.013973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f8b2d-f9af-47b5-9a9a-0301e5e6e13f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.014317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f8b2d-f9af-47b5-9a9a-0301e5e6e13f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.014568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f8b2d-f9af-47b5-9a9a-0301e5e6e13f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.015222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f8b2d-f9af-47b5-9a9a-0301e5e6e13f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:47.016633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f8b2d-f9af-47b5-9a9a-0301e5e6e13f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-1e3f8b2d-f9af-47b5-9a9a-0301e5e6e13f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:47.026360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:47.027483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:47.028450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 473/932] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 09:09:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:47.029812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b54d4-6924-412b-94b5-da5bc0802634 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.029812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b54d4-6924-412b-94b5-da5bc0802634 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.029812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b54d4-6924-412b-94b5-da5bc0802634 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.030598 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 474/933] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:47.037939 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b54d4-6924-412b-94b5-da5bc0802634 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:47.047286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:47.053131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f8c9d-d1de-4cd1-a3e0-f7d1e2e78dca None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:47.102499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2ed4a-faf9-47c6-8c8b-fc4e8c403937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:47.103159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ed4a-faf9-47c6-8c8b-fc4e8c403937 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.103449 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ed4a-faf9-47c6-8c8b-fc4e8c403937 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.103731 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ed4a-faf9-47c6-8c8b-fc4e8c403937 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.104543 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ed4a-faf9-47c6-8c8b-fc4e8c403937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:47.104870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ed4a-faf9-47c6-8c8b-fc4e8c403937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:47.106329 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ed4a-faf9-47c6-8c8b-fc4e8c403937 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:47.120213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 460/934] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:47.124487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45a6eb-9a67-4dca-a08f-aa5c74af511c None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.125779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45a6eb-9a67-4dca-a08f-aa5c74af511c None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.126387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45a6eb-9a67-4dca-a08f-aa5c74af511c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.147088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-5b45a6eb-9a67-4dca-a08f-aa5c74af511c None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:47.148419 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-5b45a6eb-9a67-4dca-a08f-aa5c74af511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:47.177665 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b54d4-6924-412b-94b5-da5bc0802634 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:47.228597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:47.229240 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.229947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.229947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.230601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:47.230915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:47.232397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:47.242504 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 475/935] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:47.243834 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:47.257401 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 461/936] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:47.259209 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45a6eb-9a67-4dca-a08f-aa5c74af511c None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:47.350278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:47.365216 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:47.390059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ec73d-ba76-4e69-a149-38177e4cd809 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:47.408060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45a6eb-9a67-4dca-a08f-aa5c74af511c None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:47.439811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0b4d3-754e-4d4d-acd8-c4c160483556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:47.439811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0b4d3-754e-4d4d-acd8-c4c160483556 None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.440402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0b4d3-754e-4d4d-acd8-c4c160483556 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.440402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0b4d3-754e-4d4d-acd8-c4c160483556 None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/41037802b8d9420f825cfda7247a52ad/roles/7e05de6606ba48c984ba5aa6420aadbc` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.445236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0b4d3-754e-4d4d-acd8-c4c160483556 None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=41037802b8d9420f825cfda7247a52ad, role_id=7e05de6606ba48c984ba5aa6420aadbc)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:47.446315 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0b4d3-754e-4d4d-acd8-c4c160483556 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:47.463079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 476/937] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:09:47 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/41037802b8d9420f825cfda7247a52ad/roles/7e05de6606ba48c984ba5aa6420aadbc => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:09:47.464068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0b4d3-754e-4d4d-acd8-c4c160483556 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:47.481772 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:47.483310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.483859 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.484680 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.486291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:47.487046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:47.489104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:47.497550 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 462/938] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:47.504264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:47.610311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0b4d3-754e-4d4d-acd8-c4c160483556 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:47.647593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154ae55-9a49-4f17-9913-87bf75151fd0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:47.656827 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:47.670083 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:47.671100 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.671457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.671890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.672614 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:47.674098 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:47.685967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:47.688548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:47.690479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 477/939] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:47.695423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:47.718493 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:47.726554 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88322-488f-443c-9c64-059687617ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:47.728836 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88322-488f-443c-9c64-059687617ddd None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.729119 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88322-488f-443c-9c64-059687617ddd None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.729393 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88322-488f-443c-9c64-059687617ddd None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.730156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d88322-488f-443c-9c64-059687617ddd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:47.731866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d88322-488f-443c-9c64-059687617ddd None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:47.757944 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 463/940] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:09:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:47.821886 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccfdd8-f29b-40ac-8282-daf352866313 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:47.870982 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:47.872015 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:47.872463 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:47.872963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:47.873848 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:47.874350 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:47.875944 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:47.882700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 478/941] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:47.890741 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88322-488f-443c-9c64-059687617ddd None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:48.040286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88322-488f-443c-9c64-059687617ddd None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:48.082852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:48.083888 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:48.084449 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:48.084941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:48.085873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:48.086539 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:48.088215 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:48.094866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 464/942] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:48.101698 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:48.206704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:48.265221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907af72e-7a49-47f2-b341-ece0645a0d8d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:48.312913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ea2b8-4caa-457f-bbab-0b507cd227ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:48.313590 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ea2b8-4caa-457f-bbab-0b507cd227ca None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:48.313885 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ea2b8-4caa-457f-bbab-0b507cd227ca None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:48.314343 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ea2b8-4caa-457f-bbab-0b507cd227ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:48.315358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ea2b8-4caa-457f-bbab-0b507cd227ca None admin] RBAC: Adding query filter params (interface=public, service_id=de88ec364b744026b19f92a47e114181, region_id=RegionOne) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:48.315756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ea2b8-4caa-457f-bbab-0b507cd227ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:48.317784 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ea2b8-4caa-457f-bbab-0b507cd227ca None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:48.325127 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 479/943] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:09:48 2022] GET /identity/v3/endpoints?service_id=de88ec364b744026b19f92a47e114181&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:48.330917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:48.389542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:48.474793 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3a7da-5218-45d4-9a65-3a7a85e1af99 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:48.526597 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c9c61-4065-4845-aa14-2b3fda11a0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:48.527361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c9c61-4065-4845-aa14-2b3fda11a0d6 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:48.527645 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c9c61-4065-4845-aa14-2b3fda11a0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:48.528064 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c9c61-4065-4845-aa14-2b3fda11a0d6 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:48.528965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c9c61-4065-4845-aa14-2b3fda11a0d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:48.530839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c9c61-4065-4845-aa14-2b3fda11a0d6 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:48.546017 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 465/944] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:09:48 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:09:48.891206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8c3fa-0a0e-4c7c-a2c0-43127f1dfc42 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:48.892266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8c3fa-0a0e-4c7c-a2c0-43127f1dfc42 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:48.892770 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8c3fa-0a0e-4c7c-a2c0-43127f1dfc42 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:48.894670 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 480/945] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:48.900479 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f81cf-aa54-4f24-a8a4-2b3a37e71997 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:48.901069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f81cf-aa54-4f24-a8a4-2b3a37e71997 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:48.901559 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f81cf-aa54-4f24-a8a4-2b3a37e71997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:48.921123 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-2a8f81cf-aa54-4f24-a8a4-2b3a37e71997 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:48.922752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-2a8f81cf-aa54-4f24-a8a4-2b3a37e71997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:49.004521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 466/946] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:49.028660 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8c3fa-0a0e-4c7c-a2c0-43127f1dfc42 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:49.150449 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8c3fa-0a0e-4c7c-a2c0-43127f1dfc42 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:49.169230 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:49.187486 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:49.188200 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:49.188485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:49.188763 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:49.189518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:49.191024 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:49.194216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: WARNING keystone.server.flask.common [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 09:09:49.216126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.sql.core [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 09:09:49.216126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 09:09:49.216126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [parameters: {'id': '44b9558354aa4000a070b1b2ba29bee3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 09:09:49.216126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=411108) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 44b9558354aa4000a070b1b2ba29bee3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 44b9558354aa4000a070b1b2ba29bee3. Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:09:49.223557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application result.read() Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 09:09:49.224508 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application yield resource Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise self.value Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application session.commit() Jul 28 09:09:49.225645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.session.flush() Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application compat.raise_( Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 09:09:49.226701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 09:09:49.227674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application result.read() Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:09:49.228568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [parameters: {'id': '44b9558354aa4000a070b1b2ba29bee3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:49.229520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:49.232485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 44b9558354aa4000a070b1b2ba29bee3. Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:49.238261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 481/947] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 09:09:49 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:49.239716 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f81cf-aa54-4f24-a8a4-2b3a37e71997 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:49.288487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:49.352836 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f81cf-aa54-4f24-a8a4-2b3a37e71997 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:49.393940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bba19b-60fd-4bef-baa4-c2e8a03b1817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:49.395463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bba19b-60fd-4bef-baa4-c2e8a03b1817 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:49.396045 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bba19b-60fd-4bef-baa4-c2e8a03b1817 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:49.396582 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bba19b-60fd-4bef-baa4-c2e8a03b1817 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:49.405570 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bba19b-60fd-4bef-baa4-c2e8a03b1817 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:49.407972 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bba19b-60fd-4bef-baa4-c2e8a03b1817 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-48bba19b-60fd-4bef-baa4-c2e8a03b1817 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:49.424167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:49.425535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:49.427096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:49.427096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:49.427096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:49.427096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:49.427096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:49.427096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:49.427096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:49.427096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:49.434334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 467/948] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:49.437607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:49.555670 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e88067-f980-48b4-9db3-c241c3e05104 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:49.593266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc091e-0885-47e2-966d-a4f5f8380f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:49.594058 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc091e-0885-47e2-966d-a4f5f8380f0f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:49.594642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc091e-0885-47e2-966d-a4f5f8380f0f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:49.594990 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc091e-0885-47e2-966d-a4f5f8380f0f None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:49.595802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc091e-0885-47e2-966d-a4f5f8380f0f None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:49.596179 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc091e-0885-47e2-966d-a4f5f8380f0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:49.597929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc091e-0885-47e2-966d-a4f5f8380f0f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:49.619918 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 482/949] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:49.679474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536f81f-fbd7-4676-95ef-b32fbd54b634 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:49.679767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536f81f-fbd7-4676-95ef-b32fbd54b634 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:49.680119 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536f81f-fbd7-4676-95ef-b32fbd54b634 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:49.681848 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 468/950] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:49.687878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612fd1-8310-4c20-9877-91faeb9d01d6 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:49.688257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612fd1-8310-4c20-9877-91faeb9d01d6 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:49.688604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612fd1-8310-4c20-9877-91faeb9d01d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:49.710237 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-c3612fd1-8310-4c20-9877-91faeb9d01d6 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:49.712011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-c3612fd1-8310-4c20-9877-91faeb9d01d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:49.744087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:49.795118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 483/951] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:49.819776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536f81f-fbd7-4676-95ef-b32fbd54b634 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:49.945625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536f81f-fbd7-4676-95ef-b32fbd54b634 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:49.950336 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:49.993661 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f487e3-38d3-4dde-8dcd-9250144a03f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:49.994370 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f487e3-38d3-4dde-8dcd-9250144a03f8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:49.994720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f487e3-38d3-4dde-8dcd-9250144a03f8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:49.995183 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f487e3-38d3-4dde-8dcd-9250144a03f8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.007491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f487e3-38d3-4dde-8dcd-9250144a03f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-80f487e3-38d3-4dde-8dcd-9250144a03f8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:09:50.014145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:50.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:50.016161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 469/952] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:09:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:50.023540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3612fd1-8310-4c20-9877-91faeb9d01d6 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.039849 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13d47-ef93-4347-aae3-96e341777702 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.040564 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13d47-ef93-4347-aae3-96e341777702 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.040564 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13d47-ef93-4347-aae3-96e341777702 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.042463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 470/953] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:50.048837 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade80d6-c57a-4176-a24a-b74765f5e86b None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.048837 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade80d6-c57a-4176-a24a-b74765f5e86b None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.049213 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade80d6-c57a-4176-a24a-b74765f5e86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.068975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-5ade80d6-c57a-4176-a24a-b74765f5e86b None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:50.070839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-5ade80d6-c57a-4176-a24a-b74765f5e86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:50.141965 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3612fd1-8310-4c20-9877-91faeb9d01d6 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:50.150494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 471/954] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:50.165122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade80d6-c57a-4176-a24a-b74765f5e86b None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.181978 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:50.183080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.183641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.184133 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.185138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:50.185668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:50.187418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:50.198001 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 484/955] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:50.203996 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.241624 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:50.308082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:50.321274 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade80d6-c57a-4176-a24a-b74765f5e86b None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:50.355272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486700f-8649-4404-b6d2-d68f0070b77b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:50.368312 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:50.369744 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.370301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.370823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] PATH_INFO: `/v3/services/compute` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.371712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:50.373373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:50.386732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:50.388305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:50.389266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:50.392710 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 472/956] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:50.399112 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.422192 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:50.423519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.424074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.425196 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.452004 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:50.454580 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:50.487986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:50.489266 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:50.490605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:50.490605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:50.490605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:50.490605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:50.497389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 485/957] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:50.501143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.559292 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd2936-2631-4cfc-99b1-37241cc32b06 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:50.614340 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:50.620617 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:50.621778 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.622270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.622726 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.623589 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] RBAC: Adding query filter params (name=compute) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:50.624233 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:50.626774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:50.638523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 473/958] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:50.641849 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.655957 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37ffd6-a994-4ea6-baea-a97ee9adcfa2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:50.672243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:50.709339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f10a8-1e6e-443d-8df5-22d0ddb0079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:50.709884 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f10a8-1e6e-443d-8df5-22d0ddb0079b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.710198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f10a8-1e6e-443d-8df5-22d0ddb0079b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.710476 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f10a8-1e6e-443d-8df5-22d0ddb0079b None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.711182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f10a8-1e6e-443d-8df5-22d0ddb0079b None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:50.711486 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f10a8-1e6e-443d-8df5-22d0ddb0079b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:50.712877 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f10a8-1e6e-443d-8df5-22d0ddb0079b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:50.725507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 486/959] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:50.727736 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac6d7-83e7-4f17-8252-e48329beef57 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.728026 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac6d7-83e7-4f17-8252-e48329beef57 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.728306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac6d7-83e7-4f17-8252-e48329beef57 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.729774 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 487/960] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:50.730930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ac6d7-83e7-4f17-8252-e48329beef57 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.776873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707f0b8-e3b0-43d9-bb8f-c2bbf3330540 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:50.822036 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf09c0f-ce0e-4ae2-b324-142aa2c269d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:50.823042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf09c0f-ce0e-4ae2-b324-142aa2c269d5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.823535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf09c0f-ce0e-4ae2-b324-142aa2c269d5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.824093 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf09c0f-ce0e-4ae2-b324-142aa2c269d5 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.824804 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf09c0f-ce0e-4ae2-b324-142aa2c269d5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:50.825115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf09c0f-ce0e-4ae2-b324-142aa2c269d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:50.826568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf09c0f-ce0e-4ae2-b324-142aa2c269d5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:50.838418 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 474/961] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:50.839973 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f958e0-bf40-47e7-a7d3-d1fda0348802 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.840266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f958e0-bf40-47e7-a7d3-d1fda0348802 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.840547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f958e0-bf40-47e7-a7d3-d1fda0348802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.861889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-d5f958e0-bf40-47e7-a7d3-d1fda0348802 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:50.862813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ac6d7-83e7-4f17-8252-e48329beef57 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:50.863530 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-d5f958e0-bf40-47e7-a7d3-d1fda0348802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:50.923566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:50.924756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:50.925301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:50.925876 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:50.935608 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:50.937697 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:50.940449 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:50.953333 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 475/962] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:50.954017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:50.955193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:50.956254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 488/963] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:50.961663 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f958e0-bf40-47e7-a7d3-d1fda0348802 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.984040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:50.996180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:51.118520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f958e0-bf40-47e7-a7d3-d1fda0348802 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:51.145935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2182cc-cae7-4173-9504-9880cb25da08 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:51.160929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:51.162024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.162447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.162798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.163976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:51.164360 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:51.166031 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:51.189677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 476/964] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:51.193071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:51.194158 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.195654 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:51.196529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.196839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.197490 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:51.198931 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:51.217659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:51.218821 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:51.219751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:51.222191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 489/965] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:51.225493 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:51.301203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:51.309835 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:51.335045 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c10c1c-6d2d-4aea-865c-d83a7aad6e58 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:51.365491 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b1330-fdb9-4b5e-ab1f-fde3dd14c2c8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:51.384186 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a53e6-d200-4da7-aa74-996456c6aaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:51.385459 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a53e6-d200-4da7-aa74-996456c6aaf9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.386022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a53e6-d200-4da7-aa74-996456c6aaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.386493 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a53e6-d200-4da7-aa74-996456c6aaf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.404632 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:51.405748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.405748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.406202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.406999 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:51.407483 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:51.409018 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:51.420273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 490/966] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:51.421472 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:51.474488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a53e6-d200-4da7-aa74-996456c6aaf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:51.476384 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a53e6-d200-4da7-aa74-996456c6aaf9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:51.529035 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e794-475a-440c-82af-ffe100f03d1e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:51.558077 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:51.563735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:51.570306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:51.571212 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.571573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.571952 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.572778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:51.573202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:51.574947 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:51.585809 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 491/967] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:51.586915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:51.597094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 477/968] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:09:51 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 09:09:51.602879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a53e6-d200-4da7-aa74-996456c6aaf9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:51.725714 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa3392-d3fb-4e82-86b4-ad547cb6313c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:51.737729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a53e6-d200-4da7-aa74-996456c6aaf9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:51.768883 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9d059-7b2d-4bfc-8236-d531202820fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:51.769791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9d059-7b2d-4bfc-8236-d531202820fc None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.770409 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9d059-7b2d-4bfc-8236-d531202820fc None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.770875 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9d059-7b2d-4bfc-8236-d531202820fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.772273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9d059-7b2d-4bfc-8236-d531202820fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:51.774157 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9d059-7b2d-4bfc-8236-d531202820fc None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:51.779549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c677f-7351-4f96-8da4-3dd8db92c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:51.780321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c677f-7351-4f96-8da4-3dd8db92c36a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.780723 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c677f-7351-4f96-8da4-3dd8db92c36a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.781272 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c677f-7351-4f96-8da4-3dd8db92c36a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.794820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c677f-7351-4f96-8da4-3dd8db92c36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:51.797305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c677f-7351-4f96-8da4-3dd8db92c36a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:51.804407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 492/969] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:09:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-f33c677f-7351-4f96-8da4-3dd8db92c36a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:09:51.818019 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:51.819219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:51.820195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:09:51.820195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:09:51.820195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:09:51.820195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:51.822748 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 478/970] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:51.828421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9d059-7b2d-4bfc-8236-d531202820fc None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:51.851965 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82ee34-6b00-4739-88ee-685cb81cf332 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.852662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82ee34-6b00-4739-88ee-685cb81cf332 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.853171 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82ee34-6b00-4739-88ee-685cb81cf332 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.855348 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 479/971] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:51.864411 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a887907-dc27-41f7-9f63-52ee8d89efea None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:51.864769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a887907-dc27-41f7-9f63-52ee8d89efea None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:51.865001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a887907-dc27-41f7-9f63-52ee8d89efea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:51.888204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-7a887907-dc27-41f7-9f63-52ee8d89efea None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:51.889472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-7a887907-dc27-41f7-9f63-52ee8d89efea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:51.900918 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:51.917606 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:51.987025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 480/972] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:51.988495 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9d059-7b2d-4bfc-8236-d531202820fc None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:52.011113 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a887907-dc27-41f7-9f63-52ee8d89efea None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:52.032715 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:52.033557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:52.033830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:52.034137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:52.034888 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:52.035206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:52.036645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:52.051583 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 493/973] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:52.057837 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:52.143431 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a887907-dc27-41f7-9f63-52ee8d89efea None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:52.180043 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:52.180768 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:52.181057 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:52.181390 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:52.182087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:52.183734 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:52.186448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: WARNING keystone.server.flask.common [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 09:09:52.187458 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3ae76-5123-4b20-ba46-450268fc74c1 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:52.189131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:52.204497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.sql.core [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 09:09:52.204497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 09:09:52.204497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [parameters: {'user_id': '6231de7989ca46cfa366bb83aaf9ff79', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 09:09:52.204497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=411107) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 09:09:52.228282 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application result.read() Jul 28 09:09:52.228546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:09:52.229652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application yield resource Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise self.value Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application session.commit() Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 09:09:52.230988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self.session.flush() Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application compat.raise_( Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 09:09:52.232224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:09:52.233432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application result.read() Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:09:52.234691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [parameters: {'user_id': '6231de7989ca46cfa366bb83aaf9ff79', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:52.235958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 09:09:52.237166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 09:09:52.238436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 09:09:52.238436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 09:09:52.238436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 09:09:52.238436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 09:09:52.238436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 09:09:52.238436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:52.238890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:52.238890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:52.238890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:52.245316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 481/974] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:09:52 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:52.247168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:52.248245 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:52.249412 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:09:52.266096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:52.267418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:52.268597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:09:52.268597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:09:52.268597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:09:52.268597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:09:52.268597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:09:52.268597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:09:52.268597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:09:52.268597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:52.269188 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:52.271759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 494/975] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:52.280809 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:52.369022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dc16b-f85e-42f4-948e-1c63211a4fae None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:52.397273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70446b-f04b-47eb-9f84-d24f6cdf6d50 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:52.408572 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:52.410068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:52.410637 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:52.411079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:52.421093 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:52.423478 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:09:52.438995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:52.440387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:09:52.441540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:52.449759 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 482/976] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:09:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:52.451330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:52.453644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:52.454404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:52.454692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:52.455018 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:52.456038 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:52.456358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:52.458043 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:52.478668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 495/977] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:09:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:52.480195 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:52.508570 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:52.540462 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:52.570250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432564-6319-4e10-9e3c-8f300d11b6a7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:52.606175 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09594d4c-55f7-45c5-80aa-59c0c1c12fae None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:52.609561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18048547-f233-462e-ab94-8ed539f5a1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:52.610245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048547-f233-462e-ab94-8ed539f5a1e9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:52.610635 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048547-f233-462e-ab94-8ed539f5a1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:52.611022 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048547-f233-462e-ab94-8ed539f5a1e9 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:52.611911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18048547-f233-462e-ab94-8ed539f5a1e9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:52.612326 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18048547-f233-462e-ab94-8ed539f5a1e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:52.613887 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18048547-f233-462e-ab94-8ed539f5a1e9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:52.647079 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ee942-96a8-4873-a844-80c9d5b668ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:52.648002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee942-96a8-4873-a844-80c9d5b668ed None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:52.648504 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee942-96a8-4873-a844-80c9d5b668ed None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:52.648936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee942-96a8-4873-a844-80c9d5b668ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:52.649934 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ee942-96a8-4873-a844-80c9d5b668ed None admin] RBAC: Adding query filter params (role.id=7e05de6606ba48c984ba5aa6420aadbc, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=41037802b8d9420f825cfda7247a52ad) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:52.650457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ee942-96a8-4873-a844-80c9d5b668ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:52.652205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ee942-96a8-4873-a844-80c9d5b668ed None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:52.666639 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 483/978] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:09:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:52.667189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 496/979] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:09:52 2022] GET /identity/v3/role_assignments?user.id=41037802b8d9420f825cfda7247a52ad&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=7e05de6606ba48c984ba5aa6420aadbc => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:53.032513 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0f6f-ea5a-4b46-b878-6aacbd71035d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:53.032833 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0f6f-ea5a-4b46-b878-6aacbd71035d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:53.033101 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0f6f-ea5a-4b46-b878-6aacbd71035d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:53.034871 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 484/980] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:53.040512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcead3-b04d-47e7-90b4-85742b928a17 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:53.040936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcead3-b04d-47e7-90b4-85742b928a17 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:53.041501 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcead3-b04d-47e7-90b4-85742b928a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:53.062137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-fcdcead3-b04d-47e7-90b4-85742b928a17 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:53.063548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-fcdcead3-b04d-47e7-90b4-85742b928a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:53.146970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 497/981] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:53.169492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e0f6f-ea5a-4b46-b878-6aacbd71035d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:53.305816 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e0f6f-ea5a-4b46-b878-6aacbd71035d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:53.339676 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:53.347999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c065dbf-a4ac-4bf8-b1bc-8a3832292a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:53.349042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c065dbf-a4ac-4bf8-b1bc-8a3832292a02 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:53.349170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c065dbf-a4ac-4bf8-b1bc-8a3832292a02 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:53.349534 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c065dbf-a4ac-4bf8-b1bc-8a3832292a02 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:53.350536 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c065dbf-a4ac-4bf8-b1bc-8a3832292a02 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:53.352152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c065dbf-a4ac-4bf8-b1bc-8a3832292a02 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:53.378129 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 485/982] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:09:53 2022] POST /identity/v3/services => generated 242 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:53.935131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7649486-f249-4b25-9aa2-efbd55c3aa1b None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:53.935738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7649486-f249-4b25-9aa2-efbd55c3aa1b None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:53.935738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7649486-f249-4b25-9aa2-efbd55c3aa1b None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:53.937280 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 498/983] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:53.943035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5c2d4-303a-4a3c-84ca-3dcec942e460 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:53.943754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5c2d4-303a-4a3c-84ca-3dcec942e460 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:53.943754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5c2d4-303a-4a3c-84ca-3dcec942e460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:53.963512 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-7fa5c2d4-303a-4a3c-84ca-3dcec942e460 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:53.965077 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-7fa5c2d4-303a-4a3c-84ca-3dcec942e460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:54.054002 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 486/984] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:54.077087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7649486-f249-4b25-9aa2-efbd55c3aa1b None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:54.097905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:54.194784 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7649486-f249-4b25-9aa2-efbd55c3aa1b None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:54.239004 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:54.239683 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:54.239969 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:54.240250 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:54.240978 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:54.242397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:54.252561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:54.253721 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:54.255248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:54.258507 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 499/985] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:54.260011 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5c2d4-303a-4a3c-84ca-3dcec942e460 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:54.372048 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5c2d4-303a-4a3c-84ca-3dcec942e460 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:54.392796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:54.415541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:54.416160 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:54.416451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:54.416720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:54.417420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:54.417732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:54.419180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:54.426634 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 487/986] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:54.430850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:54.524248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:54.547229 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40981aa9-d76b-4a81-95ac-a5f19c43a471 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:54.584769 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe4401-703b-4719-a8bc-e9a5f7c161d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:54.585417 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe4401-703b-4719-a8bc-e9a5f7c161d4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:54.585703 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe4401-703b-4719-a8bc-e9a5f7c161d4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:54.585980 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe4401-703b-4719-a8bc-e9a5f7c161d4 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:54.586714 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe4401-703b-4719-a8bc-e9a5f7c161d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:54.587019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe4401-703b-4719-a8bc-e9a5f7c161d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:54.588486 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe4401-703b-4719-a8bc-e9a5f7c161d4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:54.598304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 500/987] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:54.600787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:54.716041 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5111d-2f05-4294-ad4b-1faa78fe08e8 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:54.775008 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:54.776254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:54.777026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:54.777782 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:54.779722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] RBAC: Adding query filter params (interface=admin, service_id=de88ec364b744026b19f92a47e114181, region_id=RegionOne) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:54.780568 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:54.780838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139adac1-f23a-45fd-9576-87e90aa84bbb None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:54.781092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139adac1-f23a-45fd-9576-87e90aa84bbb None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:54.781385 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139adac1-f23a-45fd-9576-87e90aa84bbb None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:54.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:54.783540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 501/988] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:54.794153 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 488/989] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jul 28 09:09:54 2022] GET /identity/v3/endpoints?service_id=de88ec364b744026b19f92a47e114181&interface=admin®ion_id=RegionOne => generated 193 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:54.794444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6a17-9259-442f-9c14-d05ac5460618 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:54.794444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6a17-9259-442f-9c14-d05ac5460618 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:54.794444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6a17-9259-442f-9c14-d05ac5460618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:54.799879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:54.815606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-e91c6a17-9259-442f-9c14-d05ac5460618 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:54.816957 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-e91c6a17-9259-442f-9c14-d05ac5460618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:54.856139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:54.905556 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 502/990] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:54.916032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d63091-a4b1-447f-9132-5653c35d6ff0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:54.927639 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6a17-9259-442f-9c14-d05ac5460618 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:54.955535 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762a90e-2960-42bb-b4a5-228728f4b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:54.956220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a90e-2960-42bb-b4a5-228728f4b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:54.956499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a90e-2960-42bb-b4a5-228728f4b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:54.956778 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a90e-2960-42bb-b4a5-228728f4b1f4 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:54.957451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762a90e-2960-42bb-b4a5-228728f4b1f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:54.958903 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762a90e-2960-42bb-b4a5-228728f4b1f4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:54.974145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 489/991] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:09:54 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:09:55.028977 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:55.061756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6a17-9259-442f-9c14-d05ac5460618 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:55.103248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:55.103941 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:55.104225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:55.104493 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] PATH_INFO: `/v3/services/image` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:55.105151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:55.106605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:55.116922 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:55.118382 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:09:55.119379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:55.122637 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 503/992] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 09:09:54 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:09:55.125156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762a90e-2960-42bb-b4a5-228728f4b1f4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:55.258219 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762a90e-2960-42bb-b4a5-228728f4b1f4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:55.271289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:55.303591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96838e96-2bd5-4eab-a443-f199840978d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:55.304154 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96838e96-2bd5-4eab-a443-f199840978d5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:55.304482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96838e96-2bd5-4eab-a443-f199840978d5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:55.304735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96838e96-2bd5-4eab-a443-f199840978d5 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:55.305400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96838e96-2bd5-4eab-a443-f199840978d5 None admin] RBAC: Adding query filter params (name=image) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:55.305709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96838e96-2bd5-4eab-a443-f199840978d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:55.307194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96838e96-2bd5-4eab-a443-f199840978d5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:55.318145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 490/993] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:55.322482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:55.441547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0db2f9-c9ab-46cd-9c04-87ef979cea28 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:55.442518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0db2f9-c9ab-46cd-9c04-87ef979cea28 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:55.443001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0db2f9-c9ab-46cd-9c04-87ef979cea28 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:55.444844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 491/994] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:55.445523 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2ddb-b100-411f-97bf-d833fc252425 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:55.450049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c08a4-9651-40bc-a375-21ca651db1da None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:55.450369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c08a4-9651-40bc-a375-21ca651db1da None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:55.450649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c08a4-9651-40bc-a375-21ca651db1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:55.470656 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-d81c08a4-9651-40bc-a375-21ca651db1da None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:55.471858 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-d81c08a4-9651-40bc-a375-21ca651db1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:55.483448 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:55.484115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:55.484410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:55.484674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:55.485386 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] RBAC: Adding query filter params (type=image) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:55.485693 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:55.487133 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:55.492162 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 504/995] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:09:55 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:55.556782 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 492/996] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:55.575168 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:55.578239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:55.693302 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86c8a0-d286-47dd-b610-d496caf55d8e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:55.729430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:55.730553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:55.731063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:55.731547 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] PATH_INFO: `/v3/services/compute` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:55.732385 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:55.733988 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:55.743607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:55.744773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:55.745700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:09:55.746579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 505/997] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:55.752976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c08a4-9651-40bc-a375-21ca651db1da None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:55.862045 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c08a4-9651-40bc-a375-21ca651db1da None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:55.864588 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:55.899582 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:55.900229 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:55.900510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:55.900786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:55.901477 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] RBAC: Adding query filter params (name=compute) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:55.901782 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:55.903204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:55.913182 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 493/998] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:55.914812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:55.995863 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:56.035974 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772ebb5-2870-45cf-aa4d-826f384aa897 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:56.073532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:56.074205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:56.074493 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:56.074772 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:56.075482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] RBAC: Adding query filter params (type=compute) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:56.075788 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:56.077177 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:56.087624 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 506/999] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:56.088643 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:56.194420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66dcbb-30be-4a83-8658-eb8833c07841 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:56.235379 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a067c-f547-4644-91df-62c2400c6afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:56.236017 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a067c-f547-4644-91df-62c2400c6afc None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:56.236300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a067c-f547-4644-91df-62c2400c6afc None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:56.236574 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a067c-f547-4644-91df-62c2400c6afc None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:56.237340 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a067c-f547-4644-91df-62c2400c6afc None admin] RBAC: Adding query filter params (interface=public, service_id=dfc8603e9bf0401aa6de7712a0f4699e, region_id=RegionOne) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:56.237651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a067c-f547-4644-91df-62c2400c6afc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:56.239061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a067c-f547-4644-91df-62c2400c6afc None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:56.249502 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 494/1000] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:09:56 2022] GET /identity/v3/endpoints?service_id=dfc8603e9bf0401aa6de7712a0f4699e&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:56.250406 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:56.356319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed915d-6965-4952-bfb5-be8e658ae197 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:56.385520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:56.393160 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaddaa-7a0f-4a72-8773-4bb9f4bb5662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:56.393804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaddaa-7a0f-4a72-8773-4bb9f4bb5662 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:56.394089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaddaa-7a0f-4a72-8773-4bb9f4bb5662 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:56.394397 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaddaa-7a0f-4a72-8773-4bb9f4bb5662 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:56.395072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaddaa-7a0f-4a72-8773-4bb9f4bb5662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:56.396476 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaddaa-7a0f-4a72-8773-4bb9f4bb5662 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:56.410454 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 507/1001] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:09:56 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:09:57.012578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f2f23-73b6-43f0-bab4-a352340eaaa0 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.012876 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f2f23-73b6-43f0-bab4-a352340eaaa0 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.013169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f2f23-73b6-43f0-bab4-a352340eaaa0 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.014585 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 495/1002] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:57.020717 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472d1e0-0396-49f3-bfbf-31e02255b742 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.020997 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472d1e0-0396-49f3-bfbf-31e02255b742 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.021292 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472d1e0-0396-49f3-bfbf-31e02255b742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.040487 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-8472d1e0-0396-49f3-bfbf-31e02255b742 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:57.041729 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-8472d1e0-0396-49f3-bfbf-31e02255b742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:57.115664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 508/1003] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:57.138226 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f2f23-73b6-43f0-bab4-a352340eaaa0 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:57.173652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:57.259773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f2f23-73b6-43f0-bab4-a352340eaaa0 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:57.303627 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c90a6-9af8-4fa6-80a9-b925f2f1c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:57.304205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c90a6-9af8-4fa6-80a9-b925f2f1c780 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.304493 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c90a6-9af8-4fa6-80a9-b925f2f1c780 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.304772 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c90a6-9af8-4fa6-80a9-b925f2f1c780 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.305411 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c90a6-9af8-4fa6-80a9-b925f2f1c780 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:57.306859 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c90a6-9af8-4fa6-80a9-b925f2f1c780 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-462c90a6-9af8-4fa6-80a9-b925f2f1c780 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:57.316285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:57.317795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:09:57.319170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:57.322225 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 496/1004] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:09:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:57.324596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472d1e0-0396-49f3-bfbf-31e02255b742 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:57.471387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472d1e0-0396-49f3-bfbf-31e02255b742 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:57.514220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e37a-0bca-4c94-b1d5-6db5142fb416 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.514608 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e37a-0bca-4c94-b1d5-6db5142fb416 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.514789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e37a-0bca-4c94-b1d5-6db5142fb416 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.516382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 497/1005] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:57.517447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:57.521993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a31fe-ae24-47d1-b10e-af324ee3e575 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.522753 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a31fe-ae24-47d1-b10e-af324ee3e575 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.522753 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a31fe-ae24-47d1-b10e-af324ee3e575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.531785 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:57.532449 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.532737 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.533018 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.533754 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:57.534070 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:57.535560 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:57.546497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-a19a31fe-ae24-47d1-b10e-af324ee3e575 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:57.546715 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 509/1006] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:57.546852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-a19a31fe-ae24-47d1-b10e-af324ee3e575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:57.548367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:57.629952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 498/1007] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:09:57.652148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a31fe-ae24-47d1-b10e-af324ee3e575 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:57.665443 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adced058-c817-4540-a79f-c7b93f97ed16 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:57.683398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:57.705651 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:57.706328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.706611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.706881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.707567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:57.707871 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:57.709322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:57.714660 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 510/1008] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:09:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:57.720613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:57.774065 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a31fe-ae24-47d1-b10e-af324ee3e575 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:57.779109 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:57.811676 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a291586-6959-4adc-88fa-b6d9a43fd319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:57.812310 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a291586-6959-4adc-88fa-b6d9a43fd319 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.812602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a291586-6959-4adc-88fa-b6d9a43fd319 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.812876 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a291586-6959-4adc-88fa-b6d9a43fd319 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.813681 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a291586-6959-4adc-88fa-b6d9a43fd319 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:57.815111 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a291586-6959-4adc-88fa-b6d9a43fd319 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:57.833870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 499/1009] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:09:57 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:09:57.835137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb03595-1fa3-450b-a427-35713ec935a9 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:57.891784 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f7f71-dbb0-4bcc-b5d4-a045f9ecc1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:57.892362 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f7f71-dbb0-4bcc-b5d4-a045f9ecc1df None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:57.892646 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f7f71-dbb0-4bcc-b5d4-a045f9ecc1df None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:57.892923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f7f71-dbb0-4bcc-b5d4-a045f9ecc1df None admin] PATH_INFO: `/v3/endpoints` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:57.893690 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f7f71-dbb0-4bcc-b5d4-a045f9ecc1df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:57.895269 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f7f71-dbb0-4bcc-b5d4-a045f9ecc1df None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:57.917626 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 511/1010] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:09:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:58.501795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc6565-aa22-4377-8d14-0c5ec2cf0813 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:58.502373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc6565-aa22-4377-8d14-0c5ec2cf0813 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:58.502373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc6565-aa22-4377-8d14-0c5ec2cf0813 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:58.503640 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 500/1011] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:58.509604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4061a-c5dc-41c3-81cd-5af8ca2436eb None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:58.510040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4061a-c5dc-41c3-81cd-5af8ca2436eb None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:58.510279 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4061a-c5dc-41c3-81cd-5af8ca2436eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:58.529516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-f8c4061a-c5dc-41c3-81cd-5af8ca2436eb None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:58.530759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-f8c4061a-c5dc-41c3-81cd-5af8ca2436eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:58.611026 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 512/1012] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:58 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:09:58.634914 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc6565-aa22-4377-8d14-0c5ec2cf0813 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:58.733240 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:58.751883 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc6565-aa22-4377-8d14-0c5ec2cf0813 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:58.788267 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:58.788979 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:58.789270 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:58.789547 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] PATH_INFO: `/v3/services/compute` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:58.790217 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:58.791796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:09:58.801380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:09:58.802538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:09:58.807330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 501/1013] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:09:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:09:58.814157 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4061a-c5dc-41c3-81cd-5af8ca2436eb None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:58.853974 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:58.937023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4061a-c5dc-41c3-81cd-5af8ca2436eb None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:58.980478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:58.981133 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:58.981418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:58.981685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:58.982435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] RBAC: Adding query filter params (name=compute) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:58.982745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:58.984185 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:58.996284 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 513/1014] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:58.997588 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:59.131574 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24781e63-07b7-4f90-912d-9266fe00505c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:59.168869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbd3b4-ebb1-4d15-a816-aebd1e116472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:59.169510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbd3b4-ebb1-4d15-a816-aebd1e116472 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:59.169792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbd3b4-ebb1-4d15-a816-aebd1e116472 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:59.170067 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbd3b4-ebb1-4d15-a816-aebd1e116472 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:59.170788 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbd3b4-ebb1-4d15-a816-aebd1e116472 None admin] RBAC: Adding query filter params (type=compute) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:09:59.171096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbd3b4-ebb1-4d15-a816-aebd1e116472 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:59.172556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbd3b4-ebb1-4d15-a816-aebd1e116472 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:59.177725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 502/1015] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:09:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:09:59.184887 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:59.296467 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:59.301432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fba89e-a797-4a5d-a087-042443cdf027 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:59.338435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b899c1e-5677-43f6-b78b-3c1183c1ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:59.339080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b899c1e-5677-43f6-b78b-3c1183c1ab8a None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:59.339367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b899c1e-5677-43f6-b78b-3c1183c1ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:59.339644 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b899c1e-5677-43f6-b78b-3c1183c1ab8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:59.340386 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b899c1e-5677-43f6-b78b-3c1183c1ab8a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:59.341782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b899c1e-5677-43f6-b78b-3c1183c1ab8a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:59.370146 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 514/1016] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:09:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:09:59.961975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe18f-3719-412f-b3c4-fb9e87f0173d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:59.962722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe18f-3719-412f-b3c4-fb9e87f0173d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:59.962722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe18f-3719-412f-b3c4-fb9e87f0173d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:59.964142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 503/1017] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:59.969784 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff90f2-deb1-4da9-b695-8d6f12e844fa None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:59.970524 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff90f2-deb1-4da9-b695-8d6f12e844fa None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:59.970524 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff90f2-deb1-4da9-b695-8d6f12e844fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:59.974089 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33710a9-a5c9-427c-9d9e-64822dbb259f None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:59.974476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33710a9-a5c9-427c-9d9e-64822dbb259f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:59.974813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33710a9-a5c9-427c-9d9e-64822dbb259f None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:59.976558 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 504/1018] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:09:59.982140 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abdce-6bfd-42a5-82ca-94dc75e5dc8d None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:59.982447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abdce-6bfd-42a5-82ca-94dc75e5dc8d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:59.982729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abdce-6bfd-42a5-82ca-94dc75e5dc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:59.991545 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-c1ff90f2-deb1-4da9-b695-8d6f12e844fa None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:59.992763 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-c1ff90f2-deb1-4da9-b695-8d6f12e844fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:00.004123 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-a90abdce-6bfd-42a5-82ca-94dc75e5dc8d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:00.005793 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-a90abdce-6bfd-42a5-82ca-94dc75e5dc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:00.017448 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:00.087121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 515/1019] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:10:00.102825 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 505/1020] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:09:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:00.112101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff90f2-deb1-4da9-b695-8d6f12e844fa None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:00.126774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90abdce-6bfd-42a5-82ca-94dc75e5dc8d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:00.221231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:00.246364 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90abdce-6bfd-42a5-82ca-94dc75e5dc8d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:00.249719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff90f2-deb1-4da9-b695-8d6f12e844fa None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:00.286764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e61f4-a4f8-491f-baa8-63d01a38a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:00.287578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e61f4-a4f8-491f-baa8-63d01a38a32c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:00.287934 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e61f4-a4f8-491f-baa8-63d01a38a32c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:00.288040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:00.288221 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e61f4-a4f8-491f-baa8-63d01a38a32c None admin] PATH_INFO: `/v3/domains/default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:00.288621 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:00.288901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:00.289191 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] PATH_INFO: `/v3/services/image` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:00.289889 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:00.290002 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e61f4-a4f8-491f-baa8-63d01a38a32c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:00.291329 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:00.291488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e61f4-a4f8-491f-baa8-63d01a38a32c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:00.293640 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 506/1021] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:00.300349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e61f4-a4f8-491f-baa8-63d01a38a32c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:00.306552 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:00.307741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:10:00.308713 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:00.313567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 516/1022] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 09:10:00 2022] GET /identity/v3/services/image => generated 86 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:00.314685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:00.319784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:00.427650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e61f4-a4f8-491f-baa8-63d01a38a32c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:00.435190 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47de75b-def2-43ad-bbc5-7b544f2f703e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:00.468195 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2b690-0129-49a4-a7f9-f14eae51759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:00.469023 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2b690-0129-49a4-a7f9-f14eae51759a None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:00.469311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2b690-0129-49a4-a7f9-f14eae51759a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:00.469920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2b690-0129-49a4-a7f9-f14eae51759a None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:00.470699 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2b690-0129-49a4-a7f9-f14eae51759a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:00.472285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2b690-0129-49a4-a7f9-f14eae51759a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:00.482905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:00.483556 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:00.483842 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:00.484116 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:00.484944 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] RBAC: Adding query filter params (name=image) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:00.485408 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:00.487208 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:00.499819 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 517/1023] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:00.500973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:00.513155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:00.522330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 507/1024] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:10:00 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:10:00.622512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505558-ce3d-4783-913a-844fdb9c9fa4 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:00.661450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:00.662383 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:00.662678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:00.662979 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:00.663705 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] RBAC: Adding query filter params (type=image) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:00.664011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:00.665435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:00.671506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 518/1025] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:00.677750 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2b690-0129-49a4-a7f9-f14eae51759a None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:00.747758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:00.790325 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2b690-0129-49a4-a7f9-f14eae51759a None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:00.825709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f35c4-12d4-4ffc-a44c-6ebd6a11e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:00.826387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f35c4-12d4-4ffc-a44c-6ebd6a11e680 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:00.826671 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f35c4-12d4-4ffc-a44c-6ebd6a11e680 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:00.826942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f35c4-12d4-4ffc-a44c-6ebd6a11e680 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:00.827729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f35c4-12d4-4ffc-a44c-6ebd6a11e680 None admin] RBAC: Adding query filter params (interface=public, service_id=f3fca7962c794509908a0222b02614fe, region_id=RegionOne) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:00.828041 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f35c4-12d4-4ffc-a44c-6ebd6a11e680 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:00.829455 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f35c4-12d4-4ffc-a44c-6ebd6a11e680 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:00.839663 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 508/1026] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:10:00 2022] GET /identity/v3/endpoints?service_id=f3fca7962c794509908a0222b02614fe&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:00.841380 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:00.917923 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:00.951973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff96507-f2e6-4ab8-ba31-f42efc27d947 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:00.994143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3722dc9-627a-4314-8758-2d7112e5271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:00.994829 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722dc9-627a-4314-8758-2d7112e5271f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:00.995134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722dc9-627a-4314-8758-2d7112e5271f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:00.995359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722dc9-627a-4314-8758-2d7112e5271f None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:00.996013 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3722dc9-627a-4314-8758-2d7112e5271f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:00.997430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3722dc9-627a-4314-8758-2d7112e5271f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:01.004800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 519/1027] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:10:00 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:10:01.545853 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484bf5e-0153-44d2-972d-3c724db0fdde None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:01.546457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484bf5e-0153-44d2-972d-3c724db0fdde None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:01.546457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484bf5e-0153-44d2-972d-3c724db0fdde None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:01.548001 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 509/1028] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:01.553534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75953da4-a2b1-42f1-8ffc-963a0da0aaff None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:01.553890 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75953da4-a2b1-42f1-8ffc-963a0da0aaff None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:01.554317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75953da4-a2b1-42f1-8ffc-963a0da0aaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:01.573961 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-75953da4-a2b1-42f1-8ffc-963a0da0aaff None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:01.575205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-75953da4-a2b1-42f1-8ffc-963a0da0aaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:01.647334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 520/1029] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:01.669846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484bf5e-0153-44d2-972d-3c724db0fdde None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:01.782776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484bf5e-0153-44d2-972d-3c724db0fdde None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:01.809845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:01.819692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdfdde-18a4-41c4-9d58-51613655cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:01.820354 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdfdde-18a4-41c4-9d58-51613655cb0d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:01.820641 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdfdde-18a4-41c4-9d58-51613655cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:01.820915 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdfdde-18a4-41c4-9d58-51613655cb0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:01.831540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdfdde-18a4-41c4-9d58-51613655cb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-c9fdfdde-18a4-41c4-9d58-51613655cb0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:01.836706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:01.837796 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:01.844341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 510/1030] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:10:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:01.845109 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75953da4-a2b1-42f1-8ffc-963a0da0aaff None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:01.911229 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:01.949469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75953da4-a2b1-42f1-8ffc-963a0da0aaff None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:01.987287 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:01.987913 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:01.988201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:01.988477 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:01.989219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:01.989531 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:01.991005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:02.000034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 521/1031] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:10:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:02.006146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdfdde-18a4-41c4-9d58-51613655cb0d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:02.116442 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdfdde-18a4-41c4-9d58-51613655cb0d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:02.153628 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:02.154316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:02.154603 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:02.154878 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:02.165916 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:02.167419 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:10:02.182647 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:02.183790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:02.184814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:10:02.184814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:10:02.184814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:02.184814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:02.186468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 511/1032] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:02.189877 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:02.301919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa93cd0-4d2d-43d8-8844-0ee9731f879c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:02.330061 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:02.343630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f648e-5838-4652-ac3a-5d24129b43ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:02.344277 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f648e-5838-4652-ac3a-5d24129b43ce None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:02.344604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f648e-5838-4652-ac3a-5d24129b43ce None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:02.344861 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f648e-5838-4652-ac3a-5d24129b43ce None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:02.345575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f648e-5838-4652-ac3a-5d24129b43ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:02.345878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f648e-5838-4652-ac3a-5d24129b43ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:02.347337 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f648e-5838-4652-ac3a-5d24129b43ce None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:02.360646 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 522/1033] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:02.366895 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:02.419078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:02.510481 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a075ea-8e3f-4310-a662-af14f187e773 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:02.548901 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:02.549634 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:02.549993 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:02.550231 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] PATH_INFO: `/v3/users/nova` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:02.558792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:02.560273 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:10:02.573029 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:02.579388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:02.580737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 512/1034] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:10:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:02.581574 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a9c3b-384f-42df-8284-ceee9194f659 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:02.581574 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a9c3b-384f-42df-8284-ceee9194f659 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:02.581574 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a9c3b-384f-42df-8284-ceee9194f659 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:02.583244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 523/1035] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:02.583382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:02.588648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d599a4-eb3e-46a5-a1bd-2d35292314ba None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:02.589267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d599a4-eb3e-46a5-a1bd-2d35292314ba None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:02.589267 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d599a4-eb3e-46a5-a1bd-2d35292314ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:02.607642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-b9d599a4-eb3e-46a5-a1bd-2d35292314ba None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:02.608889 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-b9d599a4-eb3e-46a5-a1bd-2d35292314ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:02.690165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 524/1036] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:10:02.697514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:02.700925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b1596-eb4f-46e7-a68f-6dc0bab8904c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:02.717276 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d599a4-eb3e-46a5-a1bd-2d35292314ba None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:02.739192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:02.739840 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:02.740121 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:02.740394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:02.741211 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:02.741518 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:02.743027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:02.764799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 513/1037] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:10:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:02.770550 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:02.841477 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d599a4-eb3e-46a5-a1bd-2d35292314ba None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:02.863586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:02.889551 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cd55-43f4-4f4a-8870-743a9d797156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:02.890294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cd55-43f4-4f4a-8870-743a9d797156 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:02.890586 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cd55-43f4-4f4a-8870-743a9d797156 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:02.891118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cd55-43f4-4f4a-8870-743a9d797156 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:02.892092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123cd55-43f4-4f4a-8870-743a9d797156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:02.893596 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123cd55-43f4-4f4a-8870-743a9d797156 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:02.894722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982531c-ada3-4d71-8671-1bc2dcf7a725 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:02.895348 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 525/1038] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:02.901670 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cd55-43f4-4f4a-8870-743a9d797156 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:02.934321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efb2a3-fc13-4e9d-ada0-9ed73a5e7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:02.935294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb2a3-fc13-4e9d-ada0-9ed73a5e7a24 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:02.935758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb2a3-fc13-4e9d-ada0-9ed73a5e7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:02.936205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb2a3-fc13-4e9d-ada0-9ed73a5e7a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:02.970200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:03.018634 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cd55-43f4-4f4a-8870-743a9d797156 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.022956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efb2a3-fc13-4e9d-ada0-9ed73a5e7a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.024684 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efb2a3-fc13-4e9d-ada0-9ed73a5e7a24 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.061195 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154be764-885f-4a52-b283-f269785a9f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.061790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154be764-885f-4a52-b283-f269785a9f24 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.062095 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154be764-885f-4a52-b283-f269785a9f24 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.062423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154be764-885f-4a52-b283-f269785a9f24 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.063248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154be764-885f-4a52-b283-f269785a9f24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.064571 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154be764-885f-4a52-b283-f269785a9f24 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.089082 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 526/1039] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:10:02 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:10:03.095855 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36b92d-4d6b-4b65-80f7-2b09ea1ff3e6 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.096178 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36b92d-4d6b-4b65-80f7-2b09ea1ff3e6 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.096489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36b92d-4d6b-4b65-80f7-2b09ea1ff3e6 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.098204 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 527/1040] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:03.105150 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dddc8b-9de8-4942-8e13-33a6b6f08757 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.105440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dddc8b-9de8-4942-8e13-33a6b6f08757 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.106041 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dddc8b-9de8-4942-8e13-33a6b6f08757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.129446 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-48dddc8b-9de8-4942-8e13-33a6b6f08757 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:03.131225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-48dddc8b-9de8-4942-8e13-33a6b6f08757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:03.162866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 514/1041] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:02 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 09:10:03.169191 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efb2a3-fc13-4e9d-ada0-9ed73a5e7a24 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.180009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:03.230236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 528/1042] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:03.251049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:03.254873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dddc8b-9de8-4942-8e13-33a6b6f08757 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.306259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efb2a3-fc13-4e9d-ada0-9ed73a5e7a24 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.348874 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.349581 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.349908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.350266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.362576 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.364084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.378346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dddc8b-9de8-4942-8e13-33a6b6f08757 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:10:03.379919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:03.381051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:03.383501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:10:03.383501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:10:03.383501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:03.383501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:03.383501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 515/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:03.387083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.418186 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.418870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.419157 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.419465 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] PATH_INFO: `/v3/services/image` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.420147 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.421553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:03.430935 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:03.432169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:03.433159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 529/1044] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 09:10:03 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:10:03.438224 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.477840 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:03.510227 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8239b5d-804f-4494-8784-d3178df42e1c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.523833 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:03.549488 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15568175-298c-4ad3-be15-9ddc2fd4e48d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.549943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.550643 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.550998 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.551285 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.552026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:03.552338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.553779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.566952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 516/1045] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:03.573084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.589814 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.590499 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.590790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.591071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.591768 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] RBAC: Adding query filter params (name=image) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:03.592074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.593501 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.599143 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 530/1046] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:03.605203 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.683047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811de771-095a-4eca-9f43-34bf69e6d007 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.722846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.723858 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.724142 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.724310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:03.724444 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.727847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb8261-fb50-41ad-86f2-fc128672190f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.732172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.733662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:10:03.745651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:03.747013 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:03.748173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 517/1047] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:10:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:10:03.752880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.769545 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.770169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.770405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.770685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.771504 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] RBAC: Adding query filter params (type=image) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:03.771830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.773220 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.782146 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 531/1048] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:03.784159 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.788923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:03.864350 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77889ff2-b2ff-49df-a6ac-d9b15f81dccc None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.897643 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cadf7-e819-4ac2-b4a1-7786bbc28524 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:03.905420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.906082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.906775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.906775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.907808 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:03.908118 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.910243 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.931746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 518/1049] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:10:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:03.932660 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43baa3b-c80e-4a07-82e0-75485ef3f838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:03.933327 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:03.933604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43baa3b-c80e-4a07-82e0-75485ef3f838 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:03.933895 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43baa3b-c80e-4a07-82e0-75485ef3f838 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:03.934332 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43baa3b-c80e-4a07-82e0-75485ef3f838 None admin] PATH_INFO: `/v3/endpoints` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:03.935072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43baa3b-c80e-4a07-82e0-75485ef3f838 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:03.936469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43baa3b-c80e-4a07-82e0-75485ef3f838 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:03.958221 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 532/1050] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:10:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:10:04.048506 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:04.060824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd683f-1cdb-4ccd-8288-257e2c4743bb None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:04.107806 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ecbb36-03ca-4b03-9a7c-5bac516f3d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:04.108455 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecbb36-03ca-4b03-9a7c-5bac516f3d2d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:04.108750 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecbb36-03ca-4b03-9a7c-5bac516f3d2d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:04.109011 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecbb36-03ca-4b03-9a7c-5bac516f3d2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:04.109820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ecbb36-03ca-4b03-9a7c-5bac516f3d2d None admin] RBAC: Adding query filter params (role.id=7e05de6606ba48c984ba5aa6420aadbc, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=1e579a84d5e34a828c5981f34b458a0d) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:04.110159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ecbb36-03ca-4b03-9a7c-5bac516f3d2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:04.111717 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ecbb36-03ca-4b03-9a7c-5bac516f3d2d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:04.118930 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 519/1051] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:03 2022] GET /identity/v3/role_assignments?user.id=1e579a84d5e34a828c5981f34b458a0d&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=7e05de6606ba48c984ba5aa6420aadbc => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:05.175869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec7d7-2c3c-4043-9509-f63a8b48ace2 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:05.176433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec7d7-2c3c-4043-9509-f63a8b48ace2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:05.176433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec7d7-2c3c-4043-9509-f63a8b48ace2 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:05.177839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 533/1052] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:05.183698 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2cec1-aaa3-4af3-b1fc-0e1017a8501f None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:05.184162 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2cec1-aaa3-4af3-b1fc-0e1017a8501f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:05.184261 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2cec1-aaa3-4af3-b1fc-0e1017a8501f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:05.206799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-e9e2cec1-aaa3-4af3-b1fc-0e1017a8501f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:05.208334 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-e9e2cec1-aaa3-4af3-b1fc-0e1017a8501f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:05.291670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 520/1053] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:05.315564 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ec7d7-2c3c-4043-9509-f63a8b48ace2 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:05.339591 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:05.432323 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ec7d7-2c3c-4043-9509-f63a8b48ace2 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:05.469432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe8c85-cef5-4444-90a7-ff082573ff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:05.469940 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe8c85-cef5-4444-90a7-ff082573ff90 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:05.470286 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe8c85-cef5-4444-90a7-ff082573ff90 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:05.470511 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe8c85-cef5-4444-90a7-ff082573ff90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:05.482344 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe8c85-cef5-4444-90a7-ff082573ff90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-3cfe8c85-cef5-4444-90a7-ff082573ff90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:05.487399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:05.488452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:05.489799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:05.494420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 534/1054] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:05.495369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2cec1-aaa3-4af3-b1fc-0e1017a8501f None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:05.587988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:05.612702 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2cec1-aaa3-4af3-b1fc-0e1017a8501f None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:05.654479 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:05.655131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:05.655419 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:05.655704 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:05.656440 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:05.656746 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:05.658164 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:05.674220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 521/1055] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:05.676699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe8c85-cef5-4444-90a7-ff082573ff90 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:05.770294 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:05.793969 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe8c85-cef5-4444-90a7-ff082573ff90 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:05.831692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d0be3-5dfa-4361-9918-85fc99a9a7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:05.832393 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d0be3-5dfa-4361-9918-85fc99a9a7b0 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:05.832676 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d0be3-5dfa-4361-9918-85fc99a9a7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:05.832959 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d0be3-5dfa-4361-9918-85fc99a9a7b0 None admin] PATH_INFO: `/v3/users/619965dfabaf452b9c2806054b868111` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:05.849423 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d0be3-5dfa-4361-9918-85fc99a9a7b0 None admin] RBAC: Authorizing `identity:get_user(user_id=619965dfabaf452b9c2806054b868111)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:05.851619 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d0be3-5dfa-4361-9918-85fc99a9a7b0 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:05.854014 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 535/1056] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:05 2022] GET /identity/v3/users/619965dfabaf452b9c2806054b868111 => generated 285 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:05.860451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:05.992855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d94fa4-b744-4d29-90ba-213465737302 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:06.024411 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d91bc8-53c3-4a61-ad4d-16ce1eff1311 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.024814 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d91bc8-53c3-4a61-ad4d-16ce1eff1311 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.025106 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d91bc8-53c3-4a61-ad4d-16ce1eff1311 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.026734 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 536/1057] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:06.032584 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5004f4-dc51-46c1-98b9-505f2c5acf25 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.032944 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5004f4-dc51-46c1-98b9-505f2c5acf25 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.033299 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5004f4-dc51-46c1-98b9-505f2c5acf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.036776 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efaa1-f413-4cda-b65c-8b16b947f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:06.037497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efaa1-f413-4cda-b65c-8b16b947f40f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.037807 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efaa1-f413-4cda-b65c-8b16b947f40f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.038170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efaa1-f413-4cda-b65c-8b16b947f40f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.054032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:06.059983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-fe5004f4-dc51-46c1-98b9-505f2c5acf25 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:06.061219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-fe5004f4-dc51-46c1-98b9-505f2c5acf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:06.136003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13efaa1-f413-4cda-b65c-8b16b947f40f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:06.136351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13efaa1-f413-4cda-b65c-8b16b947f40f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:06.141253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 537/1058] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:06.143516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405bb0-0449-404a-91dd-d5796e27d54d None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.143809 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405bb0-0449-404a-91dd-d5796e27d54d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.144090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405bb0-0449-404a-91dd-d5796e27d54d None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.145550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 538/1059] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:06.152936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f779e2-cbbe-4927-a053-f36662eff59d None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.153243 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f779e2-cbbe-4927-a053-f36662eff59d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.153497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f779e2-cbbe-4927-a053-f36662eff59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.175750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-53f779e2-cbbe-4927-a053-f36662eff59d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:06.177482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-53f779e2-cbbe-4927-a053-f36662eff59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:06.255157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 522/1060] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:10:06.256099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efaa1-f413-4cda-b65c-8b16b947f40f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:06.256253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:06.273368 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 539/1061] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:10:06.274749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f779e2-cbbe-4927-a053-f36662eff59d None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:06.307222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:06.373168 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efaa1-f413-4cda-b65c-8b16b947f40f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:06.394055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f779e2-cbbe-4927-a053-f36662eff59d None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:06.412710 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e4d3-3f7e-41ae-816c-d69426bf2a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:06.413392 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e4d3-3f7e-41ae-816c-d69426bf2a42 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.413677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e4d3-3f7e-41ae-816c-d69426bf2a42 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.414003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e4d3-3f7e-41ae-816c-d69426bf2a42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.443171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d7d65-c52b-45dc-94c7-fb63c366aaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:06.443946 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d7d65-c52b-45dc-94c7-fb63c366aaf3 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.444270 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d7d65-c52b-45dc-94c7-fb63c366aaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.444563 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d7d65-c52b-45dc-94c7-fb63c366aaf3 None admin] PATH_INFO: `/v3/projects/da56673001f545aea09c272274c5da4d` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.445992 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d7d65-c52b-45dc-94c7-fb63c366aaf3 None admin] RBAC: Authorizing `identity:get_project(project_id=da56673001f545aea09c272274c5da4d)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:06.447567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d7d65-c52b-45dc-94c7-fb63c366aaf3 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:06.449331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 540/1062] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:06 2022] GET /identity/v3/projects/da56673001f545aea09c272274c5da4d => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:06.450149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d7d65-c52b-45dc-94c7-fb63c366aaf3 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:06.500451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017e4d3-3f7e-41ae-816c-d69426bf2a42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:06.501970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017e4d3-3f7e-41ae-816c-d69426bf2a42 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:06.528841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:06.547499 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:06.574114 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d7d65-c52b-45dc-94c7-fb63c366aaf3 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:06.614544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 523/1063] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:10:06.615593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e4d3-3f7e-41ae-816c-d69426bf2a42 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:06.621659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b095738-911f-42bd-b5b5-f30c3004e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:06.622355 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b095738-911f-42bd-b5b5-f30c3004e01c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.622659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b095738-911f-42bd-b5b5-f30c3004e01c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.622939 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b095738-911f-42bd-b5b5-f30c3004e01c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.635715 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b095738-911f-42bd-b5b5-f30c3004e01c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-4b095738-911f-42bd-b5b5-f30c3004e01c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:06.640786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:06.641984 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:06.642971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 541/1064] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:10:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:06.643771 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b095738-911f-42bd-b5b5-f30c3004e01c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:06.738501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e4d3-3f7e-41ae-816c-d69426bf2a42 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:06.760021 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b095738-911f-42bd-b5b5-f30c3004e01c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:06.777657 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:06.778333 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.778622 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.778957 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.779700 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=da56673001f545aea09c272274c5da4d, user.id=619965dfabaf452b9c2806054b868111) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:06.780014 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:06.781471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:06.788122 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:06.798197 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 524/1065] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:06 2022] GET /identity/v3/role_assignments?user.id=619965dfabaf452b9c2806054b868111&scope.project.id=da56673001f545aea09c272274c5da4d&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:06.798600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:06.799924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.800109 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:06.800310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:06.800310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:06.810447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:06.817489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:06.819324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:10:06.839914 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:06.841022 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:06.842346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:10:06.842346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:10:06.842346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:06.842346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:06.842346 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 542/1066] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:06.848942 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:06.943717 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc517c0-d04e-4d55-a76a-263d4eecb01e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:06.986719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292f29e-dd35-4181-8cee-3f4b4949fc15 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:06.998885 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:06.999542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:06.999814 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:07.000106 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:07.000847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:07.001144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:07.002588 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:07.013821 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 525/1067] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:10:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:07.021523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:07.028320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:07.029052 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:07.029292 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:07.029554 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:07.030270 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:07.030588 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:07.032007 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:07.058369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 543/1068] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:07.125599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:07.180051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3046ef-f764-465e-ab55-4bb02388b126 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:07.239311 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:07.240371 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:07.240371 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:07.240739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:07.254639 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:07.257679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:10:07.282519 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:07.283751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:07.284792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:10:07.284792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:10:07.284792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:07.284792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:07.286265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 526/1069] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:10:07.292223 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:07.356137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:07.440930 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e49f3-6527-408e-8e45-4b1003258774 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:07.481270 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:07.482036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:07.482360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:07.482768 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:07.483497 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:07.483812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:07.485244 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:07.504750 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 544/1070] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:07.511561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:07.637774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c20c1d-c616-4827-a6b9-8a034be637e7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:07.661610 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:07.679621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:07.680282 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:07.680566 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:07.680845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] PATH_INFO: `/v3/users/nova` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:07.689253 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:07.690778 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:10:07.707332 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:07.709511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:07.711056 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 527/1071] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:10:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:07.718573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:07.826898 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:07.832392 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e39da6-520f-40a9-b5cc-0de6a1c1a60e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:07.878012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:07.878700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:07.878989 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:07.879273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:07.880098 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:07.880413 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:07.882028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:07.906480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 545/1072] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:10:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:07.917038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:08.047685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb606d-d6ed-42b7-89d9-95a1987b9fec None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:08.090811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:08.091494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:08.091787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:08.092069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:08.104983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:08.106739 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:10:08.124625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:08.125869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:08.127222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:10:08.127222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:10:08.127222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:08.127222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:08.127222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 528/1073] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:10:08.133477 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:08.296405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d24d3-d61f-456a-b5e7-63631f2cf3f1 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:08.334482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:08.335214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:08.335506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:08.335773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:08.336685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:08.337003 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:08.338444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:08.351332 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 546/1074] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:08.361578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:08.425296 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:08.576514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9068-9cd2-410d-92e6-b90abcd3a3f9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:08.642682 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:08.643765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:08.644266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:08.644747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:08.658621 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:08.661099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:10:08.675105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:08.677760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:08.680141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 529/1075] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:10:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:08.684194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:08.709155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:08.808018 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ee92d-fe66-4f4a-b000-500dbe9c4d8d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:08.851659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c4748-2bb8-40af-beac-391d687400f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:08.852326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4748-2bb8-40af-beac-391d687400f4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:08.852753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4748-2bb8-40af-beac-391d687400f4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:08.853137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4748-2bb8-40af-beac-391d687400f4 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:08.854083 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c4748-2bb8-40af-beac-391d687400f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:08.854577 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c4748-2bb8-40af-beac-391d687400f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:08.856209 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c4748-2bb8-40af-beac-391d687400f4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:08.872036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 547/1076] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:10:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:08.879161 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:08.921160 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d974e-c383-4885-86ab-f2eb830d3f13 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:08.921648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d974e-c383-4885-86ab-f2eb830d3f13 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:08.922051 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d974e-c383-4885-86ab-f2eb830d3f13 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:08.926152 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 548/1077] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:10:08.930324 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fa1ce-1874-46d6-8308-baa7e21ccb29 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:08.930844 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fa1ce-1874-46d6-8308-baa7e21ccb29 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:08.931122 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fa1ce-1874-46d6-8308-baa7e21ccb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:08.957851 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-983fa1ce-1874-46d6-8308-baa7e21ccb29 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:08.960158 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-983fa1ce-1874-46d6-8308-baa7e21ccb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:08.993198 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:09.005541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8b46b-e126-48e5-8772-1162bc409b9c None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:09.037880 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 549/1078] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:10:09.043703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ca465-9b8c-4e9b-b60f-b8b840119618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:09.044395 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ca465-9b8c-4e9b-b60f-b8b840119618 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:09.044679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ca465-9b8c-4e9b-b60f-b8b840119618 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:09.044955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ca465-9b8c-4e9b-b60f-b8b840119618 None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/1e579a84d5e34a828c5981f34b458a0d/roles/7e05de6606ba48c984ba5aa6420aadbc` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:09.047271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ca465-9b8c-4e9b-b60f-b8b840119618 None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=1e579a84d5e34a828c5981f34b458a0d, role_id=7e05de6606ba48c984ba5aa6420aadbc)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:09.048764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ca465-9b8c-4e9b-b60f-b8b840119618 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:09.061395 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fa1ce-1874-46d6-8308-baa7e21ccb29 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:09.063589 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 530/1079] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:10:08 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/1e579a84d5e34a828c5981f34b458a0d/roles/7e05de6606ba48c984ba5aa6420aadbc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:10:09.072181 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:09.194891 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fa1ce-1874-46d6-8308-baa7e21ccb29 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:09.244847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590c48d-993f-4dc6-89e2-e8a35b24c7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:09.245308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590c48d-993f-4dc6-89e2-e8a35b24c7a2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:09.245768 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590c48d-993f-4dc6-89e2-e8a35b24c7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:09.246268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590c48d-993f-4dc6-89e2-e8a35b24c7a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:09.258228 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590c48d-993f-4dc6-89e2-e8a35b24c7a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-b590c48d-993f-4dc6-89e2-e8a35b24c7a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:09.263275 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:09.265351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:09.267180 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 550/1080] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:10:09.270973 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ca465-9b8c-4e9b-b60f-b8b840119618 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:09.377982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ca465-9b8c-4e9b-b60f-b8b840119618 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:09.413875 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:09.414678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:09.414952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:09.414952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:09.415994 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:09.416323 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:09.417679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:09.428449 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 531/1081] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:09.434883 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590c48d-993f-4dc6-89e2-e8a35b24c7a2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:09.507068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:09.546676 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590c48d-993f-4dc6-89e2-e8a35b24c7a2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:09.588202 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea460c-ab8f-44f5-b5e5-30bdebe0215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:09.588879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea460c-ab8f-44f5-b5e5-30bdebe0215b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:09.589167 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea460c-ab8f-44f5-b5e5-30bdebe0215b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:09.589446 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea460c-ab8f-44f5-b5e5-30bdebe0215b None admin] PATH_INFO: `/v3/users/619965dfabaf452b9c2806054b868111` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:09.590723 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea460c-ab8f-44f5-b5e5-30bdebe0215b None admin] RBAC: Authorizing `identity:get_user(user_id=619965dfabaf452b9c2806054b868111)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:09.592200 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea460c-ab8f-44f5-b5e5-30bdebe0215b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:09.593907 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 551/1082] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:09 2022] GET /identity/v3/users/619965dfabaf452b9c2806054b868111 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:09.600091 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:09.618883 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:09.718412 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9efe8-7447-4a62-a4ca-287a3e4c53fb None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:09.757282 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8aba-e0f6-4d2f-b20e-48407617e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:09.757860 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8aba-e0f6-4d2f-b20e-48407617e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:09.758180 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8aba-e0f6-4d2f-b20e-48407617e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:09.758464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8aba-e0f6-4d2f-b20e-48407617e6e3 None admin] PATH_INFO: `/v3/projects/da56673001f545aea09c272274c5da4d` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:09.759675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf8aba-e0f6-4d2f-b20e-48407617e6e3 None admin] RBAC: Authorizing `identity:get_project(project_id=da56673001f545aea09c272274c5da4d)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:09.761130 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf8aba-e0f6-4d2f-b20e-48407617e6e3 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:09.762926 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 532/1083] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:09 2022] GET /identity/v3/projects/da56673001f545aea09c272274c5da4d => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:09.769696 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea460c-ab8f-44f5-b5e5-30bdebe0215b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:09.951488 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea460c-ab8f-44f5-b5e5-30bdebe0215b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:10.028791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c201596-590b-461b-9025-a321aad5991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:10.030752 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c201596-590b-461b-9025-a321aad5991f None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:10.031438 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c201596-590b-461b-9025-a321aad5991f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:10.031906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c201596-590b-461b-9025-a321aad5991f None admin] PATH_INFO: `/v3/projects/da56673001f545aea09c272274c5da4d/users/619965dfabaf452b9c2806054b868111/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:10.037627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c201596-590b-461b-9025-a321aad5991f None admin] RBAC: Authorizing `identity:create_grant(project_id=da56673001f545aea09c272274c5da4d, user_id=619965dfabaf452b9c2806054b868111, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:10.041550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c201596-590b-461b-9025-a321aad5991f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:10.045609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:10.064058 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 552/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:10:09 2022] PUT /identity/v3/projects/da56673001f545aea09c272274c5da4d/users/619965dfabaf452b9c2806054b868111/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:10:11.137662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d469d-f673-4656-8d31-d9f7f1a122ca None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:11.138339 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d469d-f673-4656-8d31-d9f7f1a122ca None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:11.138893 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d469d-f673-4656-8d31-d9f7f1a122ca None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:11.141353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 533/1085] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:11.149986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e229-88a1-4ad1-bd76-d0e967633834 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:11.150543 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e229-88a1-4ad1-bd76-d0e967633834 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:11.151017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e229-88a1-4ad1-bd76-d0e967633834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:11.181970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-3689e229-88a1-4ad1-bd76-d0e967633834 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:11.184125 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-3689e229-88a1-4ad1-bd76-d0e967633834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:11.287798 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 553/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:11.326598 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d469d-f673-4656-8d31-d9f7f1a122ca None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:11.450665 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:11.474404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d469d-f673-4656-8d31-d9f7f1a122ca None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:11.513503 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c5630-a67c-4d86-a7a1-2d387779085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:11.514223 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c5630-a67c-4d86-a7a1-2d387779085d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:11.514510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c5630-a67c-4d86-a7a1-2d387779085d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:11.514785 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c5630-a67c-4d86-a7a1-2d387779085d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:11.526812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c5630-a67c-4d86-a7a1-2d387779085d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-8e1c5630-a67c-4d86-a7a1-2d387779085d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:11.531820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:11.533706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:11.535911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 534/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:10:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:11.541362 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689e229-88a1-4ad1-bd76-d0e967633834 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:11.654575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689e229-88a1-4ad1-bd76-d0e967633834 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:11.683420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:11.696943 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:11.697632 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:11.697921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:11.698241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:11.699050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:11.699371 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:11.700882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:11.712574 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 554/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:10:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:11.719289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c5630-a67c-4d86-a7a1-2d387779085d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:11.839072 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c5630-a67c-4d86-a7a1-2d387779085d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:11.880447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:11.881250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:11.881540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:11.881816 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:11.895857 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:11.897467 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:11.899913 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:10:11.917629 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:11.919784 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:11.921959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:10:11.921959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:10:11.921959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:11.921959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:11.921959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 535/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:11.925924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:12.053584 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a7e35-bfd2-439e-8926-14b05f1f69c4 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:12.091735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404cd74-9dd3-4443-ba7d-d1c5e7161732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:12.092375 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404cd74-9dd3-4443-ba7d-d1c5e7161732 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.092663 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404cd74-9dd3-4443-ba7d-d1c5e7161732 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.092945 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404cd74-9dd3-4443-ba7d-d1c5e7161732 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.093677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404cd74-9dd3-4443-ba7d-d1c5e7161732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:12.093993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404cd74-9dd3-4443-ba7d-d1c5e7161732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:12.095419 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404cd74-9dd3-4443-ba7d-d1c5e7161732 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:12.108892 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 555/1090] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:12.114917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:12.196283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b4419-0c24-4e61-b79c-5433b2d4e3bd None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.196897 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b4419-0c24-4e61-b79c-5433b2d4e3bd None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.196897 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b4419-0c24-4e61-b79c-5433b2d4e3bd None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.198246 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 556/1091] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:10:12.203968 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdbe69-3342-43e1-ab58-9bef5c51c8a4 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.204305 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdbe69-3342-43e1-ab58-9bef5c51c8a4 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.204530 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdbe69-3342-43e1-ab58-9bef5c51c8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.227288 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-0bfdbe69-3342-43e1-ab58-9bef5c51c8a4 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:12.228547 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-0bfdbe69-3342-43e1-ab58-9bef5c51c8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:12.240320 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df82c4-c2d8-45da-b35e-e596a1b210a4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:12.278670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:12.279355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.279645 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.279773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:12.279925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] PATH_INFO: `/v3/users/nova` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.297423 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:12.299131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:10:12.312706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:12.313797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:12.314900 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 536/1092] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:10:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:12.319117 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 557/1093] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:12.321644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:12.345020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdbe69-3342-43e1-ab58-9bef5c51c8a4 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:12.399478 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:12.461382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddff9e7-9951-445a-abe1-37465f6a908e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:12.489141 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdbe69-3342-43e1-ab58-9bef5c51c8a4 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:12.512766 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:12.513838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.514450 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.515100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.516032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:12.516365 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:12.517847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:12.538413 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 537/1094] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:10:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:12.544212 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb9d66-51c7-4866-b7c7-5b80b415e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:12.544671 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:12.545305 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb9d66-51c7-4866-b7c7-5b80b415e696 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.546000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb9d66-51c7-4866-b7c7-5b80b415e696 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.546399 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb9d66-51c7-4866-b7c7-5b80b415e696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.563898 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb9d66-51c7-4866-b7c7-5b80b415e696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-ecfb9d66-51c7-4866-b7c7-5b80b415e696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:12.574847 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:12.576093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:12.577331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 558/1095] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:10:12.583061 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb9d66-51c7-4866-b7c7-5b80b415e696 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:12.624008 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:12.674258 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c59cd-54a2-4c15-9a96-65b832a3d4b4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:12.690586 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:12.710193 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb9d66-51c7-4866-b7c7-5b80b415e696 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:12.724420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da62887-5c3a-41cd-a9b6-d81ffc14bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:12.725009 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da62887-5c3a-41cd-a9b6-d81ffc14bd7d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.725280 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da62887-5c3a-41cd-a9b6-d81ffc14bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.725564 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da62887-5c3a-41cd-a9b6-d81ffc14bd7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.750254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:12.751084 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.751553 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.752093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.753092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:12.753415 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:12.755447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:12.772910 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 559/1096] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:12.778839 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:12.806925 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da62887-5c3a-41cd-a9b6-d81ffc14bd7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:12.808426 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da62887-5c3a-41cd-a9b6-d81ffc14bd7d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:12.919260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:12.923308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09064c-2c60-4ad6-8484-2ea938b7448e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:12.941048 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 538/1097] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 09:10:12.947321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da62887-5c3a-41cd-a9b6-d81ffc14bd7d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:12.962423 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:12.966614 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd52a6a-8d10-4a97-8fa7-11a80072b913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:12.967653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd52a6a-8d10-4a97-8fa7-11a80072b913 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:12.968133 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd52a6a-8d10-4a97-8fa7-11a80072b913 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:12.968678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd52a6a-8d10-4a97-8fa7-11a80072b913 None admin] PATH_INFO: `/v3/users/619965dfabaf452b9c2806054b868111` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:12.970883 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd52a6a-8d10-4a97-8fa7-11a80072b913 None admin] RBAC: Authorizing `identity:get_user(user_id=619965dfabaf452b9c2806054b868111)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:12.973452 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd52a6a-8d10-4a97-8fa7-11a80072b913 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:12.976482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 560/1098] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:12 2022] GET /identity/v3/users/619965dfabaf452b9c2806054b868111 => generated 285 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:12.982938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd52a6a-8d10-4a97-8fa7-11a80072b913 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:13.065510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da62887-5c3a-41cd-a9b6-d81ffc14bd7d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:13.117033 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:13.117697 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:13.117979 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:13.118288 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:13.129863 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:13.131672 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:13.145239 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd52a6a-8d10-4a97-8fa7-11a80072b913 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:10:13.148115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:13.149224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:13.150468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:10:13.150468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:10:13.150468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:10:13.150468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:13.150468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 539/1099] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:10:13.155852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:13.194692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d8d61-c630-48c6-be14-4c13ff187348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:13.195981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d8d61-c630-48c6-be14-4c13ff187348 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:13.196376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d8d61-c630-48c6-be14-4c13ff187348 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:13.197021 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d8d61-c630-48c6-be14-4c13ff187348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:13.243380 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:13.251034 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:13.293774 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d8d61-c630-48c6-be14-4c13ff187348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:13.294502 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d403f9-9609-4eb9-aab9-344984dd04e5 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:13.295976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d8d61-c630-48c6-be14-4c13ff187348 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:13.344109 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:13.344944 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:13.345262 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:13.345615 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:13.346882 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:13.348093 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:13.349156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:13.368669 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 540/1100] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:13.369936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:13.422573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 561/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 09:10:13.428701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d8d61-c630-48c6-be14-4c13ff187348 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:13.498378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef20b3-0f82-4c9c-902a-6d64a961f329 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:13.511973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:13.527870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:13.540447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:13.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:13.541495 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:13.541775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:13.550290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:13.552307 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:13.563026 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d8d61-c630-48c6-be14-4c13ff187348 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:10:13.569069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:13.570765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:13.571996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 541/1102] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:10:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:13.579704 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:13.615014 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23138cc5-7646-4d68-8453-0771f560ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:13.616171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138cc5-7646-4d68-8453-0771f560ef4e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:13.616504 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138cc5-7646-4d68-8453-0771f560ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:13.616889 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138cc5-7646-4d68-8453-0771f560ef4e None admin] PATH_INFO: `/v3/projects/da56673001f545aea09c272274c5da4d` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:13.618475 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23138cc5-7646-4d68-8453-0771f560ef4e None admin] RBAC: Authorizing `identity:get_project(project_id=da56673001f545aea09c272274c5da4d)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:13.620749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23138cc5-7646-4d68-8453-0771f560ef4e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:13.624402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 562/1103] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:13 2022] GET /identity/v3/projects/da56673001f545aea09c272274c5da4d => generated 311 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:13.632565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23138cc5-7646-4d68-8453-0771f560ef4e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:13.720509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91561-4000-4bbc-aa73-798d40c6e104 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:13.761069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:13.761590 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:13.761871 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:13.762183 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:13.762922 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:13.763237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:13.764736 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:13.767559 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23138cc5-7646-4d68-8453-0771f560ef4e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:13.780205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 542/1104] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:10:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:13.788727 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:13.806726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ddd08-f4cf-4f6c-ad48-717bab2d61c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:13.807576 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ddd08-f4cf-4f6c-ad48-717bab2d61c5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:13.807865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ddd08-f4cf-4f6c-ad48-717bab2d61c5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:13.808172 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ddd08-f4cf-4f6c-ad48-717bab2d61c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:13.809088 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ddd08-f4cf-4f6c-ad48-717bab2d61c5 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=da56673001f545aea09c272274c5da4d, user.id=619965dfabaf452b9c2806054b868111) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:13.809405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ddd08-f4cf-4f6c-ad48-717bab2d61c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:13.811100 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ddd08-f4cf-4f6c-ad48-717bab2d61c5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:13.813241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:13.820489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 563/1105] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:13 2022] GET /identity/v3/role_assignments?user.id=619965dfabaf452b9c2806054b868111&scope.project.id=da56673001f545aea09c272274c5da4d&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:13.852434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:13.919619 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123773f-3359-46e2-8af3-e6b06940c4c1 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:13.967898 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb1ab9-5b23-4c00-a42c-38dc4e7e7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:13.968627 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb1ab9-5b23-4c00-a42c-38dc4e7e7aac None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:13.968908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb1ab9-5b23-4c00-a42c-38dc4e7e7aac None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:13.969254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb1ab9-5b23-4c00-a42c-38dc4e7e7aac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:13.970392 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb1ab9-5b23-4c00-a42c-38dc4e7e7aac None admin] RBAC: Adding query filter params (role.id=7e05de6606ba48c984ba5aa6420aadbc, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=1e579a84d5e34a828c5981f34b458a0d) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:13.971642 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb1ab9-5b23-4c00-a42c-38dc4e7e7aac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:13.973167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb1ab9-5b23-4c00-a42c-38dc4e7e7aac None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:13.981398 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 543/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:13 2022] GET /identity/v3/role_assignments?user.id=1e579a84d5e34a828c5981f34b458a0d&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=7e05de6606ba48c984ba5aa6420aadbc => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:16.160104 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d03462-132c-4ea8-9b5e-d5f414a37d7d None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:16.160744 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d03462-132c-4ea8-9b5e-d5f414a37d7d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:16.160744 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d03462-132c-4ea8-9b5e-d5f414a37d7d None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:16.162170 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 564/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:16.171342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b31fa-270c-428b-940a-7e7dc2a49e8d None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:16.171902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b31fa-270c-428b-940a-7e7dc2a49e8d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:16.172397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b31fa-270c-428b-940a-7e7dc2a49e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:16.199725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-bf6b31fa-270c-428b-940a-7e7dc2a49e8d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:16.200953 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-bf6b31fa-270c-428b-940a-7e7dc2a49e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:16.274292 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 544/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:16.299482 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d03462-132c-4ea8-9b5e-d5f414a37d7d None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:16.411344 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d03462-132c-4ea8-9b5e-d5f414a37d7d None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:16.456668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d06af9-dcd3-4ea6-86e6-e6d921ddbb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:16.457373 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d06af9-dcd3-4ea6-86e6-e6d921ddbb0e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:16.457650 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d06af9-dcd3-4ea6-86e6-e6d921ddbb0e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:16.457949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d06af9-dcd3-4ea6-86e6-e6d921ddbb0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:16.459293 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d06af9-dcd3-4ea6-86e6-e6d921ddbb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:16.460798 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d06af9-dcd3-4ea6-86e6-e6d921ddbb0e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:16.462636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 565/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:16.469014 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b31fa-270c-428b-940a-7e7dc2a49e8d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:16.514959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:16.579500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b31fa-270c-428b-940a-7e7dc2a49e8d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:16.629025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd74c4-060a-466f-b4eb-3369822de802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:16.629942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd74c4-060a-466f-b4eb-3369822de802 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:16.630397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd74c4-060a-466f-b4eb-3369822de802 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:16.630710 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd74c4-060a-466f-b4eb-3369822de802 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:16.631673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd74c4-060a-466f-b4eb-3369822de802 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:16.633235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd74c4-060a-466f-b4eb-3369822de802 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:16.662790 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 545/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:10:16 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:10:18.766664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804219ab-0c7e-4473-9efd-fbfce7d64010 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:18.767235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804219ab-0c7e-4473-9efd-fbfce7d64010 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:18.767235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804219ab-0c7e-4473-9efd-fbfce7d64010 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:18.768619 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 566/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:18.774291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf9649-09ad-4e70-8f10-c4456cdcee05 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:18.774865 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf9649-09ad-4e70-8f10-c4456cdcee05 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:18.774970 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf9649-09ad-4e70-8f10-c4456cdcee05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:18.793342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-d1bf9649-09ad-4e70-8f10-c4456cdcee05 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:18.794620 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-d1bf9649-09ad-4e70-8f10-c4456cdcee05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:18.873297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 546/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:18.904232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804219ab-0c7e-4473-9efd-fbfce7d64010 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:18.981905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:19.034467 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804219ab-0c7e-4473-9efd-fbfce7d64010 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:19.075576 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec15d7-e5fa-448c-8f41-f740faf3e150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:19.076281 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec15d7-e5fa-448c-8f41-f740faf3e150 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:19.076556 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec15d7-e5fa-448c-8f41-f740faf3e150 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:19.076837 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec15d7-e5fa-448c-8f41-f740faf3e150 None admin] PATH_INFO: `/v3/roles/601939cb90d641e08b97775c0c12d374` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:19.078494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec15d7-e5fa-448c-8f41-f740faf3e150 None admin] RBAC: Authorizing `identity:get_role(role_id=601939cb90d641e08b97775c0c12d374)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:19.079901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec15d7-e5fa-448c-8f41-f740faf3e150 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:19.081349 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 567/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:18 2022] GET /identity/v3/roles/601939cb90d641e08b97775c0c12d374 => generated 223 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:19.089253 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf9649-09ad-4e70-8f10-c4456cdcee05 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:19.117172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:19.202609 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf9649-09ad-4e70-8f10-c4456cdcee05 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:19.252540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380d55-55dd-4de8-8fbe-79c5eeadb692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:19.253316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380d55-55dd-4de8-8fbe-79c5eeadb692 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:19.253608 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380d55-55dd-4de8-8fbe-79c5eeadb692 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:19.253959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380d55-55dd-4de8-8fbe-79c5eeadb692 None admin] PATH_INFO: `/v3/users/7001628a96814ceea2ca2bed7ded5cb5` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:19.272441 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380d55-55dd-4de8-8fbe-79c5eeadb692 None admin] RBAC: Authorizing `identity:get_user(user_id=7001628a96814ceea2ca2bed7ded5cb5)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:19.274087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380d55-55dd-4de8-8fbe-79c5eeadb692 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:19.276082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 547/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:19 2022] GET /identity/v3/users/7001628a96814ceea2ca2bed7ded5cb5 => generated 286 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:19.282274 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec15d7-e5fa-448c-8f41-f740faf3e150 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:19.403301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec15d7-e5fa-448c-8f41-f740faf3e150 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:19.446014 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b2b1e-206b-431a-bcf2-0ad44994d031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:19.446868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b2b1e-206b-431a-bcf2-0ad44994d031 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:19.447295 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b2b1e-206b-431a-bcf2-0ad44994d031 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:19.447579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b2b1e-206b-431a-bcf2-0ad44994d031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:19.451333 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:19.526854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b2b1e-206b-431a-bcf2-0ad44994d031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:19.528388 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b2b1e-206b-431a-bcf2-0ad44994d031 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:19.637121 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 568/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:10:19.643847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380d55-55dd-4de8-8fbe-79c5eeadb692 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:19.750810 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:19.776289 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380d55-55dd-4de8-8fbe-79c5eeadb692 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:19.819904 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa8e8a-211f-4ceb-b87d-a031af4d8507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:19.820620 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa8e8a-211f-4ceb-b87d-a031af4d8507 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:19.820983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa8e8a-211f-4ceb-b87d-a031af4d8507 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:19.821269 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa8e8a-211f-4ceb-b87d-a031af4d8507 None admin] PATH_INFO: `/v3/projects/da56673001f545aea09c272274c5da4d` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:19.822725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa8e8a-211f-4ceb-b87d-a031af4d8507 None admin] RBAC: Authorizing `identity:get_project(project_id=da56673001f545aea09c272274c5da4d)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:19.824443 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa8e8a-211f-4ceb-b87d-a031af4d8507 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:19.826490 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 548/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:19 2022] GET /identity/v3/projects/da56673001f545aea09c272274c5da4d => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:19.833780 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b2b1e-206b-431a-bcf2-0ad44994d031 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:19.872533 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:19.945924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b2b1e-206b-431a-bcf2-0ad44994d031 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:19.983184 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91287f59-831b-4f0b-9f2a-94812398a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:19.983828 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287f59-831b-4f0b-9f2a-94812398a084 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:19.984115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287f59-831b-4f0b-9f2a-94812398a084 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:19.984395 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287f59-831b-4f0b-9f2a-94812398a084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:19.985223 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91287f59-831b-4f0b-9f2a-94812398a084 None admin] RBAC: Adding query filter params (role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=da56673001f545aea09c272274c5da4d, user.id=7001628a96814ceea2ca2bed7ded5cb5) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:19.985548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91287f59-831b-4f0b-9f2a-94812398a084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:19.987101 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91287f59-831b-4f0b-9f2a-94812398a084 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:19.993254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 569/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:19 2022] GET /identity/v3/role_assignments?user.id=7001628a96814ceea2ca2bed7ded5cb5&scope.project.id=da56673001f545aea09c272274c5da4d&role.id=601939cb90d641e08b97775c0c12d374 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:22.063251 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86669-b3eb-4707-9eba-4945cc7edcd3 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:22.063855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86669-b3eb-4707-9eba-4945cc7edcd3 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:22.063855 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86669-b3eb-4707-9eba-4945cc7edcd3 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:22.065189 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 549/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:22.071272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4dfe5-e458-4e2c-9909-be690108a143 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:22.071741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4dfe5-e458-4e2c-9909-be690108a143 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:22.071868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4dfe5-e458-4e2c-9909-be690108a143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:22.090578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-b0d4dfe5-e458-4e2c-9909-be690108a143 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:22.091788 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-b0d4dfe5-e458-4e2c-9909-be690108a143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:22.161514 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 570/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:22.184717 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e86669-b3eb-4707-9eba-4945cc7edcd3 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:22.291858 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e86669-b3eb-4707-9eba-4945cc7edcd3 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:22.327074 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1af1ca-1fab-4653-a1d5-def62b9f282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:22.327645 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af1ca-1fab-4653-a1d5-def62b9f282d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:22.327928 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af1ca-1fab-4653-a1d5-def62b9f282d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:22.328205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af1ca-1fab-4653-a1d5-def62b9f282d None admin] PATH_INFO: `/v3/roles/601939cb90d641e08b97775c0c12d374` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:22.329704 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1af1ca-1fab-4653-a1d5-def62b9f282d None admin] RBAC: Authorizing `identity:get_role(role_id=601939cb90d641e08b97775c0c12d374)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:22.331137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1af1ca-1fab-4653-a1d5-def62b9f282d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:22.332527 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 550/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:22 2022] GET /identity/v3/roles/601939cb90d641e08b97775c0c12d374 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:22.338794 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4dfe5-e458-4e2c-9909-be690108a143 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:22.367567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:22.445611 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4dfe5-e458-4e2c-9909-be690108a143 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:22.484603 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9fc4f-b1d0-4825-9640-63586ef0eb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:22.485283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9fc4f-b1d0-4825-9640-63586ef0eb9c None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:22.485565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9fc4f-b1d0-4825-9640-63586ef0eb9c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:22.485850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9fc4f-b1d0-4825-9640-63586ef0eb9c None admin] PATH_INFO: `/v3/users/7001628a96814ceea2ca2bed7ded5cb5` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:22.487092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9fc4f-b1d0-4825-9640-63586ef0eb9c None admin] RBAC: Authorizing `identity:get_user(user_id=7001628a96814ceea2ca2bed7ded5cb5)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:22.488574 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9fc4f-b1d0-4825-9640-63586ef0eb9c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:22.490332 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 571/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:22 2022] GET /identity/v3/users/7001628a96814ceea2ca2bed7ded5cb5 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:22.496575 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1af1ca-1fab-4653-a1d5-def62b9f282d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:22.517061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:22.599382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1af1ca-1fab-4653-a1d5-def62b9f282d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:22.634861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d784141-8fc6-48d8-bbc7-a635c660fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:22.635540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784141-8fc6-48d8-bbc7-a635c660fa45 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:22.635821 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784141-8fc6-48d8-bbc7-a635c660fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:22.636094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784141-8fc6-48d8-bbc7-a635c660fa45 None admin] PATH_INFO: `/v3/projects/da56673001f545aea09c272274c5da4d` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:22.637345 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d784141-8fc6-48d8-bbc7-a635c660fa45 None admin] RBAC: Authorizing `identity:get_project(project_id=da56673001f545aea09c272274c5da4d)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:22.638872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d784141-8fc6-48d8-bbc7-a635c660fa45 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:22.640615 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 551/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:22 2022] GET /identity/v3/projects/da56673001f545aea09c272274c5da4d => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:22.647049 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9fc4f-b1d0-4825-9640-63586ef0eb9c None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:22.744410 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:22.750653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9fc4f-b1d0-4825-9640-63586ef0eb9c None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:22.786538 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fe534-3c3d-47ee-a23b-e121574872ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:22.787257 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe534-3c3d-47ee-a23b-e121574872ac None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:22.787542 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe534-3c3d-47ee-a23b-e121574872ac None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:22.787815 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe534-3c3d-47ee-a23b-e121574872ac None admin] PATH_INFO: `/v3/projects/da56673001f545aea09c272274c5da4d/users/7001628a96814ceea2ca2bed7ded5cb5/roles/601939cb90d641e08b97775c0c12d374` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:22.790258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fe534-3c3d-47ee-a23b-e121574872ac None admin] RBAC: Authorizing `identity:create_grant(project_id=da56673001f545aea09c272274c5da4d, user_id=7001628a96814ceea2ca2bed7ded5cb5, role_id=601939cb90d641e08b97775c0c12d374)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:22.791790 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fe534-3c3d-47ee-a23b-e121574872ac None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:22.807951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 572/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:10:22 2022] PUT /identity/v3/projects/da56673001f545aea09c272274c5da4d/users/7001628a96814ceea2ca2bed7ded5cb5/roles/601939cb90d641e08b97775c0c12d374 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:10:24.801133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79daa5f-1d08-4d5d-ba97-feb797a6c0c2 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:24.801868 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79daa5f-1d08-4d5d-ba97-feb797a6c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:24.801868 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79daa5f-1d08-4d5d-ba97-feb797a6c0c2 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:24.803247 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 552/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:24.809413 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cae01-e9e6-4b67-828d-a898198168b8 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:24.809892 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cae01-e9e6-4b67-828d-a898198168b8 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:24.810007 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cae01-e9e6-4b67-828d-a898198168b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:24.831766 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-bf2cae01-e9e6-4b67-828d-a898198168b8 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:24.833084 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-bf2cae01-e9e6-4b67-828d-a898198168b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:24.917389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 573/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:24.942933 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79daa5f-1d08-4d5d-ba97-feb797a6c0c2 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:25.039565 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:25.057905 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79daa5f-1d08-4d5d-ba97-feb797a6c0c2 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:25.096137 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb20528-9edc-449a-b070-63c69fabb476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:25.096806 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20528-9edc-449a-b070-63c69fabb476 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:25.097093 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20528-9edc-449a-b070-63c69fabb476 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:25.097368 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20528-9edc-449a-b070-63c69fabb476 None admin] PATH_INFO: `/v3/roles/601939cb90d641e08b97775c0c12d374` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:25.098976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb20528-9edc-449a-b070-63c69fabb476 None admin] RBAC: Authorizing `identity:get_role(role_id=601939cb90d641e08b97775c0c12d374)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:25.100489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb20528-9edc-449a-b070-63c69fabb476 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:25.101919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 553/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:24 2022] GET /identity/v3/roles/601939cb90d641e08b97775c0c12d374 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:25.108614 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cae01-e9e6-4b67-828d-a898198168b8 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:25.209258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:25.228092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cae01-e9e6-4b67-828d-a898198168b8 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:25.264961 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244bf5d-d62b-4839-bf5c-3aa549e2fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:25.265647 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244bf5d-d62b-4839-bf5c-3aa549e2fabb None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:25.265926 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244bf5d-d62b-4839-bf5c-3aa549e2fabb None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:25.266245 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244bf5d-d62b-4839-bf5c-3aa549e2fabb None admin] PATH_INFO: `/v3/users/7001628a96814ceea2ca2bed7ded5cb5` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:25.267555 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244bf5d-d62b-4839-bf5c-3aa549e2fabb None admin] RBAC: Authorizing `identity:get_user(user_id=7001628a96814ceea2ca2bed7ded5cb5)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:25.269032 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244bf5d-d62b-4839-bf5c-3aa549e2fabb None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:25.270811 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 574/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:25 2022] GET /identity/v3/users/7001628a96814ceea2ca2bed7ded5cb5 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:25.276911 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb20528-9edc-449a-b070-63c69fabb476 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:25.385330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb20528-9edc-449a-b070-63c69fabb476 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:25.420977 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10767835-b6a4-4cde-be95-abd614eb7a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:25.421649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10767835-b6a4-4cde-be95-abd614eb7a25 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:25.421939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10767835-b6a4-4cde-be95-abd614eb7a25 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:25.422249 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10767835-b6a4-4cde-be95-abd614eb7a25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:25.449493 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:25.496890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10767835-b6a4-4cde-be95-abd614eb7a25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:25.498879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10767835-b6a4-4cde-be95-abd614eb7a25 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:25.598725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 554/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:10:25.605042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244bf5d-d62b-4839-bf5c-3aa549e2fabb None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:25.703270 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244bf5d-d62b-4839-bf5c-3aa549e2fabb None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:25.737059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1aa3f-e689-4f74-b613-bee8e58dcd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:25.737722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1aa3f-e689-4f74-b613-bee8e58dcd94 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:25.738009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1aa3f-e689-4f74-b613-bee8e58dcd94 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:25.738320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1aa3f-e689-4f74-b613-bee8e58dcd94 None admin] PATH_INFO: `/v3/projects/da56673001f545aea09c272274c5da4d` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:25.739557 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1aa3f-e689-4f74-b613-bee8e58dcd94 None admin] RBAC: Authorizing `identity:get_project(project_id=da56673001f545aea09c272274c5da4d)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:25.741089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1aa3f-e689-4f74-b613-bee8e58dcd94 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:25.742858 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 575/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:25 2022] GET /identity/v3/projects/da56673001f545aea09c272274c5da4d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:25.749438 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10767835-b6a4-4cde-be95-abd614eb7a25 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:25.811341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:25.863771 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10767835-b6a4-4cde-be95-abd614eb7a25 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:25.902623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa0708-bb5f-47ce-8a4f-e5d263654aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:25.903253 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0708-bb5f-47ce-8a4f-e5d263654aea None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:25.903557 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0708-bb5f-47ce-8a4f-e5d263654aea None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:25.903835 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0708-bb5f-47ce-8a4f-e5d263654aea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:25.904649 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa0708-bb5f-47ce-8a4f-e5d263654aea None admin] RBAC: Adding query filter params (role.id=601939cb90d641e08b97775c0c12d374, scope.project.id=da56673001f545aea09c272274c5da4d, user.id=7001628a96814ceea2ca2bed7ded5cb5) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:25.905347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa0708-bb5f-47ce-8a4f-e5d263654aea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:25.906460 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa0708-bb5f-47ce-8a4f-e5d263654aea None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:25.912890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 555/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:25 2022] GET /identity/v3/role_assignments?user.id=7001628a96814ceea2ca2bed7ded5cb5&scope.project.id=da56673001f545aea09c272274c5da4d&role.id=601939cb90d641e08b97775c0c12d374 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:28.058087 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54eb02-b798-40a0-ba1c-cd610af2247c None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:28.058685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54eb02-b798-40a0-ba1c-cd610af2247c None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:28.058685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54eb02-b798-40a0-ba1c-cd610af2247c None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:28.060051 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 576/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:28.065549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507dce-dd68-4646-859b-60906cb5f0de None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:28.066014 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507dce-dd68-4646-859b-60906cb5f0de None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:28.066189 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507dce-dd68-4646-859b-60906cb5f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:28.086476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-5c507dce-dd68-4646-859b-60906cb5f0de None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:28.087701 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-5c507dce-dd68-4646-859b-60906cb5f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:28.159319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 556/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:28.182391 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54eb02-b798-40a0-ba1c-cd610af2247c None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:28.210808 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:28.296071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54eb02-b798-40a0-ba1c-cd610af2247c None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:28.332504 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1931e6-250e-4081-9da4-b72e089823f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:28.333080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1931e6-250e-4081-9da4-b72e089823f4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:28.333400 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1931e6-250e-4081-9da4-b72e089823f4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:28.333686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1931e6-250e-4081-9da4-b72e089823f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:28.334893 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1931e6-250e-4081-9da4-b72e089823f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:28.336353 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1931e6-250e-4081-9da4-b72e089823f4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:28.338036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 577/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:28.344264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c507dce-dd68-4646-859b-60906cb5f0de None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:28.397321 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:28.471623 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c507dce-dd68-4646-859b-60906cb5f0de None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:28.512560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6279d-b351-4df6-a186-66879308523a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:28.513194 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6279d-b351-4df6-a186-66879308523a None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:28.513487 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6279d-b351-4df6-a186-66879308523a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:28.513764 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6279d-b351-4df6-a186-66879308523a None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:28.514510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6279d-b351-4df6-a186-66879308523a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:28.515979 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6279d-b351-4df6-a186-66879308523a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:28.545041 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 557/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:10:28 2022] POST /identity/v3/projects => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:10:30.690925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5c9b-c917-40d9-9f5a-7d741d72d934 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:30.691492 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5c9b-c917-40d9-9f5a-7d741d72d934 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:30.691492 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5c9b-c917-40d9-9f5a-7d741d72d934 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:30.693349 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 578/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:30.699268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0844-e550-4982-9c31-670f6e7ced0f None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:30.699791 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0844-e550-4982-9c31-670f6e7ced0f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:30.699791 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0844-e550-4982-9c31-670f6e7ced0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:30.718880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-97ce0844-e550-4982-9c31-670f6e7ced0f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:30.720118 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-97ce0844-e550-4982-9c31-670f6e7ced0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:30.799224 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 558/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:30.826161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5c9b-c917-40d9-9f5a-7d741d72d934 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:30.928047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:30.939810 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5c9b-c917-40d9-9f5a-7d741d72d934 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:30.977529 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9f8dd-5a9d-4c90-be9b-fa140b52e203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:30.978142 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9f8dd-5a9d-4c90-be9b-fa140b52e203 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:30.978444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9f8dd-5a9d-4c90-be9b-fa140b52e203 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:30.978735 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9f8dd-5a9d-4c90-be9b-fa140b52e203 None admin] PATH_INFO: `/v3/domains/default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:30.980089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9f8dd-5a9d-4c90-be9b-fa140b52e203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:30.981600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9f8dd-5a9d-4c90-be9b-fa140b52e203 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:30.983420 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 579/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:10:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:30.991059 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce0844-e550-4982-9c31-670f6e7ced0f None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:31.027408 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:31.111433 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce0844-e550-4982-9c31-670f6e7ced0f None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:31.149333 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e9d49-af87-4f04-9c7c-052b2f467130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:31.150083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e9d49-af87-4f04-9c7c-052b2f467130 None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:31.150498 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e9d49-af87-4f04-9c7c-052b2f467130 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:31.150829 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e9d49-af87-4f04-9c7c-052b2f467130 None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:31.151622 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e9d49-af87-4f04-9c7c-052b2f467130 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:31.153208 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e9d49-af87-4f04-9c7c-052b2f467130 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:31.179279 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 559/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:10:30 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:10:33.275943 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103c56-a601-4f5f-8561-020e64d0618d None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:33.276636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103c56-a601-4f5f-8561-020e64d0618d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:33.276636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103c56-a601-4f5f-8561-020e64d0618d None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:33.277886 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 580/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:33.283847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b809c-67cb-4fc6-9930-10b7c54d8cfb None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:33.284361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b809c-67cb-4fc6-9930-10b7c54d8cfb None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:33.284479 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b809c-67cb-4fc6-9930-10b7c54d8cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:33.304959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-b95b809c-67cb-4fc6-9930-10b7c54d8cfb None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:33.306456 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-b95b809c-67cb-4fc6-9930-10b7c54d8cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:33.378012 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 560/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:33.401873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c103c56-a601-4f5f-8561-020e64d0618d None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:33.528282 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c103c56-a601-4f5f-8561-020e64d0618d None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:33.569460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37f6a-8f64-437f-9755-a0b8e9755254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:33.570147 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37f6a-8f64-437f-9755-a0b8e9755254 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:33.570441 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37f6a-8f64-437f-9755-a0b8e9755254 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:33.570724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37f6a-8f64-437f-9755-a0b8e9755254 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:33.582466 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a37f6a-8f64-437f-9755-a0b8e9755254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-57a37f6a-8f64-437f-9755-a0b8e9755254 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:33.588656 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:33.591149 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:33.593674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 581/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:33.602058 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b809c-67cb-4fc6-9930-10b7c54d8cfb None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:33.621411 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:33.720688 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b809c-67cb-4fc6-9930-10b7c54d8cfb None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:33.755969 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:33.756640 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:33.756932 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:33.757210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:33.757958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:33.758304 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:33.759754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:33.769303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 561/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:33.776207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37f6a-8f64-437f-9755-a0b8e9755254 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:33.799927 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:33.914393 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37f6a-8f64-437f-9755-a0b8e9755254 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:33.979803 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce49577-f038-4565-a56c-cba5a0cbff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:33.980481 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce49577-f038-4565-a56c-cba5a0cbff80 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:33.980767 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce49577-f038-4565-a56c-cba5a0cbff80 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:33.981046 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce49577-f038-4565-a56c-cba5a0cbff80 None admin] PATH_INFO: `/v3/users/c96b43471fd54875a6e3820671628164` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:33.996450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce49577-f038-4565-a56c-cba5a0cbff80 None admin] RBAC: Authorizing `identity:get_user(user_id=c96b43471fd54875a6e3820671628164)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:33.998347 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce49577-f038-4565-a56c-cba5a0cbff80 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:34.000376 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 582/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:33 2022] GET /identity/v3/users/c96b43471fd54875a6e3820671628164 => generated 286 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:34.007252 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:34.092467 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:34.114267 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de79ea0-f38c-487c-8161-42a1056b6a2e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:34.147512 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7adfc-33db-4c24-a649-3c45c88e0eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:34.148146 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7adfc-33db-4c24-a649-3c45c88e0eb4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:34.148434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7adfc-33db-4c24-a649-3c45c88e0eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:34.148714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7adfc-33db-4c24-a649-3c45c88e0eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:34.220850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7adfc-33db-4c24-a649-3c45c88e0eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:34.222445 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7adfc-33db-4c24-a649-3c45c88e0eb4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:34.302643 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:34.327801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 562/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:10:34.333957 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce49577-f038-4565-a56c-cba5a0cbff80 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:34.416055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:34.441164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce49577-f038-4565-a56c-cba5a0cbff80 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:34.488524 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e507700-b4bd-41f7-b6d8-87d51c8c92ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:34.489240 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e507700-b4bd-41f7-b6d8-87d51c8c92ac None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:34.489641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e507700-b4bd-41f7-b6d8-87d51c8c92ac None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:34.489986 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e507700-b4bd-41f7-b6d8-87d51c8c92ac None admin] PATH_INFO: `/v3/projects/6d3d4be75af64ef4865f5cb6fbd5b3c9` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:34.491365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e507700-b4bd-41f7-b6d8-87d51c8c92ac None admin] RBAC: Authorizing `identity:get_project(project_id=6d3d4be75af64ef4865f5cb6fbd5b3c9)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:34.493012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e507700-b4bd-41f7-b6d8-87d51c8c92ac None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:34.494971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 583/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:34 2022] GET /identity/v3/projects/6d3d4be75af64ef4865f5cb6fbd5b3c9 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:34.501393 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7adfc-33db-4c24-a649-3c45c88e0eb4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:34.625301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7adfc-33db-4c24-a649-3c45c88e0eb4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:34.660343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e5cca-ae9c-4c00-936d-9d3ae65bd124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:34.661003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5cca-ae9c-4c00-936d-9d3ae65bd124 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:34.661290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5cca-ae9c-4c00-936d-9d3ae65bd124 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:34.661599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5cca-ae9c-4c00-936d-9d3ae65bd124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:34.662434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e5cca-ae9c-4c00-936d-9d3ae65bd124 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=6d3d4be75af64ef4865f5cb6fbd5b3c9, user.id=c96b43471fd54875a6e3820671628164) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:34.662760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e5cca-ae9c-4c00-936d-9d3ae65bd124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:34.664233 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e5cca-ae9c-4c00-936d-9d3ae65bd124 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:34.670093 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 563/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:34 2022] GET /identity/v3/role_assignments?user.id=c96b43471fd54875a6e3820671628164&scope.project.id=6d3d4be75af64ef4865f5cb6fbd5b3c9&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:36.712550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f063d5-1261-4c7a-be84-067839d75768 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:36.713168 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f063d5-1261-4c7a-be84-067839d75768 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:36.713168 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f063d5-1261-4c7a-be84-067839d75768 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:36.714516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 584/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:36.719922 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643198aa-8361-47fc-8a38-04642796a082 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:36.720428 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643198aa-8361-47fc-8a38-04642796a082 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:36.720552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643198aa-8361-47fc-8a38-04642796a082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:36.742416 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-643198aa-8361-47fc-8a38-04642796a082 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:36.743831 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-643198aa-8361-47fc-8a38-04642796a082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:36.787233 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:36.825067 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 564/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:36.847976 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f063d5-1261-4c7a-be84-067839d75768 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:36.958272 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f063d5-1261-4c7a-be84-067839d75768 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:36.997824 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2f453-ce87-46e4-8a7a-1a7eb70a5bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:36.998504 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2f453-ce87-46e4-8a7a-1a7eb70a5bc4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:36.998794 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2f453-ce87-46e4-8a7a-1a7eb70a5bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:36.999073 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2f453-ce87-46e4-8a7a-1a7eb70a5bc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:37.011700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2f453-ce87-46e4-8a7a-1a7eb70a5bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-f9e2f453-ce87-46e4-8a7a-1a7eb70a5bc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:37.016879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:37.017956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:37.019099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 585/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:37.024799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643198aa-8361-47fc-8a38-04642796a082 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:37.138847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643198aa-8361-47fc-8a38-04642796a082 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:37.177671 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:37.178328 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:37.178611 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:37.178884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:37.179616 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:37.179917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:37.181303 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:37.190333 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 565/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:37.197618 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2f453-ce87-46e4-8a7a-1a7eb70a5bc4 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:37.205677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:37.353424 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2f453-ce87-46e4-8a7a-1a7eb70a5bc4 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:37.413039 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61aa6-72ca-4615-ab76-6f75d883ebda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:37.413719 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61aa6-72ca-4615-ab76-6f75d883ebda None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:37.414009 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61aa6-72ca-4615-ab76-6f75d883ebda None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:37.414320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61aa6-72ca-4615-ab76-6f75d883ebda None admin] PATH_INFO: `/v3/users/c96b43471fd54875a6e3820671628164` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:37.415597 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61aa6-72ca-4615-ab76-6f75d883ebda None admin] RBAC: Authorizing `identity:get_user(user_id=c96b43471fd54875a6e3820671628164)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:37.417096 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61aa6-72ca-4615-ab76-6f75d883ebda None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:37.418900 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 586/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:37 2022] GET /identity/v3/users/c96b43471fd54875a6e3820671628164 => generated 286 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:37.425052 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:37.465571 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:37.540679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470894e7-2b40-4db9-a465-583563134ac0 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:37.575725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f712242-d87e-4a08-b2dc-b573c766ede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:37.576426 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712242-d87e-4a08-b2dc-b573c766ede7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:37.576706 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712242-d87e-4a08-b2dc-b573c766ede7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:37.576981 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712242-d87e-4a08-b2dc-b573c766ede7 None admin] PATH_INFO: `/v3/projects/6d3d4be75af64ef4865f5cb6fbd5b3c9` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:37.578206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f712242-d87e-4a08-b2dc-b573c766ede7 None admin] RBAC: Authorizing `identity:get_project(project_id=6d3d4be75af64ef4865f5cb6fbd5b3c9)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:37.579687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f712242-d87e-4a08-b2dc-b573c766ede7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:37.581439 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 566/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:37 2022] GET /identity/v3/projects/6d3d4be75af64ef4865f5cb6fbd5b3c9 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:37.587887 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61aa6-72ca-4615-ab76-6f75d883ebda None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:37.663338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:37.698086 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61aa6-72ca-4615-ab76-6f75d883ebda None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:37.733777 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093472cb-6768-4c39-8f56-5456c55f6cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:37.734559 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093472cb-6768-4c39-8f56-5456c55f6cf7 None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:37.734800 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093472cb-6768-4c39-8f56-5456c55f6cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:37.735076 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093472cb-6768-4c39-8f56-5456c55f6cf7 None admin] PATH_INFO: `/v3/projects/6d3d4be75af64ef4865f5cb6fbd5b3c9/users/c96b43471fd54875a6e3820671628164/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:37.737304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093472cb-6768-4c39-8f56-5456c55f6cf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d3d4be75af64ef4865f5cb6fbd5b3c9, user_id=c96b43471fd54875a6e3820671628164, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:37.738813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093472cb-6768-4c39-8f56-5456c55f6cf7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:37.753971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 587/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:10:37 2022] PUT /identity/v3/projects/6d3d4be75af64ef4865f5cb6fbd5b3c9/users/c96b43471fd54875a6e3820671628164/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:10:39.745368 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dca5a9-7829-4842-8a86-b4350193d6a8 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:39.745976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dca5a9-7829-4842-8a86-b4350193d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:39.745976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dca5a9-7829-4842-8a86-b4350193d6a8 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:39.747351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 567/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:39.752849 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d52706-339d-4840-92b2-b512c079f2eb None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:39.753303 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d52706-339d-4840-92b2-b512c079f2eb None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:39.753414 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d52706-339d-4840-92b2-b512c079f2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:39.774000 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-f3d52706-339d-4840-92b2-b512c079f2eb None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:39.775273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-f3d52706-339d-4840-92b2-b512c079f2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:39.847232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 588/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:39.875797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dca5a9-7829-4842-8a86-b4350193d6a8 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:39.989299 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dca5a9-7829-4842-8a86-b4350193d6a8 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:39.990940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:40.027390 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ae74-c020-4b6d-bc57-be058e5736a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:40.027958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ae74-c020-4b6d-bc57-be058e5736a3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:40.028248 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ae74-c020-4b6d-bc57-be058e5736a3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:40.028529 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ae74-c020-4b6d-bc57-be058e5736a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:40.039454 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322ae74-c020-4b6d-bc57-be058e5736a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-f322ae74-c020-4b6d-bc57-be058e5736a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:40.044464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:40.045631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:40.046846 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:40.050305 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 568/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:40.052233 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d52706-339d-4840-92b2-b512c079f2eb None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:40.114671 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:40.156241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d52706-339d-4840-92b2-b512c079f2eb None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:40.192997 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:40.193621 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:40.193903 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:40.194216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:40.194953 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:40.195260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:40.196677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:40.205421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 589/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:10:40.211920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ae74-c020-4b6d-bc57-be058e5736a3 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:40.317542 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ae74-c020-4b6d-bc57-be058e5736a3 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:40.352927 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2403d3-27d9-495e-8735-b63a41fd43a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:40.353567 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2403d3-27d9-495e-8735-b63a41fd43a1 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:40.353856 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2403d3-27d9-495e-8735-b63a41fd43a1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:40.354148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2403d3-27d9-495e-8735-b63a41fd43a1 None admin] PATH_INFO: `/v3/users/c96b43471fd54875a6e3820671628164` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:40.355374 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2403d3-27d9-495e-8735-b63a41fd43a1 None admin] RBAC: Authorizing `identity:get_user(user_id=c96b43471fd54875a6e3820671628164)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:40.356826 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2403d3-27d9-495e-8735-b63a41fd43a1 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:40.358493 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 569/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:40 2022] GET /identity/v3/users/c96b43471fd54875a6e3820671628164 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:40.364774 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:40.472403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1b471-c1fd-45b7-9fdc-569336ea84bb None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:40.496664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:40.510126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2ba8c-1849-495b-8f72-21459226893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:40.510797 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2ba8c-1849-495b-8f72-21459226893f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:40.511068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2ba8c-1849-495b-8f72-21459226893f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:40.511362 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2ba8c-1849-495b-8f72-21459226893f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:40.585307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2ba8c-1849-495b-8f72-21459226893f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:40.586722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2ba8c-1849-495b-8f72-21459226893f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:40.693802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 590/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:10:40.700349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2403d3-27d9-495e-8735-b63a41fd43a1 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:40.733297 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:40.811309 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2403d3-27d9-495e-8735-b63a41fd43a1 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:40.848509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900072f5-25aa-46cc-89e0-455451d716d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:40.849118 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900072f5-25aa-46cc-89e0-455451d716d5 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:40.849406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900072f5-25aa-46cc-89e0-455451d716d5 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:40.849690 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900072f5-25aa-46cc-89e0-455451d716d5 None admin] PATH_INFO: `/v3/projects/6d3d4be75af64ef4865f5cb6fbd5b3c9` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:40.850935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900072f5-25aa-46cc-89e0-455451d716d5 None admin] RBAC: Authorizing `identity:get_project(project_id=6d3d4be75af64ef4865f5cb6fbd5b3c9)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:40.852400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900072f5-25aa-46cc-89e0-455451d716d5 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:40.854120 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 570/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:40 2022] GET /identity/v3/projects/6d3d4be75af64ef4865f5cb6fbd5b3c9 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:40.860642 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2ba8c-1849-495b-8f72-21459226893f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:40.886851 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:40.968995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2ba8c-1849-495b-8f72-21459226893f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:41.005230 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa3bb2-a0a9-48b3-a9b1-e1acf265a2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:41.005822 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa3bb2-a0a9-48b3-a9b1-e1acf265a2d6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:41.006137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa3bb2-a0a9-48b3-a9b1-e1acf265a2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:41.006406 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa3bb2-a0a9-48b3-a9b1-e1acf265a2d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:41.007201 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa3bb2-a0a9-48b3-a9b1-e1acf265a2d6 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=6d3d4be75af64ef4865f5cb6fbd5b3c9, user.id=c96b43471fd54875a6e3820671628164) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:41.007516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa3bb2-a0a9-48b3-a9b1-e1acf265a2d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:41.008956 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa3bb2-a0a9-48b3-a9b1-e1acf265a2d6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:41.015099 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 591/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:40 2022] GET /identity/v3/role_assignments?user.id=c96b43471fd54875a6e3820671628164&scope.project.id=6d3d4be75af64ef4865f5cb6fbd5b3c9&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:43.043051 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35c9a-4b15-4631-8b93-d929b5a8ccde None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:43.043692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35c9a-4b15-4631-8b93-d929b5a8ccde None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:43.043692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35c9a-4b15-4631-8b93-d929b5a8ccde None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:43.045639 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 571/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:43.053405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbe2e-2247-4883-b0f9-2c6aafe65aca None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:43.053868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbe2e-2247-4883-b0f9-2c6aafe65aca None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:43.053995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbe2e-2247-4883-b0f9-2c6aafe65aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:43.077965 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-d40fbe2e-2247-4883-b0f9-2c6aafe65aca None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:43.079560 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-d40fbe2e-2247-4883-b0f9-2c6aafe65aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:43.155756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 592/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:43.178461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb35c9a-4b15-4631-8b93-d929b5a8ccde None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:43.290972 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:43.297824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb35c9a-4b15-4631-8b93-d929b5a8ccde None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:43.336671 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf0c5-1c46-4b61-9b89-e3a4a31b5119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:43.337290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf0c5-1c46-4b61-9b89-e3a4a31b5119 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:43.337570 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf0c5-1c46-4b61-9b89-e3a4a31b5119 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:43.337847 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf0c5-1c46-4b61-9b89-e3a4a31b5119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:43.415538 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cf0c5-1c46-4b61-9b89-e3a4a31b5119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:43.417006 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cf0c5-1c46-4b61-9b89-e3a4a31b5119 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:43.527115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:43.532683 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 572/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:10:43.538949 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fbe2e-2247-4883-b0f9-2c6aafe65aca None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:43.565321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:43.652309 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fbe2e-2247-4883-b0f9-2c6aafe65aca None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:43.689254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e057f-a333-441f-9800-cfee9a01111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:43.689912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e057f-a333-441f-9800-cfee9a01111b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:43.690230 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e057f-a333-441f-9800-cfee9a01111b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:43.690506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e057f-a333-441f-9800-cfee9a01111b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:43.698252 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e057f-a333-441f-9800-cfee9a01111b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:43.699725 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e057f-a333-441f-9800-cfee9a01111b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-4a5e057f-a333-441f-9800-cfee9a01111b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:43.711870 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:43.713365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:43.714527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 593/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:10:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:43.719509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf0c5-1c46-4b61-9b89-e3a4a31b5119 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:43.832133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf0c5-1c46-4b61-9b89-e3a4a31b5119 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:43.871385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991f139-54f6-4b01-9d9a-c1f4f76cce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:43.872025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f139-54f6-4b01-9d9a-c1f4f76cce16 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:43.872309 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f139-54f6-4b01-9d9a-c1f4f76cce16 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:43.872587 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f139-54f6-4b01-9d9a-c1f4f76cce16 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:43.873309 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991f139-54f6-4b01-9d9a-c1f4f76cce16 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:43.873614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991f139-54f6-4b01-9d9a-c1f4f76cce16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:43.875098 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991f139-54f6-4b01-9d9a-c1f4f76cce16 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:43.883266 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 573/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:10:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:45.923906 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f2772-fade-4358-98e2-29581b204ec8 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:45.924494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f2772-fade-4358-98e2-29581b204ec8 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:45.924494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f2772-fade-4358-98e2-29581b204ec8 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:45.925907 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 594/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:45.931463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d76a-85fe-4cc6-9b0d-c208830c7190 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:45.931932 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d76a-85fe-4cc6-9b0d-c208830c7190 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:45.932042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d76a-85fe-4cc6-9b0d-c208830c7190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:45.951687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-cec1d76a-85fe-4cc6-9b0d-c208830c7190 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:45.952889 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-cec1d76a-85fe-4cc6-9b0d-c208830c7190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:46.017273 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:46.028768 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 574/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:46.051724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f2772-fade-4358-98e2-29581b204ec8 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:46.137904 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:46.163674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f2772-fade-4358-98e2-29581b204ec8 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:46.198478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef8735-2b84-4018-a954-c4fce24bb387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:46.199039 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8735-2b84-4018-a954-c4fce24bb387 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:46.199330 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8735-2b84-4018-a954-c4fce24bb387 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:46.199610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8735-2b84-4018-a954-c4fce24bb387 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:46.200340 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef8735-2b84-4018-a954-c4fce24bb387 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:46.201749 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef8735-2b84-4018-a954-c4fce24bb387 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:46.222097 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 595/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jul 28 09:10:46 2022] POST /identity/v3/domains => generated 267 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:10:48.272870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa09d41-4278-46a4-87bc-c51c46e09c5f None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:48.273489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa09d41-4278-46a4-87bc-c51c46e09c5f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:48.273489 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa09d41-4278-46a4-87bc-c51c46e09c5f None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:48.274894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 575/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:48.281506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c1b1b-a552-415f-8ce0-e9194389aa3c None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:48.282151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c1b1b-a552-415f-8ce0-e9194389aa3c None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:48.282151 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c1b1b-a552-415f-8ce0-e9194389aa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:48.302300 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-663c1b1b-a552-415f-8ce0-e9194389aa3c None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:48.303537 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-663c1b1b-a552-415f-8ce0-e9194389aa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:48.376655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 596/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:48.400839 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa09d41-4278-46a4-87bc-c51c46e09c5f None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:48.512600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa09d41-4278-46a4-87bc-c51c46e09c5f None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:48.549174 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eb8e8-6bf1-4568-a392-faee2bd75fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:48.549865 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eb8e8-6bf1-4568-a392-faee2bd75fa3 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:48.550147 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eb8e8-6bf1-4568-a392-faee2bd75fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:48.550439 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eb8e8-6bf1-4568-a392-faee2bd75fa3 None admin] PATH_INFO: `/v3/domains/8ab1b2caf58648a8b3bf5cb65c9439ce` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:48.561375 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319eb8e8-6bf1-4568-a392-faee2bd75fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8ab1b2caf58648a8b3bf5cb65c9439ce)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:48.562879 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319eb8e8-6bf1-4568-a392-faee2bd75fa3 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:48.564541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 576/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 09:10:48 2022] GET /identity/v3/domains/8ab1b2caf58648a8b3bf5cb65c9439ce => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:48.571329 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c1b1b-a552-415f-8ce0-e9194389aa3c None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:48.643301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:48.678495 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c1b1b-a552-415f-8ce0-e9194389aa3c None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:48.714182 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062b175-2eb0-4089-9de5-d227b106d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:48.714818 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062b175-2eb0-4089-9de5-d227b106d90b None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:48.715105 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062b175-2eb0-4089-9de5-d227b106d90b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:48.715389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062b175-2eb0-4089-9de5-d227b106d90b None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:48.716092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062b175-2eb0-4089-9de5-d227b106d90b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:48.717562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062b175-2eb0-4089-9de5-d227b106d90b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:48.748733 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 597/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:10:48 2022] POST /identity/v3/projects => generated 361 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:10:50.756084 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d47a78-2fea-4af0-89a9-0d5c2dcc137d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:50.756692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d47a78-2fea-4af0-89a9-0d5c2dcc137d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:50.756692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d47a78-2fea-4af0-89a9-0d5c2dcc137d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:50.758026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 577/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:50.763668 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f4ce-c0ad-48c9-8296-a67cbe09f79a None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:50.764154 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f4ce-c0ad-48c9-8296-a67cbe09f79a None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:50.764273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f4ce-c0ad-48c9-8296-a67cbe09f79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:50.783871 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-6dc8f4ce-c0ad-48c9-8296-a67cbe09f79a None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:50.786042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-6dc8f4ce-c0ad-48c9-8296-a67cbe09f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:50.872083 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 598/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:50.899110 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d47a78-2fea-4af0-89a9-0d5c2dcc137d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:50.960803 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:51.021312 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d47a78-2fea-4af0-89a9-0d5c2dcc137d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:51.059291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a0548-1402-4055-aeca-76fe51d9e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:51.059960 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a0548-1402-4055-aeca-76fe51d9e7bb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:51.060245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a0548-1402-4055-aeca-76fe51d9e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:51.060528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a0548-1402-4055-aeca-76fe51d9e7bb None admin] PATH_INFO: `/v3/domains/8ab1b2caf58648a8b3bf5cb65c9439ce` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:51.061733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a0548-1402-4055-aeca-76fe51d9e7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=8ab1b2caf58648a8b3bf5cb65c9439ce)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:51.063251 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a0548-1402-4055-aeca-76fe51d9e7bb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:51.064959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 578/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 09:10:50 2022] GET /identity/v3/domains/8ab1b2caf58648a8b3bf5cb65c9439ce => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:51.072351 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f4ce-c0ad-48c9-8296-a67cbe09f79a None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:51.129042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:51.193098 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f4ce-c0ad-48c9-8296-a67cbe09f79a None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:51.231356 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ed2a3-fa11-4673-b82a-df86e9314143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:51.232010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ed2a3-fa11-4673-b82a-df86e9314143 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:51.232301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ed2a3-fa11-4673-b82a-df86e9314143 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:51.232582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ed2a3-fa11-4673-b82a-df86e9314143 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:51.233316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ed2a3-fa11-4673-b82a-df86e9314143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:51.234845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ed2a3-fa11-4673-b82a-df86e9314143 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:51.258746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 599/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:10:51 2022] POST /identity/v3/users => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:10:53.354685 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc937-ae29-4ed4-8418-77d8dffbd4f7 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:53.355268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc937-ae29-4ed4-8418-77d8dffbd4f7 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:53.355268 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc937-ae29-4ed4-8418-77d8dffbd4f7 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:53.356620 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 579/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:53.362295 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc02ab6-502d-4d01-a5f7-9256f09e0f94 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:53.362783 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc02ab6-502d-4d01-a5f7-9256f09e0f94 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:53.362907 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc02ab6-502d-4d01-a5f7-9256f09e0f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:53.381797 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-3fc02ab6-502d-4d01-a5f7-9256f09e0f94 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:53.383055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-3fc02ab6-502d-4d01-a5f7-9256f09e0f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:53.454314 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 600/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:10:53.477945 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dc937-ae29-4ed4-8418-77d8dffbd4f7 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:53.594058 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dc937-ae29-4ed4-8418-77d8dffbd4f7 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:53.614962 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:53.630000 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c806cb-dba2-4394-84bb-ef617d51ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:53.630699 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c806cb-dba2-4394-84bb-ef617d51ef59 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:53.630984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c806cb-dba2-4394-84bb-ef617d51ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:53.631260 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c806cb-dba2-4394-84bb-ef617d51ef59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:53.642133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c806cb-dba2-4394-84bb-ef617d51ef59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-41c806cb-dba2-4394-84bb-ef617d51ef59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:53.647115 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:53.648291 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:53.649220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 580/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:53.654868 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc02ab6-502d-4d01-a5f7-9256f09e0f94 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:53.690773 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:53.761598 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc02ab6-502d-4d01-a5f7-9256f09e0f94 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:53.797697 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:53.798360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:53.798648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:53.798928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:53.799659 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:53.799971 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:53.801361 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:53.809793 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 601/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:53.815986 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c806cb-dba2-4394-84bb-ef617d51ef59 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:53.922037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c806cb-dba2-4394-84bb-ef617d51ef59 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:53.958521 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f2b4a-0b10-414c-b960-25fcf80cb539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:53.959177 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f2b4a-0b10-414c-b960-25fcf80cb539 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:53.959471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f2b4a-0b10-414c-b960-25fcf80cb539 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:53.959745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f2b4a-0b10-414c-b960-25fcf80cb539 None admin] PATH_INFO: `/v3/users/2f149006d518482cb8fcca89b34f5f08` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:53.976805 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f2b4a-0b10-414c-b960-25fcf80cb539 None admin] RBAC: Authorizing `identity:get_user(user_id=2f149006d518482cb8fcca89b34f5f08)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:53.978340 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f2b4a-0b10-414c-b960-25fcf80cb539 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:53.980088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 581/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:53 2022] GET /identity/v3/users/2f149006d518482cb8fcca89b34f5f08 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:53.986268 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:54.091904 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:54.096133 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae449d60-01c9-4da7-b259-c560984fb145 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:54.134760 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ff3fe-b948-42df-920b-b51b0c5c76d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:54.135393 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ff3fe-b948-42df-920b-b51b0c5c76d1 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:54.135675 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ff3fe-b948-42df-920b-b51b0c5c76d1 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:54.135951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ff3fe-b948-42df-920b-b51b0c5c76d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:54.212430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ff3fe-b948-42df-920b-b51b0c5c76d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:54.213915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ff3fe-b948-42df-920b-b51b0c5c76d1 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:54.312240 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:54.316815 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 602/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:10:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:10:54.324123 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f2b4a-0b10-414c-b960-25fcf80cb539 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:54.366818 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:54.431169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f2b4a-0b10-414c-b960-25fcf80cb539 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:54.467121 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50154c7-dc32-480e-8557-38ee68351bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:54.467771 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50154c7-dc32-480e-8557-38ee68351bbb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:54.468058 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50154c7-dc32-480e-8557-38ee68351bbb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:54.468343 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50154c7-dc32-480e-8557-38ee68351bbb None admin] PATH_INFO: `/v3/projects/addb6ad02bdb4fd78ece8c64c8ccd3e9` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:54.469567 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50154c7-dc32-480e-8557-38ee68351bbb None admin] RBAC: Authorizing `identity:get_project(project_id=addb6ad02bdb4fd78ece8c64c8ccd3e9)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:54.471050 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50154c7-dc32-480e-8557-38ee68351bbb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:54.472753 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 582/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:54 2022] GET /identity/v3/projects/addb6ad02bdb4fd78ece8c64c8ccd3e9 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:54.479326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ff3fe-b948-42df-920b-b51b0c5c76d1 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:54.583704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ff3fe-b948-42df-920b-b51b0c5c76d1 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:54.619297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17f32f-54cb-41d5-a3e6-8ac365e58fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:54.619844 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17f32f-54cb-41d5-a3e6-8ac365e58fcb None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:54.620128 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17f32f-54cb-41d5-a3e6-8ac365e58fcb None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:54.620413 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17f32f-54cb-41d5-a3e6-8ac365e58fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:54.621214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17f32f-54cb-41d5-a3e6-8ac365e58fcb None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=addb6ad02bdb4fd78ece8c64c8ccd3e9, user.id=2f149006d518482cb8fcca89b34f5f08) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:54.621530 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17f32f-54cb-41d5-a3e6-8ac365e58fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:54.623031 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17f32f-54cb-41d5-a3e6-8ac365e58fcb None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:54.628756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 603/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:10:54 2022] GET /identity/v3/role_assignments?user.id=2f149006d518482cb8fcca89b34f5f08&scope.project.id=addb6ad02bdb4fd78ece8c64c8ccd3e9&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:56.639068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a2a9-422c-439d-83f3-227750a37889 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:56.639675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a2a9-422c-439d-83f3-227750a37889 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:56.639675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a2a9-422c-439d-83f3-227750a37889 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:56.640999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 583/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:56.646601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c9ff-74c2-467b-84bb-dcdc07b6fa1d None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:56.647068 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c9ff-74c2-467b-84bb-dcdc07b6fa1d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:56.647168 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c9ff-74c2-467b-84bb-dcdc07b6fa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:56.666753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-3cd0c9ff-74c2-467b-84bb-dcdc07b6fa1d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:56.667963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-3cd0c9ff-74c2-467b-84bb-dcdc07b6fa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:56.728937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:56.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 604/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:56.762686 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444a2a9-422c-439d-83f3-227750a37889 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:56.879455 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444a2a9-422c-439d-83f3-227750a37889 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:56.888027 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:56.917141 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdcb7f-fc1a-44b0-911b-abd87ac57ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:56.917761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdcb7f-fc1a-44b0-911b-abd87ac57ce2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:56.918044 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdcb7f-fc1a-44b0-911b-abd87ac57ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:56.918351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdcb7f-fc1a-44b0-911b-abd87ac57ce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:56.930074 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdcb7f-fc1a-44b0-911b-abd87ac57ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-4fcdcb7f-fc1a-44b0-911b-abd87ac57ce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:56.935076 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:56.936342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:10:56.937471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 584/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:56.943061 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0c9ff-74c2-467b-84bb-dcdc07b6fa1d None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:57.050518 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0c9ff-74c2-467b-84bb-dcdc07b6fa1d None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:57.093778 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:57.094467 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:57.094762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:57.095015 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:57.095786 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:10:57.096097 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:57.097496 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:57.108205 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 605/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:57.114624 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdcb7f-fc1a-44b0-911b-abd87ac57ce2 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:57.223294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdcb7f-fc1a-44b0-911b-abd87ac57ce2 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:57.260520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c5505-fcf7-453e-8729-d1fb79a90f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:57.261187 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c5505-fcf7-453e-8729-d1fb79a90f2a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:57.261467 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c5505-fcf7-453e-8729-d1fb79a90f2a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:57.261741 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c5505-fcf7-453e-8729-d1fb79a90f2a None admin] PATH_INFO: `/v3/users/2f149006d518482cb8fcca89b34f5f08` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:57.263172 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c5505-fcf7-453e-8729-d1fb79a90f2a None admin] RBAC: Authorizing `identity:get_user(user_id=2f149006d518482cb8fcca89b34f5f08)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:57.264675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c5505-fcf7-453e-8729-d1fb79a90f2a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:57.266463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 585/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:10:57 2022] GET /identity/v3/users/2f149006d518482cb8fcca89b34f5f08 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:57.272859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:57.337407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:57.384562 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538251e-3975-4f39-b153-7a1b5c7c2cbf None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:57.420278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fc199-4d63-443c-a72d-8d008a52e1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:57.420938 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fc199-4d63-443c-a72d-8d008a52e1b8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:57.421225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fc199-4d63-443c-a72d-8d008a52e1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:57.421506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fc199-4d63-443c-a72d-8d008a52e1b8 None admin] PATH_INFO: `/v3/projects/addb6ad02bdb4fd78ece8c64c8ccd3e9` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:57.422762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fc199-4d63-443c-a72d-8d008a52e1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=addb6ad02bdb4fd78ece8c64c8ccd3e9)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:57.424247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fc199-4d63-443c-a72d-8d008a52e1b8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:57.425928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 606/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:10:57 2022] GET /identity/v3/projects/addb6ad02bdb4fd78ece8c64c8ccd3e9 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:10:57.432404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c5505-fcf7-453e-8729-d1fb79a90f2a None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:57.471966 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:57.543858 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c5505-fcf7-453e-8729-d1fb79a90f2a None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:57.584525 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9d038-9604-4224-ad77-dcfd68c659b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:57.585328 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9d038-9604-4224-ad77-dcfd68c659b1 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:57.585737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9d038-9604-4224-ad77-dcfd68c659b1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:57.586265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9d038-9604-4224-ad77-dcfd68c659b1 None admin] PATH_INFO: `/v3/projects/addb6ad02bdb4fd78ece8c64c8ccd3e9/users/2f149006d518482cb8fcca89b34f5f08/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:57.588892 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9d038-9604-4224-ad77-dcfd68c659b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=addb6ad02bdb4fd78ece8c64c8ccd3e9, user_id=2f149006d518482cb8fcca89b34f5f08, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:10:57.590680 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9d038-9604-4224-ad77-dcfd68c659b1 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:10:57.605384 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 586/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:10:57 2022] PUT /identity/v3/projects/addb6ad02bdb4fd78ece8c64c8ccd3e9/users/2f149006d518482cb8fcca89b34f5f08/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:10:59.602582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310fa3e-baeb-4b90-948e-7463603b1fe8 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:59.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310fa3e-baeb-4b90-948e-7463603b1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:59.603147 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310fa3e-baeb-4b90-948e-7463603b1fe8 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:59.604439 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 607/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:10:59.609737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1636d-aa7b-4947-97cd-ae48f8c5af23 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:59.610250 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1636d-aa7b-4947-97cd-ae48f8c5af23 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:59.610351 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1636d-aa7b-4947-97cd-ae48f8c5af23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:59.629415 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-f3d1636d-aa7b-4947-97cd-ae48f8c5af23 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:10:59.630714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-f3d1636d-aa7b-4947-97cd-ae48f8c5af23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:10:59.704196 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 587/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:10:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:10:59.728354 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310fa3e-baeb-4b90-948e-7463603b1fe8 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:59.841404 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310fa3e-baeb-4b90-948e-7463603b1fe8 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:10:59.865401 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:10:59.880011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde255e-4347-4f9c-902a-9f0173e72b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:10:59.880685 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde255e-4347-4f9c-902a-9f0173e72b9a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:10:59.880978 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde255e-4347-4f9c-902a-9f0173e72b9a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:10:59.881260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde255e-4347-4f9c-902a-9f0173e72b9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:10:59.893478 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde255e-4347-4f9c-902a-9f0173e72b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-3fde255e-4347-4f9c-902a-9f0173e72b9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:10:59.899059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:10:59.900308 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:10:59.901440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 608/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:10:59.906631 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1636d-aa7b-4947-97cd-ae48f8c5af23 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:10:59.933111 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:00.015192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1636d-aa7b-4947-97cd-ae48f8c5af23 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:00.056369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:00.057024 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:00.057308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:00.057567 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:00.058369 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:00.058678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:00.060088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:00.069341 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 588/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:10:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:00.075759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde255e-4347-4f9c-902a-9f0173e72b9a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:00.187569 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde255e-4347-4f9c-902a-9f0173e72b9a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:00.234517 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fce3-c72a-45ce-80d9-2ceed8a3fbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:00.235214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fce3-c72a-45ce-80d9-2ceed8a3fbb2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:00.235502 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fce3-c72a-45ce-80d9-2ceed8a3fbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:00.235756 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fce3-c72a-45ce-80d9-2ceed8a3fbb2 None admin] PATH_INFO: `/v3/users/2f149006d518482cb8fcca89b34f5f08` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:00.237131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fce3-c72a-45ce-80d9-2ceed8a3fbb2 None admin] RBAC: Authorizing `identity:get_user(user_id=2f149006d518482cb8fcca89b34f5f08)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:00.238690 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fce3-c72a-45ce-80d9-2ceed8a3fbb2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:00.240403 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 609/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:11:00 2022] GET /identity/v3/users/2f149006d518482cb8fcca89b34f5f08 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:11:00.246508 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:00.345883 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:00.363757 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f1267-74e5-43f6-872e-c57cc6b07256 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:00.402167 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd8931-dc0a-467b-b85c-aa1185c9ccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:00.402834 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd8931-dc0a-467b-b85c-aa1185c9ccce None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:00.403118 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd8931-dc0a-467b-b85c-aa1185c9ccce None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:00.403400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd8931-dc0a-467b-b85c-aa1185c9ccce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:00.480218 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd8931-dc0a-467b-b85c-aa1185c9ccce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:00.481620 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd8931-dc0a-467b-b85c-aa1185c9ccce None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:00.573553 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:00.591873 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 589/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:11:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:11:00.598407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fce3-c72a-45ce-80d9-2ceed8a3fbb2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:00.659040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:00.720081 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fce3-c72a-45ce-80d9-2ceed8a3fbb2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:00.762960 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b380a-8d7d-48b9-a0f1-3a686750fb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:00.763534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b380a-8d7d-48b9-a0f1-3a686750fb0b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:00.763819 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b380a-8d7d-48b9-a0f1-3a686750fb0b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:00.764261 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b380a-8d7d-48b9-a0f1-3a686750fb0b None admin] PATH_INFO: `/v3/projects/addb6ad02bdb4fd78ece8c64c8ccd3e9` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:00.765548 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b380a-8d7d-48b9-a0f1-3a686750fb0b None admin] RBAC: Authorizing `identity:get_project(project_id=addb6ad02bdb4fd78ece8c64c8ccd3e9)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:00.767036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b380a-8d7d-48b9-a0f1-3a686750fb0b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:00.768780 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 610/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:11:00 2022] GET /identity/v3/projects/addb6ad02bdb4fd78ece8c64c8ccd3e9 => generated 361 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:00.775506 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd8931-dc0a-467b-b85c-aa1185c9ccce None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:00.889035 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd8931-dc0a-467b-b85c-aa1185c9ccce None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:00.924552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f24bc-9bcb-44d0-b56b-19b630ac239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:00.925183 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f24bc-9bcb-44d0-b56b-19b630ac239d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:00.925464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f24bc-9bcb-44d0-b56b-19b630ac239d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:00.925726 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f24bc-9bcb-44d0-b56b-19b630ac239d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:00.926628 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f24bc-9bcb-44d0-b56b-19b630ac239d None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=addb6ad02bdb4fd78ece8c64c8ccd3e9, user.id=2f149006d518482cb8fcca89b34f5f08) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:00.926941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f24bc-9bcb-44d0-b56b-19b630ac239d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:00.928404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f24bc-9bcb-44d0-b56b-19b630ac239d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:00.935103 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 590/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:11:00 2022] GET /identity/v3/role_assignments?user.id=2f149006d518482cb8fcca89b34f5f08&scope.project.id=addb6ad02bdb4fd78ece8c64c8ccd3e9&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:10.754488 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=41 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:10.932047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=75 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:57.618751 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b157-9347-4fd5-a813-570b79762021 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.619357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b157-9347-4fd5-a813-570b79762021 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.619357 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b157-9347-4fd5-a813-570b79762021 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.620607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 611/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:57.626906 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945912-b48c-477d-a5b5-81da282f9979 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.627729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945912-b48c-477d-a5b5-81da282f9979 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.627729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945912-b48c-477d-a5b5-81da282f9979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.646989 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-dd945912-b48c-477d-a5b5-81da282f9979 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:57.648233 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-dd945912-b48c-477d-a5b5-81da282f9979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:57.739094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 591/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:11:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:11:57.764799 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0b157-9347-4fd5-a813-570b79762021 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.901670 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0b157-9347-4fd5-a813-570b79762021 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.958846 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.959605 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.960042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.960427 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.977787 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.979901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:11:58.005919 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:11:58.008433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:11:58.010238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:11:58.010238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:11:58.010238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:11:58.010238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:11:58.010238 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 612/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:11:58.014227 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd945912-b48c-477d-a5b5-81da282f9979 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:58.157872 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd945912-b48c-477d-a5b5-81da282f9979 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:58.188549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:58.196974 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1e27d-1aeb-4cb0-957c-21fa92d47651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:58.197664 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1e27d-1aeb-4cb0-957c-21fa92d47651 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:58.197950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1e27d-1aeb-4cb0-957c-21fa92d47651 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:58.198254 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1e27d-1aeb-4cb0-957c-21fa92d47651 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:58.198955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1e27d-1aeb-4cb0-957c-21fa92d47651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:58.199265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1e27d-1aeb-4cb0-957c-21fa92d47651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:58.200671 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1e27d-1aeb-4cb0-957c-21fa92d47651 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:58.224585 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 592/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:11:58.226023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:58.282699 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:58.340460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6677e0-a126-4be5-9fd1-fd1c756b32c5 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:58.375985 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b2233-8508-4576-b0cf-2d0a902d058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:58.376674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b2233-8508-4576-b0cf-2d0a902d058c None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:58.376963 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b2233-8508-4576-b0cf-2d0a902d058c None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:58.377235 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b2233-8508-4576-b0cf-2d0a902d058c None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:58.377915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b2233-8508-4576-b0cf-2d0a902d058c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:58.379569 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b2233-8508-4576-b0cf-2d0a902d058c None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:58.404172 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 613/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:11:58 2022] POST /identity/v3/users => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:00.907239 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefea0b-cea5-4ef8-b4b9-3034a853bd8e None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:00.908173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefea0b-cea5-4ef8-b4b9-3034a853bd8e None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:00.908173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefea0b-cea5-4ef8-b4b9-3034a853bd8e None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:00.909587 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 593/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:00.918208 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7464-ecfa-43e9-b0e6-e09696f57b94 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:00.918208 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7464-ecfa-43e9-b0e6-e09696f57b94 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:00.918208 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7464-ecfa-43e9-b0e6-e09696f57b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:00.949968 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-5fdc7464-ecfa-43e9-b0e6-e09696f57b94 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:00.960945 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-5fdc7464-ecfa-43e9-b0e6-e09696f57b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:01.102169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 614/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:01.142222 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefea0b-cea5-4ef8-b4b9-3034a853bd8e None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:01.283358 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefea0b-cea5-4ef8-b4b9-3034a853bd8e None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:01.330182 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a06599-c114-4d7c-8786-5393b73cfe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:01.331377 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a06599-c114-4d7c-8786-5393b73cfe63 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:01.331721 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a06599-c114-4d7c-8786-5393b73cfe63 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:01.332032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a06599-c114-4d7c-8786-5393b73cfe63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:01.349971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a06599-c114-4d7c-8786-5393b73cfe63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-59a06599-c114-4d7c-8786-5393b73cfe63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:12:01.356779 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:01.357999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:01.359397 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 594/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:12:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:01.365260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc7464-ecfa-43e9-b0e6-e09696f57b94 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:01.421407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:01.515369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc7464-ecfa-43e9-b0e6-e09696f57b94 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:01.576154 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:01.576796 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:01.577089 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:01.577402 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:01.578301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:01.578664 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:01.580260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:01.600456 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 615/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:12:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:01.603722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a06599-c114-4d7c-8786-5393b73cfe63 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:01.720326 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:01.819870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a06599-c114-4d7c-8786-5393b73cfe63 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:01.890532 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:01.891871 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:01.892366 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:01.892944 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:01.913049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:01.915394 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:01.946509 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:01.948295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:01.950174 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:01.950174 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:01.950174 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:01.950174 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:01.950174 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 595/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:01.958777 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:02.141920 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:02.147638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c32de-046f-44b7-a59e-402d742ab3d8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:02.225655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:02.226728 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:02.227352 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:02.227804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:02.229123 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:02.229736 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:02.232567 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:02.252485 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 616/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:02.259304 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:02.420235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721388a2-400b-4863-a417-1a926a138c0b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:02.422775 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:02.484088 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:02.485211 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:02.485770 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:02.486236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:02.502613 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:02.505496 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:12:02.529302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:02.530712 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:02.531941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:02.535152 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 596/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:12:02.547432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:02.779712 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76365bd5-f804-4898-81e3-92b0dcbb465a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:02.842321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:02.843098 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:02.843627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:02.844085 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:02.845503 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:02.845998 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:02.848341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:02.901144 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 617/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:12:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:02.902606 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.118235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b91ed-5565-4ffe-aa6e-4fdb228a98aa None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.194032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccfff4-d662-43ab-bc22-1258618a2ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.195158 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccfff4-d662-43ab-bc22-1258618a2ed4 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.195599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccfff4-d662-43ab-bc22-1258618a2ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.196023 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccfff4-d662-43ab-bc22-1258618a2ed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.245331 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:03.324984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccfff4-d662-43ab-bc22-1258618a2ed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.327529 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccfff4-d662-43ab-bc22-1258618a2ed4 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.525800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 597/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:12:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:12:03.534899 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.565601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:03.689993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24f742-5cf9-466c-99ac-b301193a2c9f None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.757450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.758367 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.758928 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.759144 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.774515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.776430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:03.795609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:03.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:03.799937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:03.799937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:03.799937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:03.799937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:03.799937 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 618/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:03.807734 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccfff4-d662-43ab-bc22-1258618a2ed4 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.889436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:03.973346 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccfff4-d662-43ab-bc22-1258618a2ed4 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.014792 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.015593 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.015967 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.016385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.017263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:04.017670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.019262 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.033225 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 598/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:04.039391 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.155332 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:04.200319 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd828d81-0d50-4d65-97c3-735b8eedc56b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.249311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.250359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.250692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.251040 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.261421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.263575 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:12:04.277137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:04.279437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:04.281613 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 619/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:12:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:04.285128 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.423156 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac7945-5ee7-4899-8861-9a54bc97da7a None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.457641 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:04.474908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787041e0-eead-474d-9013-1dad1e62fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.475728 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787041e0-eead-474d-9013-1dad1e62fb6a None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.476103 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787041e0-eead-474d-9013-1dad1e62fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.476510 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787041e0-eead-474d-9013-1dad1e62fb6a None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.477580 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787041e0-eead-474d-9013-1dad1e62fb6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:04.478049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787041e0-eead-474d-9013-1dad1e62fb6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.480083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787041e0-eead-474d-9013-1dad1e62fb6a None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.503683 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 599/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:12:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:04.513804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.665418 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511637d3-1bb7-4d3b-8883-b1e00400bd46 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.709363 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:04.719648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e18137-4c21-4445-921c-2a1acfa8c9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.720835 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e18137-4c21-4445-921c-2a1acfa8c9e4 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.721321 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e18137-4c21-4445-921c-2a1acfa8c9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.721879 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e18137-4c21-4445-921c-2a1acfa8c9e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.723689 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e18137-4c21-4445-921c-2a1acfa8c9e4 None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=fef14e50634a457c82ea65c7662b5a0a) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:04.724312 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e18137-4c21-4445-921c-2a1acfa8c9e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.726637 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e18137-4c21-4445-921c-2a1acfa8c9e4 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.750144 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 620/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:12:04 2022] GET /identity/v3/role_assignments?user.id=fef14e50634a457c82ea65c7662b5a0a&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:07.089046 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5be7-0b47-422b-8f73-664784d4c83d None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.090012 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5be7-0b47-422b-8f73-664784d4c83d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.090012 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5be7-0b47-422b-8f73-664784d4c83d None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.091037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 600/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:07.097741 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365b30c-1ebf-4307-9cf8-f2adf8f24d12 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.098606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365b30c-1ebf-4307-9cf8-f2adf8f24d12 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.098606 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365b30c-1ebf-4307-9cf8-f2adf8f24d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.122622 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-9365b30c-1ebf-4307-9cf8-f2adf8f24d12 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:07.124318 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-9365b30c-1ebf-4307-9cf8-f2adf8f24d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:07.241269 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 621/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:12:07.269045 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d5be7-0b47-422b-8f73-664784d4c83d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.417813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d5be7-0b47-422b-8f73-664784d4c83d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.470164 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0354fc-9c32-46e2-8e7a-204f51e13c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.471030 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0354fc-9c32-46e2-8e7a-204f51e13c9b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.471333 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0354fc-9c32-46e2-8e7a-204f51e13c9b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.471653 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0354fc-9c32-46e2-8e7a-204f51e13c9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.490155 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0354fc-9c32-46e2-8e7a-204f51e13c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-4e0354fc-9c32-46e2-8e7a-204f51e13c9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:12:07.497021 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:07.503985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:07.505039 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 601/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:07.505893 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365b30c-1ebf-4307-9cf8-f2adf8f24d12 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.644779 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365b30c-1ebf-4307-9cf8-f2adf8f24d12 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.672042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:07.684625 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.685292 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.685724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.685981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.686762 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] RBAC: Adding query filter params (name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:07.688214 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.688453 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.703467 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 622/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:07.707316 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0354fc-9c32-46e2-8e7a-204f51e13c9b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.787249 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:07.869602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0354fc-9c32-46e2-8e7a-204f51e13c9b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.922878 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.924055 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.924734 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.925063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.939323 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.941342 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:07.960740 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:07.963037 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:07.964988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:07.964988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:07.964988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:07.964988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:07.964988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 602/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:07.971688 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.105808 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1e3ab-3392-4ecc-a7dc-b9573c07d4ed None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.155520 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.156195 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.156460 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.156732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.157459 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:08.157767 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.159317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.173748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 623/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:08.179867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.320985 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:08.336697 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35e820-6806-4ba4-859f-3329d08d6901 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.379436 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.380173 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.380461 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.380848 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] PATH_INFO: `/v3/users/placement` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.391411 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.393169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:12:08.408122 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:08.410294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:08.411852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 603/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:08.417146 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.483770 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:08.558391 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc859ba9-4682-4c2e-ba96-e8b33f4e248d None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.598904 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.599631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.599994 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.600328 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.601254 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:08.601617 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.604045 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.629649 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 624/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:12:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:08.636210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.788073 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5efea-b192-433f-9769-ccb24e3b3f56 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.836612 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.837722 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.838264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.838754 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.856586 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.858400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:08.877434 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:08.879107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:08.880210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:08.880210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:08.880210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:08.880210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:08.880210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 604/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:08.886174 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.032860 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6954-33b5-49df-9a46-0fbbcdbfcde2 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.036960 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:09.093253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.094074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.094534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.095789 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.095908 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:09.096359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.099010 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.115171 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 625/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:09.122809 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.135848 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:09.279976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd375c8-fc7a-4cd1-b058-3600b42f830e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.324783 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.325935 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.326457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.326969 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.339975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.342765 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:12:09.364921 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:09.366242 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:09.367472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 605/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:12:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:09.373417 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.492625 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d60896-2c18-4309-b852-8d445b197bce None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.537194 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67325272-8875-43a5-8c0a-4594147c9f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.538373 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325272-8875-43a5-8c0a-4594147c9f55 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.538700 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325272-8875-43a5-8c0a-4594147c9f55 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.539085 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325272-8875-43a5-8c0a-4594147c9f55 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.540224 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67325272-8875-43a5-8c0a-4594147c9f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:09.540854 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67325272-8875-43a5-8c0a-4594147c9f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.543017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67325272-8875-43a5-8c0a-4594147c9f55 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.565331 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 626/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:12:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:09.566760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.640683 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:09.683866 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2341-530e-4625-8869-3b832cf48fa1 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.724264 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8d04d-eb25-4337-9137-d7c804e971c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.724969 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8d04d-eb25-4337-9137-d7c804e971c3 None admin] REQUEST_METHOD: `PUT` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.725278 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8d04d-eb25-4337-9137-d7c804e971c3 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.725541 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8d04d-eb25-4337-9137-d7c804e971c3 None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/fef14e50634a457c82ea65c7662b5a0a/roles/5d22bf96fee74e9eaf2b9cdba5b26ead` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.742148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8d04d-eb25-4337-9137-d7c804e971c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=fef14e50634a457c82ea65c7662b5a0a, role_id=5d22bf96fee74e9eaf2b9cdba5b26ead)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.743988 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8d04d-eb25-4337-9137-d7c804e971c3 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.766432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 606/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:12:09 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/fef14e50634a457c82ea65c7662b5a0a/roles/5d22bf96fee74e9eaf2b9cdba5b26ead => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:11.960389 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f98b2-8e1d-4c5f-9ffd-54d866fdbd9a None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:11.961080 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f98b2-8e1d-4c5f-9ffd-54d866fdbd9a None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:11.961358 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f98b2-8e1d-4c5f-9ffd-54d866fdbd9a None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:11.963901 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 627/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:11.970378 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec105a0-23a3-457c-b707-cb53f45b821d None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:11.970936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec105a0-23a3-457c-b707-cb53f45b821d None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:11.970936 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec105a0-23a3-457c-b707-cb53f45b821d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:11.990620 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-0ec105a0-23a3-457c-b707-cb53f45b821d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:11.991476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-0ec105a0-23a3-457c-b707-cb53f45b821d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:12.076599 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 607/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:12.109663 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f98b2-8e1d-4c5f-9ffd-54d866fdbd9a None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:12.141115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:12.268011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f98b2-8e1d-4c5f-9ffd-54d866fdbd9a None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:12.312602 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98cb55-ba72-4f56-abbe-d6f54ce3d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:12.313329 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98cb55-ba72-4f56-abbe-d6f54ce3d0dd None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:12.313585 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98cb55-ba72-4f56-abbe-d6f54ce3d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:12.313897 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98cb55-ba72-4f56-abbe-d6f54ce3d0dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:12.327230 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98cb55-ba72-4f56-abbe-d6f54ce3d0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-8a98cb55-ba72-4f56-abbe-d6f54ce3d0dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:12:12.332801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:12.335515 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:12.337724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 628/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:12:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:12.340880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec105a0-23a3-457c-b707-cb53f45b821d None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:12.396308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:12.465549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec105a0-23a3-457c-b707-cb53f45b821d None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:12.507724 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:12.508355 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:12.508640 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:12.508918 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:12.509671 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] RBAC: Adding query filter params (name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:12.510017 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:12.511652 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:12.522573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 608/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:12:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:12.529258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98cb55-ba72-4f56-abbe-d6f54ce3d0dd None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:12.644153 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:12.672490 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98cb55-ba72-4f56-abbe-d6f54ce3d0dd None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:12.718012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:12.718880 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:12.719317 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:12.719791 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:12.735639 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:12.737231 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:12.776881 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:12.778813 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:12.779838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:12.779838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:12.779838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:12.779838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:12.779838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 629/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:12.784875 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:12.927556 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6edc8c-8c69-4824-97fd-09f6c266bcfd None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:12.938951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:12.992495 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:12.993244 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:12.993416 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:12.993573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:12.994338 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:12.994654 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:12.996069 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:13.016863 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 609/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:13.018161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:13.162339 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d914ba-ba0c-4202-a8f7-6c55ba1f4d75 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:13.206519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:13.207212 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:13.207491 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:13.207798 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:13.215867 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:13.217470 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:13.219164 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:12:13.233925 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:13.235307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:13.236636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:13.240320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 630/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:13.241870 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:13.363716 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2660a4a-b99b-4dbb-abd2-36758814ce47 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:13.400514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:13.401174 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:13.401435 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:13.401711 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:13.402552 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:13.402871 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:13.404298 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:13.428774 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 610/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:12:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:13.434993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:13.553625 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef8dc4-3dbb-4aeb-a869-ac093c4c43f8 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:13.597020 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57438b17-81f6-4428-8e8a-88d1dd7bcce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:13.598249 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57438b17-81f6-4428-8e8a-88d1dd7bcce1 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:13.598714 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57438b17-81f6-4428-8e8a-88d1dd7bcce1 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:13.599316 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57438b17-81f6-4428-8e8a-88d1dd7bcce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:13.692571 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:13.703726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57438b17-81f6-4428-8e8a-88d1dd7bcce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:13.706632 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57438b17-81f6-4428-8e8a-88d1dd7bcce1 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:13.876059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 631/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:12:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:12:13.882890 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:13.915506 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:14.007349 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf961b-755e-4f48-bf6f-12a6ff3f1c8e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:14.046975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:14.047650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:14.047939 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:14.048220 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:14.061189 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:14.062770 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:14.080038 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:14.081464 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:14.082673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:14.082673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:14.082673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:14.082673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:14.082673 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 611/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:14.087704 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57438b17-81f6-4428-8e8a-88d1dd7bcce1 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:14.237320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:14.246042 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57438b17-81f6-4428-8e8a-88d1dd7bcce1 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:14.301429 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:14.302088 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:14.302498 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:14.302856 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:14.303965 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:14.304386 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:14.306074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:14.321686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 632/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:14.332431 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:14.445725 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:14.473639 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa43e6-5f2b-4577-96e3-d6fc010060b7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:14.515048 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:14.515714 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:14.516026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:14.516337 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:14.524054 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:14.525523 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:12:14.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:14.541451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:14.543773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 612/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:14.547587 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:14.700334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74a05-0cc3-4add-9c45-16da5cc9b2ff None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:14.746759 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b0e47-1bf5-46ef-b5f1-9435f1ba5ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:14.747905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b0e47-1bf5-46ef-b5f1-9435f1ba5ca6 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:14.748784 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b0e47-1bf5-46ef-b5f1-9435f1ba5ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:14.748784 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b0e47-1bf5-46ef-b5f1-9435f1ba5ca6 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:14.750060 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b0e47-1bf5-46ef-b5f1-9435f1ba5ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:14.750805 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b0e47-1bf5-46ef-b5f1-9435f1ba5ca6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:14.753537 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b0e47-1bf5-46ef-b5f1-9435f1ba5ca6 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:14.772077 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 633/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:14.778657 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:14.899032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db740a-4aec-4a1c-9d23-0c6c68eddf7d None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:14.953923 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abead49-bea7-463e-a60d-66fa732fe7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:14.954573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abead49-bea7-463e-a60d-66fa732fe7b9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:14.954815 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abead49-bea7-463e-a60d-66fa732fe7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:14.955100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abead49-bea7-463e-a60d-66fa732fe7b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:14.955991 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abead49-bea7-463e-a60d-66fa732fe7b9 None admin] RBAC: Adding query filter params (role.id=5d22bf96fee74e9eaf2b9cdba5b26ead, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=fef14e50634a457c82ea65c7662b5a0a) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:14.956131 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abead49-bea7-463e-a60d-66fa732fe7b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:14.957602 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abead49-bea7-463e-a60d-66fa732fe7b9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:14.964753 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 613/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:12:14 2022] GET /identity/v3/role_assignments?user.id=fef14e50634a457c82ea65c7662b5a0a&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=5d22bf96fee74e9eaf2b9cdba5b26ead => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:17.141189 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa49ca-3757-46f3-9bac-e0b237dedf98 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:17.142629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa49ca-3757-46f3-9bac-e0b237dedf98 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:17.142629 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa49ca-3757-46f3-9bac-e0b237dedf98 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:17.144225 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 634/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:17.152211 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b2bf1-bf87-4231-9363-fb63c04467a0 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:17.153059 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b2bf1-bf87-4231-9363-fb63c04467a0 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:17.153832 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b2bf1-bf87-4231-9363-fb63c04467a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:17.164678 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:17.181887 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-a69b2bf1-bf87-4231-9363-fb63c04467a0 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:17.183275 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-a69b2bf1-bf87-4231-9363-fb63c04467a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:17.277010 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 614/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:17.300587 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa49ca-3757-46f3-9bac-e0b237dedf98 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:17.321072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:17.452354 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa49ca-3757-46f3-9bac-e0b237dedf98 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:17.504875 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52ec3f-283d-45d3-8fb5-c65d814787b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:17.506411 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52ec3f-283d-45d3-8fb5-c65d814787b9 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:17.507104 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52ec3f-283d-45d3-8fb5-c65d814787b9 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:17.507731 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52ec3f-283d-45d3-8fb5-c65d814787b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:17.528427 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52ec3f-283d-45d3-8fb5-c65d814787b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-eb52ec3f-283d-45d3-8fb5-c65d814787b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:12:17.539304 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:17.541134 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:17.543247 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 635/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:12:17.553149 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b2bf1-bf87-4231-9363-fb63c04467a0 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:17.697962 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b2bf1-bf87-4231-9363-fb63c04467a0 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:17.733032 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:17.739932 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:17.740600 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:17.740882 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:17.741159 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:17.741920 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:17.742382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:17.743823 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:17.759942 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 615/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:17.769081 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52ec3f-283d-45d3-8fb5-c65d814787b9 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:17.888277 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:17.932447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52ec3f-283d-45d3-8fb5-c65d814787b9 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:17.979671 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:17.980471 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:17.980844 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:17.981430 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:17.995929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:17.997444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:18.018578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:18.020306 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:18.021489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:18.021489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:18.021489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:18.021489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:18.021489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 636/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:18.027007 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:18.198085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269500-1994-43f8-b535-6ab646659845 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:18.241475 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:18.242511 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:18.242950 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:18.243387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:18.244406 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:18.244871 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:18.246812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:18.263424 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 616/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:18.269835 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:18.405341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e1c3e-7f06-466a-86af-32c9f0408eaa None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:18.442775 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:18.453148 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:18.453977 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:18.454432 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:18.454568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] PATH_INFO: `/v3/users/placement` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:18.465092 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:18.466793 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:12:18.486374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:18.488653 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:18.490063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 637/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:12:18.498645 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:18.580650 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:18.649182 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b5f36-f36e-47a3-b022-dd1c76b08367 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:18.697958 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:18.698656 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:18.698934 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:18.699204 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:18.700066 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:18.700347 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:18.701917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:18.733105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 617/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:12:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:18.735055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:18.875989 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018540f0-f193-425f-92b4-b6ae0e9cb75e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:18.922213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897de8a-6f76-43a4-9d1c-64775ab63e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:18.922845 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897de8a-6f76-43a4-9d1c-64775ab63e51 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:18.923208 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897de8a-6f76-43a4-9d1c-64775ab63e51 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:18.923521 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897de8a-6f76-43a4-9d1c-64775ab63e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:18.996374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:19.012537 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897de8a-6f76-43a4-9d1c-64775ab63e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:19.015213 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897de8a-6f76-43a4-9d1c-64775ab63e51 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:19.171692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 638/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:12:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:12:19.181658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:19.325201 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2693de-dc00-4779-aeaf-a4298231711b None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:19.342747 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:19.369720 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:19.370517 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:19.370824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:19.371096 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:19.384579 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:19.386245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:19.405668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:19.407696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:19.409940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:19.409940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:19.409940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:19.409940 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:19.414148 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 618/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:19.414360 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897de8a-6f76-43a4-9d1c-64775ab63e51 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:19.544065 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:19.564763 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897de8a-6f76-43a4-9d1c-64775ab63e51 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:19.627431 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:19.628459 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:19.628891 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:19.629291 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:19.630439 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:19.630801 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:19.632310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:19.648630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 639/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:19.658948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:19.809246 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ce68-9486-41e9-a630-e8760220e464 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:19.853956 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:19.854734 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:19.855048 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:19.855367 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:19.863822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:19.865539 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:12:19.881170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:19.882850 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:19.884801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 619/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:19.890639 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:20.038228 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7411-809d-4eed-b80e-644ea0babd5e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:20.083860 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ee53d-348a-4b6f-a3ba-3420c1194a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:20.084405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ee53d-348a-4b6f-a3ba-3420c1194a30 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:20.084689 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ee53d-348a-4b6f-a3ba-3420c1194a30 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:20.084967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ee53d-348a-4b6f-a3ba-3420c1194a30 None admin] PATH_INFO: `/v3/projects` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:20.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ee53d-348a-4b6f-a3ba-3420c1194a30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:20.086047 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ee53d-348a-4b6f-a3ba-3420c1194a30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:20.087535 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ee53d-348a-4b6f-a3ba-3420c1194a30 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:20.102678 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 640/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:20.109093 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:20.151899 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:20.273111 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce73e81-aef2-4e07-9a07-6e3687236a18 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:20.320899 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c2486-7db3-401e-9496-f3ac1d08e4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:20.321679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c2486-7db3-401e-9496-f3ac1d08e4fb None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:20.321990 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c2486-7db3-401e-9496-f3ac1d08e4fb None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:20.322451 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c2486-7db3-401e-9496-f3ac1d08e4fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:20.323308 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c2486-7db3-401e-9496-f3ac1d08e4fb None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=fef14e50634a457c82ea65c7662b5a0a) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:20.323801 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c2486-7db3-401e-9496-f3ac1d08e4fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:20.325948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c2486-7db3-401e-9496-f3ac1d08e4fb None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:20.333107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 620/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:12:20 2022] GET /identity/v3/role_assignments?user.id=fef14e50634a457c82ea65c7662b5a0a&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:22.788772 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebd1b3-171a-4c62-a803-8578df33d502 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:22.789450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebd1b3-171a-4c62-a803-8578df33d502 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:22.789450 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebd1b3-171a-4c62-a803-8578df33d502 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:22.791232 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 641/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:22.797109 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a48ae-08ad-4793-9dc6-13cb8c4bcf7b None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:22.797756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a48ae-08ad-4793-9dc6-13cb8c4bcf7b None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:22.797756 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a48ae-08ad-4793-9dc6-13cb8c4bcf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:22.820203 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-755a48ae-08ad-4793-9dc6-13cb8c4bcf7b None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:22.821472 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-755a48ae-08ad-4793-9dc6-13cb8c4bcf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:22.910068 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 621/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:22.939795 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebd1b3-171a-4c62-a803-8578df33d502 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:22.957436 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:23.098459 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebd1b3-171a-4c62-a803-8578df33d502 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:23.141631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba06a2-64d2-44ad-8b88-bf7e26400545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:23.142359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba06a2-64d2-44ad-8b88-bf7e26400545 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:23.142649 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba06a2-64d2-44ad-8b88-bf7e26400545 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:23.142939 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba06a2-64d2-44ad-8b88-bf7e26400545 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:23.157001 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba06a2-64d2-44ad-8b88-bf7e26400545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-49ba06a2-64d2-44ad-8b88-bf7e26400545 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:12:23.162283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:23.164338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:23.166273 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 642/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:12:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:12:23.171604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a48ae-08ad-4793-9dc6-13cb8c4bcf7b None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:23.198036 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:23.303512 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a48ae-08ad-4793-9dc6-13cb8c4bcf7b None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:23.352888 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:23.353458 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:23.353744 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:23.354030 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] PATH_INFO: `/v3/roles` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:23.355237 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] RBAC: Adding query filter params (name=admin) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:23.355713 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:23.358192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:23.370047 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 622/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:12:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:23.377160 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba06a2-64d2-44ad-8b88-bf7e26400545 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:23.464926 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:23.501983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba06a2-64d2-44ad-8b88-bf7e26400545 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:23.539302 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:23.539967 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:23.540248 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:23.540526 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:23.552549 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:23.554044 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:23.571222 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:23.573155 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:23.575071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:23.575071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:23.575071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:23.575071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:23.575071 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 643/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:23.578773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:23.678813 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:23.704974 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ece674-7dee-468a-a412-d703d7bc2847 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:23.745139 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:23.745919 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:23.746356 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:23.746733 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:23.747555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:23.747975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:23.749474 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:23.762811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 623/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:23.768869 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:23.874640 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646710ec-d7f9-4e13-be9b-ed11df03d6ab None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:23.896987 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:23.917456 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:23.918710 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:23.919264 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:23.919686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] PATH_INFO: `/v3/users/placement` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:23.931662 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:23.933090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:12:23.949036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:23.952601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:23.955002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 644/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:23.957099 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:24.076954 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93de98-6f91-4a13-bf54-4b40ff593e65 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:24.113886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:24.114625 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:24.114955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:24.115277 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] PATH_INFO: `/v3/users` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:24.116133 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:24.116488 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:24.117995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:24.128158 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:24.138083 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 624/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:12:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:24.145676 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:24.268882 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206b884-9c1b-48f7-98ae-a31e88d1d187 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:24.316737 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:24.317745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:24.318035 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:24.318350 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:24.332198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:24.333746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:24.351512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:24.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:24.354445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:24.354445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:24.354445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:24.354445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:24.354445 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 645/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:24.359165 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:24.482816 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe5353-6382-4c28-bcc7-3c64e56ee05e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:24.524025 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:24.524687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:24.524975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:24.525253 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:24.525976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:24.526322 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:24.527789 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:24.541236 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 625/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:24.548322 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:24.586738 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:24.664561 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d508a-9094-4d69-a0e2-73f60b88a93e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:24.702449 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:24.703154 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:24.703449 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:24.703727 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:24.711369 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:24.712880 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:12:24.725118 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:24.726298 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:24.727310 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 646/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:12:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:24.732633 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:24.798211 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:24.844943 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635108-d317-4ef4-b81f-4c25e8bf503e None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:24.882545 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7d599-e17d-4e5a-ae48-307cdb7bd581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:24.883404 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d599-e17d-4e5a-ae48-307cdb7bd581 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:24.883799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d599-e17d-4e5a-ae48-307cdb7bd581 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:24.884206 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d599-e17d-4e5a-ae48-307cdb7bd581 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:24.885111 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7d599-e17d-4e5a-ae48-307cdb7bd581 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:24.885544 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7d599-e17d-4e5a-ae48-307cdb7bd581 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:24.887225 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7d599-e17d-4e5a-ae48-307cdb7bd581 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:24.901696 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 626/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:12:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:24.909600 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:25.033362 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:25.044012 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ae7aa-4e8c-4173-b4c2-156dbfc2e272 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:25.083342 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b08d58-2251-4c8e-9672-bd008d59f1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:25.083961 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08d58-2251-4c8e-9672-bd008d59f1ea None admin] REQUEST_METHOD: `PUT` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:25.084245 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08d58-2251-4c8e-9672-bd008d59f1ea None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:25.084522 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08d58-2251-4c8e-9672-bd008d59f1ea None admin] PATH_INFO: `/v3/projects/17fa8be5ea424cf694846517df55b68e/users/fef14e50634a457c82ea65c7662b5a0a/roles/6406fa458ddc48c1a792bdce744f2d09` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:25.086871 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b08d58-2251-4c8e-9672-bd008d59f1ea None admin] RBAC: Authorizing `identity:create_grant(project_id=17fa8be5ea424cf694846517df55b68e, user_id=fef14e50634a457c82ea65c7662b5a0a, role_id=6406fa458ddc48c1a792bdce744f2d09)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:25.088374 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b08d58-2251-4c8e-9672-bd008d59f1ea None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:25.104021 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 647/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:12:24 2022] PUT /identity/v3/projects/17fa8be5ea424cf694846517df55b68e/users/fef14e50634a457c82ea65c7662b5a0a/roles/6406fa458ddc48c1a792bdce744f2d09 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:27.153585 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f76c6-b8df-4175-bb35-b09852a257ae None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.154561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f76c6-b8df-4175-bb35-b09852a257ae None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.154561 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f76c6-b8df-4175-bb35-b09852a257ae None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.156604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 627/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:27.162953 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814417c-ba3d-4aeb-905b-aa776cbb3c39 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.163641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814417c-ba3d-4aeb-905b-aa776cbb3c39 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.163874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814417c-ba3d-4aeb-905b-aa776cbb3c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.187428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-7814417c-ba3d-4aeb-905b-aa776cbb3c39 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:27.188645 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-7814417c-ba3d-4aeb-905b-aa776cbb3c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:27.267489 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 648/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:27.290884 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f76c6-b8df-4175-bb35-b09852a257ae None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:27.401391 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f76c6-b8df-4175-bb35-b09852a257ae None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:27.419974 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:27.438709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2274d-b492-4a73-9a24-23f698caf304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:27.439359 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2274d-b492-4a73-9a24-23f698caf304 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.439634 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2274d-b492-4a73-9a24-23f698caf304 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.439929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2274d-b492-4a73-9a24-23f698caf304 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.450941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2274d-b492-4a73-9a24-23f698caf304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-9ae2274d-b492-4a73-9a24-23f698caf304 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:12:27.456294 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:27.457971 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:27.459430 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 628/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:12:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:27.464274 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814417c-ba3d-4aeb-905b-aa776cbb3c39 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:27.553662 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:27.574748 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814417c-ba3d-4aeb-905b-aa776cbb3c39 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:27.612467 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:27.613106 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.613388 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.613666 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] PATH_INFO: `/v3/roles` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.614437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] RBAC: Adding query filter params (name=admin) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:27.614747 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:27.616141 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:27.625341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 649/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:12:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:27.631457 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2274d-b492-4a73-9a24-23f698caf304 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:27.746762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2274d-b492-4a73-9a24-23f698caf304 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:27.787082 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:27.787732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.788012 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.788284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.798859 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:27.800388 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:27.815402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:27.816520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:27.817984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:27.817984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:27.817984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:27.817984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:27.817984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 629/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:27.823253 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:27.960128 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf57b8e-342f-48c3-9afb-61484990b25a None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:28.004722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:28.005283 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:28.005572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:28.005850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] PATH_INFO: `/v3/domains` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:28.006595 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:28.006905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:28.008305 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:28.022173 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 650/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:28.028926 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:28.070074 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:28.157190 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006b3c1-f570-44b1-8086-23e348c05aee None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:28.198078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:28.198767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:28.199062 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:28.199331 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] PATH_INFO: `/v3/users/placement` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:28.207604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:28.209075 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:12:28.225592 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:28.226812 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:28.228100 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 630/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:28.232970 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:28.237198 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:28.346400 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e90b3-0458-419d-96f7-08336ebfd8f1 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:28.383206 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:28.383838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:28.384128 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:28.384405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] PATH_INFO: `/v3/users` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:28.385220 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:28.385534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:28.387023 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:28.404834 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 651/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:12:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:28.411668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:28.530550 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:28.543614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76938a0-43ab-41a6-9b6d-15949a19d90f None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:28.581337 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da1788-a203-47d1-abd3-7e91f61220d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:28.582060 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da1788-a203-47d1-abd3-7e91f61220d7 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:28.582390 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da1788-a203-47d1-abd3-7e91f61220d7 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:28.582667 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da1788-a203-47d1-abd3-7e91f61220d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:28.653594 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da1788-a203-47d1-abd3-7e91f61220d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:28.655144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da1788-a203-47d1-abd3-7e91f61220d7 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:28.752376 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:28.764144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 631/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:12:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:12:28.770425 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:28.847677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:28.891021 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f47f63-0667-41c2-b5cc-b00c94d3de35 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:28.932802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:28.933387 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:28.933677 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:28.933959 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:28.945627 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:28.947141 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:12:28.962655 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:28.963745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:28.964802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:12:28.964802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:12:28.964802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:12:28.964802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:28.964802 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 652/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:28.970200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da1788-a203-47d1-abd3-7e91f61220d7 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.088798 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da1788-a203-47d1-abd3-7e91f61220d7 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.141867 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.142797 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.143160 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.143528 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] PATH_INFO: `/v3/domains` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.144578 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:29.144891 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.146323 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.159692 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 632/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:29.168951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.291838 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30bc5f-b9cc-4e38-bd84-96060434cf5b None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.307055 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:29.331995 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.332654 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.332936 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.333210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] PATH_INFO: `/v3/projects/service` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.340228 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.341694 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:12:29.353428 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:29.354601 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: ERROR keystone.server.flask.application Jul 28 09:12:29.355830 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 653/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:29.361345 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.421508 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:29.475773 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76a693-c617-4ac5-93ee-20a72b972ffe None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.511476 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933b112-c49c-4f33-923e-8d08d312e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.512098 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b112-c49c-4f33-923e-8d08d312e560 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.512383 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b112-c49c-4f33-923e-8d08d312e560 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.512662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b112-c49c-4f33-923e-8d08d312e560 None admin] PATH_INFO: `/v3/projects` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.513438 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933b112-c49c-4f33-923e-8d08d312e560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:29.513749 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933b112-c49c-4f33-923e-8d08d312e560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.515265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933b112-c49c-4f33-923e-8d08d312e560 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.527492 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 633/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:29.534139 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.642686 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b15a1c-a188-4364-b6da-da41f1d46680 None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.681675 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef58dc7-3d47-4fa0-9acf-2c0a4be072a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.682320 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef58dc7-3d47-4fa0-9acf-2c0a4be072a7 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.682604 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef58dc7-3d47-4fa0-9acf-2c0a4be072a7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.683005 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef58dc7-3d47-4fa0-9acf-2c0a4be072a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.683753 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef58dc7-3d47-4fa0-9acf-2c0a4be072a7 None admin] RBAC: Adding query filter params (role.id=6406fa458ddc48c1a792bdce744f2d09, scope.project.id=17fa8be5ea424cf694846517df55b68e, user.id=fef14e50634a457c82ea65c7662b5a0a) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:29.684061 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef58dc7-3d47-4fa0-9acf-2c0a4be072a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.685535 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef58dc7-3d47-4fa0-9acf-2c0a4be072a7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.692503 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 654/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:12:29 2022] GET /identity/v3/role_assignments?user.id=fef14e50634a457c82ea65c7662b5a0a&scope.project.id=17fa8be5ea424cf694846517df55b68e&role.id=6406fa458ddc48c1a792bdce744f2d09 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:31.739386 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be8ee-9ccb-4318-b0af-1675011f2e12 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:31.739999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be8ee-9ccb-4318-b0af-1675011f2e12 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:31.739999 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be8ee-9ccb-4318-b0af-1675011f2e12 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:31.741295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 634/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:31.747126 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59252-7782-4b6a-9543-b9f067238c6d None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:31.747657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59252-7782-4b6a-9543-b9f067238c6d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:31.747657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59252-7782-4b6a-9543-b9f067238c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:31.769981 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-63a59252-7782-4b6a-9543-b9f067238c6d None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:31.771265 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-63a59252-7782-4b6a-9543-b9f067238c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:31.804494 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:31.845641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 655/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:31.868065 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17be8ee-9ccb-4318-b0af-1675011f2e12 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:31.999501 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17be8ee-9ccb-4318-b0af-1675011f2e12 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:32.010838 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:32.038307 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:32.038959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:32.039255 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:32.039529 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] PATH_INFO: `/v3/services/placement` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:32.040171 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:32.041582 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:32.051104 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:32.052494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:32.053514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 635/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:12:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:32.058866 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a59252-7782-4b6a-9543-b9f067238c6d None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:32.169581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a59252-7782-4b6a-9543-b9f067238c6d None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:32.207812 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f6761-a8ee-4183-a430-ae9d26b09459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:32.208479 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6761-a8ee-4183-a430-ae9d26b09459 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:32.208764 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6761-a8ee-4183-a430-ae9d26b09459 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:32.209039 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6761-a8ee-4183-a430-ae9d26b09459 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:32.209746 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f6761-a8ee-4183-a430-ae9d26b09459 None admin] RBAC: Adding query filter params (name=placement) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:32.210650 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f6761-a8ee-4183-a430-ae9d26b09459 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:32.211804 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f6761-a8ee-4183-a430-ae9d26b09459 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:32.217633 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 656/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 09:12:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:32.223917 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:32.339482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab03a7-871b-46aa-b172-3785cb973373 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:32.376011 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc0ef1-0ca6-4265-ae8d-9394196f8ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:32.376643 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc0ef1-0ca6-4265-ae8d-9394196f8ac2 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:32.376929 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc0ef1-0ca6-4265-ae8d-9394196f8ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:32.377210 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc0ef1-0ca6-4265-ae8d-9394196f8ac2 None admin] PATH_INFO: `/v3/services` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:32.377902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc0ef1-0ca6-4265-ae8d-9394196f8ac2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:32.378241 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc0ef1-0ca6-4265-ae8d-9394196f8ac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:32.379735 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc0ef1-0ca6-4265-ae8d-9394196f8ac2 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:32.384402 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 636/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 09:12:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:34.403334 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126cf3c-f34c-4721-90d0-7948337bb517 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.403945 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126cf3c-f34c-4721-90d0-7948337bb517 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.403945 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126cf3c-f34c-4721-90d0-7948337bb517 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.405638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 657/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:34.413606 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232612db-8b2a-4ccb-8119-559c8e6d8c80 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.414078 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232612db-8b2a-4ccb-8119-559c8e6d8c80 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.414200 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232612db-8b2a-4ccb-8119-559c8e6d8c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.433114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-232612db-8b2a-4ccb-8119-559c8e6d8c80 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:34.434371 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-232612db-8b2a-4ccb-8119-559c8e6d8c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:34.466393 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:34.511125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 637/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:34.533665 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126cf3c-f34c-4721-90d0-7948337bb517 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.577386 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:34.661444 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126cf3c-f34c-4721-90d0-7948337bb517 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.698663 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab474e8-e796-4dbe-9177-7c9e931504d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.699300 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab474e8-e796-4dbe-9177-7c9e931504d2 None admin] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.699589 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab474e8-e796-4dbe-9177-7c9e931504d2 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.699865 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab474e8-e796-4dbe-9177-7c9e931504d2 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.700581 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab474e8-e796-4dbe-9177-7c9e931504d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.701983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab474e8-e796-4dbe-9177-7c9e931504d2 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.720338 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 658/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:12:34 2022] POST /identity/v3/services => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:36.814888 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1151fd-c270-4efd-87a1-e425ab48d637 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:36.815604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1151fd-c270-4efd-87a1-e425ab48d637 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:36.815604 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1151fd-c270-4efd-87a1-e425ab48d637 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:36.817559 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 638/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:36.823016 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347874d7-c218-4265-a4ce-1b80da8e7401 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:36.823531 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347874d7-c218-4265-a4ce-1b80da8e7401 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:36.823531 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347874d7-c218-4265-a4ce-1b80da8e7401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:36.843097 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-347874d7-c218-4265-a4ce-1b80da8e7401 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:36.844303 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-347874d7-c218-4265-a4ce-1b80da8e7401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:36.928131 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 659/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:36.953087 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1151fd-c270-4efd-87a1-e425ab48d637 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:37.084531 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1151fd-c270-4efd-87a1-e425ab48d637 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:37.128433 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:37.129123 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:37.129413 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:37.129691 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:37.130382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:37.131802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:37.142150 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:37.143573 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:37.144662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 639/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:12:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:12:37.150401 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347874d7-c218-4265-a4ce-1b80da8e7401 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:37.174519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:37.287161 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347874d7-c218-4265-a4ce-1b80da8e7401 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:37.332630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:37.333297 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:37.333582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:37.333863 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:37.334692 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] RBAC: Adding query filter params (name=placement) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:37.334912 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:37.336377 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:37.342641 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 660/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 09:12:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:37.349326 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:37.393844 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:37.480145 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa160c-360f-409e-b8f3-e2e1e4556cf9 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:37.519718 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e9add-800e-4959-82bf-9f045e66e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:37.520352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e9add-800e-4959-82bf-9f045e66e5dc None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:37.520630 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e9add-800e-4959-82bf-9f045e66e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:37.520902 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e9add-800e-4959-82bf-9f045e66e5dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:37.521668 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e9add-800e-4959-82bf-9f045e66e5dc None admin] RBAC: Adding query filter params (interface=public, service_id=cbfcc1559f26402697e14706c80f93f6, region_id=RegionOne) {{(pid=411107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:37.521982 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e9add-800e-4959-82bf-9f045e66e5dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:37.523400 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e9add-800e-4959-82bf-9f045e66e5dc None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:37.528665 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 640/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:12:37 2022] GET /identity/v3/endpoints?service_id=cbfcc1559f26402697e14706c80f93f6&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:37.537612 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:37.629745 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:37.653711 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44a548-b734-4b40-bb40-6a5f002d2f6e None admin] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:37.690911 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e835c-6f8e-41b5-9ad5-d614dffde6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:37.691543 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e835c-6f8e-41b5-9ad5-d614dffde6ea None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:37.691836 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e835c-6f8e-41b5-9ad5-d614dffde6ea None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:37.692447 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e835c-6f8e-41b5-9ad5-d614dffde6ea None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:37.693115 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e835c-6f8e-41b5-9ad5-d614dffde6ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:37.694544 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e835c-6f8e-41b5-9ad5-d614dffde6ea None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:37.701619 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 661/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:12:37 2022] GET /identity/v3/services => generated 2191 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:39.800732 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd210720-6323-46fd-ba3f-6753d47ef398 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.801352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd210720-6323-46fd-ba3f-6753d47ef398 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.801352 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd210720-6323-46fd-ba3f-6753d47ef398 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.802727 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 641/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:39.808111 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f515-af68-462f-bd14-0ab817da0674 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.808582 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f515-af68-462f-bd14-0ab817da0674 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.808701 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f515-af68-462f-bd14-0ab817da0674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.828657 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-79c6f515-af68-462f-bd14-0ab817da0674 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:39.829908 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-79c6f515-af68-462f-bd14-0ab817da0674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:39.902573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 662/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:12:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:12:39.925715 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd210720-6323-46fd-ba3f-6753d47ef398 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.027555 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:40.034996 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd210720-6323-46fd-ba3f-6753d47ef398 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.070947 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.071515 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.071799 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.072071 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] PATH_INFO: `/v3/services/placement` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.072701 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.074086 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:12:40.083271 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:12:40.084546 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application with Lock( Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: ERROR keystone.server.flask.application Jul 28 09:12:40.085738 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 642/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:12:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:12:40.090817 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6f515-af68-462f-bd14-0ab817da0674 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.175359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:40.208156 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6f515-af68-462f-bd14-0ab817da0674 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.247405 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a32d3a-00f2-43f0-8d37-d7f528f639c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.248044 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a32d3a-00f2-43f0-8d37-d7f528f639c7 None admin] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.248326 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a32d3a-00f2-43f0-8d37-d7f528f639c7 None admin] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.248603 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a32d3a-00f2-43f0-8d37-d7f528f639c7 None admin] PATH_INFO: `/v3/services` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.249302 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a32d3a-00f2-43f0-8d37-d7f528f639c7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=411108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:40.249609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a32d3a-00f2-43f0-8d37-d7f528f639c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.251031 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a32d3a-00f2-43f0-8d37-d7f528f639c7 None admin] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.256476 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 663/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 09:12:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:40.267263 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.382470 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cc242-7c9e-41b5-bc4d-8ef802fc8094 None admin] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.417042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b835963-6741-4e07-bded-e727e1f2db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65605145c2d84cce8746bb3e047916a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.417644 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b835963-6741-4e07-bded-e727e1f2db8f None admin] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.417951 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b835963-6741-4e07-bded-e727e1f2db8f None admin] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.418259 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b835963-6741-4e07-bded-e727e1f2db8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.418983 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b835963-6741-4e07-bded-e727e1f2db8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.420382 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b835963-6741-4e07-bded-e727e1f2db8f None admin] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.440353 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 643/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:50.253236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=30 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:50.436062 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:49.558365 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e85b-7851-488c-9c19-2a4b8dbafc7b None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:49.558951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e85b-7851-488c-9c19-2a4b8dbafc7b None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:49.558951 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e85b-7851-488c-9c19-2a4b8dbafc7b None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:49.560707 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 664/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:13:49.565836 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f751-4b98-412e-9db5-a5ffb6236919 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:49.566387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f751-4b98-412e-9db5-a5ffb6236919 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:49.566498 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f751-4b98-412e-9db5-a5ffb6236919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:49.718586 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-c339f751-4b98-412e-9db5-a5ffb6236919 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:49.720007 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-c339f751-4b98-412e-9db5-a5ffb6236919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:49.828926 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 644/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:49.894565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50d69-f0f5-44eb-8f8c-5a018e8701c1 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:49.894853 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50d69-f0f5-44eb-8f8c-5a018e8701c1 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:49.895138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50d69-f0f5-44eb-8f8c-5a018e8701c1 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:49.896512 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 665/1309] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 09:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:13:49.911435 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe2a10-1a57-4ff1-84c8-3e9428bb1ebd None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:49.911719 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe2a10-1a57-4ff1-84c8-3e9428bb1ebd None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:49.911997 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe2a10-1a57-4ff1-84c8-3e9428bb1ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:50.037703 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-fafe2a10-1a57-4ff1-84c8-3e9428bb1ebd None None] MFA Rules not processed for user `b3b1a6c31cc64159847f7ca39efe8105`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:50.039335 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-fafe2a10-1a57-4ff1-84c8-3e9428bb1ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:50.085385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:50.170459 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 645/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 09:13:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:50.179356 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c50d69-f0f5-44eb-8f8c-5a018e8701c1 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:50.181139 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-06c50d69-f0f5-44eb-8f8c-5a018e8701c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:50.357270 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c50d69-f0f5-44eb-8f8c-5a018e8701c1 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:50.459806 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d59442-ebf5-4a4e-944a-41a73008b0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:50.460776 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59442-ebf5-4a4e-944a-41a73008b0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:50.461419 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59442-ebf5-4a4e-944a-41a73008b0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:50.462074 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59442-ebf5-4a4e-944a-41a73008b0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:50.464407 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-15d59442-ebf5-4a4e-944a-41a73008b0b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:50.524610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:50.607227 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d59442-ebf5-4a4e-944a-41a73008b0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:50.610375 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d59442-ebf5-4a4e-944a-41a73008b0b7 service neutron] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:50.648216 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 666/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:53.158995 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424ceea-3446-405d-bd68-2c9995103547 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:53.159549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424ceea-3446-405d-bd68-2c9995103547 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:53.159549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424ceea-3446-405d-bd68-2c9995103547 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:53.161125 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 646/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:13:53.166532 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cda57-2ce4-4bc9-b8fa-c5a771b142bc None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:53.167018 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cda57-2ce4-4bc9-b8fa-c5a771b142bc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:53.167137 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cda57-2ce4-4bc9-b8fa-c5a771b142bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:53.266540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-289cda57-2ce4-4bc9-b8fa-c5a771b142bc None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:53.268433 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-289cda57-2ce4-4bc9-b8fa-c5a771b142bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:53.372998 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 667/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:53.411672 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424ceea-3446-405d-bd68-2c9995103547 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:53.413301 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-0424ceea-3446-405d-bd68-2c9995103547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:53.531769 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424ceea-3446-405d-bd68-2c9995103547 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:53.612034 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:53.614886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ad59d-989f-448e-ac66-1fbc532eb76c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:53.615534 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ad59d-989f-448e-ac66-1fbc532eb76c service neutron] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:53.615787 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ad59d-989f-448e-ac66-1fbc532eb76c service neutron] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:53.616094 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ad59d-989f-448e-ac66-1fbc532eb76c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:53.618429 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-bf4ad59d-989f-448e-ac66-1fbc532eb76c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:53.738093 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ad59d-989f-448e-ac66-1fbc532eb76c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:53.739493 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ad59d-989f-448e-ac66-1fbc532eb76c service neutron] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:53.758869 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 647/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:56.044457 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5754-5bb0-4f50-9e78-664e13b868e9 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5754-5bb0-4f50-9e78-664e13b868e9 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.045093 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5754-5bb0-4f50-9e78-664e13b868e9 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.046435 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 668/1315] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:13:56.051694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9ee36-7254-444c-afb3-0a9a93e7a10b None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.052166 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9ee36-7254-444c-afb3-0a9a93e7a10b None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.052278 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9ee36-7254-444c-afb3-0a9a93e7a10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.120909 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-17f9ee36-7254-444c-afb3-0a9a93e7a10b None None] MFA Rules not processed for user `a5086738a1284cb6a9c74c681b71749c`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:56.122149 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-17f9ee36-7254-444c-afb3-0a9a93e7a10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:56.176085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:56.224637 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 648/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:56.264090 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e5754-5bb0-4f50-9e78-664e13b868e9 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.265716 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-7e5e5754-5bb0-4f50-9e78-664e13b868e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:56.276359 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:56.374665 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e5754-5bb0-4f50-9e78-664e13b868e9 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.454638 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c651d24-79c5-4013-8bfe-6378ad2418e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.455279 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c651d24-79c5-4013-8bfe-6378ad2418e7 service neutron] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.455569 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c651d24-79c5-4013-8bfe-6378ad2418e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.455848 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c651d24-79c5-4013-8bfe-6378ad2418e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.457514 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-1c651d24-79c5-4013-8bfe-6378ad2418e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:56.524258 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:56.573050 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c651d24-79c5-4013-8bfe-6378ad2418e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.574565 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c651d24-79c5-4013-8bfe-6378ad2418e7 service neutron] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.593610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 669/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:59.277157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9c5f-4f41-4640-8d6b-ff151a5d4be7 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.277758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9c5f-4f41-4640-8d6b-ff151a5d4be7 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.277758 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9c5f-4f41-4640-8d6b-ff151a5d4be7 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.279157 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 649/1318] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:13:59.284415 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f403f5-0eca-4453-ac4d-b360912828c1 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.284902 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f403f5-0eca-4453-ac4d-b360912828c1 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.285057 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f403f5-0eca-4453-ac4d-b360912828c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.362278 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-30f403f5-0eca-4453-ac4d-b360912828c1 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:59.363566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-30f403f5-0eca-4453-ac4d-b360912828c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:59.469905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 670/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:13:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:59.508861 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc9c5f-4f41-4640-8d6b-ff151a5d4be7 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:59.510481 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-57dc9c5f-4f41-4640-8d6b-ff151a5d4be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:59.625187 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc9c5f-4f41-4640-8d6b-ff151a5d4be7 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:59.703713 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af813a-2677-4ab8-af67-391798759311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:59.704391 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af813a-2677-4ab8-af67-391798759311 service neutron] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.704679 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af813a-2677-4ab8-af67-391798759311 service neutron] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.704955 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af813a-2677-4ab8-af67-391798759311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.706672 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-10af813a-2677-4ab8-af67-391798759311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:59.715745 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:59.827494 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af813a-2677-4ab8-af67-391798759311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:59.828946 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af813a-2677-4ab8-af67-391798759311 service neutron] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:59.847658 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 650/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:02.596168 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac1e4-24af-4910-89e0-aee6bbea0add None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:02.596915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac1e4-24af-4910-89e0-aee6bbea0add None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:02.596915 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac1e4-24af-4910-89e0-aee6bbea0add None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:02.598072 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 671/1321] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:02.603777 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b330-1d67-42a4-8fca-aac766217015 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:02.604361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b330-1d67-42a4-8fca-aac766217015 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:02.604361 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b330-1d67-42a4-8fca-aac766217015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:02.681482 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-e591b330-1d67-42a4-8fca-aac766217015 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:02.682795 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-e591b330-1d67-42a4-8fca-aac766217015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:02.752540 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:02.797253 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 651/1322] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:02.840689 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ac1e4-24af-4910-89e0-aee6bbea0add None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:02.842519 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-5c2ac1e4-24af-4910-89e0-aee6bbea0add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:02.867230 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:02.976997 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ac1e4-24af-4910-89e0-aee6bbea0add None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:03.060220 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1c8a9-e802-43e8-bfcd-646ce39f6429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:03.060856 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1c8a9-e802-43e8-bfcd-646ce39f6429 service neutron] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:03.061146 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1c8a9-e802-43e8-bfcd-646ce39f6429 service neutron] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:03.061422 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1c8a9-e802-43e8-bfcd-646ce39f6429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:03.063163 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-1dd1c8a9-e802-43e8-bfcd-646ce39f6429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:03.140361 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:03.184525 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1c8a9-e802-43e8-bfcd-646ce39f6429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:03.186018 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1c8a9-e802-43e8-bfcd-646ce39f6429 service neutron] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:03.209579 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 672/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:05.287314 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f35d-15b5-438f-bcce-9cf1ed69b831 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:05.287886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f35d-15b5-438f-bcce-9cf1ed69b831 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:05.287886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f35d-15b5-438f-bcce-9cf1ed69b831 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:05.289302 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 652/1324] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:05.295077 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107fe0-e2c1-429c-9a20-2da2d4312a67 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:05.295574 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107fe0-e2c1-429c-9a20-2da2d4312a67 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:05.295673 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107fe0-e2c1-429c-9a20-2da2d4312a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:05.368706 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-21107fe0-e2c1-429c-9a20-2da2d4312a67 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:05.369973 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-21107fe0-e2c1-429c-9a20-2da2d4312a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:05.482615 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 673/1325] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:05.519328 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0f35d-15b5-438f-bcce-9cf1ed69b831 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:05.520897 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-d7a0f35d-15b5-438f-bcce-9cf1ed69b831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:05.636648 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0f35d-15b5-438f-bcce-9cf1ed69b831 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:05.712123 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b90594-5a72-4e30-907c-24d026839ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:05.712748 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b90594-5a72-4e30-907c-24d026839ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:05.713043 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b90594-5a72-4e30-907c-24d026839ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:05.713322 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b90594-5a72-4e30-907c-24d026839ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:05.715026 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-70b90594-5a72-4e30-907c-24d026839ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:05.731786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:05.835752 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b90594-5a72-4e30-907c-24d026839ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:05.837235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b90594-5a72-4e30-907c-24d026839ea9 service neutron] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:05.856868 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 653/1326] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:07.892957 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9def6-263f-41f7-8023-4ccaab269458 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:07.893517 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9def6-263f-41f7-8023-4ccaab269458 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:07.893517 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9def6-263f-41f7-8023-4ccaab269458 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:07.894942 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 674/1327] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:07.900265 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c69c-bc22-4741-a1e6-26d4cdcde978 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:07.900726 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c69c-bc22-4741-a1e6-26d4cdcde978 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:07.900824 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c69c-bc22-4741-a1e6-26d4cdcde978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:07.969226 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-9652c69c-bc22-4741-a1e6-26d4cdcde978 None None] MFA Rules not processed for user `a5086738a1284cb6a9c74c681b71749c`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:07.970500 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-9652c69c-bc22-4741-a1e6-26d4cdcde978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:08.036894 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:08.071277 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 654/1328] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:08.109028 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9def6-263f-41f7-8023-4ccaab269458 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:08.110675 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-29c9def6-263f-41f7-8023-4ccaab269458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:08.144676 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:08.224392 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9def6-263f-41f7-8023-4ccaab269458 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:08.301342 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8e824-f653-4e28-be11-d24529a12d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:08.301982 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8e824-f653-4e28-be11-d24529a12d98 service neutron] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:08.302313 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8e824-f653-4e28-be11-d24529a12d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:08.302584 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8e824-f653-4e28-be11-d24529a12d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:08.304249 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-40d8e824-f653-4e28-be11-d24529a12d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:08.391100 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:08.424036 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8e824-f653-4e28-be11-d24529a12d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:08.425486 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8e824-f653-4e28-be11-d24529a12d98 service neutron] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:08.444549 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 675/1329] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:10.683755 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd317676-333d-44aa-aa70-ee8aa979599e None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:10.684447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd317676-333d-44aa-aa70-ee8aa979599e None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:10.684447 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd317676-333d-44aa-aa70-ee8aa979599e None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:10.685762 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 655/1330] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:10.691775 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d2f63-a5cd-4800-aa1a-dd874f19fabc None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:10.692610 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d2f63-a5cd-4800-aa1a-dd874f19fabc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:10.692722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d2f63-a5cd-4800-aa1a-dd874f19fabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:10.769683 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-3a8d2f63-a5cd-4800-aa1a-dd874f19fabc None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:10.770987 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-3a8d2f63-a5cd-4800-aa1a-dd874f19fabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:10.877859 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 676/1331] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:10.929562 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b729-5fe2-4564-9843-9fdb075e417c None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:10.929852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b729-5fe2-4564-9843-9fdb075e417c None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:10.930169 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b729-5fe2-4564-9843-9fdb075e417c None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:10.931808 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 656/1332] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 09:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:10.940527 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b18537-a397-4cc6-8ebc-8b8f92da065d None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:10.940776 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b18537-a397-4cc6-8ebc-8b8f92da065d None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:10.941167 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b18537-a397-4cc6-8ebc-8b8f92da065d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:10.989379 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:11.021630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-a1b18537-a397-4cc6-8ebc-8b8f92da065d None None] MFA Rules not processed for user `b3b1a6c31cc64159847f7ca39efe8105`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:11.022911 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-a1b18537-a397-4cc6-8ebc-8b8f92da065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:11.107934 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 677/1333] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 09:14:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:11.117300 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9b729-5fe2-4564-9843-9fdb075e417c None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:11.118901 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-92b9b729-5fe2-4564-9843-9fdb075e417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:11.232841 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9b729-5fe2-4564-9843-9fdb075e417c None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:11.312267 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54618057-54eb-4d2d-9c97-17a884e28647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:11.312908 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54618057-54eb-4d2d-9c97-17a884e28647 service neutron] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:11.313192 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54618057-54eb-4d2d-9c97-17a884e28647 service neutron] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:11.313468 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54618057-54eb-4d2d-9c97-17a884e28647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:11.315185 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-54618057-54eb-4d2d-9c97-17a884e28647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:11.345800 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:11.438330 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54618057-54eb-4d2d-9c97-17a884e28647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:11.439794 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54618057-54eb-4d2d-9c97-17a884e28647 service neutron] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:11.460079 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 657/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:13.724807 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d41df-fcf0-44b9-a293-5eae16212cf2 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:13.725437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d41df-fcf0-44b9-a293-5eae16212cf2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:13.725437 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d41df-fcf0-44b9-a293-5eae16212cf2 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:13.727011 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 678/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:13.732852 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ebaa-b5c6-4854-a52b-a22e26de42d3 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:13.733385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ebaa-b5c6-4854-a52b-a22e26de42d3 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:13.733385 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ebaa-b5c6-4854-a52b-a22e26de42d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:13.804662 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-42d4ebaa-b5c6-4854-a52b-a22e26de42d3 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:13.805248 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-42d4ebaa-b5c6-4854-a52b-a22e26de42d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:13.885680 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:13.910674 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 658/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:13.951027 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d41df-fcf0-44b9-a293-5eae16212cf2 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:13.952609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-cc6d41df-fcf0-44b9-a293-5eae16212cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:14.069434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d41df-fcf0-44b9-a293-5eae16212cf2 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:14.098993 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:14.146526 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d2608-5109-4119-bdde-c5a952a6b722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:14.147156 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2608-5109-4119-bdde-c5a952a6b722 service neutron] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:14.147440 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2608-5109-4119-bdde-c5a952a6b722 service neutron] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:14.147709 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2608-5109-4119-bdde-c5a952a6b722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:14.149412 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-287d2608-5109-4119-bdde-c5a952a6b722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:14.266075 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d2608-5109-4119-bdde-c5a952a6b722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:14.267559 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d2608-5109-4119-bdde-c5a952a6b722 service neutron] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:14.286138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 679/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:16.971737 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416989-9dc1-40f9-a5d6-8cbc2c1dcaf4 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:16.972344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416989-9dc1-40f9-a5d6-8cbc2c1dcaf4 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:16.972344 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416989-9dc1-40f9-a5d6-8cbc2c1dcaf4 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:16.973677 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 659/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:16.979311 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fcae6-0ce4-4d7d-838d-5754daf9c139 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:16.979831 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fcae6-0ce4-4d7d-838d-5754daf9c139 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:16.979965 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fcae6-0ce4-4d7d-838d-5754daf9c139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:17.049636 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-2f5fcae6-0ce4-4d7d-838d-5754daf9c139 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:17.050905 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-2f5fcae6-0ce4-4d7d-838d-5754daf9c139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:17.059609 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:17.153409 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 680/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:17.190682 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa416989-9dc1-40f9-a5d6-8cbc2c1dcaf4 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:17.192628 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-fa416989-9dc1-40f9-a5d6-8cbc2c1dcaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:17.301319 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa416989-9dc1-40f9-a5d6-8cbc2c1dcaf4 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:17.380734 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86416150-147a-4ac2-a318-3641e2a65d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:17.381370 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86416150-147a-4ac2-a318-3641e2a65d2f service neutron] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:17.381651 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86416150-147a-4ac2-a318-3641e2a65d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:17.381922 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86416150-147a-4ac2-a318-3641e2a65d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:17.383729 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-86416150-147a-4ac2-a318-3641e2a65d2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:17.422491 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:17.500595 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86416150-147a-4ac2-a318-3641e2a65d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:17.502042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86416150-147a-4ac2-a318-3641e2a65d2f service neutron] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:17.521290 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 660/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:21.538183 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5281-034d-4209-935f-51f330d694b4 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:21.538729 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5281-034d-4209-935f-51f330d694b4 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:21.539039 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5281-034d-4209-935f-51f330d694b4 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:21.540578 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 681/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:21.546373 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2631e7a-f9f0-4e83-8c2c-b760d1d9a572 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:21.546876 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2631e7a-f9f0-4e83-8c2c-b760d1d9a572 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:21.546972 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2631e7a-f9f0-4e83-8c2c-b760d1d9a572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:21.618003 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-a2631e7a-f9f0-4e83-8c2c-b760d1d9a572 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:21.619235 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-a2631e7a-f9f0-4e83-8c2c-b760d1d9a572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:21.716783 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:21.728975 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 661/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:21.768652 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a5281-034d-4209-935f-51f330d694b4 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:21.770296 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-2c5a5281-034d-4209-935f-51f330d694b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:21.882019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a5281-034d-4209-935f-51f330d694b4 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:21.923534 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:21.962715 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cce08a-8977-4db7-bf7f-e1ab043961fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:21.963350 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cce08a-8977-4db7-bf7f-e1ab043961fb service neutron] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:21.963632 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cce08a-8977-4db7-bf7f-e1ab043961fb service neutron] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:21.963907 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cce08a-8977-4db7-bf7f-e1ab043961fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:21.965549 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-01cce08a-8977-4db7-bf7f-e1ab043961fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:22.087921 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cce08a-8977-4db7-bf7f-e1ab043961fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:22.089391 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cce08a-8977-4db7-bf7f-e1ab043961fb service neutron] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:22.109307 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 682/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:24.738948 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0bc06-2cd7-4b0c-8dcd-6bb031833588 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:24.739591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0bc06-2cd7-4b0c-8dcd-6bb031833588 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:24.739591 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0bc06-2cd7-4b0c-8dcd-6bb031833588 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:24.740880 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 662/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:24.746703 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90db10-848e-4c65-8dde-1fd6f0d18fd2 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:24.747184 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90db10-848e-4c65-8dde-1fd6f0d18fd2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:24.747301 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90db10-848e-4c65-8dde-1fd6f0d18fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:24.828104 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-3d90db10-848e-4c65-8dde-1fd6f0d18fd2 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:24.829422 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-3d90db10-848e-4c65-8dde-1fd6f0d18fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:24.852572 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:24.950550 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 683/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:24.990903 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0bc06-2cd7-4b0c-8dcd-6bb031833588 None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:24.992417 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-9fd0bc06-2cd7-4b0c-8dcd-6bb031833588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:25.107387 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0bc06-2cd7-4b0c-8dcd-6bb031833588 None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:25.189959 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0067af-a758-47f7-8bd8-8d916646035d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:25.190549 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0067af-a758-47f7-8bd8-8d916646035d service neutron] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:25.190843 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0067af-a758-47f7-8bd8-8d916646035d service neutron] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:25.191112 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0067af-a758-47f7-8bd8-8d916646035d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:25.193120 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-8b0067af-a758-47f7-8bd8-8d916646035d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:25.246432 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:25.320767 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0067af-a758-47f7-8bd8-8d916646035d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:25.322497 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0067af-a758-47f7-8bd8-8d916646035d service neutron] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:25.343809 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 663/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:29.833385 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12cac-0af5-4423-9477-c1225d1b90e2 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:29.833924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12cac-0af5-4423-9477-c1225d1b90e2 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:29.833924 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12cac-0af5-4423-9477-c1225d1b90e2 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:29.839516 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 684/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:29.841670 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aff87-25d0-41d3-bf16-4494899fc78c None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:29.842295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aff87-25d0-41d3-bf16-4494899fc78c None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:29.842295 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aff87-25d0-41d3-bf16-4494899fc78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:29.928144 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-7f1aff87-25d0-41d3-bf16-4494899fc78c None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:29.929560 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-7f1aff87-25d0-41d3-bf16-4494899fc78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:30.078345 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:30.081205 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 664/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:30.123903 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa12cac-0af5-4423-9477-c1225d1b90e2 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:30.125477 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-1aa12cac-0af5-4423-9477-c1225d1b90e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:30.242001 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa12cac-0af5-4423-9477-c1225d1b90e2 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:30.291831 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:30.321176 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6823768-9024-41f0-af70-d171adb4100f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b1a6c31cc64159847f7ca39efe8105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:30.321736 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6823768-9024-41f0-af70-d171adb4100f service neutron] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:30.322024 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6823768-9024-41f0-af70-d171adb4100f service neutron] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:30.322341 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6823768-9024-41f0-af70-d171adb4100f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:30.324122 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-e6823768-9024-41f0-af70-d171adb4100f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:30.443540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6823768-9024-41f0-af70-d171adb4100f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:30.445127 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6823768-9024-41f0-af70-d171adb4100f service neutron] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:30.463841 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 685/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:35.557802 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115697ad-d415-4af1-bdf6-6c71a42823d9 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:35.558374 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115697ad-d415-4af1-bdf6-6c71a42823d9 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:35.558471 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115697ad-d415-4af1-bdf6-6c71a42823d9 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:35.559976 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 665/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 09:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:35.570573 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5b8c3-1a23-4a5d-b0f8-9a7c48e149bc None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:35.571436 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5b8c3-1a23-4a5d-b0f8-9a7c48e149bc None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:35.572030 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5b8c3-1a23-4a5d-b0f8-9a7c48e149bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:35.661568 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-41d5b8c3-1a23-4a5d-b0f8-9a7c48e149bc None None] MFA Rules not processed for user `fef14e50634a457c82ea65c7662b5a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:35.663073 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-41d5b8c3-1a23-4a5d-b0f8-9a7c48e149bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:35.692722 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:35.807271 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 686/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 09:14:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:40.436058 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8174a-aa16-450d-b599-c5651ceb81d8 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:40.436694 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8174a-aa16-450d-b599-c5651ceb81d8 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:40.437426 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8174a-aa16-450d-b599-c5651ceb81d8 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:40.438952 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 666/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 09:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:40.444782 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d62af4-46be-496e-835c-66f4a6ed59e4 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:40.445256 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d62af4-46be-496e-835c-66f4a6ed59e4 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:40.445635 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d62af4-46be-496e-835c-66f4a6ed59e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:40.528540 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-03d62af4-46be-496e-835c-66f4a6ed59e4 None None] MFA Rules not processed for user `fef14e50634a457c82ea65c7662b5a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:40.529878 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-03d62af4-46be-496e-835c-66f4a6ed59e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:40.612480 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:40.653794 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 687/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:42.118092 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888521-92bd-4f56-96d6-ed4eba9402ec None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:42.118675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888521-92bd-4f56-96d6-ed4eba9402ec None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:42.118675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888521-92bd-4f56-96d6-ed4eba9402ec None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:42.119949 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 667/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 09:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:42.125166 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a459fe-1a33-49ec-81f0-6fc517862108 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:42.125623 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a459fe-1a33-49ec-81f0-6fc517862108 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:42.125723 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a459fe-1a33-49ec-81f0-6fc517862108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:42.209690 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-63a459fe-1a33-49ec-81f0-6fc517862108 None None] MFA Rules not processed for user `fef14e50634a457c82ea65c7662b5a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:42.211002 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-63a459fe-1a33-49ec-81f0-6fc517862108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:42.330832 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 688/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 09:14:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:45.602231 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848a54-623a-47ee-a78d-7dc4c8946523 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.602761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848a54-623a-47ee-a78d-7dc4c8946523 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.602761 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848a54-623a-47ee-a78d-7dc4c8946523 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.604181 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 668/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:45.609726 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbf45-4e79-42f2-80a3-5a928b621d5f None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.610158 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbf45-4e79-42f2-80a3-5a928b621d5f None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.610368 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbf45-4e79-42f2-80a3-5a928b621d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.650132 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:45.687695 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-60dbbf45-4e79-42f2-80a3-5a928b621d5f None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:45.688999 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-60dbbf45-4e79-42f2-80a3-5a928b621d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:45.809333 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 689/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:45.871719 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766a01-929f-48a9-a964-d634410e9ffd None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.872132 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766a01-929f-48a9-a964-d634410e9ffd None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.872520 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766a01-929f-48a9-a964-d634410e9ffd None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.874286 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 669/1358] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 09:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:45.889724 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ead1f8-bc84-4221-b0da-db708fdfc971 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.890016 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ead1f8-bc84-4221-b0da-db708fdfc971 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.890327 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ead1f8-bc84-4221-b0da-db708fdfc971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.009669 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-59ead1f8-bc84-4221-b0da-db708fdfc971 None None] MFA Rules not processed for user `1e579a84d5e34a828c5981f34b458a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:46.010607 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-59ead1f8-bc84-4221-b0da-db708fdfc971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.070434 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:46.146566 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 690/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:46.154886 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f766a01-929f-48a9-a964-d634410e9ffd None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.156420 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-9f766a01-929f-48a9-a964-d634410e9ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.325820 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f766a01-929f-48a9-a964-d634410e9ffd None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.410284 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:46.455984 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0840059-94f8-4e09-ae11-71875edcd5bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e579a84d5e34a828c5981f34b458a0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.456630 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0840059-94f8-4e09-ae11-71875edcd5bb service nova] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.456912 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0840059-94f8-4e09-ae11-71875edcd5bb service nova] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.457188 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0840059-94f8-4e09-ae11-71875edcd5bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.458845 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-a0840059-94f8-4e09-ae11-71875edcd5bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.558630 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0840059-94f8-4e09-ae11-71875edcd5bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.560298 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0840059-94f8-4e09-ae11-71875edcd5bb service nova] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.580114 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 670/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:48.818486 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c748d63-fe3e-4a4c-8156-76a03e89473e None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:48.819025 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c748d63-fe3e-4a4c-8156-76a03e89473e None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:48.819025 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c748d63-fe3e-4a4c-8156-76a03e89473e None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:48.821207 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 691/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:48.827105 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097491d-6380-4105-82ed-7d55802f4dd1 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:48.827588 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097491d-6380-4105-82ed-7d55802f4dd1 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:48.827675 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097491d-6380-4105-82ed-7d55802f4dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:48.906049 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-d097491d-6380-4105-82ed-7d55802f4dd1 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:48.907379 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-d097491d-6380-4105-82ed-7d55802f4dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:48.952085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:49.033107 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 671/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:49.079648 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c748d63-fe3e-4a4c-8156-76a03e89473e None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:49.081219 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-4c748d63-fe3e-4a4c-8156-76a03e89473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:49.288263 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c748d63-fe3e-4a4c-8156-76a03e89473e None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:49.302138 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:49.477620 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34226fb-3eb2-4270-8082-e1930c71c18a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e579a84d5e34a828c5981f34b458a0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:49.478337 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34226fb-3eb2-4270-8082-e1930c71c18a service nova] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:49.478630 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34226fb-3eb2-4270-8082-e1930c71c18a service nova] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:49.478918 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34226fb-3eb2-4270-8082-e1930c71c18a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:49.480680 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-c34226fb-3eb2-4270-8082-e1930c71c18a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:49.637063 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34226fb-3eb2-4270-8082-e1930c71c18a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:49.638983 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34226fb-3eb2-4270-8082-e1930c71c18a service nova] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:49.664850 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 692/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:52.038914 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd9228-75b4-4d9a-a83d-5b7f79b2ae04 None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:52.039463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd9228-75b4-4d9a-a83d-5b7f79b2ae04 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:52.039463 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd9228-75b4-4d9a-a83d-5b7f79b2ae04 None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:52.041042 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 672/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:52.046929 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707eb6-6cb9-4017-83fe-1dc9052d82bb None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:52.047454 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707eb6-6cb9-4017-83fe-1dc9052d82bb None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:52.047454 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707eb6-6cb9-4017-83fe-1dc9052d82bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:52.070506 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:52.130037 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-1e707eb6-6cb9-4017-83fe-1dc9052d82bb None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:52.131327 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-1e707eb6-6cb9-4017-83fe-1dc9052d82bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:52.244017 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 693/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:52.290247 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea0fe-8599-4b17-aa53-fef1bcc0697f None None] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:52.290467 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea0fe-8599-4b17-aa53-fef1bcc0697f None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:52.290709 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea0fe-8599-4b17-aa53-fef1bcc0697f None None] PATH_INFO: `/` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:52.292588 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 673/1366] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 09:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:52.298169 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda856-d494-4b03-90d3-bcf136cec995 None None] REQUEST_METHOD: `POST` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:52.299558 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda856-d494-4b03-90d3-bcf136cec995 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:52.299558 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda856-d494-4b03-90d3-bcf136cec995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:52.405732 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.auth.core [None req-93cda856-d494-4b03-90d3-bcf136cec995 None None] MFA Rules not processed for user `1e579a84d5e34a828c5981f34b458a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=411108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:52.407481 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-93cda856-d494-4b03-90d3-bcf136cec995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:52.466236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:52.579417 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 694/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 09:14:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:52.593643 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ea0fe-8599-4b17-aa53-fef1bcc0697f None None] Authenticating user token {{(pid=411107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:52.595751 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-472ea0fe-8599-4b17-aa53-fef1bcc0697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:52.761160 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ea0fe-8599-4b17-aa53-fef1bcc0697f None None] Validating token access rules against request {{(pid=411107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:52.786061 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:52.887614 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bc6cf-d4d6-45dd-b1df-72581a74568c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e579a84d5e34a828c5981f34b458a0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:52.888245 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc6cf-d4d6-45dd-b1df-72581a74568c service nova] REQUEST_METHOD: `GET` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:52.888530 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc6cf-d4d6-45dd-b1df-72581a74568c service nova] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:52.888811 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc6cf-d4d6-45dd-b1df-72581a74568c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:52.890514 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-354bc6cf-d4d6-45dd-b1df-72581a74568c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:52.988584 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bc6cf-d4d6-45dd-b1df-72581a74568c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:52.990287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bc6cf-d4d6-45dd-b1df-72581a74568c service nova] RBAC: Authorization granted {{(pid=411107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:53.011786 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 674/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:53.889631 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6932b-3178-4941-9af7-9bc800714ec9 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:53.890210 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6932b-3178-4941-9af7-9bc800714ec9 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:53.890464 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6932b-3178-4941-9af7-9bc800714ec9 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:53.892019 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 695/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:53.897579 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e760f-f1b5-48f7-87d6-b8f74aaa2860 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:53.898085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e760f-f1b5-48f7-87d6-b8f74aaa2860 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:53.898085 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e760f-f1b5-48f7-87d6-b8f74aaa2860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:53.932108 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:53.970941 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-9a5e760f-f1b5-48f7-87d6-b8f74aaa2860 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:53.972170 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-9a5e760f-f1b5-48f7-87d6-b8f74aaa2860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:54.077063 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 675/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:55.112874 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fde4f-2955-488d-872c-bb7d70b65b17 None None] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:55.113421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fde4f-2955-488d-872c-bb7d70b65b17 None None] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:55.113421 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fde4f-2955-488d-872c-bb7d70b65b17 None None] PATH_INFO: `/` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:55.115059 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 696/1371] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:14:55.121281 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd16770-1c27-4b68-a332-aff60244dcd5 None None] REQUEST_METHOD: `POST` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:55.121822 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd16770-1c27-4b68-a332-aff60244dcd5 None None] SCRIPT_NAME: `/identity` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:55.121907 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd16770-1c27-4b68-a332-aff60244dcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=411107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:55.212687 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.auth.core [None req-dcd16770-1c27-4b68-a332-aff60244dcd5 None None] MFA Rules not processed for user `65605145c2d84cce8746bb3e047916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=411107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:55.213536 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG keystone.common.fernet_utils [None req-dcd16770-1c27-4b68-a332-aff60244dcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:55.284287 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=100 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:55.349323 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: [pid: 411107|app: 0|req: 676/1372] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:55.395805 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fde4f-2955-488d-872c-bb7d70b65b17 None None] Authenticating user token {{(pid=411108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:55.397413 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-9a6fde4f-2955-488d-872c-bb7d70b65b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:55.546373 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:55.552262 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fde4f-2955-488d-872c-bb7d70b65b17 None None] Validating token access rules against request {{(pid=411108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:55.675230 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923f072-b021-422e-8ce0-afcb2493f364 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e579a84d5e34a828c5981f34b458a0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17fa8be5ea424cf694846517df55b68e', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=411108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:55.675873 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f072-b021-422e-8ce0-afcb2493f364 service nova] REQUEST_METHOD: `GET` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:55.676229 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f072-b021-422e-8ce0-afcb2493f364 service nova] SCRIPT_NAME: `/identity` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:55.676576 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f072-b021-422e-8ce0-afcb2493f364 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=411108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:55.678388 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.fernet_utils [None req-c923f072-b021-422e-8ce0-afcb2493f364 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=411108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:55.910260 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923f072-b021-422e-8ce0-afcb2493f364 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:55.912236 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923f072-b021-422e-8ce0-afcb2493f364 service nova] RBAC: Authorization granted {{(pid=411108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:55.942362 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: DEBUG dbcounter [-] [411108] Writing DB stats keystone:SELECT=100 {{(pid=411108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:55.966674 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: [pid: 411108|app: 0|req: 697/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:15:05.347381 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: DEBUG dbcounter [-] [411107] Writing DB stats keystone:SELECT=19 {{(pid=411107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:16.516154 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: Thu Jul 28 09:15:16 2022 - graceful shutdown triggered... Jul 28 09:15:16.516760 dv-cinder-851351-1-iscsi devstack@keystone.service[411107]: Gracefully killing worker 1 (pid: 411107)... Jul 28 09:15:16.517165 dv-cinder-851351-1-iscsi devstack@keystone.service[411108]: Gracefully killing worker 2 (pid: 411108)... Jul 28 09:15:16.517777 dv-cinder-851351-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Jul 28 09:15:17.518045 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: worker 1 buried after 1 seconds Jul 28 09:15:17.518045 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: worker 2 buried after 1 seconds Jul 28 09:15:17.518045 dv-cinder-851351-1-iscsi devstack@keystone.service[411097]: goodbye to uWSGI. Jul 28 09:15:17.519727 dv-cinder-851351-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Jul 28 09:15:17.520299 dv-cinder-851351-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. Jul 28 09:20:08.490634 dv-cinder-851351-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 09:20:08.501104 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 09:20:08 2022] *** Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: nodename: dv-cinder-851351-1-iscsi Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: machine: x86_64 Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: clock source: unix Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: pcre jit disabled Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: detected number of CPU cores: 4 Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: current working directory: / Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: detected binary path: /usr/bin/uwsgi-core Jul 28 09:20:08.503221 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: your processes number limit is 47680 Jul 28 09:20:08.504147 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: your memory page size is 4096 bytes Jul 28 09:20:08.504147 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: detected max file descriptor number: 2048 Jul 28 09:20:08.504147 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: lock engine: pthread robust mutexes Jul 28 09:20:08.504147 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: thunder lock: enabled Jul 28 09:20:08.504147 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 09:20:08.504147 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 09:20:08.542482 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: Python main interpreter initialized at 0x556898e7e9d0 Jul 28 09:20:08.542482 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: python threads support enabled Jul 28 09:20:08.542482 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: your server socket listen backlog is limited to 100 connections Jul 28 09:20:08.542482 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: your mercy for graceful operations on workers is 90 seconds Jul 28 09:20:08.542969 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 09:20:08.542969 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: *** Operational MODE: preforking *** Jul 28 09:20:08.543233 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: *** uWSGI is running in multiple interpreter mode *** Jul 28 09:20:08.543233 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: spawned uWSGI master process (pid: 474594) Jul 28 09:20:08.543528 dv-cinder-851351-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 09:20:08.545069 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: spawned uWSGI worker 1 (pid: 474607, cores: 1) Jul 28 09:20:08.547773 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: spawned uWSGI worker 2 (pid: 474608, cores: 1) Jul 28 09:20:08.547773 dv-cinder-851351-1-iscsi devstack@keystone.service[474594]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 09:20:14.111019 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 09:20:14.111019 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: from cryptography.utils import int_from_bytes Jul 28 09:20:14.111997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 09:20:14.111997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: from cryptography.utils import int_from_bytes Jul 28 09:20:14.138686 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 09:20:14.138686 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: from cryptography.utils import int_from_bytes Jul 28 09:20:14.139449 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 09:20:14.139449 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: from cryptography.utils import int_from_bytes Jul 28 09:20:14.161385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 09:20:14.161638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 09:20:14.161742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] command line args: [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 09:20:14.162010 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 09:20:14.162195 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ================================================================================ {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 09:20:14.162491 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] admin_token = **** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.162749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] config_dir = [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.162913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.163084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] config_source = [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.163295 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] debug = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.163481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.163899 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.164049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] insecure_debug = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.164301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.164501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.164770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] list_limit = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.164968 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] log_config_append = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.165118 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.165259 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] log_dir = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.165406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] log_file = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.165677 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.165883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.166165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.166428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.166684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.166889 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.167058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.167221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.167470 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.167666 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.167905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.168185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.168441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.168668 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] notification_format = cadf {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.168947 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.169476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.169658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] publish_errors = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.169894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.170132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.170330 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.170575 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.170828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.171100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] standard_threads = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.171334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] strict_password_check = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.171580 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.171728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] use_eventlog = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.171924 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] use_journal = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.172076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] use_json = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.172265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] use_stderr = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.172407 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] use_syslog = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.172661 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] watch_log_file = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.172923 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.173168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.173434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.173680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.173916 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.174188 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.174445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.174687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.174939 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.175191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.175449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.175691 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.175949 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.176185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.176437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.176688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.176930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.177190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.177449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.177702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.177945 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.178230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.178504 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.178764 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.179023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.179264 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.179501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.179738 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.179979 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.180236 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.180489 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.180725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.180961 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.181217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.181469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.181711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.181964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.182226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.182472 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.182719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.182957 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.183201 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.183439 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.183677 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.183919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.184172 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.184410 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.184654 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.184891 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.185132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.185373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.185626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.185862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.186143 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.186369 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.186678 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.186854 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 09:20:14.186995 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.187134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 09:20:14.187273 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.187412 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] command line args: [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 09:20:14.187559 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.187699 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 09:20:14.187838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.187977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ================================================================================ {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 09:20:14.188116 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.188259 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] admin_token = **** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.188404 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.188544 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] config_dir = [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.188682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.188826 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.188826 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] config_source = [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.189090 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.189228 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] debug = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.189366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.189497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.189673 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.189812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.189951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.190089 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] insecure_debug = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.190255 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.190400 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.190540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.190693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.190833 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.190971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] list_limit = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.190971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] log_config_append = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.191250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.191700 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.191848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.191987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] log_dir = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.191987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] log_file = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.192241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.192385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.192516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.192646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.192780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.192918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.193065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.enabled = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.193210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.193362 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.193503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.193642 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.193792 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.193936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.194219 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.194358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.194489 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.194627 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.194767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.194909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.195053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.195182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.195302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.195423 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.195545 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.195674 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.196433 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.197009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.197138 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] notification_format = cadf {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.197267 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.197388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.197516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.197516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.197746 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.197866 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.197994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] publish_errors = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.198177 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.198311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.198432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.198553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.198553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.198778 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.198905 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.199028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.199148 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.199268 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.200919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] standard_threads = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.201097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.201234 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] strict_password_check = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.201365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.201494 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.201616 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.201749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] use_eventlog = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.201898 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.202042 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] use_journal = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.202191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.202335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] use_json = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.202335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] use_stderr = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.202597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.202728 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] use_syslog = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.202877 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.203027 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] watch_log_file = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 09:20:14.203169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.203309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.203441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.203578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.203723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] auth.external = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.203870 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.204009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] auth.mapped = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.204147 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.204287 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.204426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.204571 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.204571 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] auth.password = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.204841 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.204978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] auth.token = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.205115 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.205254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.205399 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.205531 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] catalog.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.205662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.205848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.205999 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.206175 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.206321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.206453 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.206599 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.206731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.206861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.206998 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.207143 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.207276 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] credential.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.207417 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.207556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] credential.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.207694 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.207855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.207989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.208127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.208266 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.208405 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.208537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.208681 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.208819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.208951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.209088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.209226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.209402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.209546 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.209686 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.209817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.209955 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.210093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.210278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.210435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.210576 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.210709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.210855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.211008 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.211261 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.211261 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.211555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.211698 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.211837 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.211976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.212132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] federation.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.212285 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.212426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.212565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.212698 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] federation.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.212838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.212976 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.213118 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.213267 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.213399 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.213529 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.213661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.213798 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.213940 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.214079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.214239 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.214391 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.214531 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.214670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.214813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.214945 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.215075 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.215212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.215343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.215487 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.215629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.215774 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.215913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.216052 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.216197 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.216337 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.216493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.216634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.216773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.216913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.217052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.217198 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.217337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.217487 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.217627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.217859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.218000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.218243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.218399 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.218535 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.218700 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.218850 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.218991 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.219137 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.219280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.219429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.219570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.219710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.219849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.219989 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.220121 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.220265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.220406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.220406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.220671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.220817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.221037 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.221174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.221322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.221534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.221682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.221822 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.221956 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.222087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.222251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.222391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.222535 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.222675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.222822 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.222973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.223134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.223286 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.223431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.223572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.223704 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.223704 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.223987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.224133 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.224272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.224272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.224529 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.224662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.224799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.224954 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.enabled = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.225094 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.225234 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.225366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.225504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.225649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.225649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.225923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.226055 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.password = **** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.226261 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.226402 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.226535 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.226692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.226839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.226972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.227110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.227242 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.227385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.227531 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.227671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.227819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.227968 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.228101 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.228240 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.228386 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.228526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.228665 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.228797 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.228934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.229066 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.229210 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.229357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.229491 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.229629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.229769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.229908 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.230051 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.230197 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.230340 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.230474 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.230612 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.230744 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.230888 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.231020 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.231158 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.231297 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.231435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.231581 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.231727 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.231875 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.231875 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.232131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.232131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.232402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.232549 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.232683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.232814 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.232956 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.233095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.233234 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] auth.external = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.233384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.233523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] auth.mapped = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.233663 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.233802 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.233950 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.234153 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.234308 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.234450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] auth.password = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.234590 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.234730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] auth.token = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.234730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.234994 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.235140 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] catalog.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.235273 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.235404 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.235542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.235681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.235820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.235974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.236108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.236246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.236393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.236534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.236681 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.236822 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] credential.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.236955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.237100 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] credential.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.237233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.237371 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.237524 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.237664 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.237804 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.237943 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.238082 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] policy.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.238230 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.238380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.238534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.238667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.238806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.238944 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.239082 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.239214 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] receipt.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.239345 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.239484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.239641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.239774 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.239913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.240060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.240060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.240317 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.240456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.240595 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.240734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] resource.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.240879 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.241021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.241153 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.241291 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] resource.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.241430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] federation.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.241562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.241712 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.241845 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.241984 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] federation.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.242137 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.242270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.242400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] revoke.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.242548 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.242689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.242822 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.242964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.243098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.243228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] role.cache_time = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.243371 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.243503 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] role.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.243640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.243778 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] role.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.243911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.244058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] role.list_limit = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.244204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.244343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.244488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.244636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.244768 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.244899 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.245049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.245183 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.245321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.245470 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.245604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.245734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.245878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.246011 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.246172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.246318 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.246457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.246604 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.246807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.246949 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.247082 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.247213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.247351 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.247574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.247723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.247863 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.248002 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.248002 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.248257 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.248401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.248535 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.248666 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.248802 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.248940 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.249161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.249309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.249449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.249582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.249720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.249860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.249992 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.250202 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.250345 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.250477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.250619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.250751 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.250880 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.251025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.251164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.251295 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.251428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.251558 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.251696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.251853 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.251992 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.252135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.252290 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.252429 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.252429 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.252689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.252837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.252974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.252974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.253228 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.253365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] token.cache_time = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.253515 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.253656 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] token.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.253794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.253932 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.254081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.254285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] token.provider = fernet {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.254434 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.254575 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.254707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.254707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.password = **** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.254979 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.254979 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.255246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.255385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.255524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.255664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.255803 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.255942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.256091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.256091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.256366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] trust.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.256366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.256612 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.256758 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.256904 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.257037 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.257168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.257168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.257545 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.257545 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.257816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.257956 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.258119 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.258248 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.258378 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.258378 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.258647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.258779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.258909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.connection = **** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.259040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.259181 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.259319 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.259458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.259589 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.259727 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.259874 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.260013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.260151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.260290 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.260422 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.260552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.260696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.260829 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.260960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.261097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.261229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.261366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.261508 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.261640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.261777 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.261910 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.262040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.262247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.262396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.262536 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.262676 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.262813 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.262949 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.263080 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.263224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.263364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.263504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.263635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.263765 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.263956 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.263956 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=474608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.264233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] policy.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] receipt.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.265401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] resource.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] resource.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] revoke.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.266553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] role.cache_time = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] role.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] role.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] role.list_limit = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.267690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.268898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.270038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.271201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] token.cache_time = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] token.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] token.provider = fernet {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.272326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] trust.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.273565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.connection = **** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.274697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.275846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.276968 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.276968 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 09:20:14.276968 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=474607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 09:20:14.278721 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.278959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.279190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.287909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.298469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.298706 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.298941 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.307496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.321577 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.321801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.330651 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.330865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.374251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.375446 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.425611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.425901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.426001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.426201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.492344 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.492588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.492747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.492961 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.493205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.493347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.493527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.493626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.493757 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.493997 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.494230 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.494479 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.516876 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.517093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.517278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.517389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.517539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.517640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.517783 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.517878 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.518018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.518145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.518284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.518436 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.518537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.518537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.529465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.529633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.542300 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.542516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.542642 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.542802 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.542900 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543249 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543354 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543500 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543599 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=474608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543726 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543726 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.543912 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=474607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 09:20:14.560558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.560784 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 09:20:14.561103 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.561268 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.561384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 09:20:14.561477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 09:20:14.561569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.561692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.561858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.562010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 09:20:14.562223 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.562382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.562549 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.562702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.562702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.562934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.572454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.572737 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.572967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.573111 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.573267 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.573387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.573540 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.573772 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.580643 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.580875 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.585383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.585674 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.585905 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.586084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.586269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.586401 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.586526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.586619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.586748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.586846 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.587049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.587206 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.587314 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.587423 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.587521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.587617 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.587826 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.587979 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.588075 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.588164 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.588313 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.588616 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.611281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.611884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.612825 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.613113 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.613431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.613721 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.617765 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.618049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.618367 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.618486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.618661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.618952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.625116 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.625344 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.625622 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.625915 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.629523 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.629819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.630136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.630241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.630484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.630580 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.630672 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.630758 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.630967 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.631085 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.631254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.631357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.631535 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.631693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.631861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.632091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.647829 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.648133 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.648435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.648741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.652895 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.653071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.656185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.656330 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.656483 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.656608 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.664787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.664967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.664967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.665142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.667351 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.667492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.667645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.667738 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.667928 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.668129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.668231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.668366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.668476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.668615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.668735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.668872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.685817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.686122 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.686446 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.686733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.687115 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.687442 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.687716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.687958 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.694243 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.695326 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.695810 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.696081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.696885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.697161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.698950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.699251 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.699542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.699835 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.700002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.700113 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.700288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.700595 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.700890 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.701181 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.711844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.712115 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.712410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.712675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.712806 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.713045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.713165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.713307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.713400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.713526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.713639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.713866 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.714153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.714449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.729324 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.730153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.732714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.733144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.735390 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.736286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.738085 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.738858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.739617 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.739905 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.740189 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.740358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.740660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.740978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.746772 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.751152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.752407 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.752545 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.752682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.752682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.752682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.753021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.753144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.753279 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.753279 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.753279 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.760311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.760594 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.760938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.761070 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.761368 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.761650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.768618 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=474608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.768908 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.769034 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=474607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 09:20:14.769193 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.769295 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.769467 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.769704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.769813 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.770006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.770595 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.770743 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.770743 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.770910 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.771021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.771241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.771340 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.771524 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.772105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.772236 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.772236 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.772408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.772515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=474608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.772613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.772820 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.773120 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.773410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.773690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.773959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=474607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 09:20:14.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556898e7e9d0 pid: 474608 (default app) Jul 28 09:20:14.807572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556898e7e9d0 pid: 474607 (default app) Jul 28 09:20:14.810040 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acbb0-a088-46e8-a99b-67c35c5249ac None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:14.810365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acbb0-a088-46e8-a99b-67c35c5249ac None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:14.810644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acbb0-a088-46e8-a99b-67c35c5249ac None None] PATH_INFO: `/v3` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:14.815097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jul 28 09:20:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:22.517003 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43936ab3-7c81-4f8b-8c75-d9bfc4d6e2b1 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:22.517679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43936ab3-7c81-4f8b-8c75-d9bfc4d6e2b1 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:22.517679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43936ab3-7c81-4f8b-8c75-d9bfc4d6e2b1 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:22.522088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:22.527549 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:22.528011 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:22.528125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:22.598409 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: INFO dbcounter [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] Registered counter for database keystone Jul 28 09:20:22.620127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG oslo_db.sqlalchemy.engines [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=474608) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 09:20:22.627636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writer thread running {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 09:20:22.805962 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG passlib.handlers.bcrypt [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=474608) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 09:20:22.806315 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG passlib.handlers.bcrypt [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=474608) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 09:20:22.861420 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:22.863106 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:22.995404 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 09:20:23.019533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43936ab3-7c81-4f8b-8c75-d9bfc4d6e2b1 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:23.089747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: INFO dbcounter [None req-43936ab3-7c81-4f8b-8c75-d9bfc4d6e2b1 None None] Registered counter for database keystone Jul 28 09:20:23.109467 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG oslo_db.sqlalchemy.engines [None req-43936ab3-7c81-4f8b-8c75-d9bfc4d6e2b1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=474607) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 09:20:23.117075 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writer thread running {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 09:20:23.242411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43936ab3-7c81-4f8b-8c75-d9bfc4d6e2b1 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:23.247447 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f1104-c5dc-4884-8eab-ed7c2ad3b9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a32e32f35d6b4f64983733c445917938', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:23.248105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f1104-c5dc-4884-8eab-ed7c2ad3b9ec admin admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:23.248393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f1104-c5dc-4884-8eab-ed7c2ad3b9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:23.248675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f1104-c5dc-4884-8eab-ed7c2ad3b9ec admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:23.257511 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f1104-c5dc-4884-8eab-ed7c2ad3b9ec admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:23.309817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2a0f1104-c5dc-4884-8eab-ed7c2ad3b9ec admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 09:20:23.310156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f1104-c5dc-4884-8eab-ed7c2ad3b9ec admin admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:23.322141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 09:20:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:20:23.328702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:23.344934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571bf87-a726-4470-a7a8-f29b3a1c5599 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:23.349627 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77971d-abec-4504-8888-5c6a344138d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a32e32f35d6b4f64983733c445917938', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:23.350809 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77971d-abec-4504-8888-5c6a344138d9 admin admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:23.351100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77971d-abec-4504-8888-5c6a344138d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:23.351380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77971d-abec-4504-8888-5c6a344138d9 admin admin] PATH_INFO: `/v3/projects/a32e32f35d6b4f64983733c445917938` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:23.353016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77971d-abec-4504-8888-5c6a344138d9 admin admin] RBAC: Authorizing `identity:get_project(project_id=a32e32f35d6b4f64983733c445917938)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:23.403427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8a77971d-abec-4504-8888-5c6a344138d9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 09:20:23.403744 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77971d-abec-4504-8888-5c6a344138d9 admin admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:23.408687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jul 28 09:20:23 2022] GET /identity/v3/projects/a32e32f35d6b4f64983733c445917938 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:25.393976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824732b5-632e-467f-b87e-11f929994334 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:25.394583 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824732b5-632e-467f-b87e-11f929994334 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:25.394583 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824732b5-632e-467f-b87e-11f929994334 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:25.398806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:25.404857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f9c66-5eb0-4824-b3d2-bc1fd3e93581 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:25.405353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f9c66-5eb0-4824-b3d2-bc1fd3e93581 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:25.405465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f9c66-5eb0-4824-b3d2-bc1fd3e93581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:25.426088 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-634f9c66-5eb0-4824-b3d2-bc1fd3e93581 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:25.427652 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-634f9c66-5eb0-4824-b3d2-bc1fd3e93581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:25.436264 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 09:20:25.458988 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824732b5-632e-467f-b87e-11f929994334 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:25.473952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824732b5-632e-467f-b87e-11f929994334 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:25.479226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b299a1a-fec8-45f9-9167-0304fbd3cffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a32e32f35d6b4f64983733c445917938', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:25.479852 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299a1a-fec8-45f9-9167-0304fbd3cffc admin admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:25.480139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299a1a-fec8-45f9-9167-0304fbd3cffc admin admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:25.480415 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299a1a-fec8-45f9-9167-0304fbd3cffc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:25.488607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b299a1a-fec8-45f9-9167-0304fbd3cffc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:25.490458 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b299a1a-fec8-45f9-9167-0304fbd3cffc admin admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:25.500387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 09:20:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:20:25.506934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f9c66-5eb0-4824-b3d2-bc1fd3e93581 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:25.517146 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f9c66-5eb0-4824-b3d2-bc1fd3e93581 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:25.523062 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8ba9c-ee9a-4620-b046-b985a17226a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a32e32f35d6b4f64983733c445917938', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:25.523732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8ba9c-ee9a-4620-b046-b985a17226a3 admin admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:25.524019 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8ba9c-ee9a-4620-b046-b985a17226a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:25.524297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8ba9c-ee9a-4620-b046-b985a17226a3 admin admin] PATH_INFO: `/v3/users/530dca3bb7524c5cbcfad367381348a5` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:25.525828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8ba9c-ee9a-4620-b046-b985a17226a3 admin admin] RBAC: Authorizing `identity:get_user(user_id=530dca3bb7524c5cbcfad367381348a5)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:25.527805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8ba9c-ee9a-4620-b046-b985a17226a3 admin admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:25.529448 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jul 28 09:20:25 2022] GET /identity/v3/users/530dca3bb7524c5cbcfad367381348a5 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:27.610220 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14265-179b-4053-85c8-78a8c5d0f6ee None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:27.610773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14265-179b-4053-85c8-78a8c5d0f6ee None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:27.610773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14265-179b-4053-85c8-78a8c5d0f6ee None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:27.612131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:27.617868 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa12dee-08d2-4693-9963-0cc3a31162af None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:27.618407 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa12dee-08d2-4693-9963-0cc3a31162af None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:27.618514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa12dee-08d2-4693-9963-0cc3a31162af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:27.638749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-ffa12dee-08d2-4693-9963-0cc3a31162af None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:27.640013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-ffa12dee-08d2-4693-9963-0cc3a31162af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:27.644235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baeb13-0074-4ef8-a0cc-acae9de747ab None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:27.644527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baeb13-0074-4ef8-a0cc-acae9de747ab None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:27.644802 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baeb13-0074-4ef8-a0cc-acae9de747ab None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:27.646158 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:27.651503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:27.651789 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:27.652066 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:27.695461 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG passlib.handlers.bcrypt [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=474607) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 09:20:27.695875 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG passlib.handlers.bcrypt [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=474607) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 09:20:27.724505 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:20:27.740983 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:27.742691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:27.750167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa12dee-08d2-4693-9963-0cc3a31162af None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:27.776756 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:27.846741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:27.861897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa12dee-08d2-4693-9963-0cc3a31162af None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:27.868989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:27.900140 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482b674-edef-4237-a77b-618643cebf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:27.900819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482b674-edef-4237-a77b-618643cebf0b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:27.901106 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482b674-edef-4237-a77b-618643cebf0b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:27.901386 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482b674-edef-4237-a77b-618643cebf0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:27.913531 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482b674-edef-4237-a77b-618643cebf0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-a482b674-edef-4237-a77b-618643cebf0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:27.922511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:27.923689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:27.924657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:20:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:27.929869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482b674-edef-4237-a77b-618643cebf0b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:27.973826 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9503e-5dad-459b-b341-bb41bdd70220 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:27.976411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:28.011678 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63658c09-7e6f-4763-89c7-528943e29507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:28.012562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63658c09-7e6f-4763-89c7-528943e29507 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:28.012851 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63658c09-7e6f-4763-89c7-528943e29507 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:28.013148 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63658c09-7e6f-4763-89c7-528943e29507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:28.014952 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:28.035011 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482b674-edef-4237-a77b-618643cebf0b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:28.070835 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:28.071454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:28.071742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:28.072006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:28.072774 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:28.073080 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:28.074522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:28.083748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63658c09-7e6f-4763-89c7-528943e29507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:28.085308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63658c09-7e6f-4763-89c7-528943e29507 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:28.086483 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:28.092212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:28.185368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:28.191703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63658c09-7e6f-4763-89c7-528943e29507 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:28.194140 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9426cd5-68ff-4983-8eb0-16460b0349bf None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:28.195273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:28.303687 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63658c09-7e6f-4763-89c7-528943e29507 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:28.340635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:28.341520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:28.341802 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:28.342076 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:28.342397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df4f3e-9ea9-4efa-9172-769a8b82c604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:28.343201 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4f3e-9ea9-4efa-9172-769a8b82c604 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:28.343487 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4f3e-9ea9-4efa-9172-769a8b82c604 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:28.343771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4f3e-9ea9-4efa-9172-769a8b82c604 None admin] PATH_INFO: `/v3/users/530dca3bb7524c5cbcfad367381348a5` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:28.344987 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df4f3e-9ea9-4efa-9172-769a8b82c604 None admin] RBAC: Authorizing `identity:get_user(user_id=530dca3bb7524c5cbcfad367381348a5)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:28.346482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df4f3e-9ea9-4efa-9172-769a8b82c604 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:28.348142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:28 2022] GET /identity/v3/users/530dca3bb7524c5cbcfad367381348a5 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:28.353892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df4f3e-9ea9-4efa-9172-769a8b82c604 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:28.361140 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:28.362684 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:28.369045 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:20:28.381639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:28.382850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:28.383964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:20:28.383964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:20:28.383964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:20:28.383964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:28.383964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:20:28.389590 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:28.447794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:28.468200 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df4f3e-9ea9-4efa-9172-769a8b82c604 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:28.493509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e2fb4-18b3-4288-9a81-7936896289c0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:28.504750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057f02-43e6-4de8-9b4f-418873247540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:28.505409 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057f02-43e6-4de8-9b4f-418873247540 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:28.505698 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057f02-43e6-4de8-9b4f-418873247540 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:28.505971 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057f02-43e6-4de8-9b4f-418873247540 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:28.507207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057f02-43e6-4de8-9b4f-418873247540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:28.508663 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057f02-43e6-4de8-9b4f-418873247540 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:28.510323 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:28.516219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057f02-43e6-4de8-9b4f-418873247540 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:28.528684 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ee01d-2727-4b18-9bae-4e4f56d1276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:28.529323 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee01d-2727-4b18-9bae-4e4f56d1276e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:28.529605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee01d-2727-4b18-9bae-4e4f56d1276e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:28.530001 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee01d-2727-4b18-9bae-4e4f56d1276e None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:28.530849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ee01d-2727-4b18-9bae-4e4f56d1276e None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:28.531141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ee01d-2727-4b18-9bae-4e4f56d1276e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:28.532550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ee01d-2727-4b18-9bae-4e4f56d1276e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:28.549814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:28.602073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:28.624743 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057f02-43e6-4de8-9b4f-418873247540 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:28.662574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f6e58-75e9-461f-a7f2-00021499f4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:28.663213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f6e58-75e9-461f-a7f2-00021499f4b3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:28.663502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f6e58-75e9-461f-a7f2-00021499f4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:28.663779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f6e58-75e9-461f-a7f2-00021499f4b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:28.664605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f6e58-75e9-461f-a7f2-00021499f4b3 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.domain.id=default, user.id=530dca3bb7524c5cbcfad367381348a5) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:28.664920 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f6e58-75e9-461f-a7f2-00021499f4b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:28.666481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f6e58-75e9-461f-a7f2-00021499f4b3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:28.674578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 09:20:28 2022] GET /identity/v3/role_assignments?user.id=530dca3bb7524c5cbcfad367381348a5&scope.domain.id=default&role.id=af958d22229a4a0297bbe2e643f55181 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:31.289875 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba0317-53cc-4270-99d9-3dc6910c37f0 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.291126 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba0317-53cc-4270-99d9-3dc6910c37f0 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.291701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba0317-53cc-4270-99d9-3dc6910c37f0 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.293441 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:31.299626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd026f-810d-4539-83a7-803bff9b8cc9 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.300623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd026f-810d-4539-83a7-803bff9b8cc9 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.301149 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd026f-810d-4539-83a7-803bff9b8cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.319934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd174-43ec-49e2-919a-1d4381030494 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.320648 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd174-43ec-49e2-919a-1d4381030494 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.321190 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd174-43ec-49e2-919a-1d4381030494 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.322855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:20:31.328395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fbaae-0080-4231-a59b-57c99f351811 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.329020 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fbaae-0080-4231-a59b-57c99f351811 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.329560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fbaae-0080-4231-a59b-57c99f351811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.335753 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-b0dd026f-810d-4539-83a7-803bff9b8cc9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:31.337277 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-b0dd026f-810d-4539-83a7-803bff9b8cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:31.359736 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-8c8fbaae-0080-4231-a59b-57c99f351811 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:31.361348 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-8c8fbaae-0080-4231-a59b-57c99f351811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:31.477941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:31.504158 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 13/25] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:31.512702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd026f-810d-4539-83a7-803bff9b8cc9 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:31.538593 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fbaae-0080-4231-a59b-57c99f351811 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:31.648968 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:31.679025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:31.717511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd026f-810d-4539-83a7-803bff9b8cc9 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:31.732815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fbaae-0080-4231-a59b-57c99f351811 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:31.759945 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804cb18-e743-4665-ae1c-13224928eebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:31.760955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804cb18-e743-4665-ae1c-13224928eebb None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.761459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804cb18-e743-4665-ae1c-13224928eebb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.762021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804cb18-e743-4665-ae1c-13224928eebb None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.763125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804cb18-e743-4665-ae1c-13224928eebb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:31.764949 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804cb18-e743-4665-ae1c-13224928eebb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:31.776646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43ef25-8fa0-40d7-bdec-3a98c239b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:31.777463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43ef25-8fa0-40d7-bdec-3a98c239b46b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.777807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43ef25-8fa0-40d7-bdec-3a98c239b46b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.778000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43ef25-8fa0-40d7-bdec-3a98c239b46b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.897932 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 13/26] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:20:31 2022] POST /identity/v3/roles => generated 225 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:31.904172 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a93b-818d-40cd-939b-3d4dfebc2356 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.904385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43ef25-8fa0-40d7-bdec-3a98c239b46b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:31.904987 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a93b-818d-40cd-939b-3d4dfebc2356 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.905514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a93b-818d-40cd-939b-3d4dfebc2356 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-1f43ef25-8fa0-40d7-bdec-3a98c239b46b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:31.910953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:31.912043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:31.912951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 14/27] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:20:31.914214 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca152-8685-4f2d-8596-6d2918b2f4b4 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.914331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca152-8685-4f2d-8596-6d2918b2f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.914546 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca152-8685-4f2d-8596-6d2918b2f4b4 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.915209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 14/28] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:31 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:31.918470 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 15/29] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:31.921657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3a93b-818d-40cd-939b-3d4dfebc2356 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:31.922378 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4481f9d-de72-4c8a-b9d0-d4560a8263f5 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:31.922720 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4481f9d-de72-4c8a-b9d0-d4560a8263f5 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:31.922928 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4481f9d-de72-4c8a-b9d0-d4560a8263f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:31.943629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-a4481f9d-de72-4c8a-b9d0-d4560a8263f5 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:31.944901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-a4481f9d-de72-4c8a-b9d0-d4560a8263f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:32.025788 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 16/30] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:20:32.027168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027c17-a430-40d1-b82b-c767806ec420 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.027460 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027c17-a430-40d1-b82b-c767806ec420 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.027741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027c17-a430-40d1-b82b-c767806ec420 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.029110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 17/31] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:32.034800 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a9f4-14ff-4cf2-96f7-80030d7331d7 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.035161 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a9f4-14ff-4cf2-96f7-80030d7331d7 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.035450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a9f4-14ff-4cf2-96f7-80030d7331d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.055230 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-6b06a9f4-14ff-4cf2-96f7-80030d7331d7 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:32.056464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-6b06a9f4-14ff-4cf2-96f7-80030d7331d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:32.087395 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3a93b-818d-40cd-939b-3d4dfebc2356 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:32.094958 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:32.136481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:32.137421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.137719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.138009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.138823 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:32.139122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:32.140719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:32.141926 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 18/32] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:32.142454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:32.143053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b6bf-63d4-4ce1-bdb3-e3885e9d9cbf None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.143350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b6bf-63d4-4ce1-bdb3-e3885e9d9cbf None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.143699 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b6bf-63d4-4ce1-bdb3-e3885e9d9cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.155100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 15/33] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:32.155976 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:32.163780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-91c6b6bf-63d4-4ce1-bdb3-e3885e9d9cbf None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:32.165146 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-91c6b6bf-63d4-4ce1-bdb3-e3885e9d9cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:32.246484 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 19/34] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:32.247387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b6bf-63d4-4ce1-bdb3-e3885e9d9cbf None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:32.279856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9f181-b2c2-49dc-86ca-3efab1a3a0c7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:32.318610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd89de-67d2-4661-bf9a-26e63195f973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:32.319268 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd89de-67d2-4661-bf9a-26e63195f973 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.319820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd89de-67d2-4661-bf9a-26e63195f973 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.319820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd89de-67d2-4661-bf9a-26e63195f973 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.320532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd89de-67d2-4661-bf9a-26e63195f973 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:32.321953 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd89de-67d2-4661-bf9a-26e63195f973 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:32.338551 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 16/35] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:20:32 2022] POST /identity/v3/roles => generated 219 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:32.339744 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd89de-67d2-4661-bf9a-26e63195f973 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:32.381226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:32.392748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b6bf-63d4-4ce1-bdb3-e3885e9d9cbf None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:32.421481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:32.438898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74972fda-fa87-4b32-97b6-001258d03f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:32.439613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74972fda-fa87-4b32-97b6-001258d03f7b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.439880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74972fda-fa87-4b32-97b6-001258d03f7b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.440160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74972fda-fa87-4b32-97b6-001258d03f7b None admin] PATH_INFO: `/v3/users/530dca3bb7524c5cbcfad367381348a5` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.442222 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74972fda-fa87-4b32-97b6-001258d03f7b None admin] RBAC: Authorizing `identity:get_user(user_id=530dca3bb7524c5cbcfad367381348a5)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:32.443759 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74972fda-fa87-4b32-97b6-001258d03f7b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:32.445524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 20/36] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:32 2022] GET /identity/v3/users/530dca3bb7524c5cbcfad367381348a5 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:32.446345 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74972fda-fa87-4b32-97b6-001258d03f7b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:32.480343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd89de-67d2-4661-bf9a-26e63195f973 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:32.523391 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea64b8-d329-4571-98c9-7c57382dda61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:32.524424 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea64b8-d329-4571-98c9-7c57382dda61 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.524907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea64b8-d329-4571-98c9-7c57382dda61 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.525290 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea64b8-d329-4571-98c9-7c57382dda61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.539232 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea64b8-d329-4571-98c9-7c57382dda61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:32.540688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea64b8-d329-4571-98c9-7c57382dda61 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c2ea64b8-d329-4571-98c9-7c57382dda61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:20:32.559990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:32.561107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:32.562127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:20:32.562127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:20:32.562127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:20:32.562127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:32.562127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 17/37] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:32.562127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f000-9432-471a-b910-519bce4a2f6d None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.562538 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f000-9432-471a-b910-519bce4a2f6d None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.562787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f000-9432-471a-b910-519bce4a2f6d None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.565043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 18/38] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:32.566092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6f000-9432-471a-b910-519bce4a2f6d None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:32.582528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74972fda-fa87-4b32-97b6-001258d03f7b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:32.622424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85aab3-1f67-48ae-a534-8261d6788068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:32.623065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85aab3-1f67-48ae-a534-8261d6788068 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.623346 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85aab3-1f67-48ae-a534-8261d6788068 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.623609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85aab3-1f67-48ae-a534-8261d6788068 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.624353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85aab3-1f67-48ae-a534-8261d6788068 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:32.625753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85aab3-1f67-48ae-a534-8261d6788068 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:32.699623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6f000-9432-471a-b910-519bce4a2f6d None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:32.709136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:32.736280 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a7461-5b59-406b-81c7-6bc1247e3be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:32.736951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a7461-5b59-406b-81c7-6bc1247e3be0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.737244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a7461-5b59-406b-81c7-6bc1247e3be0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.737589 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a7461-5b59-406b-81c7-6bc1247e3be0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.738849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a7461-5b59-406b-81c7-6bc1247e3be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:32.740334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a7461-5b59-406b-81c7-6bc1247e3be0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:32.741993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 19/39] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:32.747085 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a7461-5b59-406b-81c7-6bc1247e3be0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:32.748857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 21/40] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:20:32 2022] POST /identity/v3/roles => generated 223 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:32.750406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f49d2-a913-455c-a875-2f4f8ed5862f None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.750737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f49d2-a913-455c-a875-2f4f8ed5862f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.750955 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f49d2-a913-455c-a875-2f4f8ed5862f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.787774 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-3b6f49d2-a913-455c-a875-2f4f8ed5862f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:32.789609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-3b6f49d2-a913-455c-a875-2f4f8ed5862f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:32.813165 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:32.874156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 22/41] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:32.878424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f49d2-a913-455c-a875-2f4f8ed5862f None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:32.886566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a7461-5b59-406b-81c7-6bc1247e3be0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:32.931010 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:32.931682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:32.931955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:32.932242 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:32.933077 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:32.933393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:32.934836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:32.958908 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 20/42] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:32.960287 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:33.011739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f49d2-a913-455c-a875-2f4f8ed5862f None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:33.013365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:33.049696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a740a0f-e75e-44fb-9daf-0ffcc95bdc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:33.050473 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a740a0f-e75e-44fb-9daf-0ffcc95bdc08 None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:33.050763 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a740a0f-e75e-44fb-9daf-0ffcc95bdc08 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:33.051051 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a740a0f-e75e-44fb-9daf-0ffcc95bdc08 None admin] PATH_INFO: `/v3/domains/default/users/530dca3bb7524c5cbcfad367381348a5/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:33.053254 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a740a0f-e75e-44fb-9daf-0ffcc95bdc08 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=530dca3bb7524c5cbcfad367381348a5, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:33.054991 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a740a0f-e75e-44fb-9daf-0ffcc95bdc08 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:33.072697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 23/43] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jul 28 09:20:32 2022] PUT /identity/v3/domains/default/users/530dca3bb7524c5cbcfad367381348a5/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:33.083612 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a740a0f-e75e-44fb-9daf-0ffcc95bdc08 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:33.092720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61785e3c-58e2-40e4-ad07-0d8d5eb93dae None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:33.105959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:33.143712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbed191-4fc3-4076-b00a-00d1d3f5f158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:33.144403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbed191-4fc3-4076-b00a-00d1d3f5f158 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:33.144707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbed191-4fc3-4076-b00a-00d1d3f5f158 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:33.144975 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbed191-4fc3-4076-b00a-00d1d3f5f158 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:33.146403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbed191-4fc3-4076-b00a-00d1d3f5f158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:33.147968 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbed191-4fc3-4076-b00a-00d1d3f5f158 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:33.150012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 21/44] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:33.155974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbed191-4fc3-4076-b00a-00d1d3f5f158 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:33.218485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a740a0f-e75e-44fb-9daf-0ffcc95bdc08 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:33.269233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5174749-1c4e-4571-9343-9fccd1073a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:33.269899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5174749-1c4e-4571-9343-9fccd1073a3a None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:33.270372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5174749-1c4e-4571-9343-9fccd1073a3a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:33.270665 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5174749-1c4e-4571-9343-9fccd1073a3a None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:33.271372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5174749-1c4e-4571-9343-9fccd1073a3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:33.274156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5174749-1c4e-4571-9343-9fccd1073a3a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:33.286050 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbed191-4fc3-4076-b00a-00d1d3f5f158 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:33.290307 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:33.306872 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 24/45] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:20:33 2022] POST /identity/v3/projects => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:33.335812 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a848f4e-ace2-4ee1-a639-ff9b5811f04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:33.336567 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848f4e-ace2-4ee1-a639-ff9b5811f04f None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:33.336871 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848f4e-ace2-4ee1-a639-ff9b5811f04f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:33.337209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848f4e-ace2-4ee1-a639-ff9b5811f04f None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:33.337953 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a848f4e-ace2-4ee1-a639-ff9b5811f04f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:33.339653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a848f4e-ace2-4ee1-a639-ff9b5811f04f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:33.373084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 22/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:20:33 2022] POST /identity/v3/projects => generated 312 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:20:35.084166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1f92-3cef-41ea-bbbe-ae4ece301d57 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.084767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1f92-3cef-41ea-bbbe-ae4ece301d57 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.084767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1f92-3cef-41ea-bbbe-ae4ece301d57 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.086294 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 25/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:35.091996 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d73440-c8f4-48db-93e7-2eb3d9d88f4c None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.092627 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d73440-c8f4-48db-93e7-2eb3d9d88f4c None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.092627 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d73440-c8f4-48db-93e7-2eb3d9d88f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.112806 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-50d73440-c8f4-48db-93e7-2eb3d9d88f4c None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:35.114400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-50d73440-c8f4-48db-93e7-2eb3d9d88f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:35.189032 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 23/48] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:35.211668 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc1f92-3cef-41ea-bbbe-ae4ece301d57 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:35.288706 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:35.327894 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc1f92-3cef-41ea-bbbe-ae4ece301d57 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:35.362798 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ade049-2864-4db0-80ec-65963d4d2d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:35.363390 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade049-2864-4db0-80ec-65963d4d2d7e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.363681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade049-2864-4db0-80ec-65963d4d2d7e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.363970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade049-2864-4db0-80ec-65963d4d2d7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.375637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ade049-2864-4db0-80ec-65963d4d2d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-13ade049-2864-4db0-80ec-65963d4d2d7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:35.380865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:35.382497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:35.384098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 26/49] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:20:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:35.388447 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d73440-c8f4-48db-93e7-2eb3d9d88f4c None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:35.415184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896310c-c0a8-44b6-ae8c-fd2c9be24631 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.415476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896310c-c0a8-44b6-ae8c-fd2c9be24631 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.415739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896310c-c0a8-44b6-ae8c-fd2c9be24631 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.417127 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 27/50] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:20:35.422479 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54970f61-f3e7-4430-b267-61c7d25a4348 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.422772 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54970f61-f3e7-4430-b267-61c7d25a4348 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.423053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54970f61-f3e7-4430-b267-61c7d25a4348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.441680 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-54970f61-f3e7-4430-b267-61c7d25a4348 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:35.443147 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-54970f61-f3e7-4430-b267-61c7d25a4348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:35.477793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:35.494894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d73440-c8f4-48db-93e7-2eb3d9d88f4c None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:35.516027 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 28/51] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:20:35.531378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:35.532026 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.532313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.532598 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.533343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:35.533655 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:35.535114 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:35.539133 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54970f61-f3e7-4430-b267-61c7d25a4348 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:35.544874 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 24/52] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:35.550765 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:35.600525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:35.651267 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54970f61-f3e7-4430-b267-61c7d25a4348 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:35.664488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8023c7-8317-4e93-aa9a-dc23fb15109f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:35.690967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae830ef4-2c30-4d37-8610-5e05875e1d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:35.691966 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae830ef4-2c30-4d37-8610-5e05875e1d42 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.692337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae830ef4-2c30-4d37-8610-5e05875e1d42 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.692638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae830ef4-2c30-4d37-8610-5e05875e1d42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.693988 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae830ef4-2c30-4d37-8610-5e05875e1d42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:35.695599 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae830ef4-2c30-4d37-8610-5e05875e1d42 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:35.697385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 29/53] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:35.705192 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae830ef4-2c30-4d37-8610-5e05875e1d42 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:35.707799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fdb8a-bdd9-4970-84e9-b816d345c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:35.708527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fdb8a-bdd9-4970-84e9-b816d345c1ea None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.708793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fdb8a-bdd9-4970-84e9-b816d345c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.709148 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fdb8a-bdd9-4970-84e9-b816d345c1ea None admin] PATH_INFO: `/v3/users/530dca3bb7524c5cbcfad367381348a5` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.710510 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fdb8a-bdd9-4970-84e9-b816d345c1ea None admin] RBAC: Authorizing `identity:get_user(user_id=530dca3bb7524c5cbcfad367381348a5)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:35.712014 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fdb8a-bdd9-4970-84e9-b816d345c1ea None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:35.713774 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 25/54] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:35 2022] GET /identity/v3/users/530dca3bb7524c5cbcfad367381348a5 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:35.720008 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fdb8a-bdd9-4970-84e9-b816d345c1ea None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:35.740322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:35.812649 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae830ef4-2c30-4d37-8610-5e05875e1d42 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:35.829253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fdb8a-bdd9-4970-84e9-b816d345c1ea None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:35.837707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:35.853124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d887e8-fd48-4432-92d4-b7b720a1a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:35.853947 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d887e8-fd48-4432-92d4-b7b720a1a33d None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.854483 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d887e8-fd48-4432-92d4-b7b720a1a33d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.854961 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d887e8-fd48-4432-92d4-b7b720a1a33d None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.856148 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d887e8-fd48-4432-92d4-b7b720a1a33d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:35.858432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d887e8-fd48-4432-92d4-b7b720a1a33d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:35.870395 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e7921-0866-4a43-8b61-46c812768bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:35.871190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e7921-0866-4a43-8b61-46c812768bfb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:35.871482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e7921-0866-4a43-8b61-46c812768bfb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:35.871767 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e7921-0866-4a43-8b61-46c812768bfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:35.872999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e7921-0866-4a43-8b61-46c812768bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:35.874611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e7921-0866-4a43-8b61-46c812768bfb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:35.876303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 26/55] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:35.882555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e7921-0866-4a43-8b61-46c812768bfb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:35.888756 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 30/56] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 09:20:35 2022] POST /identity/v3/projects => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:20:35.981796 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:35.999838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e7921-0866-4a43-8b61-46c812768bfb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:36.039345 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be86ffd-8ad4-45df-8dfe-caee41a38ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:36.039993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86ffd-8ad4-45df-8dfe-caee41a38ea5 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:36.040280 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86ffd-8ad4-45df-8dfe-caee41a38ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:36.040562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86ffd-8ad4-45df-8dfe-caee41a38ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:36.041374 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be86ffd-8ad4-45df-8dfe-caee41a38ea5 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.domain.id=default, user.id=530dca3bb7524c5cbcfad367381348a5) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:36.041687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be86ffd-8ad4-45df-8dfe-caee41a38ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:36.043239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be86ffd-8ad4-45df-8dfe-caee41a38ea5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:36.050428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 27/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 09:20:35 2022] GET /identity/v3/role_assignments?user.id=530dca3bb7524c5cbcfad367381348a5&scope.domain.id=default&role.id=af958d22229a4a0297bbe2e643f55181 => generated 530 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:37.912419 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf100fa1-07af-41b8-bdc0-545850e7c239 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:37.912995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf100fa1-07af-41b8-bdc0-545850e7c239 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:37.912995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf100fa1-07af-41b8-bdc0-545850e7c239 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:37.914389 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 31/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:37.920477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c34cf5-01f7-407f-ba53-5ba432b82d96 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:37.921336 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c34cf5-01f7-407f-ba53-5ba432b82d96 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:37.921553 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c34cf5-01f7-407f-ba53-5ba432b82d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:37.944022 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-99c34cf5-01f7-407f-ba53-5ba432b82d96 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:37.945252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-99c34cf5-01f7-407f-ba53-5ba432b82d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:38.021671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 28/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:38.047072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf100fa1-07af-41b8-bdc0-545850e7c239 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:38.162590 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf100fa1-07af-41b8-bdc0-545850e7c239 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:38.198742 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a210405-a6a9-488a-b2e5-5d801c18bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:38.199406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a210405-a6a9-488a-b2e5-5d801c18bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:38.199698 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a210405-a6a9-488a-b2e5-5d801c18bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:38.199977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a210405-a6a9-488a-b2e5-5d801c18bdf9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:38.201186 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a210405-a6a9-488a-b2e5-5d801c18bdf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:38.202749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a210405-a6a9-488a-b2e5-5d801c18bdf9 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:38.204369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 32/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:38.210435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c34cf5-01f7-407f-ba53-5ba432b82d96 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:38.294125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:38.319626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c34cf5-01f7-407f-ba53-5ba432b82d96 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:38.358184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fe441-94f6-42d5-a9c6-5af59092a438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:38.358831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe441-94f6-42d5-a9c6-5af59092a438 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:38.359111 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe441-94f6-42d5-a9c6-5af59092a438 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:38.359393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe441-94f6-42d5-a9c6-5af59092a438 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:38.360086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fe441-94f6-42d5-a9c6-5af59092a438 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:38.361568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fe441-94f6-42d5-a9c6-5af59092a438 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:38.389019 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 29/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:20:38 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:41.600029 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339a1fc-bc67-43a0-a2f7-79ed0e78eb42 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:41.600866 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339a1fc-bc67-43a0-a2f7-79ed0e78eb42 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:41.601302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339a1fc-bc67-43a0-a2f7-79ed0e78eb42 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:41.603227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 33/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:41.609121 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d981c3-9a45-4c72-8960-969b0293d5a6 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:41.609988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d981c3-9a45-4c72-8960-969b0293d5a6 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:41.610471 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d981c3-9a45-4c72-8960-969b0293d5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:41.636700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-00d981c3-9a45-4c72-8960-969b0293d5a6 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:41.637941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-00d981c3-9a45-4c72-8960-969b0293d5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:41.723631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dcc5c-7642-4457-9282-91f9b1cecb6f None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:41.724302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dcc5c-7642-4457-9282-91f9b1cecb6f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:41.724575 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 30/63] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:41.725068 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dcc5c-7642-4457-9282-91f9b1cecb6f None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:41.727965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 34/64] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:41.733456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8330f-eeb3-440d-b1d9-61481d34ff61 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:41.733779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8330f-eeb3-440d-b1d9-61481d34ff61 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:41.734025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8330f-eeb3-440d-b1d9-61481d34ff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:41.751859 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dcc5c-7642-4457-9282-91f9b1cecb6f None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:41.773937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-dbb8330f-eeb3-440d-b1d9-61481d34ff61 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:41.775220 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-dbb8330f-eeb3-440d-b1d9-61481d34ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:41.862696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:41.910896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dcc5c-7642-4457-9282-91f9b1cecb6f None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:41.930497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:41.939834 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 31/65] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:20:41.963353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905a18-7229-44fc-ba37-076bb15e1ba8 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:41.963718 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905a18-7229-44fc-ba37-076bb15e1ba8 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:41.963937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905a18-7229-44fc-ba37-076bb15e1ba8 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:41.965357 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 32/66] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:41.972206 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c36f3-2b1c-4d35-8025-ed277a9fa051 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:41.972427 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963a96b-7a9c-45ec-8a64-9e615a2014d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:41.972551 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c36f3-2b1c-4d35-8025-ed277a9fa051 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:41.972785 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c36f3-2b1c-4d35-8025-ed277a9fa051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:41.973113 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a96b-7a9c-45ec-8a64-9e615a2014d9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:41.973404 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a96b-7a9c-45ec-8a64-9e615a2014d9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:41.973687 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a96b-7a9c-45ec-8a64-9e615a2014d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:41.985904 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963a96b-7a9c-45ec-8a64-9e615a2014d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-b963a96b-7a9c-45ec-8a64-9e615a2014d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:41.991739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:41.992816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:41.993714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:41.995369 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-3c7c36f3-2b1c-4d35-8025-ed277a9fa051 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:41.996895 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-3c7c36f3-2b1c-4d35-8025-ed277a9fa051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:41.997964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 35/67] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:41.999105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963a96b-7a9c-45ec-8a64-9e615a2014d9 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.086004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 33/68] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:42.087231 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c36f3-2b1c-4d35-8025-ed277a9fa051 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.127278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963a96b-7a9c-45ec-8a64-9e615a2014d9 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.130285 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:42.167890 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7faa7-9fa6-43d2-b96e-f8264263a929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.168562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7faa7-9fa6-43d2-b96e-f8264263a929 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.168850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7faa7-9fa6-43d2-b96e-f8264263a929 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.169128 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7faa7-9fa6-43d2-b96e-f8264263a929 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.181159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7faa7-9fa6-43d2-b96e-f8264263a929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-abc7faa7-9fa6-43d2-b96e-f8264263a929 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:42.186423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:42.187519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:42.188455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 36/69] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:20:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:42.189684 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bddbb-2547-4614-b36c-2cbb90c2b239 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.189684 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bddbb-2547-4614-b36c-2cbb90c2b239 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.189975 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bddbb-2547-4614-b36c-2cbb90c2b239 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.191697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 37/70] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:42.196938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bddbb-2547-4614-b36c-2cbb90c2b239 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.221534 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c36f3-2b1c-4d35-8025-ed277a9fa051 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.252177 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:42.276193 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.277334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.277740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.278027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.279098 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] RBAC: Adding query filter params (name=member) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:42.279840 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:42.281556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:42.298469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 34/71] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:42.298469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.326084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bddbb-2547-4614-b36c-2cbb90c2b239 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.366136 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde5cd2-d5cd-4229-ae6e-2754ecc7772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.366820 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde5cd2-d5cd-4229-ae6e-2754ecc7772b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.367104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde5cd2-d5cd-4229-ae6e-2754ecc7772b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.367386 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde5cd2-d5cd-4229-ae6e-2754ecc7772b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.378424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde5cd2-d5cd-4229-ae6e-2754ecc7772b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-4bde5cd2-d5cd-4229-ae6e-2754ecc7772b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:42.383604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:42.384858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:42.385987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 38/72] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:42.388677 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a7e4-f9b9-461c-b9bb-ed0982bc1d3c None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.389096 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a7e4-f9b9-461c-b9bb-ed0982bc1d3c None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.389383 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a7e4-f9b9-461c-b9bb-ed0982bc1d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.406405 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8ae3a-f5b2-4ad2-868d-d708c16bb460 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.407793 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-5fd9a7e4-f9b9-461c-b9bb-ed0982bc1d3c None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:42.409024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-5fd9a7e4-f9b9-461c-b9bb-ed0982bc1d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:42.428341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:42.443325 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.443977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.444270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.444557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.445307 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:42.445626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:42.447080 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:42.455885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 35/73] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:42.456730 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.485334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 39/74] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:42.486690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7bda9-0dba-4656-afeb-eae171feafc1 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.486984 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7bda9-0dba-4656-afeb-eae171feafc1 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.487263 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7bda9-0dba-4656-afeb-eae171feafc1 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.488803 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 40/75] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:42.493284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7bda9-0dba-4656-afeb-eae171feafc1 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.511281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:42.563952 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd50161-4562-4870-960a-4fc47e3fff12 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.606093 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7bda9-0dba-4656-afeb-eae171feafc1 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.607155 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805606f-cef7-4d20-a331-ecacb32638bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.607836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805606f-cef7-4d20-a331-ecacb32638bd None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.608127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805606f-cef7-4d20-a331-ecacb32638bd None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.608410 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805606f-cef7-4d20-a331-ecacb32638bd None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.623966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805606f-cef7-4d20-a331-ecacb32638bd None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:42.625529 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805606f-cef7-4d20-a331-ecacb32638bd None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:42.627353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 36/76] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:42.628151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805606f-cef7-4d20-a331-ecacb32638bd None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.644095 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b146-2844-47bf-aba9-8172118d67bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.644641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b146-2844-47bf-aba9-8172118d67bb None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.644927 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b146-2844-47bf-aba9-8172118d67bb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.645226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b146-2844-47bf-aba9-8172118d67bb None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.645954 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114b146-2844-47bf-aba9-8172118d67bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:42.646308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114b146-2844-47bf-aba9-8172118d67bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:42.647712 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114b146-2844-47bf-aba9-8172118d67bb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:42.659988 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 41/77] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:42.661306 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2d4a-3c57-46b2-9d73-f394b77364d7 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.661608 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2d4a-3c57-46b2-9d73-f394b77364d7 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.661916 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2d4a-3c57-46b2-9d73-f394b77364d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.679616 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-d4eb2d4a-3c57-46b2-9d73-f394b77364d7 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:42.681091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-d4eb2d4a-3c57-46b2-9d73-f394b77364d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:42.692295 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:42.736659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805606f-cef7-4d20-a331-ecacb32638bd None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.744870 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:42.750526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 42/78] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:42.751400 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb2d4a-3c57-46b2-9d73-f394b77364d7 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.779547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268da63-0140-4d4e-82ae-5b878a04d886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.780331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268da63-0140-4d4e-82ae-5b878a04d886 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.780615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268da63-0140-4d4e-82ae-5b878a04d886 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.780894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268da63-0140-4d4e-82ae-5b878a04d886 None admin] PATH_INFO: `/v3/users/530dca3bb7524c5cbcfad367381348a5` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.782135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268da63-0140-4d4e-82ae-5b878a04d886 None admin] RBAC: Authorizing `identity:get_user(user_id=530dca3bb7524c5cbcfad367381348a5)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:42.783603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268da63-0140-4d4e-82ae-5b878a04d886 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:42.785242 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 37/79] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/users/530dca3bb7524c5cbcfad367381348a5 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:42.787082 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268da63-0140-4d4e-82ae-5b878a04d886 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.851624 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb2d4a-3c57-46b2-9d73-f394b77364d7 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.885691 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268da63-0140-4d4e-82ae-5b878a04d886 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:42.886324 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4966c9-16ad-4adb-ad72-8dd0e5646c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.886985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4966c9-16ad-4adb-ad72-8dd0e5646c8e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.887290 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4966c9-16ad-4adb-ad72-8dd0e5646c8e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.887549 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4966c9-16ad-4adb-ad72-8dd0e5646c8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.897915 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4966c9-16ad-4adb-ad72-8dd0e5646c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-0b4966c9-16ad-4adb-ad72-8dd0e5646c8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:42.902799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:42.904040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 43/80] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:42.905061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4966c9-16ad-4adb-ad72-8dd0e5646c8e None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:42.913685 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:42.928629 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c83fb7-4b61-4316-9507-b06eb0d6121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:42.929274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83fb7-4b61-4316-9507-b06eb0d6121e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:42.929564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83fb7-4b61-4316-9507-b06eb0d6121e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:42.929850 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83fb7-4b61-4316-9507-b06eb0d6121e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:42.974133 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.001564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c83fb7-4b61-4316-9507-b06eb0d6121e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.003120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c83fb7-4b61-4316-9507-b06eb0d6121e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.004144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4966c9-16ad-4adb-ad72-8dd0e5646c8e None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.038510 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bbc10-3549-4533-8b8b-08af2c4a2d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.039165 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bbc10-3549-4533-8b8b-08af2c4a2d6c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.039455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bbc10-3549-4533-8b8b-08af2c4a2d6c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.039742 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bbc10-3549-4533-8b8b-08af2c4a2d6c None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.040982 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bbc10-3549-4533-8b8b-08af2c4a2d6c None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.042495 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bbc10-3549-4533-8b8b-08af2c4a2d6c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.044152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 44/81] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:43.044971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bbc10-3549-4533-8b8b-08af2c4a2d6c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.108095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 38/82] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:43.113035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c83fb7-4b61-4316-9507-b06eb0d6121e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.130388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.158084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bbc10-3549-4533-8b8b-08af2c4a2d6c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.185126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.193976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630351c-752b-4a4b-b9d8-acd9a655b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.194688 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630351c-752b-4a4b-b9d8-acd9a655b258 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.194973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630351c-752b-4a4b-b9d8-acd9a655b258 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.195252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630351c-752b-4a4b-b9d8-acd9a655b258 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.196524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630351c-752b-4a4b-b9d8-acd9a655b258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.198043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630351c-752b-4a4b-b9d8-acd9a655b258 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.200056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 45/83] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:43.200964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630351c-752b-4a4b-b9d8-acd9a655b258 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.214872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c83fb7-4b61-4316-9507-b06eb0d6121e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.250310 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb5ee1-f189-4cb2-af7d-5395f3f86466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.250980 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb5ee1-f189-4cb2-af7d-5395f3f86466 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.251269 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb5ee1-f189-4cb2-af7d-5395f3f86466 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.251550 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb5ee1-f189-4cb2-af7d-5395f3f86466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.301346 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630351c-752b-4a4b-b9d8-acd9a655b258 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.322168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb5ee1-f189-4cb2-af7d-5395f3f86466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.323691 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb5ee1-f189-4cb2-af7d-5395f3f86466 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.336039 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.336674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.336963 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.337227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.337970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:43.338360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.339745 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.352656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 46/84] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:43.352656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.354181 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.398697 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.430817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 39/85] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:43.430817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb5ee1-f189-4cb2-af7d-5395f3f86466 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.456264 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532d9e4-0ce4-4fa1-b864-152c5146fc05 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.491400 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115189d-10b9-4a7b-8b52-73225ba4c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.492040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115189d-10b9-4a7b-8b52-73225ba4c293 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.492334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115189d-10b9-4a7b-8b52-73225ba4c293 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.492646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115189d-10b9-4a7b-8b52-73225ba4c293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.525892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb5ee1-f189-4cb2-af7d-5395f3f86466 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.561788 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115189d-10b9-4a7b-8b52-73225ba4c293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.562016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac32d1b-ddb9-4034-96a5-05d752391f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.562517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac32d1b-ddb9-4034-96a5-05d752391f85 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.562801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac32d1b-ddb9-4034-96a5-05d752391f85 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.563087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac32d1b-ddb9-4034-96a5-05d752391f85 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.563289 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115189d-10b9-4a7b-8b52-73225ba4c293 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.564261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac32d1b-ddb9-4034-96a5-05d752391f85 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.565710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac32d1b-ddb9-4034-96a5-05d752391f85 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.566157 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.567730 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 40/86] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:43.573570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac32d1b-ddb9-4034-96a5-05d752391f85 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.618960 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.674503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 47/87] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:43.675993 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115189d-10b9-4a7b-8b52-73225ba4c293 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.681447 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac32d1b-ddb9-4034-96a5-05d752391f85 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.716619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f901343-330d-4bb9-a114-22009a1ffefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.717289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f901343-330d-4bb9-a114-22009a1ffefb None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.717579 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f901343-330d-4bb9-a114-22009a1ffefb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.717862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f901343-330d-4bb9-a114-22009a1ffefb None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.718602 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f901343-330d-4bb9-a114-22009a1ffefb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.720087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f901343-330d-4bb9-a114-22009a1ffefb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.742421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 41/88] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:20:43 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:43.743208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f901343-330d-4bb9-a114-22009a1ffefb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.788684 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115189d-10b9-4a7b-8b52-73225ba4c293 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.792701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.834728 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bda83-ced7-465b-9ef2-f1e9c06f2741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.835411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bda83-ced7-465b-9ef2-f1e9c06f2741 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.835701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bda83-ced7-465b-9ef2-f1e9c06f2741 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.835985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bda83-ced7-465b-9ef2-f1e9c06f2741 None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.837265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bda83-ced7-465b-9ef2-f1e9c06f2741 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.838827 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bda83-ced7-465b-9ef2-f1e9c06f2741 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.845503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 48/89] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:43.845503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bda83-ced7-465b-9ef2-f1e9c06f2741 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.872620 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.876510 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f901343-330d-4bb9-a114-22009a1ffefb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:43.919790 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276ed18-1114-410c-aef9-2ae633b94b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:43.920954 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ed18-1114-410c-aef9-2ae633b94b0d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.920954 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ed18-1114-410c-aef9-2ae633b94b0d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.921172 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ed18-1114-410c-aef9-2ae633b94b0d None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.922548 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276ed18-1114-410c-aef9-2ae633b94b0d None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:43.924568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276ed18-1114-410c-aef9-2ae633b94b0d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:43.926060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 42/90] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:43.926881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276ed18-1114-410c-aef9-2ae633b94b0d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:43.964821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bda83-ced7-465b-9ef2-f1e9c06f2741 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:44.011692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:44.012673 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:44.013105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:44.013567 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:44.014787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] RBAC: Adding query filter params (role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e, user.id=2ef76fedfde848929345c5e73369d36f) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:44.015262 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:44.016990 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:44.025730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 49/91] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/role_assignments?user.id=2ef76fedfde848929345c5e73369d36f&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:44.026922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:44.072991 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276ed18-1114-410c-aef9-2ae633b94b0d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:44.091549 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:44.138573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c9644-a50c-46ac-84af-f9c72e2780ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:44.139424 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c9644-a50c-46ac-84af-f9c72e2780ed None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:44.139776 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c9644-a50c-46ac-84af-f9c72e2780ed None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:44.140128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c9644-a50c-46ac-84af-f9c72e2780ed None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:44.142463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c9644-a50c-46ac-84af-f9c72e2780ed None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:44.144443 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c9644-a50c-46ac-84af-f9c72e2780ed None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:44.148108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 43/92] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:43 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:44.149120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c9644-a50c-46ac-84af-f9c72e2780ed None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:44.214787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89d825-578b-4ccc-87aa-3a8b7ecef233 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:44.222607 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:44.305433 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe783ec-7858-4349-91c4-44e3c4dc3b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:44.309667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe783ec-7858-4349-91c4-44e3c4dc3b99 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:44.310315 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe783ec-7858-4349-91c4-44e3c4dc3b99 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:44.311261 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe783ec-7858-4349-91c4-44e3c4dc3b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:44.386349 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c9644-a50c-46ac-84af-f9c72e2780ed None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:44.446600 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:44.448183 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:44.448851 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:44.450994 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:44.453065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e, user.id=530dca3bb7524c5cbcfad367381348a5) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:44.453824 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:44.456346 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:44.461065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe783ec-7858-4349-91c4-44e3c4dc3b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:44.462806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe783ec-7858-4349-91c4-44e3c4dc3b99 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:44.473263 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 44/93] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:44 2022] GET /identity/v3/role_assignments?user.id=530dca3bb7524c5cbcfad367381348a5&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=af958d22229a4a0297bbe2e643f55181 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:44.479340 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:44.565681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:44.647523 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:44.652797 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a3c89-9c81-4b96-b7fd-3ba78a3b2f55 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:44.704194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:44.705941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:44.706407 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:44.706959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:44.707991 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] RBAC: Adding query filter params (role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e, user.id=2ef76fedfde848929345c5e73369d36f) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:44.708482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:44.710939 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:44.728563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 45/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:44 2022] GET /identity/v3/role_assignments?user.id=2ef76fedfde848929345c5e73369d36f&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:44.771789 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 50/95] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:44.778498 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:44.937474 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc691-2f05-4f0b-8dc1-96aa07b40535 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:44.984448 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a80a6-9d2a-4a46-a099-fefb86742dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:44.985058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a80a6-9d2a-4a46-a099-fefb86742dde None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:44.985346 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a80a6-9d2a-4a46-a099-fefb86742dde None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:44.985627 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a80a6-9d2a-4a46-a099-fefb86742dde None admin] PATH_INFO: `/v3/projects/86673a26b27f4085bb0b9d324d982eac` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:44.986920 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a80a6-9d2a-4a46-a099-fefb86742dde None admin] RBAC: Authorizing `identity:get_project(project_id=86673a26b27f4085bb0b9d324d982eac)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:44.988448 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a80a6-9d2a-4a46-a099-fefb86742dde None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:44.990614 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 46/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:44 2022] GET /identity/v3/projects/86673a26b27f4085bb0b9d324d982eac => generated 312 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:44.996609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe783ec-7858-4349-91c4-44e3c4dc3b99 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:45.137309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe783ec-7858-4349-91c4-44e3c4dc3b99 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:45.177674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091f7ec-3024-4442-b646-3f3074960d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:45.178753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091f7ec-3024-4442-b646-3f3074960d0b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:45.179248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091f7ec-3024-4442-b646-3f3074960d0b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:45.179759 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091f7ec-3024-4442-b646-3f3074960d0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:45.180772 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091f7ec-3024-4442-b646-3f3074960d0b None admin] RBAC: Adding query filter params (role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=86673a26b27f4085bb0b9d324d982eac, user.id=2ef76fedfde848929345c5e73369d36f) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:45.181302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091f7ec-3024-4442-b646-3f3074960d0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:45.183028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091f7ec-3024-4442-b646-3f3074960d0b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:45.187009 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:45.190892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 51/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:44 2022] GET /identity/v3/role_assignments?user.id=2ef76fedfde848929345c5e73369d36f&scope.project.id=86673a26b27f4085bb0b9d324d982eac&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:46.457979 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f94f3-76e3-4a03-b668-d8dbdff68d4c None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:46.459059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f94f3-76e3-4a03-b668-d8dbdff68d4c None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:46.459567 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f94f3-76e3-4a03-b668-d8dbdff68d4c None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:46.461175 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 47/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:46.466868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d2908-af93-4fb6-8a62-85819128330a None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:46.467378 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d2908-af93-4fb6-8a62-85819128330a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:46.467567 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d2908-af93-4fb6-8a62-85819128330a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:46.495208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-af9d2908-af93-4fb6-8a62-85819128330a None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:46.496533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-af9d2908-af93-4fb6-8a62-85819128330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:46.586016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 52/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:46.623689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f94f3-76e3-4a03-b668-d8dbdff68d4c None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:46.656838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:46.742410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cddf7-a6d1-4082-95b1-27d7d54679ca None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:46.742604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cddf7-a6d1-4082-95b1-27d7d54679ca None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:46.742885 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cddf7-a6d1-4082-95b1-27d7d54679ca None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:46.744385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 53/100] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:46.751136 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20acff1-77b9-4d8a-be18-6ea9e5c6beb9 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:46.751524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20acff1-77b9-4d8a-be18-6ea9e5c6beb9 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:46.751722 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20acff1-77b9-4d8a-be18-6ea9e5c6beb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:46.776788 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-c20acff1-77b9-4d8a-be18-6ea9e5c6beb9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:46.778152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-c20acff1-77b9-4d8a-be18-6ea9e5c6beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:46.791515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f94f3-76e3-4a03-b668-d8dbdff68d4c None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:46.856593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52a411-86bf-454c-b9a8-5c697d3e32fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:46.857267 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a411-86bf-454c-b9a8-5c697d3e32fe None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:46.857592 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a411-86bf-454c-b9a8-5c697d3e32fe None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:46.857907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a411-86bf-454c-b9a8-5c697d3e32fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:46.877043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52a411-86bf-454c-b9a8-5c697d3e32fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ee52a411-86bf-454c-b9a8-5c697d3e32fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:46.882621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:46.883662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:20:46.884574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:46.885779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 48/101] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:46.890669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 54/102] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:20:46.891508 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52a411-86bf-454c-b9a8-5c697d3e32fe None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:46.948749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20acff1-77b9-4d8a-be18-6ea9e5c6beb9 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:47.042284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:47.077429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:47.101981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52a411-86bf-454c-b9a8-5c697d3e32fe None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:47.102521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20acff1-77b9-4d8a-be18-6ea9e5c6beb9 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:47.149814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965979a8-d7e2-4c05-ac5b-e7757a6ce380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:47.150548 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965979a8-d7e2-4c05-ac5b-e7757a6ce380 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.150836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965979a8-d7e2-4c05-ac5b-e7757a6ce380 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.151116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965979a8-d7e2-4c05-ac5b-e7757a6ce380 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.154923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965979a8-d7e2-4c05-ac5b-e7757a6ce380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:47.155410 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:47.156325 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.156459 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965979a8-d7e2-4c05-ac5b-e7757a6ce380 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:47.156965 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.157458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.158330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 55/103] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:47.158704 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] RBAC: Adding query filter params (name=reader) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:47.159236 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:47.160954 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:47.175049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 49/104] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:47.177148 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965979a8-d7e2-4c05-ac5b-e7757a6ce380 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:47.181334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:47.307639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f1052-af87-4270-a65a-66536525a236 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:47.343302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965979a8-d7e2-4c05-ac5b-e7757a6ce380 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:47.350467 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77c661-9e4c-4df5-b81d-8de24e2666a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:47.352328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77c661-9e4c-4df5-b81d-8de24e2666a9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.352856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77c661-9e4c-4df5-b81d-8de24e2666a9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.353373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77c661-9e4c-4df5-b81d-8de24e2666a9 None admin] PATH_INFO: `/v3/users/10b4e12de60a4288b72e15ab6c89f6ea` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.370052 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77c661-9e4c-4df5-b81d-8de24e2666a9 None admin] RBAC: Authorizing `identity:get_user(user_id=10b4e12de60a4288b72e15ab6c89f6ea)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:47.372083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77c661-9e4c-4df5-b81d-8de24e2666a9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:47.374269 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 50/105] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:47 2022] GET /identity/v3/users/10b4e12de60a4288b72e15ab6c89f6ea => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:47.380819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77c661-9e4c-4df5-b81d-8de24e2666a9 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:47.396413 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:47.403949 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:47.427444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7967866-ef57-43e4-8ba8-6ee540e3a2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:47.428098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967866-ef57-43e4-8ba8-6ee540e3a2cb None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.428386 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967866-ef57-43e4-8ba8-6ee540e3a2cb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.428665 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967866-ef57-43e4-8ba8-6ee540e3a2cb None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.429390 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7967866-ef57-43e4-8ba8-6ee540e3a2cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:47.430942 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7967866-ef57-43e4-8ba8-6ee540e3a2cb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:47.469527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 56/106] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:20:47 2022] POST /identity/v3/projects => generated 302 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:47.476537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8bad3-e970-4f07-8963-9b2f0fc1cae9 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.476537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8bad3-e970-4f07-8963-9b2f0fc1cae9 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.476537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8bad3-e970-4f07-8963-9b2f0fc1cae9 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.477972 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 57/107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:47.484935 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7ac60-334c-4cd2-be6b-3c85d88e388a None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.485270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7ac60-334c-4cd2-be6b-3c85d88e388a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.485496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7ac60-334c-4cd2-be6b-3c85d88e388a None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.490065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 58/108] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:47.494033 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb1a8-86d2-443d-af12-c28761662eaa None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.494349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb1a8-86d2-443d-af12-c28761662eaa None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.494641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb1a8-86d2-443d-af12-c28761662eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.535766 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-03abb1a8-86d2-443d-af12-c28761662eaa None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:47.537004 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-03abb1a8-86d2-443d-af12-c28761662eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:47.605507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77c661-9e4c-4df5-b81d-8de24e2666a9 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:47.657232 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 59/109] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:47.663810 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec78d44-8953-4e1a-81b4-07d632337ca1 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.664170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec78d44-8953-4e1a-81b4-07d632337ca1 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.664365 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec78d44-8953-4e1a-81b4-07d632337ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.684152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-7ec78d44-8953-4e1a-81b4-07d632337ca1 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:47.685364 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-7ec78d44-8953-4e1a-81b4-07d632337ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:47.703852 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea03f4-cc37-4753-8eda-d2f614503fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:47.704521 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea03f4-cc37-4753-8eda-d2f614503fdf None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.704813 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea03f4-cc37-4753-8eda-d2f614503fdf None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.705102 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea03f4-cc37-4753-8eda-d2f614503fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.771354 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 60/110] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:47.772077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec78d44-8953-4e1a-81b4-07d632337ca1 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:47.795151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea03f4-cc37-4753-8eda-d2f614503fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:47.796804 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea03f4-cc37-4753-8eda-d2f614503fdf None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:47.817242 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:47.825366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:47.903902 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec78d44-8953-4e1a-81b4-07d632337ca1 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:47.927646 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 51/111] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:47.928687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea03f4-cc37-4753-8eda-d2f614503fdf None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:47.954878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d0048-e255-41fa-8085-ac85d7c6ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:47.955565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d0048-e255-41fa-8085-ac85d7c6ab0d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:47.955857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d0048-e255-41fa-8085-ac85d7c6ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:47.956139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d0048-e255-41fa-8085-ac85d7c6ab0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:47.976125 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d0048-e255-41fa-8085-ac85d7c6ab0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-165d0048-e255-41fa-8085-ac85d7c6ab0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:47.991692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:47.994560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 61/112] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:20:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:47.995562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d0048-e255-41fa-8085-ac85d7c6ab0d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:48.050292 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea03f4-cc37-4753-8eda-d2f614503fdf None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:48.068930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:48.088159 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39086277-53f1-4b88-8da4-6ccaf192f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:48.088980 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086277-53f1-4b88-8da4-6ccaf192f675 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.089381 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086277-53f1-4b88-8da4-6ccaf192f675 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.089568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086277-53f1-4b88-8da4-6ccaf192f675 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.102069 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39086277-53f1-4b88-8da4-6ccaf192f675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-39086277-53f1-4b88-8da4-6ccaf192f675 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:48.107244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:48.109371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 52/113] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:48.111380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39086277-53f1-4b88-8da4-6ccaf192f675 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:48.213703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:48.219450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d0048-e255-41fa-8085-ac85d7c6ab0d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:48.253360 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39086277-53f1-4b88-8da4-6ccaf192f675 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:48.267579 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bd528-7e90-4746-98cb-8641665e0a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:48.268402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bd528-7e90-4746-98cb-8641665e0a70 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.268821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bd528-7e90-4746-98cb-8641665e0a70 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.269191 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bd528-7e90-4746-98cb-8641665e0a70 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.270575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bd528-7e90-4746-98cb-8641665e0a70 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:48.272170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bd528-7e90-4746-98cb-8641665e0a70 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:48.274244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 62/114] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:47 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:48.275049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bd528-7e90-4746-98cb-8641665e0a70 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:48.293147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c90c9-d5b9-46e8-8940-2d21ef32ae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:48.293929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c90c9-d5b9-46e8-8940-2d21ef32ae09 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.294320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c90c9-d5b9-46e8-8940-2d21ef32ae09 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.294695 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c90c9-d5b9-46e8-8940-2d21ef32ae09 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.295610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c90c9-d5b9-46e8-8940-2d21ef32ae09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:48.295966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c90c9-d5b9-46e8-8940-2d21ef32ae09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:48.297445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c90c9-d5b9-46e8-8940-2d21ef32ae09 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:48.311660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 53/115] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:48.311660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286ceeb-c635-4d05-bc3d-fa8b48fe73e6 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.311660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286ceeb-c635-4d05-bc3d-fa8b48fe73e6 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.311660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286ceeb-c635-4d05-bc3d-fa8b48fe73e6 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.311660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 54/116] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:48.315539 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ab7d-f9a8-44dd-b404-4ac87aea8eff None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.315831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ab7d-f9a8-44dd-b404-4ac87aea8eff None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.316111 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ab7d-f9a8-44dd-b404-4ac87aea8eff None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.318527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 55/117] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:48.321421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2ab7d-f9a8-44dd-b404-4ac87aea8eff None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:48.355872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:48.400830 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bd528-7e90-4746-98cb-8641665e0a70 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:48.442634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:48.443712 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.444073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.444424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.445419 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:48.445858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2ab7d-f9a8-44dd-b404-4ac87aea8eff None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:48.446203 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:48.448321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:48.464598 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 63/118] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:48.464598 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:48.492205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bef0d1-e5d3-41c4-bf5e-235a292223e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:48.492900 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bef0d1-e5d3-41c4-bf5e-235a292223e4 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.493135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bef0d1-e5d3-41c4-bf5e-235a292223e4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.493389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bef0d1-e5d3-41c4-bf5e-235a292223e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.494224 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bef0d1-e5d3-41c4-bf5e-235a292223e4 None admin] RBAC: Adding query filter params (role.id=f6ae8eb934c74269ae9e47640f831a4e, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e, user.id=10b4e12de60a4288b72e15ab6c89f6ea) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:48.494543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bef0d1-e5d3-41c4-bf5e-235a292223e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:48.496018 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bef0d1-e5d3-41c4-bf5e-235a292223e4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:48.500108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:48.503353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 56/119] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:48 2022] GET /identity/v3/role_assignments?user.id=10b4e12de60a4288b72e15ab6c89f6ea&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=f6ae8eb934c74269ae9e47640f831a4e => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:48.514547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e89a-b5e6-49c3-ae9a-4c546d87169d None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.514547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e89a-b5e6-49c3-ae9a-4c546d87169d None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.514547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e89a-b5e6-49c3-ae9a-4c546d87169d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.534039 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-a027e89a-b5e6-49c3-ae9a-4c546d87169d None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:48.535550 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-a027e89a-b5e6-49c3-ae9a-4c546d87169d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:48.599438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc4b48-b98d-465b-9dfb-c9a49a11a819 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:48.648976 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 57/120] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:48.650570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03a06-c62f-4280-b937-6b029cbc07a7 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.650946 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03a06-c62f-4280-b937-6b029cbc07a7 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.651310 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03a06-c62f-4280-b937-6b029cbc07a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.667009 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdfd54-b383-4e7a-a8b5-6db512aa412b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:48.667744 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdfd54-b383-4e7a-a8b5-6db512aa412b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.668033 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdfd54-b383-4e7a-a8b5-6db512aa412b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.668318 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdfd54-b383-4e7a-a8b5-6db512aa412b None admin] PATH_INFO: `/v3/users/530dca3bb7524c5cbcfad367381348a5` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.673639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-54c03a06-c62f-4280-b937-6b029cbc07a7 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:48.675060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdfd54-b383-4e7a-a8b5-6db512aa412b None admin] RBAC: Authorizing `identity:get_user(user_id=530dca3bb7524c5cbcfad367381348a5)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:48.675212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-54c03a06-c62f-4280-b937-6b029cbc07a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:48.676601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdfd54-b383-4e7a-a8b5-6db512aa412b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:48.678897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 64/121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:48 2022] GET /identity/v3/users/530dca3bb7524c5cbcfad367381348a5 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:48.679721 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdfd54-b383-4e7a-a8b5-6db512aa412b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:48.710678 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:48.780169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 58/122] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:48.787535 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c03a06-c62f-4280-b937-6b029cbc07a7 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:48.832823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:48.837537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdfd54-b383-4e7a-a8b5-6db512aa412b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:48.886730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9fae1-5238-4225-bbb9-f7ec794457d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:48.887713 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9fae1-5238-4225-bbb9-f7ec794457d8 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.888186 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9fae1-5238-4225-bbb9-f7ec794457d8 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.888635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9fae1-5238-4225-bbb9-f7ec794457d8 None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.890665 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9fae1-5238-4225-bbb9-f7ec794457d8 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:48.892730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9fae1-5238-4225-bbb9-f7ec794457d8 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:48.895789 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 65/123] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:48 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:48.897244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9fae1-5238-4225-bbb9-f7ec794457d8 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:48.916629 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c03a06-c62f-4280-b937-6b029cbc07a7 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:48.964630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53df7a1-0897-449f-b1a6-8e5bf2bdae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:48.965708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53df7a1-0897-449f-b1a6-8e5bf2bdae3b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:48.966201 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53df7a1-0897-449f-b1a6-8e5bf2bdae3b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:48.967290 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53df7a1-0897-449f-b1a6-8e5bf2bdae3b None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:48.969055 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53df7a1-0897-449f-b1a6-8e5bf2bdae3b None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:48.971679 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53df7a1-0897-449f-b1a6-8e5bf2bdae3b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:48.974610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 59/124] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:48 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:48.975762 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53df7a1-0897-449f-b1a6-8e5bf2bdae3b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.009984 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:49.037174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9fae1-5238-4225-bbb9-f7ec794457d8 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.075891 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490328d-aa79-4c2e-8562-9517c1d02473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.076938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490328d-aa79-4c2e-8562-9517c1d02473 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.077995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490328d-aa79-4c2e-8562-9517c1d02473 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.078498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490328d-aa79-4c2e-8562-9517c1d02473 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.093315 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490328d-aa79-4c2e-8562-9517c1d02473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-f490328d-aa79-4c2e-8562-9517c1d02473 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:49.102769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:49.104016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:20:49.105141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:49.105934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53df7a1-0897-449f-b1a6-8e5bf2bdae3b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.107723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 66/125] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:20:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:49.111029 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490328d-aa79-4c2e-8562-9517c1d02473 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.150363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:49.151994 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc70e74-cb76-4a3e-8c9c-df47228e1da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.152839 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70e74-cb76-4a3e-8c9c-df47228e1da9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.152968 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70e74-cb76-4a3e-8c9c-df47228e1da9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.153216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70e74-cb76-4a3e-8c9c-df47228e1da9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.174296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc70e74-cb76-4a3e-8c9c-df47228e1da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-2cc70e74-cb76-4a3e-8c9c-df47228e1da9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:49.180819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:49.182161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:49.183301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:49.185914 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 60/126] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:49.191641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc70e74-cb76-4a3e-8c9c-df47228e1da9 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.241938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490328d-aa79-4c2e-8562-9517c1d02473 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.283141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e03e7-cd7d-4631-a57b-18e8b6903891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.285055 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e03e7-cd7d-4631-a57b-18e8b6903891 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.285556 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e03e7-cd7d-4631-a57b-18e8b6903891 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.286051 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e03e7-cd7d-4631-a57b-18e8b6903891 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.287613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e03e7-cd7d-4631-a57b-18e8b6903891 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:49.289898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e03e7-cd7d-4631-a57b-18e8b6903891 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:49.292983 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 67/127] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:49 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:49.294461 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e03e7-cd7d-4631-a57b-18e8b6903891 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.301307 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:49.322921 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc70e74-cb76-4a3e-8c9c-df47228e1da9 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf815670-496b-4579-8592-a0c324709eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.363144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf815670-496b-4579-8592-a0c324709eb5 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.363630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf815670-496b-4579-8592-a0c324709eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.363834 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf815670-496b-4579-8592-a0c324709eb5 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/users/530dca3bb7524c5cbcfad367381348a5/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.366720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf815670-496b-4579-8592-a0c324709eb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc405d3f9e54cbf86ac4ca906b7637e, user_id=530dca3bb7524c5cbcfad367381348a5, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:49.368515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf815670-496b-4579-8592-a0c324709eb5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:49.394274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 61/128] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:20:49 2022] PUT /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/users/530dca3bb7524c5cbcfad367381348a5/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:49.399137 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf815670-496b-4579-8592-a0c324709eb5 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.425841 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:49.435869 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e03e7-cd7d-4631-a57b-18e8b6903891 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.477024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.478032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.478553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.479062 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.480068 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:49.480654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:49.482335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:49.492478 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 68/129] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:20:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:49.493644 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.630314 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf815670-496b-4579-8592-a0c324709eb5 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.656317 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799add7-7924-4adc-894c-b385e55de834 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.676768 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.677417 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.677861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.678240 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.679067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] RBAC: Adding query filter params (name=member) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:49.679468 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:49.680963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:49.690509 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 62/130] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:49.691663 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.702990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:49.704055 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78583471-d4ee-4528-824c-b6473797202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.705053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583471-d4ee-4528-824c-b6473797202f None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.705555 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583471-d4ee-4528-824c-b6473797202f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.706196 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583471-d4ee-4528-824c-b6473797202f None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/users/2ef76fedfde848929345c5e73369d36f/roles/16a8547072ea431abbe07aad19a1fd43` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.709721 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78583471-d4ee-4528-824c-b6473797202f None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc405d3f9e54cbf86ac4ca906b7637e, user_id=2ef76fedfde848929345c5e73369d36f, role_id=16a8547072ea431abbe07aad19a1fd43)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:49.712347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78583471-d4ee-4528-824c-b6473797202f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:49.740456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 69/131] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:20:49 2022] PUT /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/users/2ef76fedfde848929345c5e73369d36f/roles/16a8547072ea431abbe07aad19a1fd43 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:49.742385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78583471-d4ee-4528-824c-b6473797202f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.784299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:49.864066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e9fad-8811-42b1-8580-fa1760111aab None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.928848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadbea2-b4fc-40f0-bba1-10f05942ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.929538 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadbea2-b4fc-40f0-bba1-10f05942ec00 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.929828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadbea2-b4fc-40f0-bba1-10f05942ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.930128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadbea2-b4fc-40f0-bba1-10f05942ec00 None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.935009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadbea2-b4fc-40f0-bba1-10f05942ec00 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:49.936637 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadbea2-b4fc-40f0-bba1-10f05942ec00 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:49.938667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 63/132] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:49 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:49.944528 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadbea2-b4fc-40f0-bba1-10f05942ec00 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:49.955517 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78583471-d4ee-4528-824c-b6473797202f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:49.995808 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e773-be19-4675-a9e8-ee48bde30c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:49.996880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e773-be19-4675-a9e8-ee48bde30c2d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:49.997431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e773-be19-4675-a9e8-ee48bde30c2d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:49.997911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e773-be19-4675-a9e8-ee48bde30c2d None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:49.999678 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307e773-be19-4675-a9e8-ee48bde30c2d None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:50.001561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307e773-be19-4675-a9e8-ee48bde30c2d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:50.003664 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 70/133] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:49 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:50.009511 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e773-be19-4675-a9e8-ee48bde30c2d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:50.048041 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:50.095659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadbea2-b4fc-40f0-bba1-10f05942ec00 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:50.110656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:50.130018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e773-be19-4675-a9e8-ee48bde30c2d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:50.134554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832f9b1-10a1-4d65-b279-95b777292504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:50.135308 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832f9b1-10a1-4d65-b279-95b777292504 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:50.135666 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832f9b1-10a1-4d65-b279-95b777292504 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:50.136020 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832f9b1-10a1-4d65-b279-95b777292504 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:50.137296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832f9b1-10a1-4d65-b279-95b777292504 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:50.138947 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832f9b1-10a1-4d65-b279-95b777292504 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:50.141057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 64/134] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:49 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:50.142190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91f4d9-5b42-490a-87a5-0636624be49a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:50.142623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91f4d9-5b42-490a-87a5-0636624be49a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:50.143210 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91f4d9-5b42-490a-87a5-0636624be49a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:50.144318 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 65/135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:50.147065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91f4d9-5b42-490a-87a5-0636624be49a None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:50.175422 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dddd1-0223-488f-9fb2-c6042eb1954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:50.176215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dddd1-0223-488f-9fb2-c6042eb1954a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:50.176570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dddd1-0223-488f-9fb2-c6042eb1954a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:50.176921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dddd1-0223-488f-9fb2-c6042eb1954a None admin] PATH_INFO: `/v3/projects/86673a26b27f4085bb0b9d324d982eac` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:50.178311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dddd1-0223-488f-9fb2-c6042eb1954a None admin] RBAC: Authorizing `identity:get_project(project_id=86673a26b27f4085bb0b9d324d982eac)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:50.179898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dddd1-0223-488f-9fb2-c6042eb1954a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:50.181718 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 71/136] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:50 2022] GET /identity/v3/projects/86673a26b27f4085bb0b9d324d982eac => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:50.186130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4db3fe-c912-4ac5-a4ff-df2ca9a018d4 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:50.186130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4db3fe-c912-4ac5-a4ff-df2ca9a018d4 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:50.186130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4db3fe-c912-4ac5-a4ff-df2ca9a018d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:50.205519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-1f4db3fe-c912-4ac5-a4ff-df2ca9a018d4 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:50.206882 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-1f4db3fe-c912-4ac5-a4ff-df2ca9a018d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:50.292883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91f4d9-5b42-490a-87a5-0636624be49a None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:50.303862 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 72/137] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:50.304877 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4db3fe-c912-4ac5-a4ff-df2ca9a018d4 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:50.335677 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4142cc1-10f6-4c3f-827b-621bf1410cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:50.336772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4142cc1-10f6-4c3f-827b-621bf1410cdd None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:50.337261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4142cc1-10f6-4c3f-827b-621bf1410cdd None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:50.337748 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4142cc1-10f6-4c3f-827b-621bf1410cdd None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/users/2ef76fedfde848929345c5e73369d36f/roles/6b79a8b00a964a8fbcbaaa8371e71edb` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:50.341089 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4142cc1-10f6-4c3f-827b-621bf1410cdd None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc405d3f9e54cbf86ac4ca906b7637e, user_id=2ef76fedfde848929345c5e73369d36f, role_id=6b79a8b00a964a8fbcbaaa8371e71edb)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:50.342931 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4142cc1-10f6-4c3f-827b-621bf1410cdd None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:50.362077 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 66/138] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:20:50 2022] PUT /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/users/2ef76fedfde848929345c5e73369d36f/roles/6b79a8b00a964a8fbcbaaa8371e71edb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:50.364038 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4142cc1-10f6-4c3f-827b-621bf1410cdd None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:50.367620 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:50.492044 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4142cc1-10f6-4c3f-827b-621bf1410cdd None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:50.534572 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c4862-6415-4a8e-8855-c1c96bf42d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:50.535640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c4862-6415-4a8e-8855-c1c96bf42d15 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:50.536153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c4862-6415-4a8e-8855-c1c96bf42d15 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:50.537195 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c4862-6415-4a8e-8855-c1c96bf42d15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:50.538707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c4862-6415-4a8e-8855-c1c96bf42d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:50.540438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c4862-6415-4a8e-8855-c1c96bf42d15 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:50.542698 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 67/139] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:50.551839 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c4862-6415-4a8e-8855-c1c96bf42d15 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:50.568545 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:50.646133 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4db3fe-c912-4ac5-a4ff-df2ca9a018d4 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:50.683173 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:50.721058 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d5ece-a5b5-4750-bd4b-9d95252b2367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:50.721775 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5ece-a5b5-4750-bd4b-9d95252b2367 None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:50.722058 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5ece-a5b5-4750-bd4b-9d95252b2367 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:50.722395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5ece-a5b5-4750-bd4b-9d95252b2367 None admin] PATH_INFO: `/v3/projects/86673a26b27f4085bb0b9d324d982eac/users/2ef76fedfde848929345c5e73369d36f/roles/16a8547072ea431abbe07aad19a1fd43` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:50.724468 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d5ece-a5b5-4750-bd4b-9d95252b2367 None admin] RBAC: Authorizing `identity:create_grant(project_id=86673a26b27f4085bb0b9d324d982eac, user_id=2ef76fedfde848929345c5e73369d36f, role_id=16a8547072ea431abbe07aad19a1fd43)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:50.726118 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d5ece-a5b5-4750-bd4b-9d95252b2367 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:50.745945 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 73/140] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:20:50 2022] PUT /identity/v3/projects/86673a26b27f4085bb0b9d324d982eac/users/2ef76fedfde848929345c5e73369d36f/roles/16a8547072ea431abbe07aad19a1fd43 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:50.750353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c4862-6415-4a8e-8855-c1c96bf42d15 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:50.806419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e7b4a-af4a-448d-9a33-d3d6f3f15c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:50.807073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7b4a-af4a-448d-9a33-d3d6f3f15c80 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:50.807355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7b4a-af4a-448d-9a33-d3d6f3f15c80 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:50.807631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7b4a-af4a-448d-9a33-d3d6f3f15c80 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:50.808370 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e7b4a-af4a-448d-9a33-d3d6f3f15c80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:50.809893 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e7b4a-af4a-448d-9a33-d3d6f3f15c80 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:50.843986 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 68/141] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:20:50 2022] POST /identity/v3/users => generated 283 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:51.074748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605848db-04b7-4d3d-9ff4-cd137eb53b75 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:51.074959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605848db-04b7-4d3d-9ff4-cd137eb53b75 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:51.075243 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605848db-04b7-4d3d-9ff4-cd137eb53b75 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:51.077018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 74/142] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:51.083039 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3d2b0-8541-4e06-a88f-7bed4400eb50 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:51.083384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3d2b0-8541-4e06-a88f-7bed4400eb50 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:51.083606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3d2b0-8541-4e06-a88f-7bed4400eb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:51.108225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-d9d3d2b0-8541-4e06-a88f-7bed4400eb50 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:51.109557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-d9d3d2b0-8541-4e06-a88f-7bed4400eb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:51.197754 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 69/143] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:51.223055 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605848db-04b7-4d3d-9ff4-cd137eb53b75 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:51.350808 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605848db-04b7-4d3d-9ff4-cd137eb53b75 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:51.394351 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb894d-5b1b-46f1-8daa-17a2541cbb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:51.395035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb894d-5b1b-46f1-8daa-17a2541cbb12 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:51.395320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb894d-5b1b-46f1-8daa-17a2541cbb12 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:51.395601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb894d-5b1b-46f1-8daa-17a2541cbb12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:51.411825 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb894d-5b1b-46f1-8daa-17a2541cbb12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-0fcb894d-5b1b-46f1-8daa-17a2541cbb12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:51.416934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:51.419828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:51.422357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 75/144] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:51.430128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3d2b0-8541-4e06-a88f-7bed4400eb50 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:51.495032 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:51.546576 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3d2b0-8541-4e06-a88f-7bed4400eb50 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:51.592930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb976592-4172-41c4-8369-76654651a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:51.598049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb976592-4172-41c4-8369-76654651a96f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:51.598049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb976592-4172-41c4-8369-76654651a96f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:51.598049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb976592-4172-41c4-8369-76654651a96f None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:51.598049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb976592-4172-41c4-8369-76654651a96f None admin] RBAC: Adding query filter params (name=reader) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:51.598049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb976592-4172-41c4-8369-76654651a96f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:51.598049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb976592-4172-41c4-8369-76654651a96f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:51.610546 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 70/145] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:51.619247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb894d-5b1b-46f1-8daa-17a2541cbb12 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:51.625362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:51.768861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb894d-5b1b-46f1-8daa-17a2541cbb12 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:51.813995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014abe8-4091-4633-b038-237fb652d0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:51.814688 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014abe8-4091-4633-b038-237fb652d0ad None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:51.814976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014abe8-4091-4633-b038-237fb652d0ad None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:51.815257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014abe8-4091-4633-b038-237fb652d0ad None admin] PATH_INFO: `/v3/users/10b4e12de60a4288b72e15ab6c89f6ea` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:51.816898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014abe8-4091-4633-b038-237fb652d0ad None admin] RBAC: Authorizing `identity:get_user(user_id=10b4e12de60a4288b72e15ab6c89f6ea)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:51.818537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014abe8-4091-4633-b038-237fb652d0ad None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:51.821509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 76/146] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:51 2022] GET /identity/v3/users/10b4e12de60a4288b72e15ab6c89f6ea => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:51.826913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb976592-4172-41c4-8369-76654651a96f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:51.968291 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb976592-4172-41c4-8369-76654651a96f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:51.994896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:52.017379 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94df301-461a-45b8-a13f-b43f51a0c927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:52.018038 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df301-461a-45b8-a13f-b43f51a0c927 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:52.018343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df301-461a-45b8-a13f-b43f51a0c927 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:52.018618 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df301-461a-45b8-a13f-b43f51a0c927 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:52.022877 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94df301-461a-45b8-a13f-b43f51a0c927 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:52.024363 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94df301-461a-45b8-a13f-b43f51a0c927 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:52.026499 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 71/147] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:51 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:52.032419 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014abe8-4091-4633-b038-237fb652d0ad None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:52.117258 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:52.152116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014abe8-4091-4633-b038-237fb652d0ad None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:52.194512 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf1f2-79c0-40ce-b774-8c725737a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:52.195675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf1f2-79c0-40ce-b774-8c725737a922 None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:52.196051 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf1f2-79c0-40ce-b774-8c725737a922 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:52.196335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf1f2-79c0-40ce-b774-8c725737a922 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/users/10b4e12de60a4288b72e15ab6c89f6ea/roles/f6ae8eb934c74269ae9e47640f831a4e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:52.198454 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaf1f2-79c0-40ce-b774-8c725737a922 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc405d3f9e54cbf86ac4ca906b7637e, user_id=10b4e12de60a4288b72e15ab6c89f6ea, role_id=f6ae8eb934c74269ae9e47640f831a4e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:52.199996 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaf1f2-79c0-40ce-b774-8c725737a922 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:52.222423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 77/148] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:20:52 2022] PUT /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/users/10b4e12de60a4288b72e15ab6c89f6ea/roles/f6ae8eb934c74269ae9e47640f831a4e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:52.407074 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7e456-de1a-4b13-a19c-87bfee58a507 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:52.407497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7e456-de1a-4b13-a19c-87bfee58a507 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:52.407630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7e456-de1a-4b13-a19c-87bfee58a507 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:52.409303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 72/149] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:52.414393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd562dc-950a-4224-9be1-e5b52729c25c None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:52.414682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd562dc-950a-4224-9be1-e5b52729c25c None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:52.414960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd562dc-950a-4224-9be1-e5b52729c25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:52.439287 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-edd562dc-950a-4224-9be1-e5b52729c25c None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:52.440537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-edd562dc-950a-4224-9be1-e5b52729c25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:52.523302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 78/150] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:52.559764 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7e456-de1a-4b13-a19c-87bfee58a507 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:52.684623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7e456-de1a-4b13-a19c-87bfee58a507 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:52.728207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9072941-fd03-4bd7-a4a2-9c2423ecd4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:52.728937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9072941-fd03-4bd7-a4a2-9c2423ecd4eb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:52.729221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9072941-fd03-4bd7-a4a2-9c2423ecd4eb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:52.729495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9072941-fd03-4bd7-a4a2-9c2423ecd4eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:52.742886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9072941-fd03-4bd7-a4a2-9c2423ecd4eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-d9072941-fd03-4bd7-a4a2-9c2423ecd4eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:52.748564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:52.749568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:52.750555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 73/151] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:20:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:20:52.755952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd562dc-950a-4224-9be1-e5b52729c25c None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:52.827986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:52.873343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd562dc-950a-4224-9be1-e5b52729c25c None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:52.909347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:52.909988 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:52.910307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:52.910661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:52.911311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:52.911618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:52.913038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:52.923013 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 79/152] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:52.931406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9072941-fd03-4bd7-a4a2-9c2423ecd4eb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:52.987476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:53.045526 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9072941-fd03-4bd7-a4a2-9c2423ecd4eb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:53.086195 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab38ee-1a84-4ad5-9218-55d8bdbfe672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:53.086804 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab38ee-1a84-4ad5-9218-55d8bdbfe672 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:53.087103 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab38ee-1a84-4ad5-9218-55d8bdbfe672 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:53.087390 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab38ee-1a84-4ad5-9218-55d8bdbfe672 None admin] PATH_INFO: `/v3/users/530dca3bb7524c5cbcfad367381348a5` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:53.088647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab38ee-1a84-4ad5-9218-55d8bdbfe672 None admin] RBAC: Authorizing `identity:get_user(user_id=530dca3bb7524c5cbcfad367381348a5)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:53.090142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab38ee-1a84-4ad5-9218-55d8bdbfe672 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:53.092272 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 74/153] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:52 2022] GET /identity/v3/users/530dca3bb7524c5cbcfad367381348a5 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:53.100205 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:53.214049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64f777-1ff9-441c-bd2c-b048acd62a5f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:53.242808 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:53.258770 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01352fc-6f61-4706-957d-5d6e6885efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:53.259421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01352fc-6f61-4706-957d-5d6e6885efaf None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:53.259701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01352fc-6f61-4706-957d-5d6e6885efaf None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:53.259986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01352fc-6f61-4706-957d-5d6e6885efaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:53.337300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01352fc-6f61-4706-957d-5d6e6885efaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:53.338704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01352fc-6f61-4706-957d-5d6e6885efaf None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:53.446545 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 80/154] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 09:20:53.455287 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab38ee-1a84-4ad5-9218-55d8bdbfe672 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:53.567847 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab38ee-1a84-4ad5-9218-55d8bdbfe672 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:53.590160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:53.627030 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857db8ba-4d9a-4e8d-9874-3f2df7783bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:53.627734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857db8ba-4d9a-4e8d-9874-3f2df7783bdb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:53.628023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857db8ba-4d9a-4e8d-9874-3f2df7783bdb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:53.628307 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857db8ba-4d9a-4e8d-9874-3f2df7783bdb None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:53.629699 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857db8ba-4d9a-4e8d-9874-3f2df7783bdb None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:53.631262 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857db8ba-4d9a-4e8d-9874-3f2df7783bdb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:53.633414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 75/155] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:53 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:53.636196 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb99f-e0e3-426d-8110-acfd125ce1e4 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:53.636587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb99f-e0e3-426d-8110-acfd125ce1e4 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:53.636778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb99f-e0e3-426d-8110-acfd125ce1e4 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:53.638761 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 81/156] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:53.642689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857db8ba-4d9a-4e8d-9874-3f2df7783bdb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:53.649032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa55a9-0047-4020-85d1-6dd7db28abd8 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:53.649439 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa55a9-0047-4020-85d1-6dd7db28abd8 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:53.649601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa55a9-0047-4020-85d1-6dd7db28abd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:53.681758 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-24fa55a9-0047-4020-85d1-6dd7db28abd8 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:53.683220 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-24fa55a9-0047-4020-85d1-6dd7db28abd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:53.722262 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:53.780888 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857db8ba-4d9a-4e8d-9874-3f2df7783bdb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:53.824164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:53.824807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:53.825086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:53.825337 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:53.826213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e, user.id=530dca3bb7524c5cbcfad367381348a5) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:53.826537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:53.828014 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:53.832042 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 82/157] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:20:53.837244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 76/158] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:53 2022] GET /identity/v3/role_assignments?user.id=530dca3bb7524c5cbcfad367381348a5&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=af958d22229a4a0297bbe2e643f55181 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:53.867631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa55a9-0047-4020-85d1-6dd7db28abd8 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:54.000878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa55a9-0047-4020-85d1-6dd7db28abd8 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:54.042862 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf048-28ad-4b87-b053-8ccae1e01883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:54.043527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf048-28ad-4b87-b053-8ccae1e01883 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:54.043790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf048-28ad-4b87-b053-8ccae1e01883 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:54.044089 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf048-28ad-4b87-b053-8ccae1e01883 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:54.055775 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acf048-28ad-4b87-b053-8ccae1e01883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a7acf048-28ad-4b87-b053-8ccae1e01883 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:54.065179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:54.069212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:54.072373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 83/159] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:20:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:54.079505 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:54.138503 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:54.196799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb4e1c-db91-4812-ae25-21594156750e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:54.254964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:54.255634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:54.257219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:54.257219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:54.257219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:54.257474 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:54.258731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:54.268527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 77/160] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:20:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:54.274140 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf048-28ad-4b87-b053-8ccae1e01883 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:54.302507 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:54.395990 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf048-28ad-4b87-b053-8ccae1e01883 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:54.434818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eab22-da1a-44be-90d9-0ccb7824d8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:54.435649 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eab22-da1a-44be-90d9-0ccb7824d8bb None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:54.435935 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eab22-da1a-44be-90d9-0ccb7824d8bb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:54.436213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eab22-da1a-44be-90d9-0ccb7824d8bb None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:54.437410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eab22-da1a-44be-90d9-0ccb7824d8bb None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:54.438926 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eab22-da1a-44be-90d9-0ccb7824d8bb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:54.442013 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 84/161] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:54 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:54.447251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:54.564497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466d77e-fd91-49df-8f78-69aa5b70a0de None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:54.564873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466d77e-fd91-49df-8f78-69aa5b70a0de None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:54.565044 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466d77e-fd91-49df-8f78-69aa5b70a0de None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:54.566937 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 85/162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:20:54.572224 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f47217-8a55-492c-b039-9d60d9faddeb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:54.576917 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b0494-7eec-4333-b0b2-64fbc140986c None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:54.577202 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b0494-7eec-4333-b0b2-64fbc140986c None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:54.577479 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b0494-7eec-4333-b0b2-64fbc140986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:54.599514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:54.614871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-445b0494-7eec-4333-b0b2-64fbc140986c None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:54.616097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-445b0494-7eec-4333-b0b2-64fbc140986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:54.710964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596957b3-6b44-41b2-9fbc-b5cc638f3340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:54.711542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596957b3-6b44-41b2-9fbc-b5cc638f3340 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:54.711832 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596957b3-6b44-41b2-9fbc-b5cc638f3340 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:54.712119 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596957b3-6b44-41b2-9fbc-b5cc638f3340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:54.746330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 86/163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:20:54.774704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b0494-7eec-4333-b0b2-64fbc140986c None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:54.806502 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:54.903170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596957b3-6b44-41b2-9fbc-b5cc638f3340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:54.904962 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596957b3-6b44-41b2-9fbc-b5cc638f3340 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:54.996595 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b0494-7eec-4333-b0b2-64fbc140986c None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:55.070042 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2331fe-60e8-4158-a70d-320513ba2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:55.071013 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2331fe-60e8-4158-a70d-320513ba2b0d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:55.072685 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2331fe-60e8-4158-a70d-320513ba2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:55.073146 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2331fe-60e8-4158-a70d-320513ba2b0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:55.095382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2331fe-60e8-4158-a70d-320513ba2b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-2f2331fe-60e8-4158-a70d-320513ba2b0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:55.100695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:55.101923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:55.103111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:55.111165 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 87/164] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:20:55.115835 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2331fe-60e8-4158-a70d-320513ba2b0d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:55.186307 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 78/165] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:55.192718 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596957b3-6b44-41b2-9fbc-b5cc638f3340 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:55.205412 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:55.243193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:55.306826 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2331fe-60e8-4158-a70d-320513ba2b0d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:55.384871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5b2e6-cce6-4fea-b2a9-eda1ef666b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:55.385520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b2e6-cce6-4fea-b2a9-eda1ef666b63 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:55.385805 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b2e6-cce6-4fea-b2a9-eda1ef666b63 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:55.386081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b2e6-cce6-4fea-b2a9-eda1ef666b63 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:55.386856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5b2e6-cce6-4fea-b2a9-eda1ef666b63 None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:55.387167 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5b2e6-cce6-4fea-b2a9-eda1ef666b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:55.388646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5b2e6-cce6-4fea-b2a9-eda1ef666b63 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:55.391916 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596957b3-6b44-41b2-9fbc-b5cc638f3340 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:55.410578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 88/166] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:55.411916 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057fd0-3479-49e6-9755-39f005fccecd None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:55.412208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057fd0-3479-49e6-9755-39f005fccecd None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:55.412488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057fd0-3479-49e6-9755-39f005fccecd None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:55.413779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 89/167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:55.419404 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22e1cd-2267-4595-bd67-e646ec48e440 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:55.419687 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22e1cd-2267-4595-bd67-e646ec48e440 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:55.420175 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22e1cd-2267-4595-bd67-e646ec48e440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:55.439355 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-cc22e1cd-2267-4595-bd67-e646ec48e440 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:55.440906 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-cc22e1cd-2267-4595-bd67-e646ec48e440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:55.467447 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61aa68-52a9-41e1-bf14-3dab44622aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:55.468412 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61aa68-52a9-41e1-bf14-3dab44622aab None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:55.469071 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61aa68-52a9-41e1-bf14-3dab44622aab None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:55.470514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61aa68-52a9-41e1-bf14-3dab44622aab None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:55.475243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61aa68-52a9-41e1-bf14-3dab44622aab None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:55.477060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61aa68-52a9-41e1-bf14-3dab44622aab None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:55.479449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 79/168] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:55 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:55.481751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61aa68-52a9-41e1-bf14-3dab44622aab None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:55.567578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 90/169] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:20:55.568716 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22e1cd-2267-4595-bd67-e646ec48e440 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:55.639671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:55.684560 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61aa68-52a9-41e1-bf14-3dab44622aab None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:55.698513 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:55.752350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd8cbd-a538-4def-bb13-dc364a75336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:55.753014 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd8cbd-a538-4def-bb13-dc364a75336d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:55.753298 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd8cbd-a538-4def-bb13-dc364a75336d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:55.753577 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd8cbd-a538-4def-bb13-dc364a75336d None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:55.755620 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd8cbd-a538-4def-bb13-dc364a75336d None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:55.757131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd8cbd-a538-4def-bb13-dc364a75336d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:55.759025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 80/170] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:55 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:55.759634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd8cbd-a538-4def-bb13-dc364a75336d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:55.771383 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22e1cd-2267-4595-bd67-e646ec48e440 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:55.837396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:55.837960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:55.838271 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:55.838550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:55.839377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] RBAC: Adding query filter params (role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e, user.id=2ef76fedfde848929345c5e73369d36f) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:55.839692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:55.841168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:55.852079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 91/171] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:55 2022] GET /identity/v3/role_assignments?user.id=2ef76fedfde848929345c5e73369d36f&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:55.852757 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:55.956721 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd8cbd-a538-4def-bb13-dc364a75336d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:56.032783 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80789941-2f39-4ce1-80d5-ec77e3173dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:56.033704 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789941-2f39-4ce1-80d5-ec77e3173dce None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.033992 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789941-2f39-4ce1-80d5-ec77e3173dce None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.034302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789941-2f39-4ce1-80d5-ec77e3173dce None admin] PATH_INFO: `/v3/roles/member` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.052310 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80789941-2f39-4ce1-80d5-ec77e3173dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-80789941-2f39-4ce1-80d5-ec77e3173dce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:56.057585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:56.058771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:20:56.059692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:56.063287 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 81/172] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:56.068621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef54d5c-4bb6-4305-8027-5dfc7f5b58f1 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.068951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef54d5c-4bb6-4305-8027-5dfc7f5b58f1 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.069238 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef54d5c-4bb6-4305-8027-5dfc7f5b58f1 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.070986 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 82/173] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:56.071701 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef54d5c-4bb6-4305-8027-5dfc7f5b58f1 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:56.078602 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:56.108427 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92678f19-be0d-48ee-a230-94e0f858b1e0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:56.141568 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:56.153209 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05df89-b5f8-452b-8015-f59b5cbc899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:56.153854 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05df89-b5f8-452b-8015-f59b5cbc899f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.154160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05df89-b5f8-452b-8015-f59b5cbc899f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.154519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05df89-b5f8-452b-8015-f59b5cbc899f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.236442 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef54d5c-4bb6-4305-8027-5dfc7f5b58f1 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:56.265270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05df89-b5f8-452b-8015-f59b5cbc899f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:56.267473 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05df89-b5f8-452b-8015-f59b5cbc899f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:56.318690 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48a2af-cd23-404b-a821-698b576a19ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:56.319349 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a2af-cd23-404b-a821-698b576a19ae None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.319634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a2af-cd23-404b-a821-698b576a19ae None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.319910 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a2af-cd23-404b-a821-698b576a19ae None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.320675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48a2af-cd23-404b-a821-698b576a19ae None admin] RBAC: Adding query filter params (name=member) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:56.320989 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48a2af-cd23-404b-a821-698b576a19ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:56.322454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48a2af-cd23-404b-a821-698b576a19ae None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:56.345272 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 83/174] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:56.346675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2a3b-03cb-420d-99e0-7f518b470a0f None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.346967 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2a3b-03cb-420d-99e0-7f518b470a0f None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.347245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2a3b-03cb-420d-99e0-7f518b470a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.383329 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-392b2a3b-03cb-420d-99e0-7f518b470a0f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:56.384588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-392b2a3b-03cb-420d-99e0-7f518b470a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:56.450120 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 92/175] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:56.451475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05df89-b5f8-452b-8015-f59b5cbc899f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:56.475205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:56.485136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 84/176] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:56.486532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2a3b-03cb-420d-99e0-7f518b470a0f None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:56.492859 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:56.581449 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05df89-b5f8-452b-8015-f59b5cbc899f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:56.610703 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2a3b-03cb-420d-99e0-7f518b470a0f None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:56.621114 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa16c0c-fd38-4c0e-8001-ea1f1d4b5f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:56.621782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa16c0c-fd38-4c0e-8001-ea1f1d4b5f74 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.622221 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa16c0c-fd38-4c0e-8001-ea1f1d4b5f74 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.622521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa16c0c-fd38-4c0e-8001-ea1f1d4b5f74 None admin] PATH_INFO: `/v3/users/2ef76fedfde848929345c5e73369d36f` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.623817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa16c0c-fd38-4c0e-8001-ea1f1d4b5f74 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef76fedfde848929345c5e73369d36f)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:56.625408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa16c0c-fd38-4c0e-8001-ea1f1d4b5f74 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:56.627520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 93/177] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:56 2022] GET /identity/v3/users/2ef76fedfde848929345c5e73369d36f => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:56.628446 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187c7da-4bf1-4324-ac34-37d4a56e7e63 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.628699 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187c7da-4bf1-4324-ac34-37d4a56e7e63 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.629007 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187c7da-4bf1-4324-ac34-37d4a56e7e63 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.634226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 94/178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:56.634226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187c7da-4bf1-4324-ac34-37d4a56e7e63 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:56.654752 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d0327-6b80-40fe-8e3d-e693ea0fb159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:56.655414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0327-6b80-40fe-8e3d-e693ea0fb159 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.655697 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0327-6b80-40fe-8e3d-e693ea0fb159 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.655973 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0327-6b80-40fe-8e3d-e693ea0fb159 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.657169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d0327-6b80-40fe-8e3d-e693ea0fb159 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:56.658666 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d0327-6b80-40fe-8e3d-e693ea0fb159 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:56.660578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 85/179] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:56 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:56.661628 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2949169-3952-4224-8c3f-36d160eb9fa6 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.661913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2949169-3952-4224-8c3f-36d160eb9fa6 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.662209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2949169-3952-4224-8c3f-36d160eb9fa6 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.663619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 86/180] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:56.664151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2949169-3952-4224-8c3f-36d160eb9fa6 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:56.737095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:56.752792 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:56.762302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187c7da-4bf1-4324-ac34-37d4a56e7e63 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:56.779557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2949169-3952-4224-8c3f-36d160eb9fa6 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:56.800133 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92c4ad-f6a5-49e3-98ce-6ed2753ed3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:56.800792 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92c4ad-f6a5-49e3-98ce-6ed2753ed3aa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.801070 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92c4ad-f6a5-49e3-98ce-6ed2753ed3aa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.801343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92c4ad-f6a5-49e3-98ce-6ed2753ed3aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.813174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92c4ad-f6a5-49e3-98ce-6ed2753ed3aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-2e92c4ad-f6a5-49e3-98ce-6ed2753ed3aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:56.818525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:56.819597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:20:56.820645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 95/181] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:20:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:56.821451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8d2f5-7f88-4241-82d4-23bcf61a80fa None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.822257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8d2f5-7f88-4241-82d4-23bcf61a80fa None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.822257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8d2f5-7f88-4241-82d4-23bcf61a80fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.827652 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05defe4-081f-4304-8223-c5364be58970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:56.828304 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05defe4-081f-4304-8223-c5364be58970 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:56.828808 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05defe4-081f-4304-8223-c5364be58970 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:56.829091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05defe4-081f-4304-8223-c5364be58970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:56.839830 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-dfb8d2f5-7f88-4241-82d4-23bcf61a80fa None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:56.841035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-dfb8d2f5-7f88-4241-82d4-23bcf61a80fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:56.901699 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05defe4-081f-4304-8223-c5364be58970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:56.903117 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05defe4-081f-4304-8223-c5364be58970 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:56.911023 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 96/182] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:56.911912 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8d2f5-7f88-4241-82d4-23bcf61a80fa None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:56.968753 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.000339 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.013728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 87/183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:57.013728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827eff3-0283-4860-b7b5-3e292d161560 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.013728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827eff3-0283-4860-b7b5-3e292d161560 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.013728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827eff3-0283-4860-b7b5-3e292d161560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.021000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8d2f5-7f88-4241-82d4-23bcf61a80fa None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.029879 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-9827eff3-0283-4860-b7b5-3e292d161560 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:57.031101 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-9827eff3-0283-4860-b7b5-3e292d161560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:57.056034 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.056688 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.056966 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.057306 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.058130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] RBAC: Adding query filter params (role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e, user.id=2ef76fedfde848929345c5e73369d36f) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:57.058450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:57.059912 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:57.065907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 97/184] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:56 2022] GET /identity/v3/role_assignments?user.id=2ef76fedfde848929345c5e73369d36f&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:57.067047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.108061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 88/185] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:57.109347 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723624c-2936-452e-8360-cc88b48e8415 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.109644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723624c-2936-452e-8360-cc88b48e8415 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.109933 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723624c-2936-452e-8360-cc88b48e8415 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.111395 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 89/186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:20:57.111965 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723624c-2936-452e-8360-cc88b48e8415 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.184493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed62c35-6ac3-4520-aadc-30ef3c2811ac None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.223258 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.223904 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.224189 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.224463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.225219 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:57.225423 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.225543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:57.227109 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:57.234989 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723624c-2936-452e-8360-cc88b48e8415 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.241193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 98/187] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:57.241193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.255475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.281251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b294f-6b13-4e37-972f-ead03a0a657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.282011 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b294f-6b13-4e37-972f-ead03a0a657a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.282350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b294f-6b13-4e37-972f-ead03a0a657a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.282572 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b294f-6b13-4e37-972f-ead03a0a657a None admin] PATH_INFO: `/v3/domains/default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.283787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b294f-6b13-4e37-972f-ead03a0a657a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:57.286008 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b294f-6b13-4e37-972f-ead03a0a657a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:57.287170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 90/188] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:57.288146 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597fad-1dac-4ae2-99cf-a8685d796362 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.288433 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597fad-1dac-4ae2-99cf-a8685d796362 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.288716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597fad-1dac-4ae2-99cf-a8685d796362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.308464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-ae597fad-1dac-4ae2-99cf-a8685d796362 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:57.309777 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-ae597fad-1dac-4ae2-99cf-a8685d796362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:57.360339 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e303f0-814c-4b6a-8c47-7b242a33417d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.387166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 91/189] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:20:57.388125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597fad-1dac-4ae2-99cf-a8685d796362 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.395246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53457e-aece-4c60-8a93-54f80dade105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.395987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53457e-aece-4c60-8a93-54f80dade105 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.397127 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53457e-aece-4c60-8a93-54f80dade105 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.397127 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53457e-aece-4c60-8a93-54f80dade105 None admin] PATH_INFO: `/v3/projects/86673a26b27f4085bb0b9d324d982eac` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.397997 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53457e-aece-4c60-8a93-54f80dade105 None admin] RBAC: Authorizing `identity:get_project(project_id=86673a26b27f4085bb0b9d324d982eac)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:57.399566 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53457e-aece-4c60-8a93-54f80dade105 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:57.401490 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 99/190] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/projects/86673a26b27f4085bb0b9d324d982eac => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:57.404879 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53457e-aece-4c60-8a93-54f80dade105 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.483041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.484265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.493282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597fad-1dac-4ae2-99cf-a8685d796362 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.505620 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53457e-aece-4c60-8a93-54f80dade105 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.527580 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d333887-9e52-4f28-9825-b0657a1da161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.528546 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d333887-9e52-4f28-9825-b0657a1da161 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.528831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d333887-9e52-4f28-9825-b0657a1da161 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.529354 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d333887-9e52-4f28-9825-b0657a1da161 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.538792 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3dce0-5794-4da0-8c38-657bc62c07ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.539422 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3dce0-5794-4da0-8c38-657bc62c07ee None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.539704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3dce0-5794-4da0-8c38-657bc62c07ee None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.539806 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d333887-9e52-4f28-9825-b0657a1da161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:57.539979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3dce0-5794-4da0-8c38-657bc62c07ee None admin] PATH_INFO: `/v3/users/b4264a3fd9194cbb8a67c14e2e369673` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-7d333887-9e52-4f28-9825-b0657a1da161 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:57.544736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:57.546004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:57.547130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 92/191] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:57.551027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d333887-9e52-4f28-9825-b0657a1da161 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.555884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3dce0-5794-4da0-8c38-657bc62c07ee None admin] RBAC: Authorizing `identity:get_user(user_id=b4264a3fd9194cbb8a67c14e2e369673)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:57.557361 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3dce0-5794-4da0-8c38-657bc62c07ee None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:57.559083 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 100/192] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/users/b4264a3fd9194cbb8a67c14e2e369673 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:57.559853 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3dce0-5794-4da0-8c38-657bc62c07ee None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.661337 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d333887-9e52-4f28-9825-b0657a1da161 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.665297 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3dce0-5794-4da0-8c38-657bc62c07ee None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.695244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5867dd3-9fc3-4b62-9836-b65f0a73a835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.695870 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5867dd3-9fc3-4b62-9836-b65f0a73a835 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.696150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5867dd3-9fc3-4b62-9836-b65f0a73a835 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.696446 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5867dd3-9fc3-4b62-9836-b65f0a73a835 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.697136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5867dd3-9fc3-4b62-9836-b65f0a73a835 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:57.698631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5867dd3-9fc3-4b62-9836-b65f0a73a835 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:57.703118 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.703760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.704049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.704330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.705164 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] RBAC: Adding query filter params (role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=86673a26b27f4085bb0b9d324d982eac, user.id=2ef76fedfde848929345c5e73369d36f) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:57.705479 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:57.706979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:57.711444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.720752 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 93/193] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:20:57 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:57.722015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5867dd3-9fc3-4b62-9836-b65f0a73a835 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.730636 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 101/194] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/role_assignments?user.id=2ef76fedfde848929345c5e73369d36f&scope.project.id=86673a26b27f4085bb0b9d324d982eac&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:57.735377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.745889 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.879871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4f8e5-bc05-4bed-87b7-beba63515773 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.897534 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5867dd3-9fc3-4b62-9836-b65f0a73a835 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:57.932175 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.933429 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.933808 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.934333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.935452 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:57.935767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:57.937352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:57.947213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 102/195] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:57.951163 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:57.959877 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b033f-a743-4576-9986-bf610f0eb45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:57.960556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b033f-a743-4576-9986-bf610f0eb45a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.960836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b033f-a743-4576-9986-bf610f0eb45a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.961109 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b033f-a743-4576-9986-bf610f0eb45a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.976768 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b033f-a743-4576-9986-bf610f0eb45a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-1f6b033f-a743-4576-9986-bf610f0eb45a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:20:57.983635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:20:57.984732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 94/196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:20:57.985706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b033f-a743-4576-9986-bf610f0eb45a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.036651 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:58.076927 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:58.093406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbc82-a62f-4849-bb20-5a2d408e5b8a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.123952 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b033f-a743-4576-9986-bf610f0eb45a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.134731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a83cf-ea30-4c13-abe9-0547abbc0b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.135424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a83cf-ea30-4c13-abe9-0547abbc0b53 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.135717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a83cf-ea30-4c13-abe9-0547abbc0b53 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.135997 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a83cf-ea30-4c13-abe9-0547abbc0b53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:58.163115 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f3bd5-f40c-4537-bfcf-1ad895946245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.164217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3bd5-f40c-4537-bfcf-1ad895946245 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.164793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3bd5-f40c-4537-bfcf-1ad895946245 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.165325 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3bd5-f40c-4537-bfcf-1ad895946245 None admin] PATH_INFO: `/v3/users/b4264a3fd9194cbb8a67c14e2e369673` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:58.166931 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f3bd5-f40c-4537-bfcf-1ad895946245 None admin] RBAC: Authorizing `identity:get_user(user_id=b4264a3fd9194cbb8a67c14e2e369673)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:58.168712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f3bd5-f40c-4537-bfcf-1ad895946245 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:58.170905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 95/197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/users/b4264a3fd9194cbb8a67c14e2e369673 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:58.172048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f3bd5-f40c-4537-bfcf-1ad895946245 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.228720 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a83cf-ea30-4c13-abe9-0547abbc0b53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:58.230290 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a83cf-ea30-4c13-abe9-0547abbc0b53 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:58.296014 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f3bd5-f40c-4537-bfcf-1ad895946245 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.304947 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:58.345213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.346394 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.346961 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.347473 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:58.348492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] RBAC: Adding query filter params (name=reader) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:58.349035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:58.350699 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:58.353838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:58.361390 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 103/198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:58.362476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a83cf-ea30-4c13-abe9-0547abbc0b53 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.364265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 96/199] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:20:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:58.367152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.492312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a83cf-ea30-4c13-abe9-0547abbc0b53 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.506004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e0b4d-77c3-49bd-a56b-e457879f6713 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.537645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d19595-dc0b-4864-9c3a-8d954cebaff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.539369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19595-dc0b-4864-9c3a-8d954cebaff8 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.539735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19595-dc0b-4864-9c3a-8d954cebaff8 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.540014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19595-dc0b-4864-9c3a-8d954cebaff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:58.550018 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed32d40-6fcf-40fc-bd22-45801314bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.550699 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed32d40-6fcf-40fc-bd22-45801314bac4 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.550988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed32d40-6fcf-40fc-bd22-45801314bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.551266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed32d40-6fcf-40fc-bd22-45801314bac4 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:58.552642 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed32d40-6fcf-40fc-bd22-45801314bac4 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:58.554292 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed32d40-6fcf-40fc-bd22-45801314bac4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:58.562252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 97/200] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:58 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:58.563491 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed32d40-6fcf-40fc-bd22-45801314bac4 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.581055 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:58.641214 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d19595-dc0b-4864-9c3a-8d954cebaff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:58.643347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d19595-dc0b-4864-9c3a-8d954cebaff8 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:58.659828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:58.701598 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed32d40-6fcf-40fc-bd22-45801314bac4 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.740511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc84d7-2cf1-4846-a502-d63d29092e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.741530 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc84d7-2cf1-4846-a502-d63d29092e2b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.742016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc84d7-2cf1-4846-a502-d63d29092e2b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.742539 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc84d7-2cf1-4846-a502-d63d29092e2b None admin] PATH_INFO: `/v3/users/10b4e12de60a4288b72e15ab6c89f6ea` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:58.744079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc84d7-2cf1-4846-a502-d63d29092e2b None admin] RBAC: Authorizing `identity:get_user(user_id=10b4e12de60a4288b72e15ab6c89f6ea)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:58.745776 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc84d7-2cf1-4846-a502-d63d29092e2b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:58.747783 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 98/201] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:20:58 2022] GET /identity/v3/users/10b4e12de60a4288b72e15ab6c89f6ea => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:58.749139 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc84d7-2cf1-4846-a502-d63d29092e2b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.774901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 104/202] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:58.781138 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d19595-dc0b-4864-9c3a-8d954cebaff8 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.856717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:58.881018 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc84d7-2cf1-4846-a502-d63d29092e2b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.903869 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d19595-dc0b-4864-9c3a-8d954cebaff8 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.926163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.926805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.927079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.927355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:58.928167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915, user.id=b4264a3fd9194cbb8a67c14e2e369673) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:58.928476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:58.929978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:58.932887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:58.938650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 99/203] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:58 2022] GET /identity/v3/role_assignments?user.id=b4264a3fd9194cbb8a67c14e2e369673&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=af958d22229a4a0297bbe2e643f55181 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:58.940176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.966903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bf32-1dc9-44c5-a361-d9476c45ca5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.967605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bf32-1dc9-44c5-a361-d9476c45ca5b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.967894 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bf32-1dc9-44c5-a361-d9476c45ca5b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.968174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bf32-1dc9-44c5-a361-d9476c45ca5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:59.114898 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75dce8-c432-4845-a6c6-420d2c1ac5ac None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:59.132899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4bf32-1dc9-44c5-a361-d9476c45ca5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:59.134940 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4bf32-1dc9-44c5-a361-d9476c45ca5b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:59.179362 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28c2a-81e8-4415-99d3-f55e782b1874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:59.181129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28c2a-81e8-4415-99d3-f55e782b1874 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:59.181783 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28c2a-81e8-4415-99d3-f55e782b1874 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:59.182878 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28c2a-81e8-4415-99d3-f55e782b1874 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:59.188016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28c2a-81e8-4415-99d3-f55e782b1874 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:59.190392 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28c2a-81e8-4415-99d3-f55e782b1874 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:59.193641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 100/204] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:58 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:59.200674 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28c2a-81e8-4415-99d3-f55e782b1874 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:59.229841 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:59.325178 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 105/205] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:20:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:20:59.333507 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bf32-1dc9-44c5-a361-d9476c45ca5b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:59.342464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:59.440346 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28c2a-81e8-4415-99d3-f55e782b1874 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:59.469261 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bf32-1dc9-44c5-a361-d9476c45ca5b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:59.479152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:59.480059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:59.480465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:59.480833 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:59.481923 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] RBAC: Adding query filter params (role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915, user.id=b4264a3fd9194cbb8a67c14e2e369673) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:59.482389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:59.483972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:59.494153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 101/206] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:59 2022] GET /identity/v3/role_assignments?user.id=b4264a3fd9194cbb8a67c14e2e369673&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:59.540575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518b63-e7fd-48ea-8f36-8d522fde4af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:59.541663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518b63-e7fd-48ea-8f36-8d522fde4af5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:59.542207 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518b63-e7fd-48ea-8f36-8d522fde4af5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:59.542706 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518b63-e7fd-48ea-8f36-8d522fde4af5 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:59.544462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518b63-e7fd-48ea-8f36-8d522fde4af5 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:59.546459 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518b63-e7fd-48ea-8f36-8d522fde4af5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:59.548718 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 106/207] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:20:59 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:20:59.555515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:59.693368 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6707d9e-e9fd-4518-94a3-ef510b98ecda None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:59.733377 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:59.742230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9735c7-b8e1-4cdd-a67f-432d785adca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:59.742892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9735c7-b8e1-4cdd-a67f-432d785adca2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:59.743186 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9735c7-b8e1-4cdd-a67f-432d785adca2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:59.743471 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9735c7-b8e1-4cdd-a67f-432d785adca2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:59.744315 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9735c7-b8e1-4cdd-a67f-432d785adca2 None admin] RBAC: Adding query filter params (role.id=f6ae8eb934c74269ae9e47640f831a4e, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e, user.id=10b4e12de60a4288b72e15ab6c89f6ea) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:59.744637 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9735c7-b8e1-4cdd-a67f-432d785adca2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:59.746164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9735c7-b8e1-4cdd-a67f-432d785adca2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:59.753720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 102/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:20:59 2022] GET /identity/v3/role_assignments?user.id=10b4e12de60a4288b72e15ab6c89f6ea&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=f6ae8eb934c74269ae9e47640f831a4e => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:00.094135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36a09-b0ae-4da1-89ef-c9e0b536bc2d None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.094135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36a09-b0ae-4da1-89ef-c9e0b536bc2d None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.094135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36a09-b0ae-4da1-89ef-c9e0b536bc2d None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.094797 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 107/209] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:00.099842 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64323e8e-b16e-4467-bd43-17e45878fd63 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.100356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64323e8e-b16e-4467-bd43-17e45878fd63 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.100639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64323e8e-b16e-4467-bd43-17e45878fd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.123692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-64323e8e-b16e-4467-bd43-17e45878fd63 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:00.125308 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-64323e8e-b16e-4467-bd43-17e45878fd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:00.200577 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 103/210] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:00.225094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a36a09-b0ae-4da1-89ef-c9e0b536bc2d None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:00.258838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:00.291156 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029af94-137f-49f1-9e5a-e79f9c23751a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.291499 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029af94-137f-49f1-9e5a-e79f9c23751a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.291726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029af94-137f-49f1-9e5a-e79f9c23751a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.293145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 104/211] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:21:00.298838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9f08-8094-48d2-893a-e9388e36daa9 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.299126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9f08-8094-48d2-893a-e9388e36daa9 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.299402 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9f08-8094-48d2-893a-e9388e36daa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.320662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-652a9f08-8094-48d2-893a-e9388e36daa9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:00.321879 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-652a9f08-8094-48d2-893a-e9388e36daa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:00.351079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a36a09-b0ae-4da1-89ef-c9e0b536bc2d None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:00.393401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0475e36-2a60-42ce-b2de-9e237feeedb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:00.394145 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0475e36-2a60-42ce-b2de-9e237feeedb4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.394443 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0475e36-2a60-42ce-b2de-9e237feeedb4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.394738 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0475e36-2a60-42ce-b2de-9e237feeedb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.404199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 105/212] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:00.406841 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0475e36-2a60-42ce-b2de-9e237feeedb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a0475e36-2a60-42ce-b2de-9e237feeedb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:00.412273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:00.413539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:00.414837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:00.418969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 108/213] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:00.420141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a9f08-8094-48d2-893a-e9388e36daa9 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:00.431907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0475e36-2a60-42ce-b2de-9e237feeedb4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:00.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:00.534893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:00.541632 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a9f08-8094-48d2-893a-e9388e36daa9 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:00.544735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0475e36-2a60-42ce-b2de-9e237feeedb4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:00.582504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f26dc0-459f-4e71-8862-268157600fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:00.583234 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26dc0-459f-4e71-8862-268157600fcf None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.583521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26dc0-459f-4e71-8862-268157600fcf None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.583850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26dc0-459f-4e71-8862-268157600fcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.585142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f26dc0-459f-4e71-8862-268157600fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:00.586511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:00.586751 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f26dc0-459f-4e71-8862-268157600fcf None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:00.587738 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.588254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.593217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.593349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 109/214] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:00.593455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] RBAC: Adding query filter params (name=member) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:00.593455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:00.593455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:00.594160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f26dc0-459f-4e71-8862-268157600fcf None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:00.610269 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 106/215] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:00.613214 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:00.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f26dc0-459f-4e71-8862-268157600fcf None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:00.745881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437c33b-b6ac-44e4-a4d5-e5d6d2645b2d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:00.762529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbbc43-fbd8-4412-8280-c13c2a354524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:00.763177 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbbc43-fbd8-4412-8280-c13c2a354524 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.763723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:00.764015 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbbc43-fbd8-4412-8280-c13c2a354524 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.765176 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbbc43-fbd8-4412-8280-c13c2a354524 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.766064 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbbc43-fbd8-4412-8280-c13c2a354524 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:00.767812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbbc43-fbd8-4412-8280-c13c2a354524 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:00.783164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94bb30-4946-40fe-be03-473312fbaeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:00.783907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94bb30-4946-40fe-be03-473312fbaeff None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.784192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94bb30-4946-40fe-be03-473312fbaeff None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.784472 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94bb30-4946-40fe-be03-473312fbaeff None admin] PATH_INFO: `/v3/users/d078cb0e3ab24966a5cd60df22fd168a` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:00.802806 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94bb30-4946-40fe-be03-473312fbaeff None admin] RBAC: Authorizing `identity:get_user(user_id=d078cb0e3ab24966a5cd60df22fd168a)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:00.804468 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94bb30-4946-40fe-be03-473312fbaeff None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:00.805938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 110/216] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:00 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:21:00.810167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 107/217] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:00 2022] GET /identity/v3/users/d078cb0e3ab24966a5cd60df22fd168a => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:00.813089 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbbc43-fbd8-4412-8280-c13c2a354524 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:00.835984 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:00.931095 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbbc43-fbd8-4412-8280-c13c2a354524 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:00.972749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78917b2-6ec7-4fc6-b63d-70a4834a7470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:00.973396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78917b2-6ec7-4fc6-b63d-70a4834a7470 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:00.973682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78917b2-6ec7-4fc6-b63d-70a4834a7470 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:00.973964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78917b2-6ec7-4fc6-b63d-70a4834a7470 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.048425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78917b2-6ec7-4fc6-b63d-70a4834a7470 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:01.050000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78917b2-6ec7-4fc6-b63d-70a4834a7470 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:01.066756 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:01.160519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 111/218] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:01.166666 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94bb30-4946-40fe-be03-473312fbaeff None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:01.285605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94bb30-4946-40fe-be03-473312fbaeff None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:01.323580 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a789e0-657d-4c2e-8468-e2feebf214dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:01.324261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a789e0-657d-4c2e-8468-e2feebf214dd None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:01.324547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a789e0-657d-4c2e-8468-e2feebf214dd None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:01.324875 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a789e0-657d-4c2e-8468-e2feebf214dd None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.326224 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a789e0-657d-4c2e-8468-e2feebf214dd None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:01.327696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a789e0-657d-4c2e-8468-e2feebf214dd None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:01.329754 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 108/219] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:01 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:01.335840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78917b2-6ec7-4fc6-b63d-70a4834a7470 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:01.486743 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78917b2-6ec7-4fc6-b63d-70a4834a7470 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:01.513252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:01.518138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9c2a6-810a-404f-87da-0cb359f53db1 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:01.518701 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9c2a6-810a-404f-87da-0cb359f53db1 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:01.518701 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9c2a6-810a-404f-87da-0cb359f53db1 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.521199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 109/220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:01.526785 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef407c9-a60f-4c75-9fbe-57c063d869c5 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:01.527074 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef407c9-a60f-4c75-9fbe-57c063d869c5 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:01.527356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef407c9-a60f-4c75-9fbe-57c063d869c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.549285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-3ef407c9-a60f-4c75-9fbe-57c063d869c5 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:01.549475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:01.550049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:01.550361 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:01.550507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-3ef407c9-a60f-4c75-9fbe-57c063d869c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:01.550701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.551459 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] RBAC: Adding query filter params (role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915, user.id=d078cb0e3ab24966a5cd60df22fd168a) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:01.551770 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:01.554430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:01.567939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 112/221] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:01 2022] GET /identity/v3/role_assignments?user.id=d078cb0e3ab24966a5cd60df22fd168a&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:01.626929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:01.683057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 110/222] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:01.717592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:01.824046 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48510440-5358-4a3a-9871-510e614ee949 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:01.824705 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48510440-5358-4a3a-9871-510e614ee949 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:01.825209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48510440-5358-4a3a-9871-510e614ee949 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.825476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da9efd-fea0-4e65-9df4-61d888de7a81 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:01.827446 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 111/223] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:21:01.834398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58975-e3fc-4f8c-ad54-107a69fa40c9 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:01.835017 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58975-e3fc-4f8c-ad54-107a69fa40c9 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:01.835522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58975-e3fc-4f8c-ad54-107a69fa40c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.856421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-a5d58975-e3fc-4f8c-ad54-107a69fa40c9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:01.858337 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-a5d58975-e3fc-4f8c-ad54-107a69fa40c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:01.875046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a5279-9e99-4138-93fc-fd1e120ed823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:01.876148 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5279-9e99-4138-93fc-fd1e120ed823 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:01.876655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5279-9e99-4138-93fc-fd1e120ed823 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:01.877149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5279-9e99-4138-93fc-fd1e120ed823 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.898926 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a5279-9e99-4138-93fc-fd1e120ed823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-c40a5279-9e99-4138-93fc-fd1e120ed823 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:01.907462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:01.908587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:01.909543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:01.914561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 113/224] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:01.921040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a5279-9e99-4138-93fc-fd1e120ed823 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:01.992410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:02.000970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 112/225] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:02.055238 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d58975-e3fc-4f8c-ad54-107a69fa40c9 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:02.084168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a5279-9e99-4138-93fc-fd1e120ed823 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:02.126815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:02.127882 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:02.128401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:02.128899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:02.129844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:02.130398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:02.132689 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:02.151719 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 114/226] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:02.155780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:02.240960 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d58975-e3fc-4f8c-ad54-107a69fa40c9 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:02.242894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:02.284692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388ea66-bd6e-4f81-a0a1-5ac28d4f5e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:02.286265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388ea66-bd6e-4f81-a0a1-5ac28d4f5e6b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:02.286555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388ea66-bd6e-4f81-a0a1-5ac28d4f5e6b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:02.286836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388ea66-bd6e-4f81-a0a1-5ac28d4f5e6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:02.310764 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388ea66-bd6e-4f81-a0a1-5ac28d4f5e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c388ea66-bd6e-4f81-a0a1-5ac28d4f5e6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:02.319664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:02.320782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:02.321745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 113/227] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:02.330746 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388ea66-bd6e-4f81-a0a1-5ac28d4f5e6b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:02.355206 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:02.359127 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8c6fc-3026-4d18-aa4e-1bfe08665309 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:02.447788 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376ab9-cb4b-4693-8c16-ab27a7ed4e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:02.448493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376ab9-cb4b-4693-8c16-ab27a7ed4e3c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:02.448774 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376ab9-cb4b-4693-8c16-ab27a7ed4e3c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:02.449057 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376ab9-cb4b-4693-8c16-ab27a7ed4e3c None admin] PATH_INFO: `/v3/users/b4264a3fd9194cbb8a67c14e2e369673` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:02.450903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61376ab9-cb4b-4693-8c16-ab27a7ed4e3c None admin] RBAC: Authorizing `identity:get_user(user_id=b4264a3fd9194cbb8a67c14e2e369673)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:02.452629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61376ab9-cb4b-4693-8c16-ab27a7ed4e3c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:02.458870 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 115/228] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:02 2022] GET /identity/v3/users/b4264a3fd9194cbb8a67c14e2e369673 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:02.459791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376ab9-cb4b-4693-8c16-ab27a7ed4e3c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:02.581985 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388ea66-bd6e-4f81-a0a1-5ac28d4f5e6b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:02.637644 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376ab9-cb4b-4693-8c16-ab27a7ed4e3c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:02.689196 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:02.689857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:02.690288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:02.690573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:02.691651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:02.691954 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:02.693497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:02.694317 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae638ab6-045c-4b3e-b233-cb43a3d05c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:02.695699 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae638ab6-045c-4b3e-b233-cb43a3d05c06 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:02.696470 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae638ab6-045c-4b3e-b233-cb43a3d05c06 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:02.697179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae638ab6-045c-4b3e-b233-cb43a3d05c06 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:02.704260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae638ab6-045c-4b3e-b233-cb43a3d05c06 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:02.706729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae638ab6-045c-4b3e-b233-cb43a3d05c06 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:02.709834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 116/229] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:02 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:02.712830 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 114/230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:02.723441 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae638ab6-045c-4b3e-b233-cb43a3d05c06 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:02.725316 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:02.778490 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:02.781070 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:02.919718 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10886d58-2719-4a12-a305-6c9fab8823c7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:02.924810 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae638ab6-045c-4b3e-b233-cb43a3d05c06 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:02.983115 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e797a-ca55-4bf1-b531-ccc12dc0acc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:02.984331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e797a-ca55-4bf1-b531-ccc12dc0acc7 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:02.986390 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e797a-ca55-4bf1-b531-ccc12dc0acc7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:02.987626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e797a-ca55-4bf1-b531-ccc12dc0acc7 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/users/b4264a3fd9194cbb8a67c14e2e369673/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:02.991143 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e797a-ca55-4bf1-b531-ccc12dc0acc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=72fb605a1a2c4ec29bdc2d9daa734915, user_id=b4264a3fd9194cbb8a67c14e2e369673, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:02.993172 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e797a-ca55-4bf1-b531-ccc12dc0acc7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:03.005647 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f307d3e-a2c4-4654-b864-39093b9e952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:03.006659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307d3e-a2c4-4654-b864-39093b9e952d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:03.007189 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307d3e-a2c4-4654-b864-39093b9e952d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:03.007620 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307d3e-a2c4-4654-b864-39093b9e952d None admin] PATH_INFO: `/v3/users/b4264a3fd9194cbb8a67c14e2e369673` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:03.009391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f307d3e-a2c4-4654-b864-39093b9e952d None admin] RBAC: Authorizing `identity:get_user(user_id=b4264a3fd9194cbb8a67c14e2e369673)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:03.011188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f307d3e-a2c4-4654-b864-39093b9e952d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:03.017326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 117/231] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:02 2022] GET /identity/v3/users/b4264a3fd9194cbb8a67c14e2e369673 => generated 283 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:03.021079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 115/232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:02 2022] PUT /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/users/b4264a3fd9194cbb8a67c14e2e369673/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:03.028696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f307d3e-a2c4-4654-b864-39093b9e952d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:03.172707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f307d3e-a2c4-4654-b864-39093b9e952d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:03.191931 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:03.221918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68904b19-f182-49e4-8cbb-a0711e2fe713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:03.222634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904b19-f182-49e4-8cbb-a0711e2fe713 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:03.222923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904b19-f182-49e4-8cbb-a0711e2fe713 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:03.223203 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904b19-f182-49e4-8cbb-a0711e2fe713 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:03.225228 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68904b19-f182-49e4-8cbb-a0711e2fe713 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:03.226760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68904b19-f182-49e4-8cbb-a0711e2fe713 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:03.228889 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 118/233] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:03 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:03.234560 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e797a-ca55-4bf1-b531-ccc12dc0acc7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:03.378606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:03.400026 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e797a-ca55-4bf1-b531-ccc12dc0acc7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:03.452119 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a23f7-b72d-424d-843d-25c23a290567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:03.452746 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a23f7-b72d-424d-843d-25c23a290567 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:03.453027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a23f7-b72d-424d-843d-25c23a290567 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:03.453322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a23f7-b72d-424d-843d-25c23a290567 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/users/b4264a3fd9194cbb8a67c14e2e369673/roles/6b79a8b00a964a8fbcbaaa8371e71edb` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:03.459973 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a23f7-b72d-424d-843d-25c23a290567 None admin] RBAC: Authorizing `identity:create_grant(project_id=72fb605a1a2c4ec29bdc2d9daa734915, user_id=b4264a3fd9194cbb8a67c14e2e369673, role_id=6b79a8b00a964a8fbcbaaa8371e71edb)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:03.461464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a23f7-b72d-424d-843d-25c23a290567 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:03.482741 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 116/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:03 2022] PUT /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/users/b4264a3fd9194cbb8a67c14e2e369673/roles/6b79a8b00a964a8fbcbaaa8371e71edb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:03.498810 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc6d4-f050-4b3a-85b0-30a5cd8c21af None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:03.499856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc6d4-f050-4b3a-85b0-30a5cd8c21af None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:03.500362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc6d4-f050-4b3a-85b0-30a5cd8c21af None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:03.502138 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 119/235] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:03.508364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c8a79-94ad-473a-a656-c58e7d340276 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:03.508659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c8a79-94ad-473a-a656-c58e7d340276 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:03.508943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c8a79-94ad-473a-a656-c58e7d340276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:03.533739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-8a0c8a79-94ad-473a-a656-c58e7d340276 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:03.535039 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-8a0c8a79-94ad-473a-a656-c58e7d340276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:03.572138 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07519d-3a7b-4e9d-8908-09d3c5bd3737 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:03.572483 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07519d-3a7b-4e9d-8908-09d3c5bd3737 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:03.572696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07519d-3a7b-4e9d-8908-09d3c5bd3737 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:03.574484 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 120/236] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:03.580286 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833cf20-4a76-4857-84ca-cb71c5d4be26 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:03.580610 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833cf20-4a76-4857-84ca-cb71c5d4be26 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:03.580843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833cf20-4a76-4857-84ca-cb71c5d4be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:03.613769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-f833cf20-4a76-4857-84ca-cb71c5d4be26 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:03.615105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-f833cf20-4a76-4857-84ca-cb71c5d4be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:03.694997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 117/237] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:03.723382 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c8a79-94ad-473a-a656-c58e7d340276 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:03.745107 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 121/238] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:03.770524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833cf20-4a76-4857-84ca-cb71c5d4be26 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:03.822635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:03.855818 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c8a79-94ad-473a-a656-c58e7d340276 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:03.896278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233387a8-3867-4380-ab31-e18040384a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:03.897324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233387a8-3867-4380-ab31-e18040384a6c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:03.897613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233387a8-3867-4380-ab31-e18040384a6c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:03.897901 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233387a8-3867-4380-ab31-e18040384a6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:03.910948 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233387a8-3867-4380-ab31-e18040384a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-233387a8-3867-4380-ab31-e18040384a6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:03.916684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:03.917838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:03.918860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:03.919647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 118/239] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:03.927153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233387a8-3867-4380-ab31-e18040384a6c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:03.942899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833cf20-4a76-4857-84ca-cb71c5d4be26 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:03.944576 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:04.007942 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9aa5a-8396-4df2-8add-38f2f8fcf212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:04.008634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9aa5a-8396-4df2-8add-38f2f8fcf212 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.008926 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9aa5a-8396-4df2-8add-38f2f8fcf212 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.009209 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9aa5a-8396-4df2-8add-38f2f8fcf212 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.010920 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9aa5a-8396-4df2-8add-38f2f8fcf212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:04.012863 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9aa5a-8396-4df2-8add-38f2f8fcf212 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:04.019488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 122/240] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:04.020906 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9aa5a-8396-4df2-8add-38f2f8fcf212 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:04.050023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233387a8-3867-4380-ab31-e18040384a6c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:04.092247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000a1a6-c0e7-4c9f-9e99-d596a93a73f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:04.093187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000a1a6-c0e7-4c9f-9e99-d596a93a73f4 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.093473 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000a1a6-c0e7-4c9f-9e99-d596a93a73f4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.093750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000a1a6-c0e7-4c9f-9e99-d596a93a73f4 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.094537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000a1a6-c0e7-4c9f-9e99-d596a93a73f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:04.094845 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000a1a6-c0e7-4c9f-9e99-d596a93a73f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:04.096262 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000a1a6-c0e7-4c9f-9e99-d596a93a73f4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:04.099389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:04.108225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 119/241] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:04.109639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea5a80-ddf1-4f81-a960-c75792159100 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.109999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea5a80-ddf1-4f81-a960-c75792159100 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.110384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea5a80-ddf1-4f81-a960-c75792159100 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.111968 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 120/242] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:04.114386 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea5a80-ddf1-4f81-a960-c75792159100 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:04.160212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9aa5a-8396-4df2-8add-38f2f8fcf212 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:04.196342 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37443b-bce9-42f4-945a-582c7ec3dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:04.197053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37443b-bce9-42f4-945a-582c7ec3dfb2 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.197349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37443b-bce9-42f4-945a-582c7ec3dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.197655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37443b-bce9-42f4-945a-582c7ec3dfb2 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.198468 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37443b-bce9-42f4-945a-582c7ec3dfb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:04.200198 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37443b-bce9-42f4-945a-582c7ec3dfb2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:04.234267 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 123/243] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:04 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:04.236361 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039b74c-6c17-43ac-b573-9d7ef1b8d1f6 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.236934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039b74c-6c17-43ac-b573-9d7ef1b8d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039b74c-6c17-43ac-b573-9d7ef1b8d1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.245007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea5a80-ddf1-4f81-a960-c75792159100 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:04.261205 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-8039b74c-6c17-43ac-b573-9d7ef1b8d1f6 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:04.263205 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-8039b74c-6c17-43ac-b573-9d7ef1b8d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:04.293634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb02c6-7ca5-4a50-bd7e-cdce091b009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:04.294399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb02c6-7ca5-4a50-bd7e-cdce091b009e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.294707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:04.294894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb02c6-7ca5-4a50-bd7e-cdce091b009e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.295059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb02c6-7ca5-4a50-bd7e-cdce091b009e None admin] PATH_INFO: `/v3/users/3075c405d74c439cb8c65efeb6abd85c` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.334511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb02c6-7ca5-4a50-bd7e-cdce091b009e None admin] RBAC: Authorizing `identity:get_user(user_id=3075c405d74c439cb8c65efeb6abd85c)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:04.336027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb02c6-7ca5-4a50-bd7e-cdce091b009e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:04.338125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 121/244] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:04 2022] GET /identity/v3/users/3075c405d74c439cb8c65efeb6abd85c => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:04.343827 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb02c6-7ca5-4a50-bd7e-cdce091b009e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:04.399661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 124/245] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:04.428848 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039b74c-6c17-43ac-b573-9d7ef1b8d1f6 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:04.435537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:04.485900 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb02c6-7ca5-4a50-bd7e-cdce091b009e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:04.532278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c46e49-7c78-4edd-bd6d-9a5d40668835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:04.532878 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c46e49-7c78-4edd-bd6d-9a5d40668835 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.533163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c46e49-7c78-4edd-bd6d-9a5d40668835 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.533442 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c46e49-7c78-4edd-bd6d-9a5d40668835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.570945 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039b74c-6c17-43ac-b573-9d7ef1b8d1f6 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:04.620752 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c46e49-7c78-4edd-bd6d-9a5d40668835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:04.623961 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c46e49-7c78-4edd-bd6d-9a5d40668835 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:04.640251 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9bbf6-5f8f-49fa-a062-6e7a11904d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:04.641330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9bbf6-5f8f-49fa-a062-6e7a11904d57 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.641938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9bbf6-5f8f-49fa-a062-6e7a11904d57 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.642566 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9bbf6-5f8f-49fa-a062-6e7a11904d57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.656142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9bbf6-5f8f-49fa-a062-6e7a11904d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-d3d9bbf6-5f8f-49fa-a062-6e7a11904d57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:04.664307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:04.665381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:04.666380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:04.669363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 125/246] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:04.675716 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9bbf6-5f8f-49fa-a062-6e7a11904d57 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:04.691151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:04.719575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:04.760834 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 122/247] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 09:21:04.763863 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c46e49-7c78-4edd-bd6d-9a5d40668835 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:04.826024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9bbf6-5f8f-49fa-a062-6e7a11904d57 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:04.876341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:04.877008 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.877296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.877572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.878542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:04.878817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:04.880406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:04.887872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c46e49-7c78-4edd-bd6d-9a5d40668835 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:04.892478 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 126/248] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:04.898979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:04.930025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efdef6-6361-4ace-a04b-34bfb5083107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:04.931013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efdef6-6361-4ace-a04b-34bfb5083107 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:04.931596 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efdef6-6361-4ace-a04b-34bfb5083107 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:04.932039 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efdef6-6361-4ace-a04b-34bfb5083107 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:04.933582 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efdef6-6361-4ace-a04b-34bfb5083107 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:04.935517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efdef6-6361-4ace-a04b-34bfb5083107 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:04.938057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 123/249] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:04 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:04.944989 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efdef6-6361-4ace-a04b-34bfb5083107 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:04.976391 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:05.000413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:05.028355 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bd395-b7c9-4a0b-b0af-d5987a2ac8e4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:05.078216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2f60-dc84-4f13-b1c1-7e110e65a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:05.078216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2f60-dc84-4f13-b1c1-7e110e65a46f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:05.078216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2f60-dc84-4f13-b1c1-7e110e65a46f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:05.078216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2f60-dc84-4f13-b1c1-7e110e65a46f None admin] PATH_INFO: `/v3/users/d078cb0e3ab24966a5cd60df22fd168a` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:05.078729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f2f60-dc84-4f13-b1c1-7e110e65a46f None admin] RBAC: Authorizing `identity:get_user(user_id=d078cb0e3ab24966a5cd60df22fd168a)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:05.078729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f2f60-dc84-4f13-b1c1-7e110e65a46f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:05.080813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 127/250] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:04 2022] GET /identity/v3/users/d078cb0e3ab24966a5cd60df22fd168a => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:05.086271 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2f60-dc84-4f13-b1c1-7e110e65a46f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:05.099352 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efdef6-6361-4ace-a04b-34bfb5083107 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:05.168028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:05.169048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:05.169495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:05.170004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:05.171026 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] RBAC: Adding query filter params (role.id=f6ae8eb934c74269ae9e47640f831a4e, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915, user.id=3075c405d74c439cb8c65efeb6abd85c) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:05.171524 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:05.173314 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:05.193923 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 124/251] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:04 2022] GET /identity/v3/role_assignments?user.id=3075c405d74c439cb8c65efeb6abd85c&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=f6ae8eb934c74269ae9e47640f831a4e => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:05.201388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2f60-dc84-4f13-b1c1-7e110e65a46f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:05.240479 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251550-e13a-4a27-893d-6cc065eef4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:05.241343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251550-e13a-4a27-893d-6cc065eef4c4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:05.241756 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251550-e13a-4a27-893d-6cc065eef4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:05.242166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251550-e13a-4a27-893d-6cc065eef4c4 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:05.243498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03251550-e13a-4a27-893d-6cc065eef4c4 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:05.245072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03251550-e13a-4a27-893d-6cc065eef4c4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:05.247312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 128/252] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:05 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:05.253460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:05.405881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:05.426555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9db1e1-f17b-4d89-8d38-7ec8a74719e6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:05.496052 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4616f8-cfb3-4b1d-941f-9f6f47959ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:05.496759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4616f8-cfb3-4b1d-941f-9f6f47959ec2 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:05.497048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4616f8-cfb3-4b1d-941f-9f6f47959ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:05.497325 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4616f8-cfb3-4b1d-941f-9f6f47959ec2 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/users/d078cb0e3ab24966a5cd60df22fd168a/roles/16a8547072ea431abbe07aad19a1fd43` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:05.499447 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4616f8-cfb3-4b1d-941f-9f6f47959ec2 None admin] RBAC: Authorizing `identity:create_grant(project_id=72fb605a1a2c4ec29bdc2d9daa734915, user_id=d078cb0e3ab24966a5cd60df22fd168a, role_id=16a8547072ea431abbe07aad19a1fd43)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:05.500930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4616f8-cfb3-4b1d-941f-9f6f47959ec2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:05.523543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 125/253] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:05 2022] PUT /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/users/d078cb0e3ab24966a5cd60df22fd168a/roles/16a8547072ea431abbe07aad19a1fd43 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:05.727104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229ad86-c83a-46e7-9981-2dcea045473f None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:05.727658 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229ad86-c83a-46e7-9981-2dcea045473f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:05.727764 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229ad86-c83a-46e7-9981-2dcea045473f None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:05.729381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 129/254] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:05.735716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6d1a3-1e98-4b81-a337-f2bf808500b3 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:05.735995 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6d1a3-1e98-4b81-a337-f2bf808500b3 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:05.736274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6d1a3-1e98-4b81-a337-f2bf808500b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:05.772144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-f8d6d1a3-1e98-4b81-a337-f2bf808500b3 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:05.773385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-f8d6d1a3-1e98-4b81-a337-f2bf808500b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:05.876974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 126/255] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:05.911778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229ad86-c83a-46e7-9981-2dcea045473f None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:05.930449 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:06.039320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229ad86-c83a-46e7-9981-2dcea045473f None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:06.087917 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192784-e496-47cc-8476-88695682ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:06.088587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192784-e496-47cc-8476-88695682ec40 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:06.088869 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192784-e496-47cc-8476-88695682ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:06.089142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192784-e496-47cc-8476-88695682ec40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:06.102993 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8192784-e496-47cc-8476-88695682ec40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-c8192784-e496-47cc-8476-88695682ec40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:06.108159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:06.109280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:06.110224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 130/256] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:06.117050 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6d1a3-1e98-4b81-a337-f2bf808500b3 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:06.232308 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:06.264512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6d1a3-1e98-4b81-a337-f2bf808500b3 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:06.310544 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:06.311488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:06.311934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:06.312380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:06.313281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:06.313742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:06.315335 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:06.329202 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 127/257] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:06.335675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192784-e496-47cc-8476-88695682ec40 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:06.492807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:06.524013 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192784-e496-47cc-8476-88695682ec40 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:06.565893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb053f96-cee8-414c-8ac6-c523bd9095d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:06.566825 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb053f96-cee8-414c-8ac6-c523bd9095d2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:06.567145 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb053f96-cee8-414c-8ac6-c523bd9095d2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:06.567380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb053f96-cee8-414c-8ac6-c523bd9095d2 None admin] PATH_INFO: `/v3/users/b4264a3fd9194cbb8a67c14e2e369673` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:06.568649 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb053f96-cee8-414c-8ac6-c523bd9095d2 None admin] RBAC: Authorizing `identity:get_user(user_id=b4264a3fd9194cbb8a67c14e2e369673)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:06.570226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb053f96-cee8-414c-8ac6-c523bd9095d2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:06.572267 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 131/258] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:06 2022] GET /identity/v3/users/b4264a3fd9194cbb8a67c14e2e369673 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:06.582818 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:06.701417 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaf02c-61ec-41ef-9947-ded5df77265a None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:06.701692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaf02c-61ec-41ef-9947-ded5df77265a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:06.701934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaf02c-61ec-41ef-9947-ded5df77265a None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:06.703572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 132/259] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:21:06.710057 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2a909-2c4f-47b1-8fe5-476b5ff9e9af None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:06.710389 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2a909-2c4f-47b1-8fe5-476b5ff9e9af None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:06.710649 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2a909-2c4f-47b1-8fe5-476b5ff9e9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:06.751987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-92a2a909-2c4f-47b1-8fe5-476b5ff9e9af None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:06.753212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-92a2a909-2c4f-47b1-8fe5-476b5ff9e9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:06.769787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825322ff-687f-4d18-8eb9-c2408e6e54b8 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:06.844438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba221a36-4b45-4a06-90c4-cb1354b6b1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:06.845121 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba221a36-4b45-4a06-90c4-cb1354b6b1f1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:06.845372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba221a36-4b45-4a06-90c4-cb1354b6b1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:06.845565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba221a36-4b45-4a06-90c4-cb1354b6b1f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:06.862472 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:06.899488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 133/260] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:06.969973 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba221a36-4b45-4a06-90c4-cb1354b6b1f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:06.971523 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba221a36-4b45-4a06-90c4-cb1354b6b1f1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:07.021557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2a909-2c4f-47b1-8fe5-476b5ff9e9af None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:07.215973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:07.261359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 128/261] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:07.267128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba221a36-4b45-4a06-90c4-cb1354b6b1f1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:07.278831 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2a909-2c4f-47b1-8fe5-476b5ff9e9af None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:07.357609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c3e1a-745e-4699-bba8-9eec3d5b3855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:07.358725 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c3e1a-745e-4699-bba8-9eec3d5b3855 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:07.359210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c3e1a-745e-4699-bba8-9eec3d5b3855 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:07.359707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c3e1a-745e-4699-bba8-9eec3d5b3855 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:07.390531 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:07.399233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c3e1a-745e-4699-bba8-9eec3d5b3855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-b88c3e1a-745e-4699-bba8-9eec3d5b3855 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:07.409625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:07.410727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:07.411691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:07.412406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 134/262] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:07.441725 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c3e1a-745e-4699-bba8-9eec3d5b3855 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:07.477646 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba221a36-4b45-4a06-90c4-cb1354b6b1f1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:07.519108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187df45-2597-403f-abc8-7c7a4a42aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:07.520286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187df45-2597-403f-abc8-7c7a4a42aa41 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:07.520798 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187df45-2597-403f-abc8-7c7a4a42aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:07.522564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187df45-2597-403f-abc8-7c7a4a42aa41 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:07.524780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187df45-2597-403f-abc8-7c7a4a42aa41 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:07.526539 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187df45-2597-403f-abc8-7c7a4a42aa41 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:07.531964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 129/263] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:07 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:07.538916 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187df45-2597-403f-abc8-7c7a4a42aa41 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:07.722238 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c3e1a-745e-4699-bba8-9eec3d5b3855 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:07.796697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:07.803980 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187df45-2597-403f-abc8-7c7a4a42aa41 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:07.816174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a1291-40d9-4871-981c-a3961979020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:07.817212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a1291-40d9-4871-981c-a3961979020c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:07.817502 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a1291-40d9-4871-981c-a3961979020c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:07.817778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a1291-40d9-4871-981c-a3961979020c None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:07.818606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a1291-40d9-4871-981c-a3961979020c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:07.818915 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a1291-40d9-4871-981c-a3961979020c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:07.820428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a1291-40d9-4871-981c-a3961979020c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:07.829953 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:07.844733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 135/264] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:07.846154 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6387a89-47d4-4810-8507-903c5bc112dd None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:07.846462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6387a89-47d4-4810-8507-903c5bc112dd None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:07.846749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6387a89-47d4-4810-8507-903c5bc112dd None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:07.848004 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 136/265] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:07.854896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6387a89-47d4-4810-8507-903c5bc112dd None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:07.890042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ee85c-30c2-4a61-bb1c-98769fc68818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:07.892627 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ee85c-30c2-4a61-bb1c-98769fc68818 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:07.894092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ee85c-30c2-4a61-bb1c-98769fc68818 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:07.894917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ee85c-30c2-4a61-bb1c-98769fc68818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:07.897132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ee85c-30c2-4a61-bb1c-98769fc68818 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915, user.id=b4264a3fd9194cbb8a67c14e2e369673) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:07.897991 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ee85c-30c2-4a61-bb1c-98769fc68818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:07.902058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ee85c-30c2-4a61-bb1c-98769fc68818 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:07.916917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 130/266] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:07 2022] GET /identity/v3/role_assignments?user.id=b4264a3fd9194cbb8a67c14e2e369673&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=af958d22229a4a0297bbe2e643f55181 => generated 608 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:07.920836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a905-dc8e-49eb-9e90-7d80020748e7 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:07.921931 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a905-dc8e-49eb-9e90-7d80020748e7 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:07.922854 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a905-dc8e-49eb-9e90-7d80020748e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:07.958922 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-14f7a905-dc8e-49eb-9e90-7d80020748e7 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:07.962823 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-14f7a905-dc8e-49eb-9e90-7d80020748e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:08.103196 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6387a89-47d4-4810-8507-903c5bc112dd None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:08.117265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 131/267] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:08.122135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed73c7-958b-4492-9e41-0d0accf7dba8 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.122135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed73c7-958b-4492-9e41-0d0accf7dba8 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.122135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed73c7-958b-4492-9e41-0d0accf7dba8 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.127511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 132/268] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:08.138129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18207fc0-1e3f-4c7f-ad73-e8d34cee4d7b None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.138129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18207fc0-1e3f-4c7f-ad73-e8d34cee4d7b None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.138129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18207fc0-1e3f-4c7f-ad73-e8d34cee4d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.163059 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f871bb-9bd2-4dc6-88bb-b1fc0c194e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:08.163785 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f871bb-9bd2-4dc6-88bb-b1fc0c194e83 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.164093 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f871bb-9bd2-4dc6-88bb-b1fc0c194e83 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.164375 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f871bb-9bd2-4dc6-88bb-b1fc0c194e83 None admin] PATH_INFO: `/v3/users/b4264a3fd9194cbb8a67c14e2e369673` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.165621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-18207fc0-1e3f-4c7f-ad73-e8d34cee4d7b None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:08.165759 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f871bb-9bd2-4dc6-88bb-b1fc0c194e83 None admin] RBAC: Authorizing `identity:get_user(user_id=b4264a3fd9194cbb8a67c14e2e369673)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:08.166947 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-18207fc0-1e3f-4c7f-ad73-e8d34cee4d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:08.167246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f871bb-9bd2-4dc6-88bb-b1fc0c194e83 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:08.171353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 137/269] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:07 2022] GET /identity/v3/users/b4264a3fd9194cbb8a67c14e2e369673 => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:08.172073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ac45c-333e-4191-bcf1-2e5c8ccd89bd None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.172359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ac45c-333e-4191-bcf1-2e5c8ccd89bd None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.172639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ac45c-333e-4191-bcf1-2e5c8ccd89bd None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.174057 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 138/270] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:08.174643 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ac45c-333e-4191-bcf1-2e5c8ccd89bd None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:08.252817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 133/271] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:08.254084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18207fc0-1e3f-4c7f-ad73-e8d34cee4d7b None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:08.262702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:08.300993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:08.308425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ac45c-333e-4191-bcf1-2e5c8ccd89bd None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:08.346317 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bc372-243b-4a75-83c0-a80b58f88fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:08.346979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc372-243b-4a75-83c0-a80b58f88fef None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.347266 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc372-243b-4a75-83c0-a80b58f88fef None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.347540 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc372-243b-4a75-83c0-a80b58f88fef None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.348709 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bc372-243b-4a75-83c0-a80b58f88fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:08.350676 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bc372-243b-4a75-83c0-a80b58f88fef None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:08.352671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 139/272] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:08.353873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0d431-1d76-45df-8815-7d5aef03125a None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.354125 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0d431-1d76-45df-8815-7d5aef03125a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.354411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0d431-1d76-45df-8815-7d5aef03125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.373012 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-0ba0d431-1d76-45df-8815-7d5aef03125a None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:08.374266 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-0ba0d431-1d76-45df-8815-7d5aef03125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:08.387772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18207fc0-1e3f-4c7f-ad73-e8d34cee4d7b None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:08.428073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66192-53c8-4b86-a232-94cf85bf61c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:08.428880 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66192-53c8-4b86-a232-94cf85bf61c1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.429166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66192-53c8-4b86-a232-94cf85bf61c1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.429591 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66192-53c8-4b86-a232-94cf85bf61c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.452743 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 140/273] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:08.454080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0d431-1d76-45df-8815-7d5aef03125a None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:08.514895 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c66192-53c8-4b86-a232-94cf85bf61c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:08.516381 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c66192-53c8-4b86-a232-94cf85bf61c1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:08.529034 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:08.553182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:08.576927 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0d431-1d76-45df-8815-7d5aef03125a None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:08.616575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1072ca1-2602-4b54-8167-38368d02dd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:08.617173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1072ca1-2602-4b54-8167-38368d02dd18 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.617454 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1072ca1-2602-4b54-8167-38368d02dd18 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.617743 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1072ca1-2602-4b54-8167-38368d02dd18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.628756 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1072ca1-2602-4b54-8167-38368d02dd18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-f1072ca1-2602-4b54-8167-38368d02dd18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:08.633814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:08.635006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 141/274] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:08.641131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1072ca1-2602-4b54-8167-38368d02dd18 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:08.642044 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 134/275] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:08.642044 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66192-53c8-4b86-a232-94cf85bf61c1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:08.748842 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66192-53c8-4b86-a232-94cf85bf61c1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:08.753877 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1072ca1-2602-4b54-8167-38368d02dd18 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:08.777099 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:08.782241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:08.789194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a664f0e-b55c-49d1-90ff-5f3bfd55e351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:08.789869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664f0e-b55c-49d1-90ff-5f3bfd55e351 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.790184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664f0e-b55c-49d1-90ff-5f3bfd55e351 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.790470 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664f0e-b55c-49d1-90ff-5f3bfd55e351 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.795860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fbcde-0ead-47dc-9348-73d477b88328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:08.796733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fbcde-0ead-47dc-9348-73d477b88328 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.797621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fbcde-0ead-47dc-9348-73d477b88328 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.798063 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fbcde-0ead-47dc-9348-73d477b88328 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.798979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fbcde-0ead-47dc-9348-73d477b88328 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:08.800645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fbcde-0ead-47dc-9348-73d477b88328 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:08.801633 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a664f0e-b55c-49d1-90ff-5f3bfd55e351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-9a664f0e-b55c-49d1-90ff-5f3bfd55e351 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:08.806857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:08.808151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 135/276] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:08.809309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a664f0e-b55c-49d1-90ff-5f3bfd55e351 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:08.837657 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 142/277] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:08 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:08.837657 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fbcde-0ead-47dc-9348-73d477b88328 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:08.932100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a664f0e-b55c-49d1-90ff-5f3bfd55e351 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:08.954666 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fbcde-0ead-47dc-9348-73d477b88328 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:08.973768 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38769-583d-4759-a2d3-ce1f92f9c37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:08.974476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38769-583d-4759-a2d3-ce1f92f9c37a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:08.974760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38769-583d-4759-a2d3-ce1f92f9c37a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:08.975037 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38769-583d-4759-a2d3-ce1f92f9c37a None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:08.976284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a38769-583d-4759-a2d3-ce1f92f9c37a None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:08.977805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a38769-583d-4759-a2d3-ce1f92f9c37a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:08.984678 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 136/278] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:08 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:08.984678 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38769-583d-4759-a2d3-ce1f92f9c37a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:09.001512 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf8642-fabd-43f1-a858-047c62e327a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:09.002196 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf8642-fabd-43f1-a858-047c62e327a6 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.002485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf8642-fabd-43f1-a858-047c62e327a6 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.002871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf8642-fabd-43f1-a858-047c62e327a6 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.003623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf8642-fabd-43f1-a858-047c62e327a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:09.003948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf8642-fabd-43f1-a858-047c62e327a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:09.005393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf8642-fabd-43f1-a858-047c62e327a6 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:09.015120 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 143/279] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:09.016403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7fe84-7d9c-4224-837c-f05d2a8ea397 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.018233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7fe84-7d9c-4224-837c-f05d2a8ea397 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.018750 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7fe84-7d9c-4224-837c-f05d2a8ea397 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.020360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 144/280] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:09.021280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7fe84-7d9c-4224-837c-f05d2a8ea397 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:09.060334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:09.076620 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:09.127223 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38769-583d-4759-a2d3-ce1f92f9c37a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:09.169635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7fe84-7d9c-4224-837c-f05d2a8ea397 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:09.171017 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:09.171017 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.171232 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.171400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.172195 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] RBAC: Adding query filter params (name=reader) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:09.172523 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:09.173930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:09.189322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 137/281] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:09.189322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:09.226874 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a78ed-f0b5-4a71-8ae0-4fccc790697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:09.227533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a78ed-f0b5-4a71-8ae0-4fccc790697b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.227849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a78ed-f0b5-4a71-8ae0-4fccc790697b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.228105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a78ed-f0b5-4a71-8ae0-4fccc790697b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.228939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a78ed-f0b5-4a71-8ae0-4fccc790697b None admin] RBAC: Adding query filter params (role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915, user.id=b4264a3fd9194cbb8a67c14e2e369673) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:09.229260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a78ed-f0b5-4a71-8ae0-4fccc790697b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:09.230825 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a78ed-f0b5-4a71-8ae0-4fccc790697b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:09.238212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 145/282] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:09 2022] GET /identity/v3/role_assignments?user.id=b4264a3fd9194cbb8a67c14e2e369673&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:09.239387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446ddb8-9132-462b-9fa7-f46b575f47d5 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.239681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446ddb8-9132-462b-9fa7-f46b575f47d5 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.239964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446ddb8-9132-462b-9fa7-f46b575f47d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.260201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-9446ddb8-9132-462b-9fa7-f46b575f47d5 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:09.261435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-9446ddb8-9132-462b-9fa7-f46b575f47d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:09.317970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290a349-8f2a-4570-a5f7-94568ebf41da None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:09.339578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 146/283] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:09.340259 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446ddb8-9132-462b-9fa7-f46b575f47d5 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:09.354595 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:09.368154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea43c0-8d7c-4362-bed0-bd7bd83cd00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:09.369760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea43c0-8d7c-4362-bed0-bd7bd83cd00b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.369760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea43c0-8d7c-4362-bed0-bd7bd83cd00b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.369760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea43c0-8d7c-4362-bed0-bd7bd83cd00b None admin] PATH_INFO: `/v3/users/10ce74688fcb438491064c88e6948b7e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.398770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea43c0-8d7c-4362-bed0-bd7bd83cd00b None admin] RBAC: Authorizing `identity:get_user(user_id=10ce74688fcb438491064c88e6948b7e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:09.400578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea43c0-8d7c-4362-bed0-bd7bd83cd00b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:09.403056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 138/284] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:09 2022] GET /identity/v3/users/10ce74688fcb438491064c88e6948b7e => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:09.403447 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:09.405583 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea43c0-8d7c-4362-bed0-bd7bd83cd00b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:09.491818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446ddb8-9132-462b-9fa7-f46b575f47d5 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:09.539896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49c7bd-e289-439c-8ffc-bb6e97ebd7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:09.541598 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49c7bd-e289-439c-8ffc-bb6e97ebd7b4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.542131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49c7bd-e289-439c-8ffc-bb6e97ebd7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.542651 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49c7bd-e289-439c-8ffc-bb6e97ebd7b4 None admin] PATH_INFO: `/v3/users/3075c405d74c439cb8c65efeb6abd85c` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.552879 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49c7bd-e289-439c-8ffc-bb6e97ebd7b4 None admin] RBAC: Authorizing `identity:get_user(user_id=3075c405d74c439cb8c65efeb6abd85c)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:09.555942 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49c7bd-e289-439c-8ffc-bb6e97ebd7b4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:09.562686 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 147/285] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:09 2022] GET /identity/v3/users/3075c405d74c439cb8c65efeb6abd85c => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:09.564289 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49c7bd-e289-439c-8ffc-bb6e97ebd7b4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:09.609758 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea43c0-8d7c-4362-bed0-bd7bd83cd00b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:09.673567 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d176f2-0e91-4a27-9f6e-e882ac804d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:09.674431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d176f2-0e91-4a27-9f6e-e882ac804d96 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.674719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d176f2-0e91-4a27-9f6e-e882ac804d96 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.675006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d176f2-0e91-4a27-9f6e-e882ac804d96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.688143 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d176f2-0e91-4a27-9f6e-e882ac804d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-e1d176f2-0e91-4a27-9f6e-e882ac804d96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:09.693597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:09.694779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 139/286] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:09.695941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d176f2-0e91-4a27-9f6e-e882ac804d96 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:09.739588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:09.762581 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:09.770252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49c7bd-e289-439c-8ffc-bb6e97ebd7b4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:09.821762 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d176f2-0e91-4a27-9f6e-e882ac804d96 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:09.841330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:09.842039 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.842417 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.842696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.843547 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] RBAC: Adding query filter params (role.id=16a8547072ea431abbe07aad19a1fd43, user.id=10ce74688fcb438491064c88e6948b7e, scope.system=all) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:09.843867 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:09.845417 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:09.854460 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 148/287] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 09:21:09 2022] GET /identity/v3/role_assignments?user.id=10ce74688fcb438491064c88e6948b7e&scope.system=all&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:09.855113 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:09.891506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f32ed-45a6-4227-bc94-a9c67f9267cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:09.893533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f32ed-45a6-4227-bc94-a9c67f9267cb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:09.893914 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f32ed-45a6-4227-bc94-a9c67f9267cb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:09.894099 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f32ed-45a6-4227-bc94-a9c67f9267cb None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:09.896769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f32ed-45a6-4227-bc94-a9c67f9267cb None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:09.898403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f32ed-45a6-4227-bc94-a9c67f9267cb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:09.900422 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 140/288] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:09 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:09.908487 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f32ed-45a6-4227-bc94-a9c67f9267cb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:10.062531 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14fae3-a185-4a3b-9e6f-98c83b828f05 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:10.109026 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:10.109934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:10.110646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:10.112327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:10.113264 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:10.114816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:10.116230 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:10.124304 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f32ed-45a6-4227-bc94-a9c67f9267cb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:10.126305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 149/289] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:10.132099 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:10.138893 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:10.162519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:10.181127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74415-861a-4d0c-b6b9-85ec5459f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:10.182358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74415-861a-4d0c-b6b9-85ec5459f683 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:10.182854 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74415-861a-4d0c-b6b9-85ec5459f683 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:10.190132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74415-861a-4d0c-b6b9-85ec5459f683 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/users/3075c405d74c439cb8c65efeb6abd85c/roles/f6ae8eb934c74269ae9e47640f831a4e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:10.193341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d74415-861a-4d0c-b6b9-85ec5459f683 None admin] RBAC: Authorizing `identity:create_grant(project_id=72fb605a1a2c4ec29bdc2d9daa734915, user_id=3075c405d74c439cb8c65efeb6abd85c, role_id=f6ae8eb934c74269ae9e47640f831a4e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:10.195426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d74415-861a-4d0c-b6b9-85ec5459f683 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:10.218594 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 141/290] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:09 2022] PUT /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/users/3075c405d74c439cb8c65efeb6abd85c/roles/f6ae8eb934c74269ae9e47640f831a4e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:10.269845 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef18d8-5b4a-4c9f-bfd5-e44fd381ccf2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:10.323582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00d3bf-35b1-4b59-b462-cdba427b2ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:10.324648 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00d3bf-35b1-4b59-b462-cdba427b2ddd None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:10.325138 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00d3bf-35b1-4b59-b462-cdba427b2ddd None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:10.325631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00d3bf-35b1-4b59-b462-cdba427b2ddd None admin] PATH_INFO: `/v3/users/d078cb0e3ab24966a5cd60df22fd168a` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:10.327389 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c00d3bf-35b1-4b59-b462-cdba427b2ddd None admin] RBAC: Authorizing `identity:get_user(user_id=d078cb0e3ab24966a5cd60df22fd168a)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:10.329383 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c00d3bf-35b1-4b59-b462-cdba427b2ddd None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:10.331947 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 150/291] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:10 2022] GET /identity/v3/users/d078cb0e3ab24966a5cd60df22fd168a => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:10.337459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74415-861a-4d0c-b6b9-85ec5459f683 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:10.481430 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74415-861a-4d0c-b6b9-85ec5459f683 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:10.522017 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65cd5a-40c8-4c38-98dc-fef053a331e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:10.523125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65cd5a-40c8-4c38-98dc-fef053a331e5 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:10.523621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65cd5a-40c8-4c38-98dc-fef053a331e5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:10.524122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65cd5a-40c8-4c38-98dc-fef053a331e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:10.565550 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:10.608225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65cd5a-40c8-4c38-98dc-fef053a331e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:10.610038 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65cd5a-40c8-4c38-98dc-fef053a331e5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:10.722301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 142/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:10.725345 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00d3bf-35b1-4b59-b462-cdba427b2ddd None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:10.844247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:10.851321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00d3bf-35b1-4b59-b462-cdba427b2ddd None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:10.902772 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd27997-408c-4e2a-8a19-64b63f0bd206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:10.903460 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd27997-408c-4e2a-8a19-64b63f0bd206 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:10.903749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd27997-408c-4e2a-8a19-64b63f0bd206 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:10.904064 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd27997-408c-4e2a-8a19-64b63f0bd206 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:10.910976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd27997-408c-4e2a-8a19-64b63f0bd206 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:10.912494 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd27997-408c-4e2a-8a19-64b63f0bd206 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:10.914402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 151/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:10 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:10.920857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65cd5a-40c8-4c38-98dc-fef053a331e5 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:10.995513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:11.032788 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65cd5a-40c8-4c38-98dc-fef053a331e5 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:11.070285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa242ea-c5dd-4855-9715-57220d068fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:11.070925 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa242ea-c5dd-4855-9715-57220d068fd2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:11.071213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa242ea-c5dd-4855-9715-57220d068fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:11.071495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa242ea-c5dd-4855-9715-57220d068fd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:11.072305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa242ea-c5dd-4855-9715-57220d068fd2 None admin] RBAC: Adding query filter params (role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915, user.id=d078cb0e3ab24966a5cd60df22fd168a) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:11.072626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa242ea-c5dd-4855-9715-57220d068fd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:11.074091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa242ea-c5dd-4855-9715-57220d068fd2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:11.081037 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 143/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:10 2022] GET /identity/v3/role_assignments?user.id=d078cb0e3ab24966a5cd60df22fd168a&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:11.491227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7a73d-3ae7-4d42-9e5e-e04c427c0657 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:11.492025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7a73d-3ae7-4d42-9e5e-e04c427c0657 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:11.492604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7a73d-3ae7-4d42-9e5e-e04c427c0657 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:11.494602 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 152/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:11.500815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bb37a-e0c9-4833-92ce-0896d2b7120f None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:11.501768 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bb37a-e0c9-4833-92ce-0896d2b7120f None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:11.502297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bb37a-e0c9-4833-92ce-0896d2b7120f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:11.526142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-600bb37a-e0c9-4833-92ce-0896d2b7120f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:11.527418 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-600bb37a-e0c9-4833-92ce-0896d2b7120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:11.577110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f781610-d007-4b76-bd93-f7bb2e86e492 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:11.577467 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f781610-d007-4b76-bd93-f7bb2e86e492 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:11.577664 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f781610-d007-4b76-bd93-f7bb2e86e492 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:11.579366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 153/296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:11.585341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b01f4-746e-44d0-a738-231de22810b9 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:11.585657 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b01f4-746e-44d0-a738-231de22810b9 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:11.585905 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b01f4-746e-44d0-a738-231de22810b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:11.611025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-967b01f4-746e-44d0-a738-231de22810b9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:11.612244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-967b01f4-746e-44d0-a738-231de22810b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:11.617773 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 144/297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:11.644656 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bb37a-e0c9-4833-92ce-0896d2b7120f None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:11.699692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 154/298] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:11.723198 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b01f4-746e-44d0-a738-231de22810b9 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:11.727638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:11.786129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bb37a-e0c9-4833-92ce-0896d2b7120f None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:11.827104 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cf42e-91be-4374-a290-8158332e1581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:11.827770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cf42e-91be-4374-a290-8158332e1581 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:11.828086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cf42e-91be-4374-a290-8158332e1581 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:11.828365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cf42e-91be-4374-a290-8158332e1581 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:11.840273 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cf42e-91be-4374-a290-8158332e1581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:11.843607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-8e2cf42e-91be-4374-a290-8158332e1581 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:11.846886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:11.848233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:11.849176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:11.852027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 145/299] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:11.857951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b01f4-746e-44d0-a738-231de22810b9 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:11.859562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cf42e-91be-4374-a290-8158332e1581 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:11.896455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfb22-9890-4194-b9b0-3f7eec48d129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:11.897123 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfb22-9890-4194-b9b0-3f7eec48d129 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:11.897442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfb22-9890-4194-b9b0-3f7eec48d129 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:11.897723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfb22-9890-4194-b9b0-3f7eec48d129 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:11.899050 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dfb22-9890-4194-b9b0-3f7eec48d129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:11.900631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dfb22-9890-4194-b9b0-3f7eec48d129 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:11.907428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 155/300] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:11.908693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfb22-9890-4194-b9b0-3f7eec48d129 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:11.993895 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cf42e-91be-4374-a290-8158332e1581 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:12.020834 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:12.040341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:12.040891 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.041548 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.041824 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.042921 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] RBAC: Adding query filter params (name=reader) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:12.043232 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:12.045122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:12.048714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfb22-9890-4194-b9b0-3f7eec48d129 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:12.057205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 146/301] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:12.063626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:12.098744 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf743f7-3099-4815-8b3c-80940010b6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:12.099729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf743f7-3099-4815-8b3c-80940010b6f0 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.100239 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf743f7-3099-4815-8b3c-80940010b6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.100701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf743f7-3099-4815-8b3c-80940010b6f0 None admin] PATH_INFO: `/v3/groups` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.101724 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf743f7-3099-4815-8b3c-80940010b6f0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:12.103701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf743f7-3099-4815-8b3c-80940010b6f0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:12.131791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 156/302] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 09:21:11 2022] POST /identity/v3/groups => generated 229 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:12.193044 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f051c-c813-4a6d-9833-0207cf897497 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:12.239204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c576e-32b6-4f92-9030-45140f6b06cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:12.240435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c576e-32b6-4f92-9030-45140f6b06cb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.240940 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c576e-32b6-4f92-9030-45140f6b06cb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.241510 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c576e-32b6-4f92-9030-45140f6b06cb None admin] PATH_INFO: `/v3/users/c73d3d2d179b40b79efbd3b6b861c23d` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.260775 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c576e-32b6-4f92-9030-45140f6b06cb None admin] RBAC: Authorizing `identity:get_user(user_id=c73d3d2d179b40b79efbd3b6b861c23d)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:12.262741 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c576e-32b6-4f92-9030-45140f6b06cb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:12.265350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 147/303] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:12 2022] GET /identity/v3/users/c73d3d2d179b40b79efbd3b6b861c23d => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:12.271381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf743f7-3099-4815-8b3c-80940010b6f0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:12.304432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:12.386731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c18c1-0314-4f93-b3e6-fc8d11a66f78 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.387128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c18c1-0314-4f93-b3e6-fc8d11a66f78 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.387301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c18c1-0314-4f93-b3e6-fc8d11a66f78 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.389638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 148/304] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:12.396362 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a0e03-8128-468c-ad64-b5262e93173f None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.396948 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a0e03-8128-468c-ad64-b5262e93173f None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.397325 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a0e03-8128-468c-ad64-b5262e93173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.412605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf743f7-3099-4815-8b3c-80940010b6f0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:12.420609 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-5d2a0e03-8128-468c-ad64-b5262e93173f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:12.421079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-5d2a0e03-8128-468c-ad64-b5262e93173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:12.468884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:12.469474 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.471360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.471360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.471360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] RBAC: Adding query filter params (role.id=f6ae8eb934c74269ae9e47640f831a4e, user.id=c73d3d2d179b40b79efbd3b6b861c23d, scope.system=all) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:12.471360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:12.473245 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:12.475225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:12.483421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 157/305] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 09:21:12 2022] GET /identity/v3/role_assignments?user.id=c73d3d2d179b40b79efbd3b6b861c23d&scope.system=all&role.id=f6ae8eb934c74269ae9e47640f831a4e => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:12.548201 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 149/306] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:12.571693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:12.605118 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13f7e-a673-447c-9235-a0f0dbb10042 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.605457 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13f7e-a673-447c-9235-a0f0dbb10042 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.605691 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13f7e-a673-447c-9235-a0f0dbb10042 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.607174 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 150/307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:12.613014 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89570ef-ee93-4347-a67f-6d3e0e8462ac None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.613681 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89570ef-ee93-4347-a67f-6d3e0e8462ac None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.614216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89570ef-ee93-4347-a67f-6d3e0e8462ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.635390 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-f89570ef-ee93-4347-a67f-6d3e0e8462ac None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:12.637114 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-f89570ef-ee93-4347-a67f-6d3e0e8462ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:12.682043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:12.700110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4868b-a892-4168-a8dd-cbc0c6bbe898 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:12.712814 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 151/308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:12.736076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89570ef-ee93-4347-a67f-6d3e0e8462ac None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:12.740475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2492d-f17a-403a-b3e4-a295aa576ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:12.741441 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2492d-f17a-403a-b3e4-a295aa576ac6 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.741951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2492d-f17a-403a-b3e4-a295aa576ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.742475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2492d-f17a-403a-b3e4-a295aa576ac6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.762139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2492d-f17a-403a-b3e4-a295aa576ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-33c2492d-f17a-403a-b3e4-a295aa576ac6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:12.765924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:12.767037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:12.767948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 158/309] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:12.773337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2492d-f17a-403a-b3e4-a295aa576ac6 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:12.846299 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:12.849797 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89570ef-ee93-4347-a67f-6d3e0e8462ac None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:12.890733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2492d-f17a-403a-b3e4-a295aa576ac6 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:12.891536 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49856e0b-f037-4579-8ad6-6443bb954a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:12.892226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49856e0b-f037-4579-8ad6-6443bb954a56 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.892761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49856e0b-f037-4579-8ad6-6443bb954a56 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.893048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49856e0b-f037-4579-8ad6-6443bb954a56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.913210 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49856e0b-f037-4579-8ad6-6443bb954a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-49856e0b-f037-4579-8ad6-6443bb954a56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:12.920359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:12.922658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:12.924693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:12.926302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 152/310] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:12.928440 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49856e0b-f037-4579-8ad6-6443bb954a56 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:12.935058 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:12.936030 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:12.936523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:12.937026 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:12.937991 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:12.939029 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:12.941080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:12.953244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 159/311] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:12.959125 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:12.973537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:13.064218 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49856e0b-f037-4579-8ad6-6443bb954a56 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:13.095964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb3c24-74cd-48c4-a629-4b1fcc49bb8c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:13.109587 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:13.110298 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:13.110646 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:13.110933 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:13.112198 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:13.112831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:13.114623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:13.130226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 153/312] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:13.132531 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:13.149861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325101-1a6e-4c56-9dce-14134fb5cdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:13.151157 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325101-1a6e-4c56-9dce-14134fb5cdd7 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:13.152077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325101-1a6e-4c56-9dce-14134fb5cdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:13.152580 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325101-1a6e-4c56-9dce-14134fb5cdd7 None admin] PATH_INFO: `/v3/users/10ce74688fcb438491064c88e6948b7e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:13.154650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a325101-1a6e-4c56-9dce-14134fb5cdd7 None admin] RBAC: Authorizing `identity:get_user(user_id=10ce74688fcb438491064c88e6948b7e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:13.156754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a325101-1a6e-4c56-9dce-14134fb5cdd7 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:13.159673 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 160/313] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:12 2022] GET /identity/v3/users/10ce74688fcb438491064c88e6948b7e => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:13.165061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:13.165595 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325101-1a6e-4c56-9dce-14134fb5cdd7 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:13.265828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:13.270486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997fe4a-1a2c-4292-99ad-0235ab1e1b79 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:13.303400 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325101-1a6e-4c56-9dce-14134fb5cdd7 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:13.315170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e714-c146-411d-b73c-ca977bcc55db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:13.315842 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e714-c146-411d-b73c-ca977bcc55db None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:13.316141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e714-c146-411d-b73c-ca977bcc55db None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:13.316735 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e714-c146-411d-b73c-ca977bcc55db None admin] PATH_INFO: `/v3/users/3075c405d74c439cb8c65efeb6abd85c` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:13.318410 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999e714-c146-411d-b73c-ca977bcc55db None admin] RBAC: Authorizing `identity:get_user(user_id=3075c405d74c439cb8c65efeb6abd85c)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:13.320497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999e714-c146-411d-b73c-ca977bcc55db None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:13.326188 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 154/314] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:13 2022] GET /identity/v3/users/3075c405d74c439cb8c65efeb6abd85c => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:13.329965 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e714-c146-411d-b73c-ca977bcc55db None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:13.356377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cb8e-bd9b-4c1c-816e-d1609682b0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:13.357330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cb8e-bd9b-4c1c-816e-d1609682b0db None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:13.357788 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cb8e-bd9b-4c1c-816e-d1609682b0db None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:13.358079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cb8e-bd9b-4c1c-816e-d1609682b0db None admin] PATH_INFO: `/v3/system/users/10ce74688fcb438491064c88e6948b7e/roles/16a8547072ea431abbe07aad19a1fd43` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:13.359967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926cb8e-bd9b-4c1c-816e-d1609682b0db None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=10ce74688fcb438491064c88e6948b7e, role_id=16a8547072ea431abbe07aad19a1fd43)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:13.361816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926cb8e-bd9b-4c1c-816e-d1609682b0db None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:13.382220 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 161/315] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 09:21:13 2022] PUT /identity/v3/system/users/10ce74688fcb438491064c88e6948b7e/roles/16a8547072ea431abbe07aad19a1fd43 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:13.457923 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:13.475001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e714-c146-411d-b73c-ca977bcc55db None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:13.515702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b8c28-bde3-491b-9af2-d46cb5ed49e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:13.516373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b8c28-bde3-491b-9af2-d46cb5ed49e0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:13.516699 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b8c28-bde3-491b-9af2-d46cb5ed49e0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:13.516976 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b8c28-bde3-491b-9af2-d46cb5ed49e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:13.624116 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b8c28-bde3-491b-9af2-d46cb5ed49e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:13.625993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b8c28-bde3-491b-9af2-d46cb5ed49e0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:13.743158 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:13.755120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 155/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:13.761263 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cb8e-bd9b-4c1c-816e-d1609682b0db None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:13.878046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cb8e-bd9b-4c1c-816e-d1609682b0db None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:13.912806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83fc386-0e0a-4fba-97a6-c287ab074d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:13.913507 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83fc386-0e0a-4fba-97a6-c287ab074d07 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:13.913762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83fc386-0e0a-4fba-97a6-c287ab074d07 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:13.914007 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83fc386-0e0a-4fba-97a6-c287ab074d07 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:13.915227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83fc386-0e0a-4fba-97a6-c287ab074d07 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:13.916778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83fc386-0e0a-4fba-97a6-c287ab074d07 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:13.918903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 162/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:13 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:13.925101 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b8c28-bde3-491b-9af2-d46cb5ed49e0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:14.043836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b8c28-bde3-491b-9af2-d46cb5ed49e0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:14.083601 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b562571-06fe-414c-924a-6cf04ce2de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:14.084657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562571-06fe-414c-924a-6cf04ce2de6f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:14.085169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562571-06fe-414c-924a-6cf04ce2de6f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:14.085661 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562571-06fe-414c-924a-6cf04ce2de6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:14.086722 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b562571-06fe-414c-924a-6cf04ce2de6f None admin] RBAC: Adding query filter params (role.id=f6ae8eb934c74269ae9e47640f831a4e, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915, user.id=3075c405d74c439cb8c65efeb6abd85c) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:14.087282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b562571-06fe-414c-924a-6cf04ce2de6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:14.088978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b562571-06fe-414c-924a-6cf04ce2de6f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:14.101347 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 156/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:13 2022] GET /identity/v3/role_assignments?user.id=3075c405d74c439cb8c65efeb6abd85c&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=f6ae8eb934c74269ae9e47640f831a4e => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:14.402893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db4cab-1432-4271-a909-5db46c1c401d None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:14.403317 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db4cab-1432-4271-a909-5db46c1c401d None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:14.403451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db4cab-1432-4271-a909-5db46c1c401d None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:14.404972 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 163/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:14.410399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2f052-8036-45ec-b124-a4d0c0a44025 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:14.410693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2f052-8036-45ec-b124-a4d0c0a44025 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:14.410973 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2f052-8036-45ec-b124-a4d0c0a44025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:14.430147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-12a2f052-8036-45ec-b124-a4d0c0a44025 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:14.431393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-12a2f052-8036-45ec-b124-a4d0c0a44025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:14.488027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:14.508211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 157/320] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:14.530852 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db4cab-1432-4271-a909-5db46c1c401d None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:14.540671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:14.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db4cab-1432-4271-a909-5db46c1c401d None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:14.687771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7228-8b0d-4c89-809f-4d6b6711d9a5 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:14.688829 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7228-8b0d-4c89-809f-4d6b6711d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:14.689078 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b9ce1-9ab0-4865-858d-29aeb723572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:14.689586 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7228-8b0d-4c89-809f-4d6b6711d9a5 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:14.689783 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b9ce1-9ab0-4865-858d-29aeb723572a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:14.690024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b9ce1-9ab0-4865-858d-29aeb723572a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:14.690335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b9ce1-9ab0-4865-858d-29aeb723572a None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:14.691834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b9ce1-9ab0-4865-858d-29aeb723572a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:14.692789 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 158/321] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:14.693359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b9ce1-9ab0-4865-858d-29aeb723572a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:14.695226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 164/322] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:14.698090 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ca57-a77a-4d60-b257-28e4b6446ec1 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:14.698615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ca57-a77a-4d60-b257-28e4b6446ec1 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:14.699055 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ca57-a77a-4d60-b257-28e4b6446ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:14.700751 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b9ce1-9ab0-4865-858d-29aeb723572a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:14.719516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-a5b7ca57-a77a-4d60-b257-28e4b6446ec1 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:14.720712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-a5b7ca57-a77a-4d60-b257-28e4b6446ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:14.783944 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:14.801623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 159/323] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:14.816022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b9ce1-9ab0-4865-858d-29aeb723572a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:14.825927 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7ca57-a77a-4d60-b257-28e4b6446ec1 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:14.854059 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055871c-af0e-4744-8488-978ffa0393da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:14.854725 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055871c-af0e-4744-8488-978ffa0393da None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:14.854999 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055871c-af0e-4744-8488-978ffa0393da None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:14.855310 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055871c-af0e-4744-8488-978ffa0393da None admin] PATH_INFO: `/v3/groups` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:14.855982 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055871c-af0e-4744-8488-978ffa0393da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:14.857608 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055871c-af0e-4744-8488-978ffa0393da None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:14.876970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 165/324] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 09:21:14 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:14.951157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7ca57-a77a-4d60-b257-28e4b6446ec1 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:14.968227 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:14.992613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edef51-2d40-49cf-817e-2d2862b2c470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:14.993286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edef51-2d40-49cf-817e-2d2862b2c470 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:14.993574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edef51-2d40-49cf-817e-2d2862b2c470 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:14.993852 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edef51-2d40-49cf-817e-2d2862b2c470 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:15.005963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edef51-2d40-49cf-817e-2d2862b2c470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-11edef51-2d40-49cf-817e-2d2862b2c470 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:15.011229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:15.012342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:15.013332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 160/325] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:15.019649 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055871c-af0e-4744-8488-978ffa0393da None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:15.148049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055871c-af0e-4744-8488-978ffa0393da None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:15.193137 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:15.193778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:15.194060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:15.194522 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:15.195260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:15.195571 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:15.197010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:15.198907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:15.209464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 166/326] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:15.215062 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edef51-2d40-49cf-817e-2d2862b2c470 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:15.329222 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edef51-2d40-49cf-817e-2d2862b2c470 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:15.372076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7a843-7ef4-46b7-8033-04a43fefd2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:15.373283 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7a843-7ef4-46b7-8033-04a43fefd2b2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:15.373787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7a843-7ef4-46b7-8033-04a43fefd2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:15.374363 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7a843-7ef4-46b7-8033-04a43fefd2b2 None admin] PATH_INFO: `/v3/users/c73d3d2d179b40b79efbd3b6b861c23d` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:15.375883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7a843-7ef4-46b7-8033-04a43fefd2b2 None admin] RBAC: Authorizing `identity:get_user(user_id=c73d3d2d179b40b79efbd3b6b861c23d)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:15.377691 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7a843-7ef4-46b7-8033-04a43fefd2b2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:15.379898 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 161/327] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:15 2022] GET /identity/v3/users/c73d3d2d179b40b79efbd3b6b861c23d => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:15.385748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:15.506073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dc368-bdab-4e9e-9e8e-36d4cf4739f2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:15.537130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb123a12-4f64-40df-8c64-08f1a3a965bc None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:15.537513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb123a12-4f64-40df-8c64-08f1a3a965bc None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:15.537696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb123a12-4f64-40df-8c64-08f1a3a965bc None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:15.539367 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 162/328] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:21:15.546281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee705-770d-49a7-b8b6-bf0fd75df962 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:15.546281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee705-770d-49a7-b8b6-bf0fd75df962 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:15.546663 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee705-770d-49a7-b8b6-bf0fd75df962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:15.553873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04905ab9-afab-4742-b385-be1b73864a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:15.554606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04905ab9-afab-4742-b385-be1b73864a3b None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:15.554897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04905ab9-afab-4742-b385-be1b73864a3b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:15.555175 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04905ab9-afab-4742-b385-be1b73864a3b None admin] PATH_INFO: `/v3/system/users/c73d3d2d179b40b79efbd3b6b861c23d/roles/f6ae8eb934c74269ae9e47640f831a4e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:15.557461 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04905ab9-afab-4742-b385-be1b73864a3b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c73d3d2d179b40b79efbd3b6b861c23d, role_id=f6ae8eb934c74269ae9e47640f831a4e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:15.558931 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04905ab9-afab-4742-b385-be1b73864a3b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:15.569122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-369ee705-770d-49a7-b8b6-bf0fd75df962 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:15.570774 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-369ee705-770d-49a7-b8b6-bf0fd75df962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:15.570893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 167/329] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 09:21:15 2022] PUT /identity/v3/system/users/c73d3d2d179b40b79efbd3b6b861c23d/roles/f6ae8eb934c74269ae9e47640f831a4e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:15.614146 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:15.651292 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 163/330] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:15.683309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04905ab9-afab-4742-b385-be1b73864a3b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:15.761044 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:15.809295 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04905ab9-afab-4742-b385-be1b73864a3b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:15.845240 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99d0bd-c6c3-4588-86e3-6e1ac10ca09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:15.846286 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99d0bd-c6c3-4588-86e3-6e1ac10ca09b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:15.846770 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99d0bd-c6c3-4588-86e3-6e1ac10ca09b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:15.847236 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99d0bd-c6c3-4588-86e3-6e1ac10ca09b None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:15.859661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99d0bd-c6c3-4588-86e3-6e1ac10ca09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-1c99d0bd-c6c3-4588-86e3-6e1ac10ca09b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:15.865604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:15.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:15.867560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:15.869970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 168/331] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:15.876138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ee705-770d-49a7-b8b6-bf0fd75df962 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:15.991472 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ee705-770d-49a7-b8b6-bf0fd75df962 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:16.030409 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:16.031318 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:16.031762 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:16.032186 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:16.033126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] RBAC: Adding query filter params (name=member) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:16.033615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:16.035222 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:16.046225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 164/332] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:16.054397 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99d0bd-c6c3-4588-86e3-6e1ac10ca09b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:16.192651 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99d0bd-c6c3-4588-86e3-6e1ac10ca09b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:16.222146 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:16.240455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9837131-fa4b-4c9f-b532-7e3370cd407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:16.241114 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9837131-fa4b-4c9f-b532-7e3370cd407b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:16.241394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9837131-fa4b-4c9f-b532-7e3370cd407b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:16.241667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9837131-fa4b-4c9f-b532-7e3370cd407b None admin] PATH_INFO: `/v3/users/10ce74688fcb438491064c88e6948b7e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:16.246887 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9837131-fa4b-4c9f-b532-7e3370cd407b None admin] RBAC: Authorizing `identity:get_user(user_id=10ce74688fcb438491064c88e6948b7e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:16.248499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9837131-fa4b-4c9f-b532-7e3370cd407b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:16.250650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 169/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:16 2022] GET /identity/v3/users/10ce74688fcb438491064c88e6948b7e => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:16.256498 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:16.305089 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:16.395494 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a237b6-483b-4511-b198-4cd8010decdd None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:16.442059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00fb49-a1de-4064-b2d7-80aec04ef1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:16.443078 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00fb49-a1de-4064-b2d7-80aec04ef1d2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:16.443366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00fb49-a1de-4064-b2d7-80aec04ef1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:16.443648 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00fb49-a1de-4064-b2d7-80aec04ef1d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:16.444478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00fb49-a1de-4064-b2d7-80aec04ef1d2 None admin] RBAC: Adding query filter params (role.id=16a8547072ea431abbe07aad19a1fd43, user.id=10ce74688fcb438491064c88e6948b7e, scope.system=all) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:16.444795 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00fb49-a1de-4064-b2d7-80aec04ef1d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:16.449451 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00fb49-a1de-4064-b2d7-80aec04ef1d2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:16.469519 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 165/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 09:21:16 2022] GET /identity/v3/role_assignments?user.id=10ce74688fcb438491064c88e6948b7e&scope.system=all&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 510 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:17.492627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946cb74-c5e5-475f-b272-d44a8fc0782e None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:17.493752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946cb74-c5e5-475f-b272-d44a8fc0782e None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:17.494322 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946cb74-c5e5-475f-b272-d44a8fc0782e None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:17.496233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 170/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:17.501635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d10f8-94e3-41b3-8244-bd5e97ccb95b None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:17.502143 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d10f8-94e3-41b3-8244-bd5e97ccb95b None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:17.502257 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d10f8-94e3-41b3-8244-bd5e97ccb95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:17.531237 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-526d10f8-94e3-41b3-8244-bd5e97ccb95b None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:17.532532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-526d10f8-94e3-41b3-8244-bd5e97ccb95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:17.619828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 166/336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:17.657896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946cb74-c5e5-475f-b272-d44a8fc0782e None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:17.675151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c568a82-7bce-4a5f-bef8-67a602592e93 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:17.675500 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c568a82-7bce-4a5f-bef8-67a602592e93 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:17.675706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c568a82-7bce-4a5f-bef8-67a602592e93 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:17.679273 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 167/337] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:17.683891 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83650-089b-44aa-ad02-e1d37406c657 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:17.684351 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83650-089b-44aa-ad02-e1d37406c657 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:17.684773 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83650-089b-44aa-ad02-e1d37406c657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:17.727198 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-e4e83650-089b-44aa-ad02-e1d37406c657 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:17.728438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-e4e83650-089b-44aa-ad02-e1d37406c657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:17.770578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:17.859119 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946cb74-c5e5-475f-b272-d44a8fc0782e None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:17.880827 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 168/338] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:17.920408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e83650-089b-44aa-ad02-e1d37406c657 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:17.924184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f2361-0861-45f9-b007-91ce38c1015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:17.925103 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f2361-0861-45f9-b007-91ce38c1015d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:17.925614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f2361-0861-45f9-b007-91ce38c1015d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:17.926198 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f2361-0861-45f9-b007-91ce38c1015d None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:17.946550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f2361-0861-45f9-b007-91ce38c1015d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-1f1f2361-0861-45f9-b007-91ce38c1015d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:17.952036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:17.953077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:17.954018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:17.956796 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 171/339] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:17.962689 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f2361-0861-45f9-b007-91ce38c1015d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:18.058515 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:18.168838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e83650-089b-44aa-ad02-e1d37406c657 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:18.185338 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f2361-0861-45f9-b007-91ce38c1015d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:18.234210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:18.234858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:18.235139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:18.235667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:18.236449 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:18.236756 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:18.238178 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:18.259342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0c06-aea8-4eed-9c92-e6571ffa756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:18.260013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0c06-aea8-4eed-9c92-e6571ffa756f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:18.260296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0c06-aea8-4eed-9c92-e6571ffa756f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:18.260571 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0c06-aea8-4eed-9c92-e6571ffa756f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:18.263065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 172/340] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:18.269393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:18.290759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a0c06-aea8-4eed-9c92-e6571ffa756f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-204a0c06-aea8-4eed-9c92-e6571ffa756f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:18.296029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:18.297063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:18.298059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 169/341] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:18.311128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0c06-aea8-4eed-9c92-e6571ffa756f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:18.354474 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:18.492588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5204e7c-0ce6-44f4-a9e8-034f907e91aa None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:18.503674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:18.533413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93432-9c85-41a2-a675-26857409c776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:18.534067 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93432-9c85-41a2-a675-26857409c776 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:18.534423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93432-9c85-41a2-a675-26857409c776 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:18.534709 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93432-9c85-41a2-a675-26857409c776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:18.559604 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0c06-aea8-4eed-9c92-e6571ffa756f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:18.619301 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b93432-9c85-41a2-a675-26857409c776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:18.620834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b93432-9c85-41a2-a675-26857409c776 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:18.637602 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:18.638323 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:18.638612 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:18.638892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:18.639645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:18.639958 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:18.641372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:18.661165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 170/342] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:18.667859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:18.774752 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:18.812575 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494ae72-2655-4564-953d-3f37aeca2709 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:18.851034 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 173/343] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:18.859175 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93432-9c85-41a2-a675-26857409c776 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:18.863369 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37b08a-9006-433b-a22c-a42856c6c6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:18.864023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37b08a-9006-433b-a22c-a42856c6c6b2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:18.864308 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37b08a-9006-433b-a22c-a42856c6c6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:18.864591 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37b08a-9006-433b-a22c-a42856c6c6b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:18.885848 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:19.013438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93432-9c85-41a2-a675-26857409c776 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:19.054658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37b08a-9006-433b-a22c-a42856c6c6b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:19.056157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37b08a-9006-433b-a22c-a42856c6c6b2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:19.058675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027008a-6d7e-4400-b7f2-2e74d93809e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:19.059391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027008a-6d7e-4400-b7f2-2e74d93809e3 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.059680 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027008a-6d7e-4400-b7f2-2e74d93809e3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.059994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027008a-6d7e-4400-b7f2-2e74d93809e3 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.061667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027008a-6d7e-4400-b7f2-2e74d93809e3 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:19.063139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027008a-6d7e-4400-b7f2-2e74d93809e3 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:19.065080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 174/344] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:18 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:19.065997 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131df5-30e0-4919-9865-741bbbbdb595 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.066623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131df5-30e0-4919-9865-741bbbbdb595 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.066906 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131df5-30e0-4919-9865-741bbbbdb595 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.069327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 175/345] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:19.075557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18131df5-30e0-4919-9865-741bbbbdb595 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:19.210619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:19.239124 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 171/346] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 09:21:19.240507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af46d9-a697-4741-be22-fdef79844af1 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.240799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af46d9-a697-4741-be22-fdef79844af1 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.241112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af46d9-a697-4741-be22-fdef79844af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.243724 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:19.267393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-d3af46d9-a697-4741-be22-fdef79844af1 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:19.269261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-d3af46d9-a697-4741-be22-fdef79844af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:19.294759 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18131df5-30e0-4919-9865-741bbbbdb595 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:19.352083 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b16da5-c873-4200-a95e-7fb76be4f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:19.352776 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16da5-c873-4200-a95e-7fb76be4f390 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.353051 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16da5-c873-4200-a95e-7fb76be4f390 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.353328 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16da5-c873-4200-a95e-7fb76be4f390 None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.362429 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b16da5-c873-4200-a95e-7fb76be4f390 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:19.365063 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b16da5-c873-4200-a95e-7fb76be4f390 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:19.366227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 176/347] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:19 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:19.367201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b16da5-c873-4200-a95e-7fb76be4f390 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:19.379099 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 172/348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:19.380943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e6f1-8951-4dcc-bfcb-7f24528dde66 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.382171 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e6f1-8951-4dcc-bfcb-7f24528dde66 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.382719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e6f1-8951-4dcc-bfcb-7f24528dde66 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.384725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 173/349] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:19.385790 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152e6f1-8951-4dcc-bfcb-7f24528dde66 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:19.510833 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b16da5-c873-4200-a95e-7fb76be4f390 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:19.525484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152e6f1-8951-4dcc-bfcb-7f24528dde66 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:19.531727 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:19.553340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c6d8e-608a-4c6b-89da-9e836610dd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:19.554017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6d8e-608a-4c6b-89da-9e836610dd10 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.554355 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6d8e-608a-4c6b-89da-9e836610dd10 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.554640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6d8e-608a-4c6b-89da-9e836610dd10 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.556868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c6d8e-608a-4c6b-89da-9e836610dd10 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:19.557560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c6d8e-608a-4c6b-89da-9e836610dd10 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:19.560241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 177/350] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:19 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:19.560912 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139b6c0-6fe7-434e-b6f0-d35aba79f594 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.561208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139b6c0-6fe7-434e-b6f0-d35aba79f594 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.561488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139b6c0-6fe7-434e-b6f0-d35aba79f594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.581205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28763fbe-a66f-4368-99ae-2e63075c6bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:19.581854 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28763fbe-a66f-4368-99ae-2e63075c6bf2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.582195 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28763fbe-a66f-4368-99ae-2e63075c6bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.582496 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28763fbe-a66f-4368-99ae-2e63075c6bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.583320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28763fbe-a66f-4368-99ae-2e63075c6bf2 None admin] RBAC: Adding query filter params (group.id=9c3e993c8ffd471588066c5d91fa9d34, role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:19.583638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28763fbe-a66f-4368-99ae-2e63075c6bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:19.585118 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28763fbe-a66f-4368-99ae-2e63075c6bf2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:19.587217 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-8139b6c0-6fe7-434e-b6f0-d35aba79f594 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:19.588525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-8139b6c0-6fe7-434e-b6f0-d35aba79f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:19.592780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 174/351] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:19 2022] GET /identity/v3/role_assignments?group.id=9c3e993c8ffd471588066c5d91fa9d34&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:19.599096 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f378482-5556-4c4f-acdc-7b50024a6969 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.599096 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f378482-5556-4c4f-acdc-7b50024a6969 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.599297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f378482-5556-4c4f-acdc-7b50024a6969 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.600997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 175/352] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:19.601781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f378482-5556-4c4f-acdc-7b50024a6969 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:19.604360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:19.706763 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 178/353] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:19.708306 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139b6c0-6fe7-434e-b6f0-d35aba79f594 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:19.785726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f378482-5556-4c4f-acdc-7b50024a6969 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:19.834755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb34e3-be81-4d0d-9e32-d231c44da8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:19.835728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb34e3-be81-4d0d-9e32-d231c44da8ee None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.836018 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb34e3-be81-4d0d-9e32-d231c44da8ee None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.836378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb34e3-be81-4d0d-9e32-d231c44da8ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.848602 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb34e3-be81-4d0d-9e32-d231c44da8ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-03fb34e3-be81-4d0d-9e32-d231c44da8ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:19.853644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:19.854861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:19.855892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 176/354] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:19.856835 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c4682-b727-44ec-86ac-079872c579e5 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.857434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c4682-b727-44ec-86ac-079872c579e5 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.858197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c4682-b727-44ec-86ac-079872c579e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.861068 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139b6c0-6fe7-434e-b6f0-d35aba79f594 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:19.884984 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-318c4682-b727-44ec-86ac-079872c579e5 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:19.886409 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-318c4682-b727-44ec-86ac-079872c579e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:19.900014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e2e1c-7b4d-4029-99a7-7f22446ad00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:19.900686 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e2e1c-7b4d-4029-99a7-7f22446ad00f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.900978 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e2e1c-7b4d-4029-99a7-7f22446ad00f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.901259 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e2e1c-7b4d-4029-99a7-7f22446ad00f None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.902509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e2e1c-7b4d-4029-99a7-7f22446ad00f None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:19.904464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e2e1c-7b4d-4029-99a7-7f22446ad00f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:19.906022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 179/355] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:19 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:19.906847 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e2e1c-7b4d-4029-99a7-7f22446ad00f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:19.918933 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:19.923089 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:20.012036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 177/356] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:20.016184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c4682-b727-44ec-86ac-079872c579e5 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.041053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e2e1c-7b4d-4029-99a7-7f22446ad00f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.096392 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1cf03-c10e-4de7-94ec-6ffdaf676b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.096392 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1cf03-c10e-4de7-94ec-6ffdaf676b08 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.097156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1cf03-c10e-4de7-94ec-6ffdaf676b08 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.097613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1cf03-c10e-4de7-94ec-6ffdaf676b08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.119153 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1cf03-c10e-4de7-94ec-6ffdaf676b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-aaa1cf03-c10e-4de7-94ec-6ffdaf676b08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:20.142041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:20.143391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 180/357] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:20.144523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1cf03-c10e-4de7-94ec-6ffdaf676b08 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.206409 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c4682-b727-44ec-86ac-079872c579e5 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.269253 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:20.273901 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904fd14-92c4-4756-8d41-c7f17fe828d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.274612 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fd14-92c4-4756-8d41-c7f17fe828d6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.274883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fd14-92c4-4756-8d41-c7f17fe828d6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.275400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fd14-92c4-4756-8d41-c7f17fe828d6 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.278838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904fd14-92c4-4756-8d41-c7f17fe828d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:20.279036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904fd14-92c4-4756-8d41-c7f17fe828d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:20.280347 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904fd14-92c4-4756-8d41-c7f17fe828d6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:20.299981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 178/358] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:20.300443 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321202f-456c-43c4-a8e8-1ed585d72e5a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.300712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321202f-456c-43c4-a8e8-1ed585d72e5a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.301020 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321202f-456c-43c4-a8e8-1ed585d72e5a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.306168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 179/359] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:20.306168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321202f-456c-43c4-a8e8-1ed585d72e5a None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.328850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1cf03-c10e-4de7-94ec-6ffdaf676b08 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.341039 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:20.394170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.394170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.394170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.394170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.394683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] RBAC: Adding query filter params (group.id=9c3e993c8ffd471588066c5d91fa9d34, role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:20.394683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:20.394837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:20.403431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 181/360] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/role_assignments?group.id=9c3e993c8ffd471588066c5d91fa9d34&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:20.404338 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.457125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321202f-456c-43c4-a8e8-1ed585d72e5a None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.540972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48c38-f61a-45b6-8003-6f171fdd0aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.541664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48c38-f61a-45b6-8003-6f171fdd0aac None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.541955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48c38-f61a-45b6-8003-6f171fdd0aac None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.542339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48c38-f61a-45b6-8003-6f171fdd0aac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.558818 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48c38-f61a-45b6-8003-6f171fdd0aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-30c48c38-f61a-45b6-8003-6f171fdd0aac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:20.565848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:20.567416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:20.568372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 180/361] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:20.569283 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48c38-f61a-45b6-8003-6f171fdd0aac None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.577431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e645dbe-aa2a-4464-b975-61c97ddb13fa None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.616523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:20.628185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fb9ed-1efa-4dc2-aa30-7f3f6e356211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.628938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb9ed-1efa-4dc2-aa30-7f3f6e356211 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.629259 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb9ed-1efa-4dc2-aa30-7f3f6e356211 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.629542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb9ed-1efa-4dc2-aa30-7f3f6e356211 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.630318 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fb9ed-1efa-4dc2-aa30-7f3f6e356211 None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:20.630631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fb9ed-1efa-4dc2-aa30-7f3f6e356211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:20.632033 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fb9ed-1efa-4dc2-aa30-7f3f6e356211 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:20.643438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 182/362] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:20.646161 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b5dce-03c5-4ba2-9e4f-2632aa689e76 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.646540 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b5dce-03c5-4ba2-9e4f-2632aa689e76 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.646825 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b5dce-03c5-4ba2-9e4f-2632aa689e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.668269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-048b5dce-03c5-4ba2-9e4f-2632aa689e76 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:20.669665 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-048b5dce-03c5-4ba2-9e4f-2632aa689e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:20.676097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:20.699333 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48c38-f61a-45b6-8003-6f171fdd0aac None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.736964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859d9be-5dfd-4422-9740-47e7a2c759d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.737956 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859d9be-5dfd-4422-9740-47e7a2c759d9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.738533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859d9be-5dfd-4422-9740-47e7a2c759d9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.738741 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859d9be-5dfd-4422-9740-47e7a2c759d9 None admin] PATH_INFO: `/v3/users/c73d3d2d179b40b79efbd3b6b861c23d` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.740097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859d9be-5dfd-4422-9740-47e7a2c759d9 None admin] RBAC: Authorizing `identity:get_user(user_id=c73d3d2d179b40b79efbd3b6b861c23d)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:20.741687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859d9be-5dfd-4422-9740-47e7a2c759d9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:20.743517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 181/363] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/users/c73d3d2d179b40b79efbd3b6b861c23d => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:20.744421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859d9be-5dfd-4422-9740-47e7a2c759d9 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.757360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 183/364] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:20.758721 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b5dce-03c5-4ba2-9e4f-2632aa689e76 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.856537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859d9be-5dfd-4422-9740-47e7a2c759d9 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.883985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b5dce-03c5-4ba2-9e4f-2632aa689e76 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.897083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.898157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.898427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.898803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.899750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:20.900087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:20.901716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:20.911375 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 182/365] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:20.912506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.913731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:20.926603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7ec10-bb25-4ae1-be7b-69b359cdbd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.927025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7ec10-bb25-4ae1-be7b-69b359cdbd11 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.927278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7ec10-bb25-4ae1-be7b-69b359cdbd11 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.927551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7ec10-bb25-4ae1-be7b-69b359cdbd11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.932088 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:21.024950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7ec10-bb25-4ae1-be7b-69b359cdbd11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:21.027393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7ec10-bb25-4ae1-be7b-69b359cdbd11 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:21.040470 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8d834-2839-4a9b-919c-76dc34bc7e8d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:21.080126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:21.080888 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:21.081236 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:21.081579 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:21.082498 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] RBAC: Adding query filter params (role.id=f6ae8eb934c74269ae9e47640f831a4e, user.id=c73d3d2d179b40b79efbd3b6b861c23d, scope.system=all) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:21.082879 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:21.084424 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:21.099049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 183/366] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/role_assignments?user.id=c73d3d2d179b40b79efbd3b6b861c23d&scope.system=all&role.id=f6ae8eb934c74269ae9e47640f831a4e => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:21.099715 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:21.168501 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 184/367] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:21.170097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7ec10-bb25-4ae1-be7b-69b359cdbd11 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:21.188600 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:21.235591 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:21.273015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d4cf4-440a-4814-be65-23d6735d30a4 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:21.354346 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7642b2-dfed-4d0e-a1ee-899aed563d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:21.355040 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7642b2-dfed-4d0e-a1ee-899aed563d3a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:21.355327 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7642b2-dfed-4d0e-a1ee-899aed563d3a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:21.355609 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7642b2-dfed-4d0e-a1ee-899aed563d3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:21.378675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7642b2-dfed-4d0e-a1ee-899aed563d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-5e7642b2-dfed-4d0e-a1ee-899aed563d3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:21.385388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:21.386540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:21.387464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:21.388454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 184/368] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:21.388938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7642b2-dfed-4d0e-a1ee-899aed563d3a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:21.397822 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7ec10-bb25-4ae1-be7b-69b359cdbd11 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:21.451091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968c4a3-7163-4bc8-b46c-3d9ad0d2a500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:21.451653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968c4a3-7163-4bc8-b46c-3d9ad0d2a500 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:21.451942 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968c4a3-7163-4bc8-b46c-3d9ad0d2a500 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:21.452222 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968c4a3-7163-4bc8-b46c-3d9ad0d2a500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:21.512009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7642b2-dfed-4d0e-a1ee-899aed563d3a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:21.531989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968c4a3-7163-4bc8-b46c-3d9ad0d2a500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:21.533551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968c4a3-7163-4bc8-b46c-3d9ad0d2a500 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:21.543629 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:21.557057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d3afc-27f0-4965-932f-e149aca394ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:21.557812 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d3afc-27f0-4965-932f-e149aca394ac None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:21.558095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d3afc-27f0-4965-932f-e149aca394ac None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:21.558396 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d3afc-27f0-4965-932f-e149aca394ac None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:21.560837 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d3afc-27f0-4965-932f-e149aca394ac None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:21.562871 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d3afc-27f0-4965-932f-e149aca394ac None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:21.564223 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 185/369] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:21 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:21.564947 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d3afc-27f0-4965-932f-e149aca394ac None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:21.576720 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:21.664476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 185/370] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:21.665466 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968c4a3-7163-4bc8-b46c-3d9ad0d2a500 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:21.696296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d3afc-27f0-4965-932f-e149aca394ac None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:21.733309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:21.733915 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:21.734234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:21.734613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:21.735448 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:21.735761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:21.737223 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:21.747245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 186/371] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:21.748523 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:21.780599 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968c4a3-7163-4bc8-b46c-3d9ad0d2a500 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:21.808887 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:21.822320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea4ba0-0f7f-4e5a-b4df-40452290ea67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:21.822988 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea4ba0-0f7f-4e5a-b4df-40452290ea67 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:21.823268 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea4ba0-0f7f-4e5a-b4df-40452290ea67 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:21.823543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea4ba0-0f7f-4e5a-b4df-40452290ea67 None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:21.823752 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:21.825406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea4ba0-0f7f-4e5a-b4df-40452290ea67 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:21.827212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea4ba0-0f7f-4e5a-b4df-40452290ea67 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:21.828901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 186/372] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:21 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:21.829685 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea4ba0-0f7f-4e5a-b4df-40452290ea67 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:21.872697 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8617796-f07d-49fc-9ec4-034b215636b6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:21.910909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acdfdc-27e7-4a8d-a928-1fb8ccbef1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:21.911709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acdfdc-27e7-4a8d-a928-1fb8ccbef1e9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:21.912001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acdfdc-27e7-4a8d-a928-1fb8ccbef1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:21.912283 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acdfdc-27e7-4a8d-a928-1fb8ccbef1e9 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:21.913516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acdfdc-27e7-4a8d-a928-1fb8ccbef1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:21.915081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acdfdc-27e7-4a8d-a928-1fb8ccbef1e9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:21.916813 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 187/373] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:21 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:21.922647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acdfdc-27e7-4a8d-a928-1fb8ccbef1e9 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:21.943921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea4ba0-0f7f-4e5a-b4df-40452290ea67 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:21.980334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f79328-6d60-48b6-b81e-fbbe1a983e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:21.980973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79328-6d60-48b6-b81e-fbbe1a983e1f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:21.981256 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79328-6d60-48b6-b81e-fbbe1a983e1f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:21.981530 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79328-6d60-48b6-b81e-fbbe1a983e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.037656 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acdfdc-27e7-4a8d-a928-1fb8ccbef1e9 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:22.045594 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:22.056204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f79328-6d60-48b6-b81e-fbbe1a983e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:22.058865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f79328-6d60-48b6-b81e-fbbe1a983e1f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:22.073920 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:22.090152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:22.090152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.090152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.090152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.090705 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] RBAC: Adding query filter params (group.id=9c3e993c8ffd471588066c5d91fa9d34, role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:22.090705 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:22.091308 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:22.098822 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 188/374] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:21 2022] GET /identity/v3/role_assignments?group.id=9c3e993c8ffd471588066c5d91fa9d34&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:22.099768 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:22.191679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 187/375] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:22.196741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7997476-09fb-46da-8147-8c1adb009ee4 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.196741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7997476-09fb-46da-8147-8c1adb009ee4 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.196741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7997476-09fb-46da-8147-8c1adb009ee4 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.197344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 188/376] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:22.203188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7997476-09fb-46da-8147-8c1adb009ee4 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:22.259239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa94bb-9b0e-4365-b6de-858fe7d1476b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:22.317307 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0bed1-53b2-4a9e-8162-0dd9ea57ad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:22.318580 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0bed1-53b2-4a9e-8162-0dd9ea57ad1a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.320001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0bed1-53b2-4a9e-8162-0dd9ea57ad1a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.320452 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0bed1-53b2-4a9e-8162-0dd9ea57ad1a None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.321897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0bed1-53b2-4a9e-8162-0dd9ea57ad1a None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:22.323586 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0bed1-53b2-4a9e-8162-0dd9ea57ad1a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:22.326030 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 189/377] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:22 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:22.327808 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82860f25-4e40-43fb-9762-b93b012bda3e None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.328118 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82860f25-4e40-43fb-9762-b93b012bda3e None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.328397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82860f25-4e40-43fb-9762-b93b012bda3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.349754 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-82860f25-4e40-43fb-9762-b93b012bda3e None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:22.351019 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-82860f25-4e40-43fb-9762-b93b012bda3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:22.353241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:22.356433 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7997476-09fb-46da-8147-8c1adb009ee4 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:22.396112 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30046c29-3f8e-4eb9-87a4-6eed9daf84c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:22.396852 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30046c29-3f8e-4eb9-87a4-6eed9daf84c8 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.398315 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30046c29-3f8e-4eb9-87a4-6eed9daf84c8 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.398629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30046c29-3f8e-4eb9-87a4-6eed9daf84c8 None admin] PATH_INFO: `/v3/projects/a32e32f35d6b4f64983733c445917938` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.399873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30046c29-3f8e-4eb9-87a4-6eed9daf84c8 None admin] RBAC: Authorizing `identity:get_project(project_id=a32e32f35d6b4f64983733c445917938)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:22.401354 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30046c29-3f8e-4eb9-87a4-6eed9daf84c8 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:22.403788 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 189/378] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:22 2022] GET /identity/v3/projects/a32e32f35d6b4f64983733c445917938 => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:22.404663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30046c29-3f8e-4eb9-87a4-6eed9daf84c8 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:22.421754 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:22.442765 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 190/379] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:22.443817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82860f25-4e40-43fb-9762-b93b012bda3e None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:22.541501 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30046c29-3f8e-4eb9-87a4-6eed9daf84c8 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:22.571219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82860f25-4e40-43fb-9762-b93b012bda3e None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:22.588380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:22.589056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.589346 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.589632 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.590535 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] RBAC: Adding query filter params (group.id=9c3e993c8ffd471588066c5d91fa9d34, role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:22.590860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:22.592433 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:22.606890 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 190/380] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:22 2022] GET /identity/v3/role_assignments?group.id=9c3e993c8ffd471588066c5d91fa9d34&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:22.615033 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:22.630029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc41a68-c71f-4d6a-91a8-c721a8253fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:22.632180 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41a68-c71f-4d6a-91a8-c721a8253fba None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.632467 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41a68-c71f-4d6a-91a8-c721a8253fba None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.632758 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41a68-c71f-4d6a-91a8-c721a8253fba None admin] PATH_INFO: `/v3/groups/b1e09586858047a994038804e654b386` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.653016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc41a68-c71f-4d6a-91a8-c721a8253fba None admin] RBAC: Authorizing `identity:get_group(group_id=b1e09586858047a994038804e654b386)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:22.655106 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc41a68-c71f-4d6a-91a8-c721a8253fba None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:22.656953 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 191/381] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:22 2022] GET /identity/v3/groups/b1e09586858047a994038804e654b386 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:22.664614 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc41a68-c71f-4d6a-91a8-c721a8253fba None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:22.689406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:22.781007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:22.843164 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fd4f5-56ea-4a93-9051-88242ed89933 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:22.862970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc41a68-c71f-4d6a-91a8-c721a8253fba None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:22.884607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f628fe-b6b5-482f-834c-2f3599fa0386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:22.885731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f628fe-b6b5-482f-834c-2f3599fa0386 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.886030 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f628fe-b6b5-482f-834c-2f3599fa0386 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.886347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f628fe-b6b5-482f-834c-2f3599fa0386 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.901007 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f628fe-b6b5-482f-834c-2f3599fa0386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-26f628fe-b6b5-482f-834c-2f3599fa0386 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:22.907586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:22.908654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:22.909586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:22.910324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96af924-6fda-4b08-bd6c-faf1b40d5ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:22.910324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96af924-6fda-4b08-bd6c-faf1b40d5ec7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.910324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96af924-6fda-4b08-bd6c-faf1b40d5ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.910572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 191/382] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:22.910653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96af924-6fda-4b08-bd6c-faf1b40d5ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.910653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96af924-6fda-4b08-bd6c-faf1b40d5ec7 None admin] RBAC: Adding query filter params (group.id=b1e09586858047a994038804e654b386, role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=a32e32f35d6b4f64983733c445917938) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:22.910795 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354d628-16d1-44e4-9e37-01aed5fdb8bc None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.910961 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96af924-6fda-4b08-bd6c-faf1b40d5ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:22.911046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354d628-16d1-44e4-9e37-01aed5fdb8bc None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.911046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354d628-16d1-44e4-9e37-01aed5fdb8bc None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.912240 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96af924-6fda-4b08-bd6c-faf1b40d5ec7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:22.912442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 192/383] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:22.917410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354d628-16d1-44e4-9e37-01aed5fdb8bc None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:22.919596 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 192/384] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:22 2022] GET /identity/v3/role_assignments?group.id=b1e09586858047a994038804e654b386&scope.project.id=a32e32f35d6b4f64983733c445917938&role.id=af958d22229a4a0297bbe2e643f55181 => generated 260 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:22.920921 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8959cf-e470-446e-9cdc-f678a5661a84 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:22.921206 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8959cf-e470-446e-9cdc-f678a5661a84 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:22.921481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8959cf-e470-446e-9cdc-f678a5661a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:22.942838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-ac8959cf-e470-446e-9cdc-f678a5661a84 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:22.944073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-ac8959cf-e470-446e-9cdc-f678a5661a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:23.025135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 193/385] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:23.062999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8959cf-e470-446e-9cdc-f678a5661a84 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:23.138541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:23.143734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:23.188028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354d628-16d1-44e4-9e37-01aed5fdb8bc None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:23.226004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8959cf-e470-446e-9cdc-f678a5661a84 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:23.253906 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:23.254583 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:23.254866 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:23.255141 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:23.255879 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:23.256184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:23.257645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:23.270253 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 193/386] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:23.281204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:23.287927 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a518b-7034-4c53-90e2-4d8df8a1e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:23.288933 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a518b-7034-4c53-90e2-4d8df8a1e11b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:23.289403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a518b-7034-4c53-90e2-4d8df8a1e11b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:23.289862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a518b-7034-4c53-90e2-4d8df8a1e11b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:23.305827 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a518b-7034-4c53-90e2-4d8df8a1e11b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-af3a518b-7034-4c53-90e2-4d8df8a1e11b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:23.311988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:23.313160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:23.314157 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:23.317578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 194/387] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:23.335221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a518b-7034-4c53-90e2-4d8df8a1e11b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:23.428806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f12b8-c004-46a1-aebb-fb0d6f83656a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:23.471476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfccb5f-b348-4045-92ab-08f96b53cd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:23.472163 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfccb5f-b348-4045-92ab-08f96b53cd63 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:23.472444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfccb5f-b348-4045-92ab-08f96b53cd63 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:23.472726 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfccb5f-b348-4045-92ab-08f96b53cd63 None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:23.474144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfccb5f-b348-4045-92ab-08f96b53cd63 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:23.475807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfccb5f-b348-4045-92ab-08f96b53cd63 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:23.477983 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 194/388] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:23 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:23.488192 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfccb5f-b348-4045-92ab-08f96b53cd63 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:23.508763 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:23.543417 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a518b-7034-4c53-90e2-4d8df8a1e11b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:23.555158 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:23.628002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:23.628662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:23.628945 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:23.629221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:23.630043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:23.630385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:23.631846 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:23.653371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 195/389] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:23.658682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:23.665159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfccb5f-b348-4045-92ab-08f96b53cd63 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:23.711128 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11c27b-332b-46c5-a710-76b8c16c2f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:23.711878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11c27b-332b-46c5-a710-76b8c16c2f36 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:23.712106 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11c27b-332b-46c5-a710-76b8c16c2f36 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:23.712386 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11c27b-332b-46c5-a710-76b8c16c2f36 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:23.714302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11c27b-332b-46c5-a710-76b8c16c2f36 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:23.716192 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11c27b-332b-46c5-a710-76b8c16c2f36 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:23.718626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 195/390] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:23 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:23.724464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11c27b-332b-46c5-a710-76b8c16c2f36 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:23.836348 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:23.856848 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11c27b-332b-46c5-a710-76b8c16c2f36 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:23.896369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1cc9-0e31-43d5-8ccc-d5e89c6e950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:23.897079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1cc9-0e31-43d5-8ccc-d5e89c6e950b None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:23.897364 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1cc9-0e31-43d5-8ccc-d5e89c6e950b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:23.897645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1cc9-0e31-43d5-8ccc-d5e89c6e950b None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/groups/9c3e993c8ffd471588066c5d91fa9d34/roles/16a8547072ea431abbe07aad19a1fd43` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:23.900459 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c1cc9-0e31-43d5-8ccc-d5e89c6e950b None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc405d3f9e54cbf86ac4ca906b7637e, group_id=9c3e993c8ffd471588066c5d91fa9d34, role_id=16a8547072ea431abbe07aad19a1fd43)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:23.901947 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c1cc9-0e31-43d5-8ccc-d5e89c6e950b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:23.906427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58416d4-5963-4c09-bd93-48457e2527c6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:23.931645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 196/391] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:21:23 2022] PUT /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/groups/9c3e993c8ffd471588066c5d91fa9d34/roles/16a8547072ea431abbe07aad19a1fd43 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:23.959086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36a884-52f7-447a-bd23-d40b13885645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:23.959793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36a884-52f7-447a-bd23-d40b13885645 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:23.960086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36a884-52f7-447a-bd23-d40b13885645 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:23.960390 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36a884-52f7-447a-bd23-d40b13885645 None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:23.961796 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36a884-52f7-447a-bd23-d40b13885645 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:23.963515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36a884-52f7-447a-bd23-d40b13885645 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:23.965799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 196/392] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:23 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:23.972102 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1cc9-0e31-43d5-8ccc-d5e89c6e950b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:24.118675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1cc9-0e31-43d5-8ccc-d5e89c6e950b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:24.171097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc9216-3794-4871-a9d5-de181fd52d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:24.171905 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc9216-3794-4871-a9d5-de181fd52d59 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:24.172242 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc9216-3794-4871-a9d5-de181fd52d59 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:24.172544 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc9216-3794-4871-a9d5-de181fd52d59 None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:24.173991 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc9216-3794-4871-a9d5-de181fd52d59 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:24.175754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc9216-3794-4871-a9d5-de181fd52d59 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:24.177956 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 197/393] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:23 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:24.184007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36a884-52f7-447a-bd23-d40b13885645 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:24.246561 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:24.351064 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36a884-52f7-447a-bd23-d40b13885645 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:24.399385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d58810-f93f-4437-b3cf-bb86c6bf787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:24.400088 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58810-f93f-4437-b3cf-bb86c6bf787d None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:24.400426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58810-f93f-4437-b3cf-bb86c6bf787d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:24.400710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58810-f93f-4437-b3cf-bb86c6bf787d None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/groups/9c3e993c8ffd471588066c5d91fa9d34/roles/6b79a8b00a964a8fbcbaaa8371e71edb` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:24.403025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d58810-f93f-4437-b3cf-bb86c6bf787d None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc405d3f9e54cbf86ac4ca906b7637e, group_id=9c3e993c8ffd471588066c5d91fa9d34, role_id=6b79a8b00a964a8fbcbaaa8371e71edb)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:24.404578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d58810-f93f-4437-b3cf-bb86c6bf787d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:24.422031 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 197/394] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:21:24 2022] PUT /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e/groups/9c3e993c8ffd471588066c5d91fa9d34/roles/6b79a8b00a964a8fbcbaaa8371e71edb => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:24.746788 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ee11-4f69-44f0-8170-3649d925d6da None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:24.747491 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ee11-4f69-44f0-8170-3649d925d6da None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:24.747491 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ee11-4f69-44f0-8170-3649d925d6da None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:24.749165 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 198/395] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:24.755678 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213908a-5368-4502-b753-310716916007 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:24.756087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213908a-5368-4502-b753-310716916007 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:24.756466 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213908a-5368-4502-b753-310716916007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:24.776159 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-8213908a-5368-4502-b753-310716916007 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:24.777373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-8213908a-5368-4502-b753-310716916007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:24.850634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 198/396] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:24.872551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326ee11-4f69-44f0-8170-3649d925d6da None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:24.888768 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:24.997575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326ee11-4f69-44f0-8170-3649d925d6da None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:25.045256 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1849b9a-ef65-469c-90ed-4631494e9492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:25.045964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849b9a-ef65-469c-90ed-4631494e9492 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.046314 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849b9a-ef65-469c-90ed-4631494e9492 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.046551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849b9a-ef65-469c-90ed-4631494e9492 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.064280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1849b9a-ef65-469c-90ed-4631494e9492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-c1849b9a-ef65-469c-90ed-4631494e9492 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:25.070284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:25.071340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:25.072269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 199/397] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:25.077958 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213908a-5368-4502-b753-310716916007 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:25.106695 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:25.192548 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213908a-5368-4502-b753-310716916007 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:25.230994 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e61f99-dcc8-4980-87bc-7f8f1b0ff8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:25.231672 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e61f99-dcc8-4980-87bc-7f8f1b0ff8fa None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.231959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e61f99-dcc8-4980-87bc-7f8f1b0ff8fa None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.232228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e61f99-dcc8-4980-87bc-7f8f1b0ff8fa None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.232964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e61f99-dcc8-4980-87bc-7f8f1b0ff8fa None admin] RBAC: Adding query filter params (name=member) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:25.233266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e61f99-dcc8-4980-87bc-7f8f1b0ff8fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:25.234925 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e61f99-dcc8-4980-87bc-7f8f1b0ff8fa None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:25.250250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 199/398] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:25.255779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1849b9a-ef65-469c-90ed-4631494e9492 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:25.342636 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:25.385869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c09a1e-9e24-4268-9b60-63b63dd8c412 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.386267 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c09a1e-9e24-4268-9b60-63b63dd8c412 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.386459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c09a1e-9e24-4268-9b60-63b63dd8c412 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.388166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 200/399] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:25.388302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1849b9a-ef65-469c-90ed-4631494e9492 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:25.393971 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d122a0-2861-488d-aeaf-dc85d69b68e6 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.394303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d122a0-2861-488d-aeaf-dc85d69b68e6 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.394596 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d122a0-2861-488d-aeaf-dc85d69b68e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.429396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f27a3-bf8a-4227-9ea8-bd89f606ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:25.430080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f27a3-bf8a-4227-9ea8-bd89f606ade2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.430408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f27a3-bf8a-4227-9ea8-bd89f606ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.430695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f27a3-bf8a-4227-9ea8-bd89f606ade2 None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.431903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f27a3-bf8a-4227-9ea8-bd89f606ade2 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:25.433399 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f27a3-bf8a-4227-9ea8-bd89f606ade2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:25.435564 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 200/400] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:25 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:25.440940 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f27a3-bf8a-4227-9ea8-bd89f606ade2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:25.441131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-b8d122a0-2861-488d-aeaf-dc85d69b68e6 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:25.442623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-b8d122a0-2861-488d-aeaf-dc85d69b68e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:25.576636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 201/401] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:25.604033 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f27a3-bf8a-4227-9ea8-bd89f606ade2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:25.619299 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d122a0-2861-488d-aeaf-dc85d69b68e6 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:25.642499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765fe38-5cf6-49cd-939c-f1f739045715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:25.642849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765fe38-5cf6-49cd-939c-f1f739045715 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.643061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765fe38-5cf6-49cd-939c-f1f739045715 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.643343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765fe38-5cf6-49cd-939c-f1f739045715 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.644547 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765fe38-5cf6-49cd-939c-f1f739045715 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:25.646021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765fe38-5cf6-49cd-939c-f1f739045715 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:25.648191 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 201/402] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:25 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:25.653639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765fe38-5cf6-49cd-939c-f1f739045715 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:25.666131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:25.666468 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:25.768139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765fe38-5cf6-49cd-939c-f1f739045715 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:25.811575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3ed4d-026f-442a-9392-d960082ffe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:25.812461 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3ed4d-026f-442a-9392-d960082ffe6e None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.812748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3ed4d-026f-442a-9392-d960082ffe6e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.813416 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3ed4d-026f-442a-9392-d960082ffe6e None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/groups/9c3e993c8ffd471588066c5d91fa9d34/roles/16a8547072ea431abbe07aad19a1fd43` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.815958 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3ed4d-026f-442a-9392-d960082ffe6e None admin] RBAC: Authorizing `identity:create_grant(project_id=72fb605a1a2c4ec29bdc2d9daa734915, group_id=9c3e993c8ffd471588066c5d91fa9d34, role_id=16a8547072ea431abbe07aad19a1fd43)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:25.818136 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3ed4d-026f-442a-9392-d960082ffe6e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:25.834186 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d122a0-2861-488d-aeaf-dc85d69b68e6 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:25.836556 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 202/403] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:21:25 2022] PUT /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/groups/9c3e993c8ffd471588066c5d91fa9d34/roles/16a8547072ea431abbe07aad19a1fd43 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:21:25.839082 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b847f-8ef1-4ea1-b333-400f92edf905 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.839375 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b847f-8ef1-4ea1-b333-400f92edf905 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.839706 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b847f-8ef1-4ea1-b333-400f92edf905 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.841574 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 203/404] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:25.861306 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acf8ea-a96d-47ee-ae6d-33a9cc1b462a None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.861688 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acf8ea-a96d-47ee-ae6d-33a9cc1b462a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.862037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acf8ea-a96d-47ee-ae6d-33a9cc1b462a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.905838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-21acf8ea-a96d-47ee-ae6d-33a9cc1b462a None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:25.909924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-21acf8ea-a96d-47ee-ae6d-33a9cc1b462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:25.925675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20d959-9639-4ece-8d48-a8d8c5c2df88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:25.926375 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20d959-9639-4ece-8d48-a8d8c5c2df88 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:25.926836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20d959-9639-4ece-8d48-a8d8c5c2df88 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:25.927400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20d959-9639-4ece-8d48-a8d8c5c2df88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:25.964917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20d959-9639-4ece-8d48-a8d8c5c2df88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-da20d959-9639-4ece-8d48-a8d8c5c2df88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:25.973897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:25.975204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:25.976320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 202/405] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:25.984715 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20d959-9639-4ece-8d48-a8d8c5c2df88 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:26.003117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:26.013748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 204/406] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:26.040687 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acf8ea-a96d-47ee-ae6d-33a9cc1b462a None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:26.130512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:26.144506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20d959-9639-4ece-8d48-a8d8c5c2df88 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:26.185357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acf8ea-a96d-47ee-ae6d-33a9cc1b462a None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:26.210274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:26.210923 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.211205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.211480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.212236 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:26.212554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:26.213963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:26.235226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 203/407] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:26.241887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:26.258426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e51ce-daac-4d23-b944-ec5ec0544ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:26.259101 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e51ce-daac-4d23-b944-ec5ec0544ae2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.259476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e51ce-daac-4d23-b944-ec5ec0544ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.259748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e51ce-daac-4d23-b944-ec5ec0544ae2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.282457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e51ce-daac-4d23-b944-ec5ec0544ae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-cf1e51ce-daac-4d23-b944-ec5ec0544ae2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:26.293753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:26.294969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:26.295918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:26.296641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 205/408] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:26.302291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e51ce-daac-4d23-b944-ec5ec0544ae2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:26.415514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:26.430385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8d8f5-8f96-44f0-b818-7ceff23e0e72 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:26.495948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e51ce-daac-4d23-b944-ec5ec0544ae2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:26.506216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf3d6-2290-49aa-a351-c78f3895cf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:26.506901 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf3d6-2290-49aa-a351-c78f3895cf82 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.507187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf3d6-2290-49aa-a351-c78f3895cf82 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.507469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf3d6-2290-49aa-a351-c78f3895cf82 None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.508670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cf3d6-2290-49aa-a351-c78f3895cf82 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:26.510174 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cf3d6-2290-49aa-a351-c78f3895cf82 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:26.512165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 204/409] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:26 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:26.517863 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf3d6-2290-49aa-a351-c78f3895cf82 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:26.540957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f8afd-b33a-4c98-b734-b3ba8d73ca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:26.541676 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f8afd-b33a-4c98-b734-b3ba8d73ca69 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.541962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f8afd-b33a-4c98-b734-b3ba8d73ca69 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.542304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f8afd-b33a-4c98-b734-b3ba8d73ca69 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.542898 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:26.543113 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f8afd-b33a-4c98-b734-b3ba8d73ca69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:26.543929 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f8afd-b33a-4c98-b734-b3ba8d73ca69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:26.545617 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f8afd-b33a-4c98-b734-b3ba8d73ca69 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:26.560828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 206/410] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:26.562487 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32693c6-02e7-4866-b100-ab235d6eb68e None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.562779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32693c6-02e7-4866-b100-ab235d6eb68e None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.563057 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32693c6-02e7-4866-b100-ab235d6eb68e None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.569518 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 207/411] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:26.570582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32693c6-02e7-4866-b100-ab235d6eb68e None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:26.689503 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf3d6-2290-49aa-a351-c78f3895cf82 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:26.728256 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32693c6-02e7-4866-b100-ab235d6eb68e None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:26.732119 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468ba51-a15f-41e5-ae1f-0b731a06e36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:26.733024 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468ba51-a15f-41e5-ae1f-0b731a06e36b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.733488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468ba51-a15f-41e5-ae1f-0b731a06e36b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.733926 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468ba51-a15f-41e5-ae1f-0b731a06e36b None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.735389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468ba51-a15f-41e5-ae1f-0b731a06e36b None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:26.737045 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468ba51-a15f-41e5-ae1f-0b731a06e36b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:26.740258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 205/412] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:26 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:26.742734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caa959-07c8-4520-8ced-a541accebc6a None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.743275 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caa959-07c8-4520-8ced-a541accebc6a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.743774 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caa959-07c8-4520-8ced-a541accebc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.750077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:26.767684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-f4caa959-07c8-4520-8ced-a541accebc6a None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:26.769392 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-f4caa959-07c8-4520-8ced-a541accebc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:26.773889 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec44ba8-6859-454b-b75d-d7ef0c1c0dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:26.774647 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44ba8-6859-454b-b75d-d7ef0c1c0dd2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.774927 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44ba8-6859-454b-b75d-d7ef0c1c0dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.775208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44ba8-6859-454b-b75d-d7ef0c1c0dd2 None admin] PATH_INFO: `/v3/groups/b1e09586858047a994038804e654b386` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.776505 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec44ba8-6859-454b-b75d-d7ef0c1c0dd2 None admin] RBAC: Authorizing `identity:get_group(group_id=b1e09586858047a994038804e654b386)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:26.777993 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec44ba8-6859-454b-b75d-d7ef0c1c0dd2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:26.781198 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 208/413] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:26 2022] GET /identity/v3/groups/b1e09586858047a994038804e654b386 => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:26.781198 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec44ba8-6859-454b-b75d-d7ef0c1c0dd2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:26.874280 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 206/414] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:26.875864 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caa959-07c8-4520-8ced-a541accebc6a None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:26.886562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:26.927382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec44ba8-6859-454b-b75d-d7ef0c1c0dd2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:26.969132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723b6b4-72de-4268-a9d1-f0cde41996e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:26.970269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723b6b4-72de-4268-a9d1-f0cde41996e7 None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:26.970768 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723b6b4-72de-4268-a9d1-f0cde41996e7 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:26.971266 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723b6b4-72de-4268-a9d1-f0cde41996e7 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/groups/9c3e993c8ffd471588066c5d91fa9d34/roles/6b79a8b00a964a8fbcbaaa8371e71edb` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:26.973777 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723b6b4-72de-4268-a9d1-f0cde41996e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=72fb605a1a2c4ec29bdc2d9daa734915, group_id=9c3e993c8ffd471588066c5d91fa9d34, role_id=6b79a8b00a964a8fbcbaaa8371e71edb)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:26.975533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723b6b4-72de-4268-a9d1-f0cde41996e7 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:26.991084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 209/415] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:21:26 2022] PUT /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915/groups/9c3e993c8ffd471588066c5d91fa9d34/roles/6b79a8b00a964a8fbcbaaa8371e71edb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:26.992431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723b6b4-72de-4268-a9d1-f0cde41996e7 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:27.005322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caa959-07c8-4520-8ced-a541accebc6a None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:27.046740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86351e-3a7c-4eb6-8b19-9909892fde6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:27.047515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86351e-3a7c-4eb6-8b19-9909892fde6d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:27.047804 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86351e-3a7c-4eb6-8b19-9909892fde6d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:27.048081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86351e-3a7c-4eb6-8b19-9909892fde6d None admin] PATH_INFO: `/v3/projects/a32e32f35d6b4f64983733c445917938` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:27.049484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86351e-3a7c-4eb6-8b19-9909892fde6d None admin] RBAC: Authorizing `identity:get_project(project_id=a32e32f35d6b4f64983733c445917938)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:27.051132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86351e-3a7c-4eb6-8b19-9909892fde6d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:27.058267 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 207/416] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:26 2022] GET /identity/v3/projects/a32e32f35d6b4f64983733c445917938 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:27.059213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:27.060351 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86351e-3a7c-4eb6-8b19-9909892fde6d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:27.125904 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723b6b4-72de-4268-a9d1-f0cde41996e7 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:27.153655 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:27.170668 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1808730-6d16-4f4a-ab68-2bb81dcf7b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:27.171757 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1808730-6d16-4f4a-ab68-2bb81dcf7b6a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:27.172352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1808730-6d16-4f4a-ab68-2bb81dcf7b6a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:27.172936 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1808730-6d16-4f4a-ab68-2bb81dcf7b6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:27.183147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86351e-3a7c-4eb6-8b19-9909892fde6d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:27.186883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1808730-6d16-4f4a-ab68-2bb81dcf7b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-e1808730-6d16-4f4a-ab68-2bb81dcf7b6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:27.193631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:27.194836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:27.195794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:27.198920 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 210/417] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:27.209309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1808730-6d16-4f4a-ab68-2bb81dcf7b6a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:27.257449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96958b1-adfc-409f-8be7-0a5430c63e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:27.258254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96958b1-adfc-409f-8be7-0a5430c63e4c None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:27.258540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96958b1-adfc-409f-8be7-0a5430c63e4c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:27.258818 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96958b1-adfc-409f-8be7-0a5430c63e4c None admin] PATH_INFO: `/v3/projects/a32e32f35d6b4f64983733c445917938/groups/b1e09586858047a994038804e654b386/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:27.263878 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96958b1-adfc-409f-8be7-0a5430c63e4c None admin] RBAC: Authorizing `identity:create_grant(project_id=a32e32f35d6b4f64983733c445917938, group_id=b1e09586858047a994038804e654b386, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:27.265667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96958b1-adfc-409f-8be7-0a5430c63e4c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:27.298417 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 208/418] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 09:21:27 2022] PUT /identity/v3/projects/a32e32f35d6b4f64983733c445917938/groups/b1e09586858047a994038804e654b386/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:27.372443 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1808730-6d16-4f4a-ab68-2bb81dcf7b6a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:27.381769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:27.388908 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41cd0-561f-40d0-83df-5fdc0012bbd4 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:27.389632 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41cd0-561f-40d0-83df-5fdc0012bbd4 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:27.391673 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41cd0-561f-40d0-83df-5fdc0012bbd4 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:27.394954 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 209/419] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:27.399974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1c0c-69df-40a7-9c8e-288dfb6b97cb None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:27.400590 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1c0c-69df-40a7-9c8e-288dfb6b97cb None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:27.401106 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1c0c-69df-40a7-9c8e-288dfb6b97cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:27.414323 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:27.414992 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:27.415275 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:27.415556 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:27.416402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] RBAC: Adding query filter params (name=member) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:27.416800 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:27.418364 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:27.428849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 211/420] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:27.436913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:27.443147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-5b8b1c0c-69df-40a7-9c8e-288dfb6b97cb None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:27.444842 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-5b8b1c0c-69df-40a7-9c8e-288dfb6b97cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:27.523491 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 210/421] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:27.548665 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b1c0c-69df-40a7-9c8e-288dfb6b97cb None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:27.579055 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323284a-42fb-4d98-843a-881ed559793d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:27.622215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b22f6-18a5-471d-a98f-51960c98607c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:27.622215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b22f6-18a5-471d-a98f-51960c98607c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:27.622215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b22f6-18a5-471d-a98f-51960c98607c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:27.622215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b22f6-18a5-471d-a98f-51960c98607c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:27.669372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:27.680033 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:27.733105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b22f6-18a5-471d-a98f-51960c98607c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:27.734676 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b22f6-18a5-471d-a98f-51960c98607c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:27.762847 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b1c0c-69df-40a7-9c8e-288dfb6b97cb None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:27.811710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235c6e-c0d6-449e-929a-e95db9108d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:27.812302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235c6e-c0d6-449e-929a-e95db9108d6b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:27.812588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235c6e-c0d6-449e-929a-e95db9108d6b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:27.812946 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235c6e-c0d6-449e-929a-e95db9108d6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:27.831633 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70235c6e-c0d6-449e-929a-e95db9108d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-70235c6e-c0d6-449e-929a-e95db9108d6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:27.836760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:27.837786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:27.838770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:27.845086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 211/422] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:27.849351 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235c6e-c0d6-449e-929a-e95db9108d6b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:27.955516 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 212/423] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:27.962067 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b22f6-18a5-471d-a98f-51960c98607c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:28.048981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235c6e-c0d6-449e-929a-e95db9108d6b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.085056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:28.099661 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:28.117374 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d20a4-ff1a-481e-a71c-e00769af7245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:28.118049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d20a4-ff1a-481e-a71c-e00769af7245 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.118357 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d20a4-ff1a-481e-a71c-e00769af7245 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.118633 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d20a4-ff1a-481e-a71c-e00769af7245 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.119378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d20a4-ff1a-481e-a71c-e00769af7245 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:28.119687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d20a4-ff1a-481e-a71c-e00769af7245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:28.121084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d20a4-ff1a-481e-a71c-e00769af7245 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:28.138995 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 212/424] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:28.143135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047543b2-2c06-402a-a62e-5d4645411ee4 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.143438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047543b2-2c06-402a-a62e-5d4645411ee4 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.144540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047543b2-2c06-402a-a62e-5d4645411ee4 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.148592 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 213/425] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:28.149488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047543b2-2c06-402a-a62e-5d4645411ee4 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:28.155124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b22f6-18a5-471d-a98f-51960c98607c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.197117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25230d-f5d4-4786-838f-28c0e5f4142d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:28.197911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25230d-f5d4-4786-838f-28c0e5f4142d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.198231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25230d-f5d4-4786-838f-28c0e5f4142d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.198513 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25230d-f5d4-4786-838f-28c0e5f4142d None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.199935 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25230d-f5d4-4786-838f-28c0e5f4142d None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:28.201493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25230d-f5d4-4786-838f-28c0e5f4142d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:28.203715 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 213/426] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:27 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:28.205014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12f28e-bc0c-4a3a-9ce2-bb4d9c840ebe None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.205303 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12f28e-bc0c-4a3a-9ce2-bb4d9c840ebe None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.205583 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12f28e-bc0c-4a3a-9ce2-bb4d9c840ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-1f12f28e-bc0c-4a3a-9ce2-bb4d9c840ebe None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:28.232059 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-1f12f28e-bc0c-4a3a-9ce2-bb4d9c840ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:28.276362 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047543b2-2c06-402a-a62e-5d4645411ee4 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.322130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 214/427] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:28.322130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12f28e-bc0c-4a3a-9ce2-bb4d9c840ebe None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:28.324441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a46241-e697-462c-82ac-f2c3b1ae3215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:28.325404 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a46241-e697-462c-82ac-f2c3b1ae3215 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.325936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a46241-e697-462c-82ac-f2c3b1ae3215 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.326538 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a46241-e697-462c-82ac-f2c3b1ae3215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.387807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:28.407690 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:28.423022 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a46241-e697-462c-82ac-f2c3b1ae3215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:28.424983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a46241-e697-462c-82ac-f2c3b1ae3215 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:28.426608 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12f28e-bc0c-4a3a-9ce2-bb4d9c840ebe None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.463311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8138a-7843-4e6a-b17e-80a0e635269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:28.463894 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8138a-7843-4e6a-b17e-80a0e635269d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.464182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8138a-7843-4e6a-b17e-80a0e635269d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.464460 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8138a-7843-4e6a-b17e-80a0e635269d None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.465649 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8138a-7843-4e6a-b17e-80a0e635269d None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:28.467212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8138a-7843-4e6a-b17e-80a0e635269d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:28.469099 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 215/428] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:28 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:28.474609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8138a-7843-4e6a-b17e-80a0e635269d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:28.543727 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 214/429] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:28.545156 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a46241-e697-462c-82ac-f2c3b1ae3215 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:28.580708 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8138a-7843-4e6a-b17e-80a0e635269d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.617897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24aac1-f2ad-437e-aeac-2d1631147066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:28.618697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24aac1-f2ad-437e-aeac-2d1631147066 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.618992 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24aac1-f2ad-437e-aeac-2d1631147066 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.619269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24aac1-f2ad-437e-aeac-2d1631147066 None admin] PATH_INFO: `/v3/roles/member` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.622542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:28.634731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24aac1-f2ad-437e-aeac-2d1631147066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:28.637201 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-1f24aac1-f2ad-437e-aeac-2d1631147066 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:28.643681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:28.644762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 216/430] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:28.645705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24aac1-f2ad-437e-aeac-2d1631147066 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:28.666888 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a46241-e697-462c-82ac-f2c3b1ae3215 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.705841 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:28.707012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.707526 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.708027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.709151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] RBAC: Adding query filter params (group.id=9c3e993c8ffd471588066c5d91fa9d34, role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:28.709781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:28.711458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:28.722484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 215/431] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:28 2022] GET /identity/v3/role_assignments?group.id=9c3e993c8ffd471588066c5d91fa9d34&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:28.723942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:28.774137 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24aac1-f2ad-437e-aeac-2d1631147066 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.818132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44e5e-5841-4109-8cf5-d33fece1f7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:28.818132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44e5e-5841-4109-8cf5-d33fece1f7ea None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.818132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44e5e-5841-4109-8cf5-d33fece1f7ea None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.818132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44e5e-5841-4109-8cf5-d33fece1f7ea None admin] PATH_INFO: `/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.819265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44e5e-5841-4109-8cf5-d33fece1f7ea None admin] RBAC: Authorizing `identity:get_project(project_id=6fc405d3f9e54cbf86ac4ca906b7637e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:28.820734 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44e5e-5841-4109-8cf5-d33fece1f7ea None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:28.823540 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 217/432] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:28 2022] GET /identity/v3/projects/6fc405d3f9e54cbf86ac4ca906b7637e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:28.831889 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44e5e-5841-4109-8cf5-d33fece1f7ea None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:28.908581 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:28.954147 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44e5e-5841-4109-8cf5-d33fece1f7ea None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.996191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965762c-d8d7-4998-8640-7407976e0e22 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:28.996992 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57949910-2377-40b4-a2a1-cb55f7cd058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:28.997729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57949910-2377-40b4-a2a1-cb55f7cd058a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:28.998016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57949910-2377-40b4-a2a1-cb55f7cd058a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:28.998324 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57949910-2377-40b4-a2a1-cb55f7cd058a None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:28.999821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57949910-2377-40b4-a2a1-cb55f7cd058a None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:29.002361 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57949910-2377-40b4-a2a1-cb55f7cd058a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:29.010543 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 218/433] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:28 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:29.012951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57949910-2377-40b4-a2a1-cb55f7cd058a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.044344 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:29.044989 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.045273 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.045554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.047085 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] RBAC: Adding query filter params (name=member) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:29.047085 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:29.048034 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:29.061775 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 216/434] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:21:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:29.073449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.095401 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:29.159017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57949910-2377-40b4-a2a1-cb55f7cd058a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:29.202551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:29.214370 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d820-84e9-472d-9f0a-a29fb6538734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:29.215018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d820-84e9-472d-9f0a-a29fb6538734 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.215292 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d820-84e9-472d-9f0a-a29fb6538734 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.215571 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d820-84e9-472d-9f0a-a29fb6538734 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.216322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd22ac-64ac-4ac7-ab06-f8f6bed66471 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:29.216708 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d820-84e9-472d-9f0a-a29fb6538734 None admin] RBAC: Adding query filter params (group.id=9c3e993c8ffd471588066c5d91fa9d34, role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:29.217125 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d820-84e9-472d-9f0a-a29fb6538734 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:29.218818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d820-84e9-472d-9f0a-a29fb6538734 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:29.231604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 219/435] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/role_assignments?group.id=9c3e993c8ffd471588066c5d91fa9d34&scope.project.id=6fc405d3f9e54cbf86ac4ca906b7637e&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:29.262055 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd20dd-56f1-4f6f-8acc-400cdbb76a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:29.262785 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd20dd-56f1-4f6f-8acc-400cdbb76a15 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.263072 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd20dd-56f1-4f6f-8acc-400cdbb76a15 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.263350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd20dd-56f1-4f6f-8acc-400cdbb76a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.349440 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd20dd-56f1-4f6f-8acc-400cdbb76a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:29.351010 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd20dd-56f1-4f6f-8acc-400cdbb76a15 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:29.367481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:29.468234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 217/436] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:29.470226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903069bb-d6b5-4c20-b93d-5924e72c9f72 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.470520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903069bb-d6b5-4c20-b93d-5924e72c9f72 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.470802 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903069bb-d6b5-4c20-b93d-5924e72c9f72 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.474640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd20dd-56f1-4f6f-8acc-400cdbb76a15 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.474801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 220/437] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:21:29.479484 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72484f-60d5-46fc-809c-bba4b3593d63 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.479767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72484f-60d5-46fc-809c-bba4b3593d63 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.480035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72484f-60d5-46fc-809c-bba4b3593d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.502771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-4b72484f-60d5-46fc-809c-bba4b3593d63 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:29.504370 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-4b72484f-60d5-46fc-809c-bba4b3593d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:29.608171 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd20dd-56f1-4f6f-8acc-400cdbb76a15 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:29.616438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 221/438] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:29.620663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f31a-104b-4c8c-b82a-7d15404bd041 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.620960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f31a-104b-4c8c-b82a-7d15404bd041 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.621248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f31a-104b-4c8c-b82a-7d15404bd041 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.623040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 222/439] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:29.625180 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:29.628444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169fc43-930d-4c56-a6f7-667aa729657f None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.628730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169fc43-930d-4c56-a6f7-667aa729657f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.629004 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169fc43-930d-4c56-a6f7-667aa729657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.649673 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-f169fc43-930d-4c56-a6f7-667aa729657f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:29.650933 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-f169fc43-930d-4c56-a6f7-667aa729657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:29.652826 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a79d5e-8b34-4133-a573-f4497dee3d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:29.653548 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79d5e-8b34-4133-a573-f4497dee3d33 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.653835 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79d5e-8b34-4133-a573-f4497dee3d33 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.654144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79d5e-8b34-4133-a573-f4497dee3d33 None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.655470 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a79d5e-8b34-4133-a573-f4497dee3d33 None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:29.660712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a79d5e-8b34-4133-a573-f4497dee3d33 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:29.662156 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 218/440] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:29.662822 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a79d5e-8b34-4133-a573-f4497dee3d33 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.743034 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 223/441] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:29.743680 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169fc43-930d-4c56-a6f7-667aa729657f None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.784357 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a79d5e-8b34-4133-a573-f4497dee3d33 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:29.809966 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:29.831093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91b3f1-5b21-4e85-aacb-c80a3b404e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:29.832721 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91b3f1-5b21-4e85-aacb-c80a3b404e02 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.833041 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91b3f1-5b21-4e85-aacb-c80a3b404e02 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.833390 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91b3f1-5b21-4e85-aacb-c80a3b404e02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.846861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91b3f1-5b21-4e85-aacb-c80a3b404e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-4c91b3f1-5b21-4e85-aacb-c80a3b404e02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:29.852092 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:29.853219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:29.854207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 219/442] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:29.854983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91b3f1-5b21-4e85-aacb-c80a3b404e02 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.871520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169fc43-930d-4c56-a6f7-667aa729657f None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:29.900974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:29.909878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ac5d9-f3a8-44f5-b3a9-372fe0ddc9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:29.910609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac5d9-f3a8-44f5-b3a9-372fe0ddc9f0 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.910901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac5d9-f3a8-44f5-b3a9-372fe0ddc9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.911157 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac5d9-f3a8-44f5-b3a9-372fe0ddc9f0 None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.912408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ac5d9-f3a8-44f5-b3a9-372fe0ddc9f0 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:29.913907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ac5d9-f3a8-44f5-b3a9-372fe0ddc9f0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:29.915677 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 224/443] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:29.916498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ac5d9-f3a8-44f5-b3a9-372fe0ddc9f0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.972217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91b3f1-5b21-4e85-aacb-c80a3b404e02 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.014346 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27504d4d-0686-424f-bb0f-009fba351d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.014924 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27504d4d-0686-424f-bb0f-009fba351d5d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.015214 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27504d4d-0686-424f-bb0f-009fba351d5d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.015493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27504d4d-0686-424f-bb0f-009fba351d5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.024470 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ac5d9-f3a8-44f5-b3a9-372fe0ddc9f0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.026884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27504d4d-0686-424f-bb0f-009fba351d5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-27504d4d-0686-424f-bb0f-009fba351d5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:30.032438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:30.033855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 220/444] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:30.035182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27504d4d-0686-424f-bb0f-009fba351d5d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.058038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:30.074746 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.075495 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.075777 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.076060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.076859 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:30.077160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.078620 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.088333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 225/445] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:30.089002 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.152399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:30.155765 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27504d4d-0686-424f-bb0f-009fba351d5d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.191823 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c7642-680d-4027-8b79-36a504994088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.192454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c7642-680d-4027-8b79-36a504994088 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.192737 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c7642-680d-4027-8b79-36a504994088 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.193014 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c7642-680d-4027-8b79-36a504994088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.193278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f46347-9c47-4951-8cc5-561c1ec059b5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.193825 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c7642-680d-4027-8b79-36a504994088 None admin] RBAC: Adding query filter params (group.id=9c3e993c8ffd471588066c5d91fa9d34, role.id=16a8547072ea431abbe07aad19a1fd43, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:30.194166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c7642-680d-4027-8b79-36a504994088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.195640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c7642-680d-4027-8b79-36a504994088 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.201851 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 221/446] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/role_assignments?group.id=9c3e993c8ffd471588066c5d91fa9d34&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=16a8547072ea431abbe07aad19a1fd43 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:30.202495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c7642-680d-4027-8b79-36a504994088 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.247487 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.248139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.248428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.248710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.249471 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:30.249787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.251256 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.261150 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 226/447] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:30.267504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.328696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:30.332999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c7642-680d-4027-8b79-36a504994088 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.376035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78655a-169d-407b-a27c-39ae7e22db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.376749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78655a-169d-407b-a27c-39ae7e22db22 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.377068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78655a-169d-407b-a27c-39ae7e22db22 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.377427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78655a-169d-407b-a27c-39ae7e22db22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.394128 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3956759-2be7-4654-82a9-a60e69c4ab78 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.418341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:30.434828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c50ccc-2efa-40f8-ab21-ca449b3ee40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.435633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c50ccc-2efa-40f8-ab21-ca449b3ee40c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.435934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c50ccc-2efa-40f8-ab21-ca449b3ee40c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.436214 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c50ccc-2efa-40f8-ab21-ca449b3ee40c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.462191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78655a-169d-407b-a27c-39ae7e22db22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.463674 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78655a-169d-407b-a27c-39ae7e22db22 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.516709 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c50ccc-2efa-40f8-ab21-ca449b3ee40c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.519012 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c50ccc-2efa-40f8-ab21-ca449b3ee40c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.577376 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:30.587088 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 222/448] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:30.592659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78655a-169d-407b-a27c-39ae7e22db22 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.639351 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 227/449] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 09:21:30.645228 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c50ccc-2efa-40f8-ab21-ca449b3ee40c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.659358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:30.700579 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78655a-169d-407b-a27c-39ae7e22db22 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.737095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc61616-9771-4ace-bbcb-c4a0511e220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.737759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61616-9771-4ace-bbcb-c4a0511e220f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.738046 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61616-9771-4ace-bbcb-c4a0511e220f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.738344 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61616-9771-4ace-bbcb-c4a0511e220f None admin] PATH_INFO: `/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.739563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc61616-9771-4ace-bbcb-c4a0511e220f None admin] RBAC: Authorizing `identity:get_project(project_id=72fb605a1a2c4ec29bdc2d9daa734915)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.741049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc61616-9771-4ace-bbcb-c4a0511e220f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.742804 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 223/450] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/projects/72fb605a1a2c4ec29bdc2d9daa734915 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:30.747004 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c50ccc-2efa-40f8-ab21-ca449b3ee40c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.748528 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc61616-9771-4ace-bbcb-c4a0511e220f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.781502 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ce711-8d34-4561-be7e-f1707066ace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.782081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ce711-8d34-4561-be7e-f1707066ace5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.782441 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ce711-8d34-4561-be7e-f1707066ace5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.782727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ce711-8d34-4561-be7e-f1707066ace5 None admin] PATH_INFO: `/v3/projects/a32e32f35d6b4f64983733c445917938` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.783955 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ce711-8d34-4561-be7e-f1707066ace5 None admin] RBAC: Authorizing `identity:get_project(project_id=a32e32f35d6b4f64983733c445917938)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.785412 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ce711-8d34-4561-be7e-f1707066ace5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.787172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 228/451] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/projects/a32e32f35d6b4f64983733c445917938 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:30.792776 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ce711-8d34-4561-be7e-f1707066ace5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.808685 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:30.850115 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc61616-9771-4ace-bbcb-c4a0511e220f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.879859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:30.885709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c2b66-c620-4565-af3b-f27237b3f6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.886407 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2b66-c620-4565-af3b-f27237b3f6b6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.886697 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2b66-c620-4565-af3b-f27237b3f6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.886967 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2b66-c620-4565-af3b-f27237b3f6b6 None admin] PATH_INFO: `/v3/groups/9c3e993c8ffd471588066c5d91fa9d34` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.888206 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2b66-c620-4565-af3b-f27237b3f6b6 None admin] RBAC: Authorizing `identity:get_group(group_id=9c3e993c8ffd471588066c5d91fa9d34)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.889710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2b66-c620-4565-af3b-f27237b3f6b6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.891566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 224/452] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/groups/9c3e993c8ffd471588066c5d91fa9d34 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:30.893156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ce711-8d34-4561-be7e-f1707066ace5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:30.897295 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c2b66-c620-4565-af3b-f27237b3f6b6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:30.927658 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec730c-6c21-4cb5-a389-869764b94dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:30.928316 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec730c-6c21-4cb5-a389-869764b94dd5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:30.928598 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec730c-6c21-4cb5-a389-869764b94dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:30.928873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec730c-6c21-4cb5-a389-869764b94dd5 None admin] PATH_INFO: `/v3/groups/b1e09586858047a994038804e654b386` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:30.930052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec730c-6c21-4cb5-a389-869764b94dd5 None admin] RBAC: Authorizing `identity:get_group(group_id=b1e09586858047a994038804e654b386)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:30.931561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec730c-6c21-4cb5-a389-869764b94dd5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.933347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 229/453] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/groups/b1e09586858047a994038804e654b386 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:30.939344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec730c-6c21-4cb5-a389-869764b94dd5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:31.005370 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c2b66-c620-4565-af3b-f27237b3f6b6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:31.025681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:31.044740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166249e-74d0-41b7-b479-255daa6adcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:31.045293 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166249e-74d0-41b7-b479-255daa6adcbd None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:31.045584 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166249e-74d0-41b7-b479-255daa6adcbd None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:31.045691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec730c-6c21-4cb5-a389-869764b94dd5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:31.045860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166249e-74d0-41b7-b479-255daa6adcbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:31.046712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166249e-74d0-41b7-b479-255daa6adcbd None admin] RBAC: Adding query filter params (group.id=9c3e993c8ffd471588066c5d91fa9d34, role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:31.047021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166249e-74d0-41b7-b479-255daa6adcbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:31.048504 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166249e-74d0-41b7-b479-255daa6adcbd None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:31.054685 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 225/454] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/role_assignments?group.id=9c3e993c8ffd471588066c5d91fa9d34&scope.project.id=72fb605a1a2c4ec29bdc2d9daa734915&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:31.083456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfc76a-cf88-4760-9787-318a4fc20158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:31.084109 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfc76a-cf88-4760-9787-318a4fc20158 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:31.084398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfc76a-cf88-4760-9787-318a4fc20158 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:31.084674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfc76a-cf88-4760-9787-318a4fc20158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:31.085500 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfc76a-cf88-4760-9787-318a4fc20158 None admin] RBAC: Adding query filter params (group.id=b1e09586858047a994038804e654b386, role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=a32e32f35d6b4f64983733c445917938) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:31.085814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfc76a-cf88-4760-9787-318a4fc20158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:31.087335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfc76a-cf88-4760-9787-318a4fc20158 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:31.094565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 230/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 09:21:30 2022] GET /identity/v3/role_assignments?group.id=b1e09586858047a994038804e654b386&scope.project.id=a32e32f35d6b4f64983733c445917938&role.id=af958d22229a4a0297bbe2e643f55181 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:33.608376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae344-c4e0-41d2-b2cb-3c32a45fbea8 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:33.609431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae344-c4e0-41d2-b2cb-3c32a45fbea8 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:33.609942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae344-c4e0-41d2-b2cb-3c32a45fbea8 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:33.611639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 226/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:33.628396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ec808-1b4c-4f3d-bf88-091d843d1791 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:33.629395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ec808-1b4c-4f3d-bf88-091d843d1791 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:33.629884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ec808-1b4c-4f3d-bf88-091d843d1791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:33.658443 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-908ec808-1b4c-4f3d-bf88-091d843d1791 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:33.660190 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-908ec808-1b4c-4f3d-bf88-091d843d1791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:33.763154 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 231/457] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:33.804578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ae344-c4e0-41d2-b2cb-3c32a45fbea8 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:33.892156 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:33.936541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ae344-c4e0-41d2-b2cb-3c32a45fbea8 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:33.979210 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196215d-7600-4ed3-bc50-a3be99a2fd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:33.979882 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196215d-7600-4ed3-bc50-a3be99a2fd8c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:33.980166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196215d-7600-4ed3-bc50-a3be99a2fd8c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:33.980446 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196215d-7600-4ed3-bc50-a3be99a2fd8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:33.994353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196215d-7600-4ed3-bc50-a3be99a2fd8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:33.995839 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196215d-7600-4ed3-bc50-a3be99a2fd8c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-3196215d-7600-4ed3-bc50-a3be99a2fd8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:34.014429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:34.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:34.016547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:34.016547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:34.016547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:34.016547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:34.020641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 227/458] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:34.022218 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ec808-1b4c-4f3d-bf88-091d843d1791 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:34.174446 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1699e2-afc1-4751-a14c-81081318762a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.174644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1699e2-afc1-4751-a14c-81081318762a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.174920 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1699e2-afc1-4751-a14c-81081318762a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.177340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:34.178222 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 228/459] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:34.184188 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba758d3-8f4e-4f7c-9de4-29b305af2320 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.184469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba758d3-8f4e-4f7c-9de4-29b305af2320 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.184814 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba758d3-8f4e-4f7c-9de4-29b305af2320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.204273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ec808-1b4c-4f3d-bf88-091d843d1791 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:34.227220 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-cba758d3-8f4e-4f7c-9de4-29b305af2320 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:34.228479 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-cba758d3-8f4e-4f7c-9de4-29b305af2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:34.252979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:34.254000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.254518 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.255046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.256027 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:34.256558 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:34.258395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:34.276302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 232/460] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:34.285833 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:34.346133 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 229/461] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:34.347596 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74173d4b-ce42-44de-89f0-f5b37baa53a4 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.347884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74173d4b-ce42-44de-89f0-f5b37baa53a4 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.348160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74173d4b-ce42-44de-89f0-f5b37baa53a4 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.354002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 230/462] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:34.355418 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890735e4-ab79-4f5c-9249-a5a87c2a2d4f None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.355702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890735e4-ab79-4f5c-9249-a5a87c2a2d4f None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.355977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890735e4-ab79-4f5c-9249-a5a87c2a2d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.376710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-890735e4-ab79-4f5c-9249-a5a87c2a2d4f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:34.377998 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-890735e4-ab79-4f5c-9249-a5a87c2a2d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:34.420036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413c8e0-02f9-4cae-9bab-53f2823355a4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:34.439887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:34.464704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41f93d-1a8f-40a5-bbdf-da7d99819c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:34.465397 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41f93d-1a8f-40a5-bbdf-da7d99819c93 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.465682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41f93d-1a8f-40a5-bbdf-da7d99819c93 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.466076 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41f93d-1a8f-40a5-bbdf-da7d99819c93 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.466804 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41f93d-1a8f-40a5-bbdf-da7d99819c93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:34.468275 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41f93d-1a8f-40a5-bbdf-da7d99819c93 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:34.480267 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:34.489292 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 231/463] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:34.495166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890735e4-ab79-4f5c-9249-a5a87c2a2d4f None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:34.496698 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 233/464] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:34 2022] POST /identity/v3/users => generated 246 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:34.498601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3b6d-cb7f-4248-a7b5-2840f214c390 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.499474 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3b6d-cb7f-4248-a7b5-2840f214c390 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.499980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3b6d-cb7f-4248-a7b5-2840f214c390 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.502089 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 234/465] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:34.507953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905533-6c79-4cfb-9457-c2a58224cec9 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.508585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905533-6c79-4cfb-9457-c2a58224cec9 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.509093 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905533-6c79-4cfb-9457-c2a58224cec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.542719 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-7a905533-6c79-4cfb-9457-c2a58224cec9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:34.545202 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-7a905533-6c79-4cfb-9457-c2a58224cec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:34.627763 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890735e4-ab79-4f5c-9249-a5a87c2a2d4f None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:34.646276 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 235/466] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:34.648062 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a905533-6c79-4cfb-9457-c2a58224cec9 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:34.685939 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:34.686683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.686975 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.687274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.700189 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:34.701926 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:34.722003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:34.723135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:34.724122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:34.724122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:34.724122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:34.724122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:34.724122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 232/467] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:34.725168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:34.761404 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:34.783791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a905533-6c79-4cfb-9457-c2a58224cec9 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:34.818513 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:34.833016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:34.834212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.834522 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.834809 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.848382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:34.850202 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:34.861331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a0791-8e01-4cfc-85c5-50b9109ef75b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:34.868312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:34.869584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:34.870622 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:34.870622 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:34.870622 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:34.870622 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:34.870622 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 236/468] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:34.870937 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:34.905465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:34.906258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:34.906547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:34.906874 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:34.921328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:34.922887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:34.939555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:34.940626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:34.941926 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:34.941926 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:34.941926 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:34.941926 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:34.941926 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 233/469] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:34.946152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:34.984707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7adba2-a273-4454-9a4b-9f66308ad61b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:35.020530 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:35.021268 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.021633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.021985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.022805 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:35.023192 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:35.024653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:35.035881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:35.038054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 237/470] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:35.039121 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:35.060393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a63e1-2f32-4b3d-a87e-97cf77308c50 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:35.082851 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:35.101615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:35.102783 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.103106 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.103550 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.104173 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:35.104480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:35.105876 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:35.120268 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 234/471] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:35.121130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:35.165993 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f911a59-0dec-478d-93ea-3e3e138b4516 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:35.203224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f147beb-149d-479d-a779-c719cc36fb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:35.203857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f147beb-149d-479d-a779-c719cc36fb5c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.204156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f147beb-149d-479d-a779-c719cc36fb5c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.204444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f147beb-149d-479d-a779-c719cc36fb5c None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.205144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f147beb-149d-479d-a779-c719cc36fb5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:35.205460 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f147beb-149d-479d-a779-c719cc36fb5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:35.206893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f147beb-149d-479d-a779-c719cc36fb5c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:35.218986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 238/472] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:35.220299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de14ce-cf37-4105-b8e4-f52210d6f565 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.220592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de14ce-cf37-4105-b8e4-f52210d6f565 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.220875 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de14ce-cf37-4105-b8e4-f52210d6f565 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.222342 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 239/473] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:35.222855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de14ce-cf37-4105-b8e4-f52210d6f565 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:35.239451 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939c31c-f71f-4d77-8bd2-77a885649c82 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:35.274434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77be50e-4745-4124-81ac-399afaf7947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:35.275084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77be50e-4745-4124-81ac-399afaf7947d None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.275385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77be50e-4745-4124-81ac-399afaf7947d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.275666 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77be50e-4745-4124-81ac-399afaf7947d None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.276397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77be50e-4745-4124-81ac-399afaf7947d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:35.277899 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77be50e-4745-4124-81ac-399afaf7947d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:35.289530 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:35.302421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 235/474] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:35 2022] POST /identity/v3/users => generated 248 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:35.303606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77be50e-4745-4124-81ac-399afaf7947d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:35.346227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:35.355901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de14ce-cf37-4105-b8e4-f52210d6f565 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:35.397115 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827826a1-89c6-4474-ab95-a9b941d871d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:35.397849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827826a1-89c6-4474-ab95-a9b941d871d3 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.398167 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827826a1-89c6-4474-ab95-a9b941d871d3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.398451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827826a1-89c6-4474-ab95-a9b941d871d3 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.399227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827826a1-89c6-4474-ab95-a9b941d871d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:35.400774 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827826a1-89c6-4474-ab95-a9b941d871d3 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:35.434149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 240/475] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:35 2022] POST /identity/v3/users => generated 249 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:35.434149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac732-0181-4177-8f47-683b3fc551e9 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.434149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac732-0181-4177-8f47-683b3fc551e9 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.434149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac732-0181-4177-8f47-683b3fc551e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.468928 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77be50e-4745-4124-81ac-399afaf7947d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:35.471839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-973ac732-0181-4177-8f47-683b3fc551e9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:35.473228 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-973ac732-0181-4177-8f47-683b3fc551e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:35.528385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b616-66e3-48d8-8e2e-53fe3191427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:35.529084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b616-66e3-48d8-8e2e-53fe3191427c None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.529432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b616-66e3-48d8-8e2e-53fe3191427c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.529770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b616-66e3-48d8-8e2e-53fe3191427c None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.530607 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b616-66e3-48d8-8e2e-53fe3191427c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:35.532144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b616-66e3-48d8-8e2e-53fe3191427c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:35.561626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 236/476] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:35 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:35.585469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 241/477] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:35.610756 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b616-66e3-48d8-8e2e-53fe3191427c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:35.676808 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:35.738456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b616-66e3-48d8-8e2e-53fe3191427c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:35.778734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:35.779348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:35.779635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:35.779920 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:35.780669 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:35.782148 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:35.790542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.sql.core [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 09:21:35.790542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 09:21:35.790542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [parameters: {'id': '0600981472494743a3084f7c5381a512', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 09:21:35.790542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=474608) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application result.read() Jul 28 09:21:35.809060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:21:35.810983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application yield resource Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise self.value Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application session.commit() Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 09:21:35.813444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self.session.flush() Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application compat.raise_( Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 09:21:35.815816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:21:35.818988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application result.read() Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:21:35.821250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [parameters: {'id': '0600981472494743a3084f7c5381a512', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:35.825348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 09:21:35.827533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:35.829824 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 237/478] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:21:35 2022] POST /identity/v3/roles => generated 144 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:35.829987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac732-0181-4177-8f47-683b3fc551e9 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:35.938672 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:35.962430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac732-0181-4177-8f47-683b3fc551e9 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:36.000094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf5c74-1fd6-4208-997f-dcc200a69605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:36.000756 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf5c74-1fd6-4208-997f-dcc200a69605 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:36.001041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf5c74-1fd6-4208-997f-dcc200a69605 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:36.001318 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf5c74-1fd6-4208-997f-dcc200a69605 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:36.012472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf5c74-1fd6-4208-997f-dcc200a69605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-67cf5c74-1fd6-4208-997f-dcc200a69605 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:36.017740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:36.018824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:36.019753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 242/479] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:21:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:36.025210 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:36.157426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ddee-62f0-4fd1-b352-20dc6f308c46 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:36.174964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:36.194873 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8310609-0b43-4c98-89bf-ccf17a4e65e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:36.195561 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310609-0b43-4c98-89bf-ccf17a4e65e5 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:36.195851 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310609-0b43-4c98-89bf-ccf17a4e65e5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:36.196130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310609-0b43-4c98-89bf-ccf17a4e65e5 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:36.196876 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8310609-0b43-4c98-89bf-ccf17a4e65e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:36.197187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8310609-0b43-4c98-89bf-ccf17a4e65e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:36.198616 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8310609-0b43-4c98-89bf-ccf17a4e65e5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:36.207917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 238/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:36.956437 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c32e-1711-490b-ba7e-bf44b8314aab None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:36.957019 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c32e-1711-490b-ba7e-bf44b8314aab None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:36.957019 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c32e-1711-490b-ba7e-bf44b8314aab None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:36.958707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 243/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:36.964464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eecafa-3f97-4fa3-a7d5-58a2658d722b None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:36.964936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eecafa-3f97-4fa3-a7d5-58a2658d722b None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:36.965022 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eecafa-3f97-4fa3-a7d5-58a2658d722b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:36.984429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-c2eecafa-3f97-4fa3-a7d5-58a2658d722b None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:36.985646 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-c2eecafa-3f97-4fa3-a7d5-58a2658d722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:37.055779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 239/482] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:37.079097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c32e-1711-490b-ba7e-bf44b8314aab None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:37.212659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c32e-1711-490b-ba7e-bf44b8314aab None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:37.252918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494d33-3f05-4c68-a73b-08ec0dfaaa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:37.253577 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494d33-3f05-4c68-a73b-08ec0dfaaa60 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:37.253861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494d33-3f05-4c68-a73b-08ec0dfaaa60 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:37.254168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494d33-3f05-4c68-a73b-08ec0dfaaa60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:37.262749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:37.268196 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80494d33-3f05-4c68-a73b-08ec0dfaaa60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-80494d33-3f05-4c68-a73b-08ec0dfaaa60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:37.273336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:37.274428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:37.275350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 244/483] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:37.281012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eecafa-3f97-4fa3-a7d5-58a2658d722b None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:37.402306 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eecafa-3f97-4fa3-a7d5-58a2658d722b None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:37.412522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:37.438646 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:37.439286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:37.439570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:37.439848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:37.440588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:37.440898 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:37.442901 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:37.451947 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 240/484] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:37.457569 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494d33-3f05-4c68-a73b-08ec0dfaaa60 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:37.591894 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494d33-3f05-4c68-a73b-08ec0dfaaa60 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:37.645735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:37.646432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:37.646725 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:37.647010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:37.661238 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:37.662804 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:37.691534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:37.692637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:37.693604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:37.693604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:37.693604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:37.693604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:37.693604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 245/485] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:37.699508 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:37.864129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43017079-675d-4719-b99d-ceacfbfb6902 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:37.920859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21341a-3a30-4c8b-8d7e-7bb710aab851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:37.921919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21341a-3a30-4c8b-8d7e-7bb710aab851 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:37.922429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21341a-3a30-4c8b-8d7e-7bb710aab851 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:37.922907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21341a-3a30-4c8b-8d7e-7bb710aab851 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:37.923825 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21341a-3a30-4c8b-8d7e-7bb710aab851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:37.924344 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21341a-3a30-4c8b-8d7e-7bb710aab851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:37.926002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21341a-3a30-4c8b-8d7e-7bb710aab851 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:37.946378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 241/486] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:37.951994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:37.975479 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5960d6-f86f-4968-bcff-cf71cdb11816 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:37.976453 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5960d6-f86f-4968-bcff-cf71cdb11816 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:37.976953 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5960d6-f86f-4968-bcff-cf71cdb11816 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:37.978581 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 242/487] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:37.983708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1c97a-0348-46e2-90eb-519538377b17 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:37.984259 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1c97a-0348-46e2-90eb-519538377b17 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:37.984758 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1c97a-0348-46e2-90eb-519538377b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.018561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.020067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-5bc1c97a-0348-46e2-90eb-519538377b17 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:38.021931 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-5bc1c97a-0348-46e2-90eb-519538377b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:38.068663 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.111551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bff15c-6a02-45d5-adee-9efd4db3be9b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:38.152161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 243/488] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:38.175584 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1c97a-0348-46e2-90eb-519538377b17 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:38.178605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:38.180022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.180022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.180022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.192525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:38.192525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:38.204741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:38.206779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:38.208812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 246/489] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:21:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:38.214095 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:38.333408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1c97a-0348-46e2-90eb-519538377b17 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:38.355767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172904d-23e3-4356-b226-d35721d40dc2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:38.374058 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.374440 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb89048-b238-4606-a238-c05c28db5e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:38.374961 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb89048-b238-4606-a238-c05c28db5e03 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.375324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb89048-b238-4606-a238-c05c28db5e03 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.375647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb89048-b238-4606-a238-c05c28db5e03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.388735 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb89048-b238-4606-a238-c05c28db5e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-6fb89048-b238-4606-a238-c05c28db5e03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:38.394254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:38.397709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:38.401212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:38.410989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c3f98-61a7-4377-a5e1-c8a2f64964fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:38.410989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c3f98-61a7-4377-a5e1-c8a2f64964fb None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.410989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c3f98-61a7-4377-a5e1-c8a2f64964fb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.410989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c3f98-61a7-4377-a5e1-c8a2f64964fb None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.411308 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 244/490] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:38.411398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c3f98-61a7-4377-a5e1-c8a2f64964fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:38.411398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c3f98-61a7-4377-a5e1-c8a2f64964fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:38.411539 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd335-805c-42c6-8f3f-4d5793dcc962 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.411539 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd335-805c-42c6-8f3f-4d5793dcc962 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.411539 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd335-805c-42c6-8f3f-4d5793dcc962 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.411748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c3f98-61a7-4377-a5e1-c8a2f64964fb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:38.411836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 245/491] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:38.411836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cd335-805c-42c6-8f3f-4d5793dcc962 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:38.434780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.461162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 247/492] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:38.462897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee72946-dc9d-42b5-bfe8-b5e90d328fb9 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.463239 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee72946-dc9d-42b5-bfe8-b5e90d328fb9 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.463471 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee72946-dc9d-42b5-bfe8-b5e90d328fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.482464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-fee72946-dc9d-42b5-bfe8-b5e90d328fb9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:38.483698 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-fee72946-dc9d-42b5-bfe8-b5e90d328fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:38.538889 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cd335-805c-42c6-8f3f-4d5793dcc962 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:38.555138 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 248/493] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:38.556051 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee72946-dc9d-42b5-bfe8-b5e90d328fb9 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:38.601397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2c333-a092-438a-95b5-868fc64ccd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:38.602042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c333-a092-438a-95b5-868fc64ccd80 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.602362 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c333-a092-438a-95b5-868fc64ccd80 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.602650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c333-a092-438a-95b5-868fc64ccd80 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.603396 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2c333-a092-438a-95b5-868fc64ccd80 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:38.603707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2c333-a092-438a-95b5-868fc64ccd80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:38.605622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2c333-a092-438a-95b5-868fc64ccd80 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:38.616607 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 246/494] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:38.618714 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156852a-9d5c-4794-a390-0f6518075170 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.619253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156852a-9d5c-4794-a390-0f6518075170 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.619969 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156852a-9d5c-4794-a390-0f6518075170 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.623451 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 247/495] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:38.623451 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156852a-9d5c-4794-a390-0f6518075170 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:38.675074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.678432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee72946-dc9d-42b5-bfe8-b5e90d328fb9 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:38.714547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.717218 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c785b-abcc-44c6-abaf-2cf1bb79e87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:38.717917 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c785b-abcc-44c6-abaf-2cf1bb79e87f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.718307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c785b-abcc-44c6-abaf-2cf1bb79e87f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.718542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c785b-abcc-44c6-abaf-2cf1bb79e87f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.738695 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156852a-9d5c-4794-a390-0f6518075170 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:38.776877 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff63aa-c1d0-476f-9f95-6982e5425c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:38.777814 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff63aa-c1d0-476f-9f95-6982e5425c0c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.778453 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff63aa-c1d0-476f-9f95-6982e5425c0c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.778736 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff63aa-c1d0-476f-9f95-6982e5425c0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.790743 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff63aa-c1d0-476f-9f95-6982e5425c0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-91ff63aa-c1d0-476f-9f95-6982e5425c0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:38.796640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:38.797723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:38.799061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:38.799771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c785b-abcc-44c6-abaf-2cf1bb79e87f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:38.799771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c785b-abcc-44c6-abaf-2cf1bb79e87f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:38.803565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 248/496] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:38.804568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff63aa-c1d0-476f-9f95-6982e5425c0c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:38.898764 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.902961 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 249/497] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:38.904257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c96346-dfc7-46fc-a7f7-96dfaef63d1f None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.904554 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c96346-dfc7-46fc-a7f7-96dfaef63d1f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.904843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c96346-dfc7-46fc-a7f7-96dfaef63d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.910357 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff63aa-c1d0-476f-9f95-6982e5425c0c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:38.923070 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-a2c96346-dfc7-46fc-a7f7-96dfaef63d1f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:38.924592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-a2c96346-dfc7-46fc-a7f7-96dfaef63d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:38.947807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:38.948373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.948658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.948936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.960243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:38.961716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:38.963562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:38.977281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:38.979007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:38.980243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:38.980243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:38.980243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:38.980243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:38.980243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 249/498] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:38.980243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:38.996112 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 250/499] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:38.997469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67732d3f-6edf-4e93-a95c-8a7c049deecf None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.997764 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67732d3f-6edf-4e93-a95c-8a7c049deecf None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.998047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67732d3f-6edf-4e93-a95c-8a7c049deecf None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.999354 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 251/500] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:39.000051 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67732d3f-6edf-4e93-a95c-8a7c049deecf None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.088459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc31448-5d71-4ed3-8334-893d8af7261f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.104414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67732d3f-6edf-4e93-a95c-8a7c049deecf None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.131569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.132220 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.132489 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.132769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.133499 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:39.133810 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.135251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.139300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:39.143980 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 250/501] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:39.144801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.150194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597144af-53a9-4a3b-90a3-3f8284065d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.150853 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597144af-53a9-4a3b-90a3-3f8284065d07 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.151142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597144af-53a9-4a3b-90a3-3f8284065d07 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.151423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597144af-53a9-4a3b-90a3-3f8284065d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.161873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597144af-53a9-4a3b-90a3-3f8284065d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.163390 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597144af-53a9-4a3b-90a3-3f8284065d07 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-597144af-53a9-4a3b-90a3-3f8284065d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:39.178344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:39.179421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:39.180413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:39.180413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:39.180413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:39.180413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:39.180413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 252/502] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:39.180736 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6828-5774-4904-993c-8e6dc1fff941 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.181030 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6828-5774-4904-993c-8e6dc1fff941 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.181309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6828-5774-4904-993c-8e6dc1fff941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.199927 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-5c3b6828-5774-4904-993c-8e6dc1fff941 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:39.201118 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-5c3b6828-5774-4904-993c-8e6dc1fff941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:39.206491 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:39.252696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044d83-24ef-407f-8db6-07ce01acb244 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.272537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 253/503] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:39.273324 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b6828-5774-4904-993c-8e6dc1fff941 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.300437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.301070 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.301353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.301630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.302374 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:39.302685 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.304091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.316728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 251/504] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:39.317555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.382854 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b6828-5774-4904-993c-8e6dc1fff941 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.397073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:39.420381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c06eb-ecce-4358-8e9e-757e8c0f331d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.421055 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c06eb-ecce-4358-8e9e-757e8c0f331d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.421334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c06eb-ecce-4358-8e9e-757e8c0f331d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.421609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c06eb-ecce-4358-8e9e-757e8c0f331d None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.428570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5fc05-a264-46d7-b479-4aa50236c743 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.432472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c06eb-ecce-4358-8e9e-757e8c0f331d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-7c7c06eb-ecce-4358-8e9e-757e8c0f331d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:39.437565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:39.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:39.439626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 254/505] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:39.440404 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c06eb-ecce-4358-8e9e-757e8c0f331d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.448896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:39.468147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.468747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.469031 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.469312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.479703 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.481182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:39.496136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:39.497332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:39.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:39.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:39.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:39.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:39.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 252/506] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:39.498341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.547265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c06eb-ecce-4358-8e9e-757e8c0f331d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.586744 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.587372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.587663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.587944 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.588652 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:39.588959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.590370 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.602574 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 255/507] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:39.603419 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.605940 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c6c0-3d0a-457b-ab8c-435d805e72c8 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.636968 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:39.641561 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.642278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.642571 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.642854 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.655004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.656539 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:39.672905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:39.674230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:39.675358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:39.675358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:39.675358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:39.675358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:39.675358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 253/508] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:39.675358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.703268 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:39.721588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122ff69-2d49-4b02-b7aa-2b482d826c8f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.757165 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.757839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.758158 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.758430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.766672 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.768203 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:39.780561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:39.781732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 256/509] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:39.782837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.790733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8b921-8c8d-484c-9ef2-d4aa09d78f75 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.828090 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.828730 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.829016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.829322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.830064 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:39.830400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.831801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.840620 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 254/510] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:39.841455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.884441 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:39.894482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d6fb6-5e2d-443f-82d9-0aa17907c3cf None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.926694 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:39.931729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.932371 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.932661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.932945 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.933669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:39.933980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.935465 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:39.946491 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07745a-6f5e-4541-997f-551c0e086453 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:39.947907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 257/511] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:39.948823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:39.983495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:39.984070 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:39.984341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:39.984621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:39.991563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:39.993055 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:40.005881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:40.007185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 255/512] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:40.008445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.051284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e465bf1-aca1-4cd6-b873-15f4de4d7405 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.086759 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.087420 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.087703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.087987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.088695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:40.089006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.090415 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.101865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 258/513] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:40.102748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.109575 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17322065-2b46-4ab2-bcb0-3f95a582d436 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.117744 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:40.144342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.144969 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.145258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.145544 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.146409 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:40.146726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.148217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.181924 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:40.186065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 256/514] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:40.187216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.205170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10221-db98-4346-865c-abc327a268ac None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.242702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.243377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.243669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.243954 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.256176 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.257670 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:40.272669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:40.273899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:40.275248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:40.275248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:40.275248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:40.275248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:40.276798 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 259/515] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:40.277750 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.302710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bae8b3-292c-417e-a3e4-4d6ed743f199 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.339134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.339790 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.340072 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.340348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.348002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.349165 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:40.349504 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:40.363462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:40.365002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 257/516] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:40.366543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.397234 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19523f3c-9af9-4c74-b7c1-406d5a240cd5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.426912 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:40.433511 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.434173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.434519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.434918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.436277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:40.436814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.438809 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.459023 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 260/517] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:40.459739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.476912 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271fc85-65eb-4a94-8790-a0823a1cbe5b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.512436 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40206c9-56b5-4441-a5b4-e1ed77461f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.512962 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40206c9-56b5-4441-a5b4-e1ed77461f78 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.513246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40206c9-56b5-4441-a5b4-e1ed77461f78 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.513524 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40206c9-56b5-4441-a5b4-e1ed77461f78 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.514243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40206c9-56b5-4441-a5b4-e1ed77461f78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.515719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40206c9-56b5-4441-a5b4-e1ed77461f78 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.537197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 258/518] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:40 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:40.538038 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40206c9-56b5-4441-a5b4-e1ed77461f78 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.570152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6780e-ec9e-4c79-90b9-7743e0752039 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.602702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:40.616104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea9db4-4ef3-4395-91b8-9c1aac4222b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.616934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9db4-4ef3-4395-91b8-9c1aac4222b3 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.617058 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9db4-4ef3-4395-91b8-9c1aac4222b3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.617298 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9db4-4ef3-4395-91b8-9c1aac4222b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.660930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40206c9-56b5-4441-a5b4-e1ed77461f78 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.680701 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:40.703355 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea9db4-4ef3-4395-91b8-9c1aac4222b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.704855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea9db4-4ef3-4395-91b8-9c1aac4222b3 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.707081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.707809 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.708168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.708516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.709252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:40.709571 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.711110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.727225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 259/519] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:40.728378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.822241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 261/520] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:40.822241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea9db4-4ef3-4395-91b8-9c1aac4222b3 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.845159 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25faa0-5fe9-4c96-8f65-587ca311dbbb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.849475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:40.881454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.882080 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.882390 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.882672 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.883480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:40.883909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.885238 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.904314 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 260/521] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:40.906003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:40.930776 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea9db4-4ef3-4395-91b8-9c1aac4222b3 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:40.943079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:40.969395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:40.970207 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:40.970594 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:40.971004 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:40.971910 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=0970fc7ae258457ca70042acc0fc6c4c) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:40.972332 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:40.973904 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:40.990154 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 262/522] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/role_assignments?user.id=0970fc7ae258457ca70042acc0fc6c4c&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:40.990154 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.036814 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29eb87-78c9-421f-bd90-bf6254b87aa3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.088279 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.089418 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.089784 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.090284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.100323 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:41.102905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.104651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:41.118663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7952-6676-4a88-9cb7-364de1c9b13f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:41.120160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:41.121261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:41.122478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 261/523] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:41.123160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.170994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.172073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.172723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.173215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.185319 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.187425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:41.204108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:41.205180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:41.206219 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:41.206219 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:41.206219 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:41.206219 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:41.208529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 263/524] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:41.209579 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.234408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:41.241185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00d33-3cec-4636-8e2e-6ae55dfacedb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.277148 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971ad74-b362-4303-ab14-79583bd69ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.277840 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971ad74-b362-4303-ab14-79583bd69ce1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.278272 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971ad74-b362-4303-ab14-79583bd69ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.278956 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971ad74-b362-4303-ab14-79583bd69ce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.326778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a284e7-940e-4dec-b8b7-dab44dc8395f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.359118 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971ad74-b362-4303-ab14-79583bd69ce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.360677 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971ad74-b362-4303-ab14-79583bd69ce1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:41.368565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.369488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.369961 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.370425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.371401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:41.371906 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.373642 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:41.393895 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:41.394428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 264/525] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:21:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:41.405926 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.466810 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:41.476063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 262/526] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:41.477076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971ad74-b362-4303-ab14-79583bd69ce1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.539715 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed56a4-ff77-4088-8ee2-14123ba086ef None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.583075 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.583987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.584368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.584743 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.585553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:41.585957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.587480 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:41.603724 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 265/527] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:41.605069 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.611549 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971ad74-b362-4303-ab14-79583bd69ce1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.653096 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac12eb2-bf8f-4989-b199-ed35b85c7dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.653735 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac12eb2-bf8f-4989-b199-ed35b85c7dce None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.654078 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac12eb2-bf8f-4989-b199-ed35b85c7dce None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.654316 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac12eb2-bf8f-4989-b199-ed35b85c7dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.670018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:41.716411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574106-921a-4e13-a932-2fd70ad8e04c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.721628 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:41.730192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac12eb2-bf8f-4989-b199-ed35b85c7dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.732217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac12eb2-bf8f-4989-b199-ed35b85c7dce None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:41.756495 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.757629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.758135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.758658 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.770572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.772496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:41.787923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:41.788974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:41.789970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:41.789970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:41.789970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:41.789970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:41.792334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 266/528] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:41.793450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.845554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 263/529] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:41.846360 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac12eb2-bf8f-4989-b199-ed35b85c7dce None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.900291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d55be7-179a-49f4-b8d3-496967ea63a5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.911631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:41.937413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.938658 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.939153 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.939683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.943789 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:41.948814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.950926 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:41.951457 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac12eb2-bf8f-4989-b199-ed35b85c7dce None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:41.964693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:41.965839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:41.966897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:41.966897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:41.966897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:41.966897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:41.966897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:41.966897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:41.966897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:41.966897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:41.969626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 267/530] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:41.970781 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:41.987700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:41.988329 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:41.988610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:41.988886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:41.989711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:41.989909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:41.991312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.007798 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 264/531] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:42.008694 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.086645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ead3-88f2-4d06-8bb4-68a4eecbe124 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.120110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b290a-971a-49ac-8aa6-8efffb04900c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.124937 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645739b-5940-4141-967f-f319f31d223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.126212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645739b-5940-4141-967f-f319f31d223e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.126730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645739b-5940-4141-967f-f319f31d223e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.127242 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645739b-5940-4141-967f-f319f31d223e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.146152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645739b-5940-4141-967f-f319f31d223e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.146152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645739b-5940-4141-967f-f319f31d223e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.158906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.159460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.159740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.160013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.160787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:42.161094 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.162644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-1645739b-5940-4141-967f-f319f31d223e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:42.162850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:42.163821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:42.165366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:42.165366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:42.165366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:42.165366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:42.165366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 268/532] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:42.165366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645739b-5940-4141-967f-f319f31d223e None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.191885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 265/533] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:42.193283 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.195761 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:42.212043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:42.305852 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1763ca-9892-4226-a3e5-eb24dab0e149 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.308728 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645739b-5940-4141-967f-f319f31d223e None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.346205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.347224 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.347740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.348240 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.349168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:42.349706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.351438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.355271 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.356001 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.356413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.356752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.365473 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.365773 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 266/534] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:42.367338 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.371869 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:42.387572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:42.389043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:42.390225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:42.390225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:42.390225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:42.390225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:42.390225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:42.390225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:42.390225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:42.390225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:42.394714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 269/535] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:42.395604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.461934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:42.492822 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec672da-f96d-4abc-8761-9ce7fdab4241 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.497337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:42.519134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62223ff5-8fa4-4dd8-a717-3a862927ed71 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.529206 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.529966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.530371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.530761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.531780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=6174eb4be9314a51af64f926441a055d) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:42.532068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.533619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.540102 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 267/536] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/role_assignments?user.id=6174eb4be9314a51af64f926441a055d&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:42.541240 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.559823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.560782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.561257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.561718 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.570423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.572470 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:42.586094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:42.587187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:42.588174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:42.588174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:42.588174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:42.588174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:42.588174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:42.588174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:42.588174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:42.588174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:42.590903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 270/537] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:42.597012 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.664780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadb7ac-c1b6-4570-93d5-40783936ecf5 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.703521 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.704296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.704583 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.704984 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.705671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:42.705984 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.707567 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.716449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:42.721272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b649017-b115-4b6d-9bb6-eeb965561c1e None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.723087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 268/538] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:42.735347 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.795650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a0e15-d2a5-4474-b072-9c5f916d66fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.796373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a0e15-d2a5-4474-b072-9c5f916d66fb None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.796662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a0e15-d2a5-4474-b072-9c5f916d66fb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.796942 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a0e15-d2a5-4474-b072-9c5f916d66fb None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.798769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a0e15-d2a5-4474-b072-9c5f916d66fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:42.799201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a0e15-d2a5-4474-b072-9c5f916d66fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.800608 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a0e15-d2a5-4474-b072-9c5f916d66fb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.816548 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 271/539] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:42.818277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169ec7a-217c-4609-a187-76ea2ea265af None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.818496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169ec7a-217c-4609-a187-76ea2ea265af None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.818723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169ec7a-217c-4609-a187-76ea2ea265af None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.820426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 272/540] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:42.821218 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169ec7a-217c-4609-a187-76ea2ea265af None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:42.825555 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:42.885120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2d854-5284-45f2-95ab-773770a96f99 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:42.924255 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddeb76-5fd8-4291-a887-94b82a41906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:42.925303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddeb76-5fd8-4291-a887-94b82a41906f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.925817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddeb76-5fd8-4291-a887-94b82a41906f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.926378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddeb76-5fd8-4291-a887-94b82a41906f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.927514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ddeb76-5fd8-4291-a887-94b82a41906f None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=03519d64b78e4eaa9869a4e2c9e31413) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:42.928067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ddeb76-5fd8-4291-a887-94b82a41906f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:42.929746 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ddeb76-5fd8-4291-a887-94b82a41906f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:42.936955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 269/541] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/role_assignments?user.id=03519d64b78e4eaa9869a4e2c9e31413&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:42.938706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccb92-fbba-4032-a407-5654ed68701b None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:42.939234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccb92-fbba-4032-a407-5654ed68701b None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:42.939753 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccb92-fbba-4032-a407-5654ed68701b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:42.960881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-894ccb92-fbba-4032-a407-5654ed68701b None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:42.962683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-894ccb92-fbba-4032-a407-5654ed68701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:42.966034 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169ec7a-217c-4609-a187-76ea2ea265af None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:43.040967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbc4a8-49e0-4a83-9888-4225b89c5e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:43.042033 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbc4a8-49e0-4a83-9888-4225b89c5e1c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:43.042567 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbc4a8-49e0-4a83-9888-4225b89c5e1c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:43.043094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbc4a8-49e0-4a83-9888-4225b89c5e1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:43.044401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbc4a8-49e0-4a83-9888-4225b89c5e1c None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=629e1537906b4e1b9431af873d612a34) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:43.045014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbc4a8-49e0-4a83-9888-4225b89c5e1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:43.046761 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbc4a8-49e0-4a83-9888-4225b89c5e1c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:43.054900 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 273/542] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:42 2022] GET /identity/v3/role_assignments?user.id=629e1537906b4e1b9431af873d612a34&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:43.062512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:43.073973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c137c8-cfc0-4ec0-96e1-497b6995cb6f None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:43.074393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c137c8-cfc0-4ec0-96e1-497b6995cb6f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:43.074637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c137c8-cfc0-4ec0-96e1-497b6995cb6f None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:43.076381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 274/543] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:43.081038 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 270/544] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:43.084941 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc149e-65bc-4598-b917-eef0bdd9bf86 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:43.085607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc149e-65bc-4598-b917-eef0bdd9bf86 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:43.086594 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc149e-65bc-4598-b917-eef0bdd9bf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:43.107763 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-1fcc149e-65bc-4598-b917-eef0bdd9bf86 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:43.109284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-1fcc149e-65bc-4598-b917-eef0bdd9bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:43.112541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ccb92-fbba-4032-a407-5654ed68701b None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:43.191010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:43.206250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 275/545] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:43.241913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc149e-65bc-4598-b917-eef0bdd9bf86 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:43.310337 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ccb92-fbba-4032-a407-5654ed68701b None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:43.365298 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2ed30-2349-4604-8303-e012b1c54416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:43.368161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2ed30-2349-4604-8303-e012b1c54416 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:43.368868 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2ed30-2349-4604-8303-e012b1c54416 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:43.370730 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2ed30-2349-4604-8303-e012b1c54416 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:43.390823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc149e-65bc-4598-b917-eef0bdd9bf86 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:43.394324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2ed30-2349-4604-8303-e012b1c54416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-2ae2ed30-2349-4604-8303-e012b1c54416 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:43.405068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:43.406460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:43.407645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:43.412458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 271/546] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:43.420007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2ed30-2349-4604-8303-e012b1c54416 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:43.444421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f157954-4fbd-4a04-80cd-3eedea3ef6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:43.445526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f157954-4fbd-4a04-80cd-3eedea3ef6e5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:43.446030 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f157954-4fbd-4a04-80cd-3eedea3ef6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:43.446588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f157954-4fbd-4a04-80cd-3eedea3ef6e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:43.479215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f157954-4fbd-4a04-80cd-3eedea3ef6e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-3f157954-4fbd-4a04-80cd-3eedea3ef6e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:43.484871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:43.485965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:43.486962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:43.488897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:43.490540 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 276/547] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:43.497376 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f157954-4fbd-4a04-80cd-3eedea3ef6e5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:43.599715 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2ed30-2349-4604-8303-e012b1c54416 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:43.615196 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:43.682490 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:43.685640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:43.686470 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:43.686757 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:43.687690 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:43.688012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:43.689465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:43.701716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 272/548] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:43.706288 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f157954-4fbd-4a04-80cd-3eedea3ef6e5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:43.707500 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:43.793073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:43.793796 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:43.794087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:43.794403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:43.795169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:43.795464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:43.796878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:43.811655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 277/549] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:43.817208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:43.908386 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495a3f5-dad6-4f87-8088-824dbc33e23b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:43.944172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018ae73-b72f-4295-97d2-910dca0d9696 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:43.948782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:43.971464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:43.993538 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:43.994499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:43.994796 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:43.995079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:44.007800 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:44.009302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:44.016216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:44.016896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:44.017181 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:44.017461 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:44.036398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:44.037498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:44.038538 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:44.038538 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:44.038538 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:44.038538 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:44.038807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 278/550] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:44.044266 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:44.044527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:44.045882 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:44.080750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:44.083057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:44.085321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:44.085321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:44.085321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:44.085321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:44.090467 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 273/551] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:44.091513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:44.164963 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f73b4-3ef3-4c84-9b21-13c16f86f0ba None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:44.203439 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:44.204237 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:44.204458 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:44.204744 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:44.205565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:44.205885 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:44.207337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:44.214164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc76df3-f41e-4ffd-990a-fd7f244cc7d9 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:44.221598 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 279/552] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:44.226754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:44.260191 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:44.319512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:44.320240 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:44.320411 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:44.320697 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:44.321426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:44.321731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:44.323160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:44.347621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeead29-4dc3-46ab-b002-0f371ba9054b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:44.354623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 274/553] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:44.360903 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:44.388681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:44.389359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:44.389746 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:44.390029 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:44.398478 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:44.400102 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:44.414281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:44.416773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:21:44.419327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:44.422226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 280/554] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:21:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:44.422465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:44.424663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:44.544870 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b9f66-51da-4ba7-9c7f-3f5c58f2f499 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:44.562505 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:44.574415 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2d9f-441d-4d1e-a2ca-c24f0f594e25 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:44.620962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:44.621975 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:44.622377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:44.622727 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:44.622839 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:44.623402 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:44.623691 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:44.623808 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:44.623976 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:44.624079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:44.625656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:44.641937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:44.643476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:44.649860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 281/555] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:21:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:44.652131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:44.672128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:44.673241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:21:44.674486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:44.678368 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 275/556] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:44.679776 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:44.801939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b27e4-ac5b-4b39-ba50-04ce23d83b67 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:44.848024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:44.848711 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:44.848993 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:44.849266 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:44.860917 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:44.862466 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:44.878396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:44.879499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:44.880527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:44.880527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:44.880527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:44.880527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:44.882528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:44.884850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 282/557] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:44.886242 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:44.921710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:44.959437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65498f-c9fc-4b7d-9b42-5c74b2c673f1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:45.037647 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071ad1f-bb04-45d7-b832-b8dafdf075d2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:45.047510 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:45.048161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.048708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.048852 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.049686 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:45.050000 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:45.051510 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:45.080653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:45.081575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.082014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.082508 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.083392 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:45.083857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:45.085547 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:45.089084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 276/558] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:21:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:45.099228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:45.104767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 283/559] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:45.106481 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:45.175036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:45.228308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc971a-e5cb-4e31-a032-2d2c0c2c7054 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:45.236551 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c8303-d434-482c-863e-0c76744cda79 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:45.273995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bbd8b-28d3-4eef-899c-fac673454886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:45.275681 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bbd8b-28d3-4eef-899c-fac673454886 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.276128 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bbd8b-28d3-4eef-899c-fac673454886 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.276658 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bbd8b-28d3-4eef-899c-fac673454886 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.281681 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1c569-e918-44ef-885b-082a53443e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:45.282706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c569-e918-44ef-885b-082a53443e04 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.283144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c569-e918-44ef-885b-082a53443e04 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.283570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c569-e918-44ef-885b-082a53443e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.289091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bbd8b-28d3-4eef-899c-fac673454886 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:45.290786 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bbd8b-28d3-4eef-899c-fac673454886 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:45.302172 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-773bbd8b-28d3-4eef-899c-fac673454886 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:45.308442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:45.309884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:45.311952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:45.311952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:45.311952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:45.311952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:45.311952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:45.311952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:45.311952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:45.311952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:45.314265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 284/560] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:45.315939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9b48c-48e2-43e5-88ad-bf54af223c61 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.316764 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9b48c-48e2-43e5-88ad-bf54af223c61 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.317186 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9b48c-48e2-43e5-88ad-bf54af223c61 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.318973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 285/561] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:45.323622 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9b48c-48e2-43e5-88ad-bf54af223c61 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:45.393251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1c569-e918-44ef-885b-082a53443e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:45.397611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1c569-e918-44ef-885b-082a53443e04 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:45.476636 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9b48c-48e2-43e5-88ad-bf54af223c61 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:45.499499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:45.517562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378826f-0270-45f6-b2bd-de46e1c16fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:45.518233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378826f-0270-45f6-b2bd-de46e1c16fc9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.518516 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378826f-0270-45f6-b2bd-de46e1c16fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.518791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378826f-0270-45f6-b2bd-de46e1c16fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.519559 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378826f-0270-45f6-b2bd-de46e1c16fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:45.519987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378826f-0270-45f6-b2bd-de46e1c16fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:45.521332 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378826f-0270-45f6-b2bd-de46e1c16fc9 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:45.531385 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 277/562] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:45.532474 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568505b-1ab9-40d1-b095-69205402340b None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.532767 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568505b-1ab9-40d1-b095-69205402340b None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.533048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568505b-1ab9-40d1-b095-69205402340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.535926 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 286/563] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:45.537015 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e4f4-4e80-4efc-a63e-7ec31d7a4ca9 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.537309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e4f4-4e80-4efc-a63e-7ec31d7a4ca9 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.537590 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e4f4-4e80-4efc-a63e-7ec31d7a4ca9 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.539277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 287/564] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:45.540233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170f44-77d4-4f80-9714-833673d8e3c5 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.540526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170f44-77d4-4f80-9714-833673d8e3c5 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.540811 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170f44-77d4-4f80-9714-833673d8e3c5 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.542065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 288/565] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:45.542809 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50170f44-77d4-4f80-9714-833673d8e3c5 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:45.553044 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-3568505b-1ab9-40d1-b095-69205402340b None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:45.554328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-3568505b-1ab9-40d1-b095-69205402340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:45.607160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:45.637071 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 278/566] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:45.637981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568505b-1ab9-40d1-b095-69205402340b None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:45.670988 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50170f44-77d4-4f80-9714-833673d8e3c5 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:45.711491 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b4631-93e9-416d-8bbc-34e93ff1581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:45.712151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4631-93e9-416d-8bbc-34e93ff1581b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.712438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4631-93e9-416d-8bbc-34e93ff1581b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.712723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4631-93e9-416d-8bbc-34e93ff1581b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.724122 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b4631-93e9-416d-8bbc-34e93ff1581b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:45.725703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b4631-93e9-416d-8bbc-34e93ff1581b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-4a0b4631-93e9-416d-8bbc-34e93ff1581b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:45.741022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:45.742185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:45.743200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:45.743200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:45.743200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:45.743200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:45.743769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 289/567] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:45.745093 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc8b42-25f0-4b1e-a0fd-108debc07861 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.745344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc8b42-25f0-4b1e-a0fd-108debc07861 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.745619 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc8b42-25f0-4b1e-a0fd-108debc07861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.750077 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568505b-1ab9-40d1-b095-69205402340b None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:45.765016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-c5dc8b42-25f0-4b1e-a0fd-108debc07861 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:45.766290 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-c5dc8b42-25f0-4b1e-a0fd-108debc07861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:45.794689 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:45.797123 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e6ff0-24bc-4211-8730-0fb79082508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:45.797854 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6ff0-24bc-4211-8730-0fb79082508a None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.798148 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6ff0-24bc-4211-8730-0fb79082508a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.798441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6ff0-24bc-4211-8730-0fb79082508a None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/0970fc7ae258457ca70042acc0fc6c4c/roles/aec37946c06745cfb2ea73e9b3c9a547` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.815952 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e6ff0-24bc-4211-8730-0fb79082508a None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=0970fc7ae258457ca70042acc0fc6c4c, role_id=aec37946c06745cfb2ea73e9b3c9a547)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:45.817821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e6ff0-24bc-4211-8730-0fb79082508a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:45.834184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 279/568] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:45 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/0970fc7ae258457ca70042acc0fc6c4c/roles/aec37946c06745cfb2ea73e9b3c9a547 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:45.835712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ab4e4-21e0-4995-b7d4-bc71aa50ac82 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:45.836154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ab4e4-21e0-4995-b7d4-bc71aa50ac82 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:45.836442 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ab4e4-21e0-4995-b7d4-bc71aa50ac82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:45.845564 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 290/569] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:45.846345 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc8b42-25f0-4b1e-a0fd-108debc07861 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:45.870873 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-a89ab4e4-21e0-4995-b7d4-bc71aa50ac82 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:45.872504 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-a89ab4e4-21e0-4995-b7d4-bc71aa50ac82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:45.898781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:45.976662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc8b42-25f0-4b1e-a0fd-108debc07861 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:45.978027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 280/570] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:45.979154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ab4e4-21e0-4995-b7d4-bc71aa50ac82 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.035064 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05e32-dded-4be7-a77b-d78ff989c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.035646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05e32-dded-4be7-a77b-d78ff989c076 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.035928 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05e32-dded-4be7-a77b-d78ff989c076 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.036210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05e32-dded-4be7-a77b-d78ff989c076 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.048153 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e05e32-dded-4be7-a77b-d78ff989c076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-f3e05e32-dded-4be7-a77b-d78ff989c076 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:46.053349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:46.054463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:46.055435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 291/571] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:46.056210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05e32-dded-4be7-a77b-d78ff989c076 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.086097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:46.108600 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ab4e4-21e0-4995-b7d4-bc71aa50ac82 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.142482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.143111 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.143396 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.143668 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.144376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:46.144680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:46.146054 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:46.157823 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 281/572] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:46.158714 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.176768 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:46.178026 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05e32-dded-4be7-a77b-d78ff989c076 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.217060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beff476-5548-4eee-97a4-cdab9c7c3bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.218166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff476-5548-4eee-97a4-cdab9c7c3bbe None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.218601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff476-5548-4eee-97a4-cdab9c7c3bbe None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.219053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff476-5548-4eee-97a4-cdab9c7c3bbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.230943 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beff476-5548-4eee-97a4-cdab9c7c3bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-3beff476-5548-4eee-97a4-cdab9c7c3bbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:46.236440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:46.237541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 292/573] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:46.238653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beff476-5548-4eee-97a4-cdab9c7c3bbe None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.271979 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69606780-2505-48fe-b5b8-e08ed0cd7b26 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.309870 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503e48b-ca8e-498e-81de-1977cd6c10fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.310951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e48b-ca8e-498e-81de-1977cd6c10fc None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.311483 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e48b-ca8e-498e-81de-1977cd6c10fc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.311761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e48b-ca8e-498e-81de-1977cd6c10fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.325360 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503e48b-ca8e-498e-81de-1977cd6c10fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-2503e48b-ca8e-498e-81de-1977cd6c10fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:46.332761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:46.333881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:46.334856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:46.335582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:46.335675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 282/574] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:46.336303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503e48b-ca8e-498e-81de-1977cd6c10fc None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.362016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beff476-5548-4eee-97a4-cdab9c7c3bbe None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.408844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.409907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.411164 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.411603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.412497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:46.412962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:46.414544 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:46.415560 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:46.431115 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 293/575] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:46.431115 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.441656 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503e48b-ca8e-498e-81de-1977cd6c10fc None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.476667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.477331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.477620 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.477904 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.485401 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:46.486914 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:46.499253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:46.501246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:46.503501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:46.503501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:46.503501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:46.503501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:46.503501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:46.503501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:46.503501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:46.503501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:46.505303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 283/576] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:46.506135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.548694 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c64ae-d842-48e3-a06a-30835f4dd4ce None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.587423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.588232 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.588569 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.589407 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.590323 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:46.590659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:46.592103 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:46.601559 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 294/577] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:46.602415 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.613113 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:46.634777 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23e226-62dd-4e60-8232-2f08844fb6e2 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.669058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.669692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.669960 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.670266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.671002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:46.671311 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:46.672698 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:46.674209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:46.686131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 284/578] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:46.686131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.720350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49ccb1-b45a-4c8c-a266-0e0f919211d5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.753864 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.754669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.755114 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.755484 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.768220 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:46.769921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:46.785934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:46.787052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:46.788074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:46.788074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:46.788074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:46.788074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:46.788074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 295/579] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:46.788074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.796993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed251b-fe32-4d6e-9734-312e25c2fb18 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.833746 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.834429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.834710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.834994 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.835769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:46.836087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:46.837636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:46.851306 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 285/580] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:46.852230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:46.859653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:46.906372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124a490-88f7-4d48-899d-25a93bdb25d9 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.911483 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:46.947181 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:46.948299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:46.948584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:46.948863 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:46.960406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f3df-1ce5-4e70-8ca6-c03b14515ffb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:46.961890 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:46.963551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:46.980079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:46.981469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:46.982765 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:46.982765 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:46.982765 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:46.982765 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:46.982765 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 296/581] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:46.983325 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.006411 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.007951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.008711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.009190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.024274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.026651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:47.049142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:47.050317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:47.051376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:47.051376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:47.051376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:47.051376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:47.051624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 286/582] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:47.052361 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.094847 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38dc8c-833f-400f-80fa-0d62b69f24d0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.107306 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:47.133493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.134578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.134951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.135243 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.135960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:47.136631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.138434 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.152525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 297/583] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:47.153617 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.165596 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696894a-84a0-41aa-8027-a4656271454e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.175380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:47.204038 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.205026 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.205559 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.205896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.206763 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=6726b8e1313a4181bcc32ed3697f4fd0) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:47.207079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.208553 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.214311 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 287/584] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/role_assignments?user.id=6726b8e1313a4181bcc32ed3697f4fd0&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:47.215108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.287872 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31acbdd-49fe-4a8f-872e-8f8416947b96 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.342672 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.343823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.344809 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.345270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.346363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:47.346777 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.348986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.366262 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5db3b2-2e13-4ca3-8832-842a4b387d82 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.367428 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 298/585] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:47.368555 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.404252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:47.431097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.431740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.432028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.432414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.433183 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:47.433493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.435052 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.458680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 288/586] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:47.459649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.500701 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:47.501953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf0ed8-44da-4515-8833-cf9d35883bd6 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.542955 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.544336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.544790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.545237 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.554831 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.556737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:47.571359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:47.572582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:21:47.573655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:47.576626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 299/587] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:47.577646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.629029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095b958-401d-4a17-b80d-bafb1763b7af None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.688144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:47.696340 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.698427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.698996 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.699549 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.700867 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e92561-f9f4-4eb5-b04b-d2f9bdf41ca5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.714303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.717645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.741085 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.742021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.742503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.742958 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:47.743187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:47.744245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:21:47.745322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:47.746122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 289/588] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:47.748163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.756580 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.758321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:47.771235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:47.772344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:21:47.773542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:47.781509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 300/589] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:47.782864 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.874687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:47.885888 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee364c90-e4c0-4376-a044-4f4fc4596eaa None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.902568 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574011c4-4089-46ad-8cd5-268d096ab167 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.935100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.936118 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.936513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.936896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.938565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:47.939151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.940984 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.948523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.949366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.949751 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.950216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.951185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:47.951624 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.953654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.970596 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 290/590] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:47.971897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.978560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 301/591] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:47.979714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.993652 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:48.085079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba44f7-0642-4f7a-8e46-03c3ff8661e7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.093098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5baed00-e1d5-44df-b7ae-66d4fd30f1bf None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.122381 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.123030 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.123332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.123610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.124437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:48.124813 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.126252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.132149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a4b5a-85a9-4aee-926e-f960c15a5aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.132807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a4b5a-85a9-4aee-926e-f960c15a5aaa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.133080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a4b5a-85a9-4aee-926e-f960c15a5aaa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.133371 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a4b5a-85a9-4aee-926e-f960c15a5aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.144828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a4b5a-85a9-4aee-926e-f960c15a5aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.145654 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 291/592] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:48.146405 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a4b5a-85a9-4aee-926e-f960c15a5aaa None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.147426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-1d2a4b5a-85a9-4aee-926e-f960c15a5aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:48.162036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:48.163117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:48.164121 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:48.164121 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:48.164121 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:48.164121 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:48.164121 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 302/593] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:48.165224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8c63-5d9b-4a30-a7a5-f6681f7fa2b4 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.165224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8c63-5d9b-4a30-a7a5-f6681f7fa2b4 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.165224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8c63-5d9b-4a30-a7a5-f6681f7fa2b4 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.165868 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:48.170184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 303/594] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:48.170184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f8c63-5d9b-4a30-a7a5-f6681f7fa2b4 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.233578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:48.271986 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6559d-126a-4e64-803e-debe1d4946d2 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.279918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f8c63-5d9b-4a30-a7a5-f6681f7fa2b4 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.312586 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e952f-cf58-4f70-ba32-1f0a314c6368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.313341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e952f-cf58-4f70-ba32-1f0a314c6368 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.313628 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e952f-cf58-4f70-ba32-1f0a314c6368 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.313940 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e952f-cf58-4f70-ba32-1f0a314c6368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.315193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.315979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.316654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.316930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.325499 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e952f-cf58-4f70-ba32-1f0a314c6368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.326999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e952f-cf58-4f70-ba32-1f0a314c6368 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.328493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.330032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:48.345739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:48.347005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:48.348032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:48.348032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:48.348032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:48.348032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-344e952f-cf58-4f70-ba32-1f0a314c6368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:48.348284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:48.349194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:48.350256 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:48.350256 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:48.350256 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:48.350256 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:48.350529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 304/595] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:48.350529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.357849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 292/596] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:48.357849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb535db-3561-451a-96e8-a0a9cb4cd222 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.357849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb535db-3561-451a-96e8-a0a9cb4cd222 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.357849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb535db-3561-451a-96e8-a0a9cb4cd222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.374436 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-bdb535db-3561-451a-96e8-a0a9cb4cd222 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:48.375778 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-bdb535db-3561-451a-96e8-a0a9cb4cd222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:48.435425 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:48.450803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 293/597] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:48.452177 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb535db-3561-451a-96e8-a0a9cb4cd222 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.481813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408101-7123-4628-b23d-6ef74fcf4e3d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.495641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:48.521511 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.522372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.522719 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.523467 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.523774 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:48.524079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.525451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.542955 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 305/598] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:48.542955 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.563698 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb535db-3561-451a-96e8-a0a9cb4cd222 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.609057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.609702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.609989 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.610301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.611020 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:48.611345 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.612730 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.624246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 294/599] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:48.625072 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.655586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31a632-dc5e-4388-856e-3ae6d5ce580b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.688461 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:48.694372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.695018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.695304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.695584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.696291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:48.696602 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.698011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.718439 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 306/600] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:48.718439 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.747887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68864-8982-4c32-8192-8ab6710addc0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.753922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:48.786802 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e08cb-eb06-4104-a971-35b94fe2be8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.787471 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e08cb-eb06-4104-a971-35b94fe2be8e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.787757 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e08cb-eb06-4104-a971-35b94fe2be8e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.788048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e08cb-eb06-4104-a971-35b94fe2be8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.799123 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e08cb-eb06-4104-a971-35b94fe2be8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-9f8e08cb-eb06-4104-a971-35b94fe2be8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:48.804476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:48.805573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 295/601] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:48.807184 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e08cb-eb06-4104-a971-35b94fe2be8e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.833408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05a937-b1c3-4c3d-9021-7289cc539093 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.870166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.870829 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.871110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.871383 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.878471 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.879920 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:48.892356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:48.893485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 307/602] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:48.894588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:48.923745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e08cb-eb06-4104-a971-35b94fe2be8e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:48.936601 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:48.966277 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:48.967450 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:48.968048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:48.968331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:48.977029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:48.978735 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:48.993027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:48.994211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 296/603] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:48.995334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.010188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:49.021771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3aa88-d02d-4c06-aa93-680216f9008a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.061096 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.061777 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.062063 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.062379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.070673 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.072197 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:49.085211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:49.086379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 308/604] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:49.087411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.116039 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0756f-255f-415d-af7c-ca7527c0e7bc None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.154038 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.154716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.155010 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.155277 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.156047 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:49.156419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.157920 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.168260 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 297/605] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:49.169023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.202228 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62fcf6-ac59-41e7-8577-c9d0dd995942 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.205966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:49.240487 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.241488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.241778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.242053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.242869 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:49.243181 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.244666 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.261680 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 309/606] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:49.262656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.283383 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:49.291615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be8168-1474-483d-be7f-6e8d64cbaa8f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.332978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.333799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.334221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.334563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.335573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:49.335964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.337900 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.351664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 298/607] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:49.352527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.386345 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2791e7b-cdfc-4370-8267-ed702dd7ddf1 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.423835 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.424470 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.424760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.425043 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.425836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:49.426515 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.427666 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.446413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 310/608] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:49.447321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.462229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:49.473916 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f14dc-9722-427b-8a4e-ebc6592853dc None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.508868 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d2ac5-2e4a-4017-b231-98307c5f5c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.509536 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2ac5-2e4a-4017-b231-98307c5f5c14 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.509824 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2ac5-2e4a-4017-b231-98307c5f5c14 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.510130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2ac5-2e4a-4017-b231-98307c5f5c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.522743 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d2ac5-2e4a-4017-b231-98307c5f5c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.525074 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d2ac5-2e4a-4017-b231-98307c5f5c14 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.543480 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c86d2ac5-2e4a-4017-b231-98307c5f5c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:49.546321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:49.547469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:49.548610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:49.548610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:49.548610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:49.548610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:49.548610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 299/609] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:49.549476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53540d53-dbaa-4540-8bed-a11ee55438e7 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.549476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53540d53-dbaa-4540-8bed-a11ee55438e7 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.549476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53540d53-dbaa-4540-8bed-a11ee55438e7 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.550917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 300/610] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:49.551416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53540d53-dbaa-4540-8bed-a11ee55438e7 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.587562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f8c30-0f20-4dd8-ae07-d9b594788220 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.629478 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f5647-1a62-44ba-a139-ae0319f13b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.630224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f5647-1a62-44ba-a139-ae0319f13b64 None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.630513 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f5647-1a62-44ba-a139-ae0319f13b64 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.630794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f5647-1a62-44ba-a139-ae0319f13b64 None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/6174eb4be9314a51af64f926441a055d/roles/aec37946c06745cfb2ea73e9b3c9a547` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.647346 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f5647-1a62-44ba-a139-ae0319f13b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=6174eb4be9314a51af64f926441a055d, role_id=aec37946c06745cfb2ea73e9b3c9a547)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.648833 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f5647-1a62-44ba-a139-ae0319f13b64 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.663080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 311/611] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:49 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/6174eb4be9314a51af64f926441a055d/roles/aec37946c06745cfb2ea73e9b3c9a547 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:49.664007 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f5647-1a62-44ba-a139-ae0319f13b64 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.668883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53540d53-dbaa-4540-8bed-a11ee55438e7 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.713471 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d06bb-7405-419d-bdc9-f70b2fca0ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.714246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d06bb-7405-419d-bdc9-f70b2fca0ac5 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.714608 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d06bb-7405-419d-bdc9-f70b2fca0ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.714964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d06bb-7405-419d-bdc9-f70b2fca0ac5 None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/629e1537906b4e1b9431af873d612a34/roles/aec37946c06745cfb2ea73e9b3c9a547` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.734709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d06bb-7405-419d-bdc9-f70b2fca0ac5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=629e1537906b4e1b9431af873d612a34, role_id=aec37946c06745cfb2ea73e9b3c9a547)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.736477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d06bb-7405-419d-bdc9-f70b2fca0ac5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.739281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:49.757597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 301/612] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:49 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/629e1537906b4e1b9431af873d612a34/roles/aec37946c06745cfb2ea73e9b3c9a547 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:49.758615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d06bb-7405-419d-bdc9-f70b2fca0ac5 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:49.818213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f5647-1a62-44ba-a139-ae0319f13b64 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.866214 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:49.893568 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22ceba-24da-418e-a7eb-42e9af9182c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:49.895273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ceba-24da-418e-a7eb-42e9af9182c3 None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.896977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ceba-24da-418e-a7eb-42e9af9182c3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.897753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ceba-24da-418e-a7eb-42e9af9182c3 None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/03519d64b78e4eaa9869a4e2c9e31413/roles/aec37946c06745cfb2ea73e9b3c9a547` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.925731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ceba-24da-418e-a7eb-42e9af9182c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=03519d64b78e4eaa9869a4e2c9e31413, role_id=aec37946c06745cfb2ea73e9b3c9a547)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:49.927712 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ceba-24da-418e-a7eb-42e9af9182c3 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:49.945873 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d06bb-7405-419d-bdc9-f70b2fca0ac5 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:49.955679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 312/613] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:49 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/03519d64b78e4eaa9869a4e2c9e31413/roles/aec37946c06745cfb2ea73e9b3c9a547 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:49.955679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514243-0957-4457-958d-0f97e7d7efa7 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:49.955679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514243-0957-4457-958d-0f97e7d7efa7 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:49.955679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514243-0957-4457-958d-0f97e7d7efa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:49.985188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-b9514243-0957-4457-958d-0f97e7d7efa7 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:49.986763 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-b9514243-0957-4457-958d-0f97e7d7efa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:50.003890 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:50.004981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:50.005653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:50.006251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:50.007287 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:50.007817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:50.009507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:50.033911 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 302/614] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:50.041612 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:50.095246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 313/615] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:50.111974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:50.120075 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9514243-0957-4457-958d-0f97e7d7efa7 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:50.166153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469d91c-1531-4637-a3af-61ccd3c9c0ef None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:50.215808 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:50.216629 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:50.217041 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:50.217434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] PATH_INFO: `/v3/users/nova` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:50.229826 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:50.231434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:50.245462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:50.246751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:50.247860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 303/616] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:21:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:50.251240 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9514243-0957-4457-958d-0f97e7d7efa7 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:50.255175 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:50.255369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:50.301674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57609-8a79-4dea-a362-547c15842f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:50.303601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57609-8a79-4dea-a362-547c15842f36 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:50.304103 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57609-8a79-4dea-a362-547c15842f36 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:50.304578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57609-8a79-4dea-a362-547c15842f36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:50.321928 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f57609-8a79-4dea-a362-547c15842f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-66f57609-8a79-4dea-a362-547c15842f36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:50.330212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:50.331815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:50.332857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:50.339802 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 314/617] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:50.345400 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57609-8a79-4dea-a362-547c15842f36 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:50.390582 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb7578-fec2-47b2-ae2a-37ac6b3b3516 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:50.406476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:50.429410 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:50.430537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:50.431186 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:50.431676 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:50.432727 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:50.433252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:50.434983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:50.455065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 304/618] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:21:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:50.460422 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:50.613335 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c422a-0682-4723-8e22-89268e8947a8 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:50.619526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57609-8a79-4dea-a362-547c15842f36 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:50.684782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d9088-b12a-4bc5-8a83-b1c3dd1d1ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:50.685424 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d9088-b12a-4bc5-8a83-b1c3dd1d1ce4 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:50.685712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d9088-b12a-4bc5-8a83-b1c3dd1d1ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:50.685996 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d9088-b12a-4bc5-8a83-b1c3dd1d1ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:50.690614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:50.691627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:50.691914 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:50.696927 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:50.700637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:50.701266 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:50.702790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:50.734270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 315/619] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:50.755994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:50.771351 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:50.806860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d9088-b12a-4bc5-8a83-b1c3dd1d1ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:50.808376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d9088-b12a-4bc5-8a83-b1c3dd1d1ce4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:50.815175 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:50.919857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 305/620] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:50.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d9088-b12a-4bc5-8a83-b1c3dd1d1ce4 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:50.973853 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd6dc4-3bb2-40e4-821e-b6b417ee6f85 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.026502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:51.059253 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.060107 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.060402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.060696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.073977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.075894 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.088457 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d9088-b12a-4bc5-8a83-b1c3dd1d1ce4 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:51.095353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:51.096387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:51.097999 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:51.097999 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:51.097999 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:51.097999 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:51.097999 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 316/621] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:51.104484 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.172801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.173382 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.173670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.173952 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.192474 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.194360 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:51.223719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:51.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:51.225896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:51.225896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:51.225896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:51.225896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:51.235753 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 306/622] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:51.241946 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:51.243249 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.270523 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b06617-9a45-4334-8e1a-d60a1241c3d1 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.343680 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.344333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.344624 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.344904 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.345624 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:51.345935 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.347671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.372773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 317/623] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:51.379593 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.391725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c48c8-9dc5-4e4b-aefb-095fa2a19aad None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.419932 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:51.432189 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.433138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.433590 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.434057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.435643 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:51.436275 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.437892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.457051 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 307/624] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:51.467285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.594255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfc4f-85d4-4547-ad1a-401a075da9bd None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.659845 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e029760-a48e-4b3d-b836-823e90997184 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.664372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.665081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.665373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.665651 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] PATH_INFO: `/v3/users/swift` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.672089 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:51.683094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.684598 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:51.701172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:51.702557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:21:51.703903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:51.704641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 318/625] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:51.716991 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.738517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.739420 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.739852 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.740228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.753389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.756693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:51.779427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:51.780769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:51.782015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:51.782015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:51.782015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:51.782015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:51.782015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:51.782015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:51.782015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:51.782015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:51.785073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 308/626] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:51.792150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.886487 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:51.915298 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a2b05-0079-4f29-a848-d81ec4c06537 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.981324 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a4f8b-94c1-4e47-94e2-dac0582a9ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.982069 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4f8b-94c1-4e47-94e2-dac0582a9ad6 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.983057 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4f8b-94c1-4e47-94e2-dac0582a9ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.983440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4f8b-94c1-4e47-94e2-dac0582a9ad6 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.985834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a4f8b-94c1-4e47-94e2-dac0582a9ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:51.986285 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a4f8b-94c1-4e47-94e2-dac0582a9ad6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.986682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e799b24-a41a-410d-b325-08dc84947fb4 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.987966 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a4f8b-94c1-4e47-94e2-dac0582a9ad6 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:52.007534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 319/627] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:21:52.009166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9cb3-864c-4386-bd0b-857d571b3f34 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.009557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9cb3-864c-4386-bd0b-857d571b3f34 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.009966 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9cb3-864c-4386-bd0b-857d571b3f34 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.015593 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 320/628] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:52.015593 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae9cb3-864c-4386-bd0b-857d571b3f34 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:52.029310 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44493553-71d2-493a-befa-c4afd3236e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.030246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493553-71d2-493a-befa-c4afd3236e15 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.030704 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493553-71d2-493a-befa-c4afd3236e15 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.031148 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493553-71d2-493a-befa-c4afd3236e15 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.032187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44493553-71d2-493a-befa-c4afd3236e15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:52.032664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44493553-71d2-493a-befa-c4afd3236e15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:52.034633 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44493553-71d2-493a-befa-c4afd3236e15 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:52.048580 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 309/629] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:52.050263 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ac06-139e-4ccf-9c00-b4ac6f37072a None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.050711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ac06-139e-4ccf-9c00-b4ac6f37072a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.051129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ac06-139e-4ccf-9c00-b4ac6f37072a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.070876 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-ea36ac06-139e-4ccf-9c00-b4ac6f37072a None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:52.072521 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-ea36ac06-139e-4ccf-9c00-b4ac6f37072a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:52.092728 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:52.159256 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae9cb3-864c-4386-bd0b-857d571b3f34 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:52.163319 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 310/630] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:52.169086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36ac06-139e-4ccf-9c00-b4ac6f37072a None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:52.197435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bebfe-bc76-4993-b896-8179843db43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.198094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bebfe-bc76-4993-b896-8179843db43d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.198412 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bebfe-bc76-4993-b896-8179843db43d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.198693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bebfe-bc76-4993-b896-8179843db43d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.209694 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bebfe-bc76-4993-b896-8179843db43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:52.211214 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bebfe-bc76-4993-b896-8179843db43d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:52.214886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-7d7bebfe-bc76-4993-b896-8179843db43d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:52.228414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:52.229747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:52.231017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:52.231017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:52.231017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:52.231017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:52.231017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 321/631] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:21:52.231395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e284304-970b-4f6f-81ee-331530de48ca None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.231588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e284304-970b-4f6f-81ee-331530de48ca None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.231873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e284304-970b-4f6f-81ee-331530de48ca None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.233322 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 322/632] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:52.237231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e284304-970b-4f6f-81ee-331530de48ca None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:52.293266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36ac06-139e-4ccf-9c00-b4ac6f37072a None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:52.328161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.328787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.329068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.329359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.330172 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=0970fc7ae258457ca70042acc0fc6c4c) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:52.330488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:52.331943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:52.337987 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 311/633] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/role_assignments?user.id=0970fc7ae258457ca70042acc0fc6c4c&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:52.338833 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:52.363851 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e284304-970b-4f6f-81ee-331530de48ca None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:52.373842 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:52.404975 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177230f-70cf-44c2-937e-e2617ea0850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.405806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177230f-70cf-44c2-937e-e2617ea0850c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.406214 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177230f-70cf-44c2-937e-e2617ea0850c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.406591 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177230f-70cf-44c2-937e-e2617ea0850c None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.419588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177230f-70cf-44c2-937e-e2617ea0850c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-f177230f-70cf-44c2-937e-e2617ea0850c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:52.425387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:52.426844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:52.428014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:52.434914 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 323/634] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:52.436522 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a277c2-42ef-4bdd-ad84-1e6898e22616 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.436923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a277c2-42ef-4bdd-ad84-1e6898e22616 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.437299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a277c2-42ef-4bdd-ad84-1e6898e22616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.458156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-d5a277c2-42ef-4bdd-ad84-1e6898e22616 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:52.459655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-d5a277c2-42ef-4bdd-ad84-1e6898e22616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:52.476833 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:52.482696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72744ad-f41a-4d88-b375-de0092653a83 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:52.532966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.534216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.534676 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.535631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.536855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:52.537351 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:52.540019 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:52.550132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 324/635] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:52.550132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a277c2-42ef-4bdd-ad84-1e6898e22616 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:52.563219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 312/636] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:52.569819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:52.668975 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:52.677468 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a277c2-42ef-4bdd-ad84-1e6898e22616 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:52.690048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa1444-8e8e-4d00-b605-f0849a283f5c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:52.715927 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.716564 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.716847 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.717126 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.717870 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:52.718222 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:52.719654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:52.728332 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 325/637] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:52.729012 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:52.736487 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f886d4d-75a6-4318-9f8a-db6249e564ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.737077 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f886d4d-75a6-4318-9f8a-db6249e564ba None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.737357 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f886d4d-75a6-4318-9f8a-db6249e564ba None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.737629 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f886d4d-75a6-4318-9f8a-db6249e564ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.745202 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f886d4d-75a6-4318-9f8a-db6249e564ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:52.746752 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f886d4d-75a6-4318-9f8a-db6249e564ba None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-8f886d4d-75a6-4318-9f8a-db6249e564ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:52.758821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:52.759935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:52.760970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 313/638] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:52.762950 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d485-a847-4283-ac2a-53e815c0c773 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.762950 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d485-a847-4283-ac2a-53e815c0c773 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.762950 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d485-a847-4283-ac2a-53e815c0c773 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.763871 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 314/639] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:52.764534 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8d485-a847-4283-ac2a-53e815c0c773 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:52.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:52.850706 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7c81-d4bb-414f-817e-5caae0614a83 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:52.880027 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8d485-a847-4283-ac2a-53e815c0c773 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:52.889438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4f36e-0f07-40c5-a877-6e4f1e416ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.890130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f36e-0f07-40c5-a877-6e4f1e416ea9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.890434 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f36e-0f07-40c5-a877-6e4f1e416ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.890717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f36e-0f07-40c5-a877-6e4f1e416ea9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.904042 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4f36e-0f07-40c5-a877-6e4f1e416ea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:52.917183 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:52.917863 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:52.918175 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:52.918502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:52.931337 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:52.932830 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:52.950356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:52.951537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:52.952556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:52.952556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:52.952556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:52.952556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:52.953407 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 315/640] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:52.954638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.032379 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-83d4f36e-0f07-40c5-a877-6e4f1e416ea9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:53.036514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:53.044134 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 326/641] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:53.045116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d2c3b-0787-4ef5-838d-a35b36445238 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.045967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d2c3b-0787-4ef5-838d-a35b36445238 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.045967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d2c3b-0787-4ef5-838d-a35b36445238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.059388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-1b6d2c3b-0787-4ef5-838d-a35b36445238 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:53.060861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-1b6d2c3b-0787-4ef5-838d-a35b36445238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:53.072436 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87476ec6-90e6-4e79-94ea-69745d7dd00f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.078252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:53.115803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:53.116468 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.116749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.117388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.118203 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:53.118528 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:53.120056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:53.139563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 316/642] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:53.139563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.148996 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 327/643] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:53.155479 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d2c3b-0787-4ef5-838d-a35b36445238 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.257176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16121d-27f2-47e0-bffe-7fca1a7968e7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.275494 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d2c3b-0787-4ef5-838d-a35b36445238 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.289427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:53.297065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:53.297731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.297991 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.298310 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.299042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:53.299347 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:53.300760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:53.317662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:53.318262 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 317/644] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:53.318393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.318627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.318892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.319658 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:53.319962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:53.320422 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.321390 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:53.335919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 328/645] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:53.335919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.346518 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:53.443671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9380dda-1e33-44ed-9ca4-73fbccc79a85 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.447992 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25228cf8-e3d0-4cbe-9a01-d7624e73223a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.486445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15be725-d41e-4541-b594-3c47e8ffeaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:53.487139 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15be725-d41e-4541-b594-3c47e8ffeaa0 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.487434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15be725-d41e-4541-b594-3c47e8ffeaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.487719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15be725-d41e-4541-b594-3c47e8ffeaa0 None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/6726b8e1313a4181bcc32ed3697f4fd0/roles/aec37946c06745cfb2ea73e9b3c9a547` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.489209 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151bdebc-152e-4aa0-a7d8-c74e3f0f8822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:53.490089 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bdebc-152e-4aa0-a7d8-c74e3f0f8822 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.490609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bdebc-152e-4aa0-a7d8-c74e3f0f8822 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.491092 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bdebc-152e-4aa0-a7d8-c74e3f0f8822 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.502019 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151bdebc-152e-4aa0-a7d8-c74e3f0f8822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:53.506459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15be725-d41e-4541-b594-3c47e8ffeaa0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=6726b8e1313a4181bcc32ed3697f4fd0, role_id=aec37946c06745cfb2ea73e9b3c9a547)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-151bdebc-152e-4aa0-a7d8-c74e3f0f8822 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:53.507124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:53.508168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:53.514856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:53.515747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15be725-d41e-4541-b594-3c47e8ffeaa0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:53.515846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 329/646] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:53.515846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151bdebc-152e-4aa0-a7d8-c74e3f0f8822 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.534822 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 318/647] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:21:53 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/6726b8e1313a4181bcc32ed3697f4fd0/roles/aec37946c06745cfb2ea73e9b3c9a547 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:53.535950 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15be725-d41e-4541-b594-3c47e8ffeaa0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.614582 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:53.635219 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:53.688800 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151bdebc-152e-4aa0-a7d8-c74e3f0f8822 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.717113 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15be725-d41e-4541-b594-3c47e8ffeaa0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.734937 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:53.735613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.735901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.736179 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] PATH_INFO: `/v3/users/glance` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.745039 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:53.746696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:53.759747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:53.760909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 330/648] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:53.762065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.765113 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:53.765113 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.765341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.765526 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.784512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:53.786268 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:53.806399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:53.807481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:53.808518 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:53.808518 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:53.808518 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:53.808518 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:53.808518 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 319/649] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:53.808518 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.882701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3afe27-d223-4976-9c7c-e4cb586bb769 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.913094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:53.920838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:53.921931 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:53.922341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:53.922716 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:53.923655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:53.924061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:53.925648 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:53.935669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 331/650] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:53.936747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:53.937475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:53.948661 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbdb01-552f-45eb-8f20-8194252b7db1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:53.998917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.000121 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.000412 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.000694 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.001538 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:54.002099 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.004017 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.028058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 320/651] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:21:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:54.028930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.062637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c3151-0c37-472a-b82c-6f51a456ec68 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.101491 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.102585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.103129 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.103735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.104662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:54.105219 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.107149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.120868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 332/652] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:54.122333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.146416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28fa40-0eda-4148-8624-4e44daf75fd3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.180799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.182418 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.183144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.184119 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.184119 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.195076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.196585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:54.212216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:54.213252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:54.214455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:54.214455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:54.214455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:54.214455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:54.214455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 321/653] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:54.215011 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.218143 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.241713 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae0a24-536a-40bb-8d7b-d71930779f64 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.279952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436aa4de-6d15-4458-8878-7a1f9fce36fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.281105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436aa4de-6d15-4458-8878-7a1f9fce36fa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.281605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436aa4de-6d15-4458-8878-7a1f9fce36fa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.282096 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436aa4de-6d15-4458-8878-7a1f9fce36fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.325445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb902-2bae-4f71-99d5-089314af23d7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.360819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.361484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.361773 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.362058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.363431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436aa4de-6d15-4458-8878-7a1f9fce36fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.365339 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436aa4de-6d15-4458-8878-7a1f9fce36fa None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.370326 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.371850 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:54.384302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:54.385432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:54.386522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 322/654] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:54.387277 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.427169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.452242 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.484006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 333/655] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:54.485319 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436aa4de-6d15-4458-8878-7a1f9fce36fa None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.498084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92185e98-e37e-4eef-b199-7ea2ff3fc648 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.535098 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.535838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.536043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.536393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.537506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:54.538204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.540302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.553258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 323/656] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:54.554140 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.606952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436aa4de-6d15-4458-8878-7a1f9fce36fa None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.641393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30be3a4-8c46-4d56-b299-079e6cbc09aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.642025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30be3a4-8c46-4d56-b299-079e6cbc09aa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.642542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30be3a4-8c46-4d56-b299-079e6cbc09aa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.642676 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30be3a4-8c46-4d56-b299-079e6cbc09aa None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.643404 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30be3a4-8c46-4d56-b299-079e6cbc09aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:54.643867 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30be3a4-8c46-4d56-b299-079e6cbc09aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.645140 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30be3a4-8c46-4d56-b299-079e6cbc09aa None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.662664 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 334/657] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:54.663960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac84bdb-4ff2-4701-a133-2d08f11228a4 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.664245 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac84bdb-4ff2-4701-a133-2d08f11228a4 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.664519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac84bdb-4ff2-4701-a133-2d08f11228a4 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.665863 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 335/658] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:54.670683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac84bdb-4ff2-4701-a133-2d08f11228a4 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.671093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50346b-0f38-4eba-b009-04e7e6af4c6a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.683402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.701117 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.716139 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.716803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.717091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.717380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.728294 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.729885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:54.747274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:54.748383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:54.749414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:54.749414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:54.749414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:54.749414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:54.749414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 324/659] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:54.749414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.791422 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac84bdb-4ff2-4701-a133-2d08f11228a4 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.828201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd4424-fc8c-4b6b-a267-11112d696fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.828863 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd4424-fc8c-4b6b-a267-11112d696fc4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.829150 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd4424-fc8c-4b6b-a267-11112d696fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.829531 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd4424-fc8c-4b6b-a267-11112d696fc4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.836837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd4424-fc8c-4b6b-a267-11112d696fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.838335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd4424-fc8c-4b6b-a267-11112d696fc4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-89fd4424-fc8c-4b6b-a267-11112d696fc4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:54.854224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:54.855304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:54.856430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 336/660] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:54.857131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2996c6b-054f-4f66-924c-14071d8aed40 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.857131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2996c6b-054f-4f66-924c-14071d8aed40 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.857282 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2996c6b-054f-4f66-924c-14071d8aed40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.863135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403980a-98a4-4199-a530-b481761121f1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.875707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-a2996c6b-054f-4f66-924c-14071d8aed40 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:54.876912 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-a2996c6b-054f-4f66-924c-14071d8aed40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:54.906299 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6307651-f37f-4133-a889-162acf6a6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.907253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307651-f37f-4133-a889-162acf6a6c20 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.907816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307651-f37f-4133-a889-162acf6a6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.908103 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307651-f37f-4133-a889-162acf6a6c20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.945616 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.962130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 337/661] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:54.962130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2996c6b-054f-4f66-924c-14071d8aed40 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.962444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.993384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6307651-f37f-4133-a889-162acf6a6c20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.994918 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6307651-f37f-4133-a889-162acf6a6c20 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.101114 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2996c6b-054f-4f66-924c-14071d8aed40 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.108631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 325/662] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:55.114179 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6307651-f37f-4133-a889-162acf6a6c20 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.137563 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.138312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.138609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.138897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.139624 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:55.139934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.141372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.153697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 338/663] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:55.159024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.198079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.226467 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6307651-f37f-4133-a889-162acf6a6c20 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.228384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.264301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.264928 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.265228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.265477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.266350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:55.266621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.268110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.274190 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709f0cf-387a-4efa-9800-a8af3fb3fa96 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.286946 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 326/664] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:55.292139 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.313022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdd704-2299-4814-9c19-58f2a48104ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.313699 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdd704-2299-4814-9c19-58f2a48104ec None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.313969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdd704-2299-4814-9c19-58f2a48104ec None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.314556 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdd704-2299-4814-9c19-58f2a48104ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.326703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdd704-2299-4814-9c19-58f2a48104ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-8efdd704-2299-4814-9c19-58f2a48104ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:55.332309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:55.333395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:55.334385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 339/665] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:55.335170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdd704-2299-4814-9c19-58f2a48104ec None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.402213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa8ad3-b063-4c88-abd7-a868c30fdd92 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.433579 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.439420 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.440082 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.440369 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.440648 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.443949 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdd704-2299-4814-9c19-58f2a48104ec None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.452015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.453420 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.466151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:55.470435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:55.471459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:55.472484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:55.472484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:55.472484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:55.472484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:55.472484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 327/666] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:55.472484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.488202 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.489217 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.489856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.490517 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.498408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.500107 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:55.512769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:55.513819 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:55.514883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:55.514883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:55.514883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:55.514883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:55.514883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:55.514883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:55.514883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:55.514883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:55.518753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 340/667] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:55.519631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.587004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3ef84-f156-4c1e-a535-3eb19fa14294 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.624683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ff78-5e45-4215-8ae2-6a75f82c1864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.625317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ff78-5e45-4215-8ae2-6a75f82c1864 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.625599 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ff78-5e45-4215-8ae2-6a75f82c1864 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.625871 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ff78-5e45-4215-8ae2-6a75f82c1864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.627169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76173-54ce-453f-87e2-910689cc9e4f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.663204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.663840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.664127 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.664403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.665204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:55.665519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.666937 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.675264 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 341/668] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:55.676062 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.684048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.701058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6ff78-5e45-4215-8ae2-6a75f82c1864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.702693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6ff78-5e45-4215-8ae2-6a75f82c1864 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.718825 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.775807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775c1d3-885a-4e65-9fa0-aebcee93d1e8 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.809612 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 328/669] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:55.810755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ff78-5e45-4215-8ae2-6a75f82c1864 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.828246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400272ff-4bf1-437a-838d-a7a19d28c50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.828850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400272ff-4bf1-437a-838d-a7a19d28c50a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.828850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400272ff-4bf1-437a-838d-a7a19d28c50a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.828850 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400272ff-4bf1-437a-838d-a7a19d28c50a None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.829569 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400272ff-4bf1-437a-838d-a7a19d28c50a None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:55.829886 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400272ff-4bf1-437a-838d-a7a19d28c50a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.831347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400272ff-4bf1-437a-838d-a7a19d28c50a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.844279 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 342/670] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:55.845698 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800a2c-0e2c-4329-b129-f2d65bd1b28e None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.845884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800a2c-0e2c-4329-b129-f2d65bd1b28e None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.846215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800a2c-0e2c-4329-b129-f2d65bd1b28e None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.847597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 343/671] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:55.848352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800a2c-0e2c-4329-b129-f2d65bd1b28e None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.915545 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.934611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ff78-5e45-4215-8ae2-6a75f82c1864 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.970658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.971296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.971649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.971993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.972837 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:55.973215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.974767 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.975072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800a2c-0e2c-4329-b129-f2d65bd1b28e None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.976885 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.987978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 329/672] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:55.988854 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.014715 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0e94d-98ff-4362-9b75-70a1300e623d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.015380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0e94d-98ff-4362-9b75-70a1300e623d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.015656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0e94d-98ff-4362-9b75-70a1300e623d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.015931 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0e94d-98ff-4362-9b75-70a1300e623d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.026742 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0e94d-98ff-4362-9b75-70a1300e623d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.028359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0e94d-98ff-4362-9b75-70a1300e623d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-d6f0e94d-98ff-4362-9b75-70a1300e623d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:56.045787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:56.046959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:56.048035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:56.048035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:56.048035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:56.048035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:56.048035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 344/673] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:56.048368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33dd0c-ab6d-4b62-9dc9-1c09d2b30c7d None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.048877 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33dd0c-ab6d-4b62-9dc9-1c09d2b30c7d None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.048877 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33dd0c-ab6d-4b62-9dc9-1c09d2b30c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.068442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-ba33dd0c-ab6d-4b62-9dc9-1c09d2b30c7d None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:56.069668 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-ba33dd0c-ab6d-4b62-9dc9-1c09d2b30c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:56.115416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be047cf8-3200-42a8-911b-4d99163d4eaa None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.156455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 345/674] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:21:56.157416 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33dd0c-ab6d-4b62-9dc9-1c09d2b30c7d None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.158807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.159583 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.160145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.160431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.174823 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.176321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.179578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:56.194389 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:56.197622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:56.199081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:56.199081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:56.199081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:56.199081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:56.199652 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 330/675] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:56.200719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.254976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:56.277370 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33dd0c-ab6d-4b62-9dc9-1c09d2b30c7d None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.312739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd829-7f1c-440c-a36f-700b784e9420 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.319604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.320344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.320705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.321080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.329615 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.331155 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:56.343923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:56.345149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 346/676] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:56.346201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.361127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.361799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.362085 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.362402 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.363228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=6174eb4be9314a51af64f926441a055d) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:56.363546 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.365006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.371928 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 331/677] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/role_assignments?user.id=6174eb4be9314a51af64f926441a055d&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:56.372789 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.457592 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:56.510706 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d694338-e334-47d1-ae4b-9cf5e82ebbaa None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.536075 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8f486-c2be-4344-ae9e-0a85fb848ff5 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.557544 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.558685 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.558685 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.558901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.560597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:56.560597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.561230 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.578204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 347/678] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:56.582173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.587865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:56.598650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a4e7b-4552-4736-bba3-4ce42931b15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.598650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a4e7b-4552-4736-bba3-4ce42931b15c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.598650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a4e7b-4552-4736-bba3-4ce42931b15c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.598650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a4e7b-4552-4736-bba3-4ce42931b15c None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.614733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a4e7b-4552-4736-bba3-4ce42931b15c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-f64a4e7b-4552-4736-bba3-4ce42931b15c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:21:56.621886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:56.623112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 332/679] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:56.624477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a4e7b-4552-4736-bba3-4ce42931b15c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.735746 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270e87d-074f-4445-9442-897514f6a78e None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.776006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a4e7b-4552-4736-bba3-4ce42931b15c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.782478 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.783211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.783520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.783830 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.784573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:56.785012 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.786436 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.800720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:56.807157 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 348/680] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:56.808039 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.831766 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.833376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.833828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.834346 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.835581 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:56.836135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.838160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.857233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 333/681] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:56.858797 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.898742 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:56.972210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058e9d4-7f7f-4c14-b8f2-6d636bdc1857 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.015962 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18607f9f-939a-49b5-917c-f47da8d21272 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.038306 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.039433 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.039898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.040396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] PATH_INFO: `/v3/users/nova` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.055464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.058393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.061641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.062191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.062479 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.062860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.063615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:57.063925 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.065339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.082884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 334/682] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:57.082884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:57.083194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:57.084321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 349/683] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:57.085406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.159913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:57.232548 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc16958-a7a2-45b7-9c59-c85448cbc026 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.251458 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93372680-2ba7-42fc-b664-b07743300619 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.279139 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.280098 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.280487 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.280931 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.286851 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:57.293841 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.296115 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.307281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.308499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.309152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.309574 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.311352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=629e1537906b4e1b9431af873d612a34) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:57.311946 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.315027 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:57.317284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:57.319476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 335/684] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:57.323480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.347081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 350/685] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/role_assignments?user.id=629e1537906b4e1b9431af873d612a34&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:57.347081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.511807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a73ff-e13f-4af7-926f-85deacad57c7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.534563 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9d3ca-6d5c-402b-a3c9-7b29b1f952de None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.551969 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:57.575650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.577741 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.577741 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.578183 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.598677 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.601750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.604768 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.605641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.606355 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.607197 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.609213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:57.610017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.612561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:57.631305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:57.633554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:57.635869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:57.635869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:57.635869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:57.635869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:57.644869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 336/686] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:57.646645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.655760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 351/687] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:57.657963 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.742867 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:57.815876 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8002c1-1ebb-4758-a180-a460f5c3766b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.864137 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.865123 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.865417 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.865702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.866467 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:57.866785 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.868418 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.879859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa19349-1875-4bf4-8fd3-8032d04c8e32 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.888384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 352/688] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:57.889280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.962638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.963878 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.964673 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.965243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.967117 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:57.967764 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.970498 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.998809 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 337/689] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:58.000514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.033631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:58.076427 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5f8bd-bc34-4725-b38f-0472ac3ba62f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.090026 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:58.144339 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a46164-a2f5-492a-881a-cae9a5b70df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.145355 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46164-a2f5-492a-881a-cae9a5b70df6 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.145838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46164-a2f5-492a-881a-cae9a5b70df6 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.146370 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46164-a2f5-492a-881a-cae9a5b70df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.214856 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff57fd-f777-4aa5-aa42-efb0691049be None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.280681 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.281742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.282262 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.282805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.290450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a46164-a2f5-492a-881a-cae9a5b70df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.293489 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a46164-a2f5-492a-881a-cae9a5b70df6 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.297290 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.300285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:21:58.325541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:58.327867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:21:58.329999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:58.334624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 338/690] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:58.336542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.489252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:58.503400 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 353/691] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:58.505047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a46164-a2f5-492a-881a-cae9a5b70df6 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.516684 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:58.524359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83114a-38bf-4832-93a8-90fe4c2b41a6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.566462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.567116 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.567415 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.567672 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.568493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=03519d64b78e4eaa9869a4e2c9e31413) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:58.568811 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.571324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.584868 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 339/692] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/role_assignments?user.id=03519d64b78e4eaa9869a4e2c9e31413&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:58.585712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.703521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a46164-a2f5-492a-881a-cae9a5b70df6 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.715901 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41909301-1667-4f37-b567-c28bf54a4ca3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.765368 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260553e-8fd0-41f9-aae6-ac699f5636e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.767013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260553e-8fd0-41f9-aae6-ac699f5636e1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.767373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260553e-8fd0-41f9-aae6-ac699f5636e1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.767569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260553e-8fd0-41f9-aae6-ac699f5636e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.777359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:58.784313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260553e-8fd0-41f9-aae6-ac699f5636e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.785839 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260553e-8fd0-41f9-aae6-ac699f5636e1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.786256 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.788024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.788860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.789598 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.791682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:58.792920 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.796630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-5260553e-8fd0-41f9-aae6-ac699f5636e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:58.804339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:58.806565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:58.808624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:58.808624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:58.808624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:58.808624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:21:58.808624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 340/693] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:58.808624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dc1f5-f847-46c9-ab31-db19a25aadf6 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.808624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dc1f5-f847-46c9-ab31-db19a25aadf6 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.808624 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dc1f5-f847-46c9-ab31-db19a25aadf6 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.810053 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 341/694] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:21:58.818191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd03113-877e-4c5c-acbf-e27cd28a9c39 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.818191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd03113-877e-4c5c-acbf-e27cd28a9c39 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.818191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd03113-877e-4c5c-acbf-e27cd28a9c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.829372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 354/695] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:58.833064 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.847113 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-ecd03113-877e-4c5c-acbf-e27cd28a9c39 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:58.848296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-ecd03113-877e-4c5c-acbf-e27cd28a9c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:58.952161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 342/696] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:21:58.953132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd03113-877e-4c5c-acbf-e27cd28a9c39 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.965625 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:59.043914 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbcc2c-0af9-4249-b043-486ec91d2d1e None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:59.083793 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:59.085327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:59.085855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:59.086381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:59.087340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:59.087874 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:59.089516 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:59.091356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd03113-877e-4c5c-acbf-e27cd28a9c39 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:59.103318 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 355/697] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:59.104528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:59.120227 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:59.137805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d64401-a6dc-4f67-9b73-df64b99eae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:59.138559 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d64401-a6dc-4f67-9b73-df64b99eae95 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:59.138844 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d64401-a6dc-4f67-9b73-df64b99eae95 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:59.139174 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d64401-a6dc-4f67-9b73-df64b99eae95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:59.215526 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d64401-a6dc-4f67-9b73-df64b99eae95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:59.217057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d64401-a6dc-4f67-9b73-df64b99eae95 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:59.222619 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2c5dc-643c-40e2-8785-6da4be230742 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:59.249350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:59.264824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:59.265957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:59.266520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:59.266981 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:59.279675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:59.281487 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:59.298047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:59.299421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:59.300482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:59.300482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:59.300482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:59.300482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:59.302138 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 356/698] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:59.303164 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:59.344024 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 343/699] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:21:59.344024 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d64401-a6dc-4f67-9b73-df64b99eae95 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:59.366146 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:59.452841 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983ca6c-24a1-461d-b60e-ca0cc6b496af None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:59.469313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d64401-a6dc-4f67-9b73-df64b99eae95 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:59.504474 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:59.505463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:59.506014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:59.506528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:59.506818 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:59.507577 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:59.509400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:59.509687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:59.510484 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:59.510893 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:59.512754 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:59.520123 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:59.521629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:21:59.538265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:59.540410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:59.542639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:21:59.542639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:21:59.542639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:21:59.542639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:21:59.542639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:21:59.542639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:21:59.542639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:21:59.542639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:59.544591 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 357/700] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:59.545582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:59.546050 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 344/701] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:59.553171 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:59.620073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:59.723252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb08a21-a11a-40eb-af62-3ace6fd853a5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:59.735016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:59.775222 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b403c3-e49c-46d9-b661-c6bf39f26e41 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:59.795986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:59.797139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:59.797606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:59.798049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:59.819315 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:59.821905 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:59.851006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67118d45-752e-455f-8be8-0a5905997a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:21:59.851878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:21:59.853659 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:21:59.855633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:21:59.855633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:21:59.855633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:21:59.855633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:21:59.857993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118d45-752e-455f-8be8-0a5905997a4b None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:59.859138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118d45-752e-455f-8be8-0a5905997a4b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:59.860548 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118d45-752e-455f-8be8-0a5905997a4b None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:59.862738 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 358/702] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:21:59.863538 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67118d45-752e-455f-8be8-0a5905997a4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:59.864740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:59.867351 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67118d45-752e-455f-8be8-0a5905997a4b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:59.914970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 345/703] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:21:59 2022] POST /identity/v3/users => generated 254 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:59.917010 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67118d45-752e-455f-8be8-0a5905997a4b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:00.015131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:00.033672 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436854e2-57f2-4e93-b4e6-7fd31a031b51 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:00.094993 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:00.095893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:00.096368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:00.096845 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:00.098272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:00.098754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:00.101454 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:00.151074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 359/704] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:21:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:00.160461 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:00.215757 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67118d45-752e-455f-8be8-0a5905997a4b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:00.291470 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:00.292443 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:00.292885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:00.293421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:00.294903 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:00.295456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:00.298125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:00.312166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:00.327817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 346/705] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:00.337872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:00.398977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02fb06-7f56-4bdc-ab7b-315260342743 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:00.463381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:00.464048 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:00.464336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:00.464617 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:00.465538 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=0970fc7ae258457ca70042acc0fc6c4c) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:00.465938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:00.467503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:00.483291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 360/706] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:00 2022] GET /identity/v3/role_assignments?user.id=0970fc7ae258457ca70042acc0fc6c4c&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=af958d22229a4a0297bbe2e643f55181 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:00.541740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c3120-8367-42ee-9650-faf91e5c71b6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:00.588081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:00.588781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:00.589066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:00.589350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:00.598329 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:00.599863 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:00.613680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:00.615742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:00.617820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 347/707] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:00.622970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:00.684178 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:00.913106 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c00b9-8452-486a-90e9-e03dc3d1df5b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:00.987977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703f6bc-f994-4d44-b9b4-718d2cb5cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:00.988900 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703f6bc-f994-4d44-b9b4-718d2cb5cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:00.989238 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703f6bc-f994-4d44-b9b4-718d2cb5cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:00.989574 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703f6bc-f994-4d44-b9b4-718d2cb5cfe7 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:00.990592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703f6bc-f994-4d44-b9b4-718d2cb5cfe7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:00.990910 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703f6bc-f994-4d44-b9b4-718d2cb5cfe7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:00.992767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703f6bc-f994-4d44-b9b4-718d2cb5cfe7 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:01.015452 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 361/708] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:01.021600 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:01.030721 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72eb064-af17-4829-9fd0-d2e189e51b20 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:01.031215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72eb064-af17-4829-9fd0-d2e189e51b20 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:01.031506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72eb064-af17-4829-9fd0-d2e189e51b20 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:01.033665 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 362/709] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:01.045075 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f2385-146e-48d6-a69a-4b83286df15d None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:01.045075 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f2385-146e-48d6-a69a-4b83286df15d None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:01.045075 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f2385-146e-48d6-a69a-4b83286df15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:01.065122 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-359f2385-146e-48d6-a69a-4b83286df15d None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:01.067159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-359f2385-146e-48d6-a69a-4b83286df15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:01.139678 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:01.199566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad318217-6e9f-4094-9d32-feff463a129f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:01.208976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 363/710] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:22:01.237912 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:01.239240 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:01.239698 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:01.240160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:01.241154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=6726b8e1313a4181bcc32ed3697f4fd0) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:01.241650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:01.243325 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:01.251035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 348/711] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:01 2022] GET /identity/v3/role_assignments?user.id=6726b8e1313a4181bcc32ed3697f4fd0&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:01.255624 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f2385-146e-48d6-a69a-4b83286df15d None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:01.262325 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:01.413592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f2385-146e-48d6-a69a-4b83286df15d None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:01.455395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:01.455996 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:01.456280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:01.456557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:01.457238 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:01.458765 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:01.470921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:01.472408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:01.473529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 364/712] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 09:22:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:01.479313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:01.643057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f16f3f-fc97-4fda-846f-dcbdc30e5d9b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:01.683455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:01.711149 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaadc3-ed10-427a-a0f3-16acd3bb7458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:01.712123 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaadc3-ed10-427a-a0f3-16acd3bb7458 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:01.712630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaadc3-ed10-427a-a0f3-16acd3bb7458 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:01.713126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaadc3-ed10-427a-a0f3-16acd3bb7458 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:01.714465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaadc3-ed10-427a-a0f3-16acd3bb7458 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:01.715031 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaadc3-ed10-427a-a0f3-16acd3bb7458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:01.717622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaadc3-ed10-427a-a0f3-16acd3bb7458 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:01.726680 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 349/713] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:22:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:01.733860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:01.775324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a36681-e80e-41b1-b849-6d5a0ed83095 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:01.775844 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a36681-e80e-41b1-b849-6d5a0ed83095 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:01.776346 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a36681-e80e-41b1-b849-6d5a0ed83095 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:01.780560 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 350/714] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:01.786779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b8ede-1322-4627-a264-ac749d0c0f0f None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:01.787252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b8ede-1322-4627-a264-ac749d0c0f0f None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:01.787706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b8ede-1322-4627-a264-ac749d0c0f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:01.831848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-068b8ede-1322-4627-a264-ac749d0c0f0f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:01.834597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-068b8ede-1322-4627-a264-ac749d0c0f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:01.842650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:01.893236 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f700a-14b6-4192-ba4f-780714e57aae None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:01.940511 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:01.942704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:01.943008 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:01.943350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:01.944073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:01.944435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:01.946035 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:01.953172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 365/715] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:22:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:01.985966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 351/716] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:22:02.008897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:02.132460 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec42c8a-2053-40f5-a536-4bb75887aac3 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:02.165116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:02.171797 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:02.172550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:02.173246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:02.173610 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:02.174323 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:02.175754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:02.187514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:02.189737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:02.191971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:02.195840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 366/717] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:02 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:02.202888 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b8ede-1322-4627-a264-ac749d0c0f0f None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:02.417628 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b8ede-1322-4627-a264-ac749d0c0f0f None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:02.447174 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:02.495649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbab806-bd27-401a-adb0-33013178a17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:02.497202 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbab806-bd27-401a-adb0-33013178a17f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:02.497920 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbab806-bd27-401a-adb0-33013178a17f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:02.498667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbab806-bd27-401a-adb0-33013178a17f None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:02.500212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbab806-bd27-401a-adb0-33013178a17f None admin] RBAC: Adding query filter params (name=network) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:02.501002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbab806-bd27-401a-adb0-33013178a17f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:02.503886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbab806-bd27-401a-adb0-33013178a17f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:02.513463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 352/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:02.518953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:02.715933 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb955ebc-1468-4431-a58b-ead8f38071a9 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:02.784442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfab9e6-523c-48e1-bbef-5b0aaf1b0891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:02.785511 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfab9e6-523c-48e1-bbef-5b0aaf1b0891 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:02.786006 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfab9e6-523c-48e1-bbef-5b0aaf1b0891 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:02.786537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfab9e6-523c-48e1-bbef-5b0aaf1b0891 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:02.787840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfab9e6-523c-48e1-bbef-5b0aaf1b0891 None admin] RBAC: Adding query filter params (type=network) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:02.788397 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfab9e6-523c-48e1-bbef-5b0aaf1b0891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:02.790981 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfab9e6-523c-48e1-bbef-5b0aaf1b0891 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:02.800291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 367/719] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:03.068350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb511db8-f803-419c-aee7-90d7fe16a64f None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:03.068562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb511db8-f803-419c-aee7-90d7fe16a64f None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:03.068826 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb511db8-f803-419c-aee7-90d7fe16a64f None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:03.070565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 353/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:03.076402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24adf7-77e6-42fa-b850-4594ba21951f None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:03.076714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24adf7-77e6-42fa-b850-4594ba21951f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:03.076971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24adf7-77e6-42fa-b850-4594ba21951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:03.096797 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-2c24adf7-77e6-42fa-b850-4594ba21951f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:03.098038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-2c24adf7-77e6-42fa-b850-4594ba21951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:03.196431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:03.222613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 368/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:22:03.246963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb511db8-f803-419c-aee7-90d7fe16a64f None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:03.407560 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb511db8-f803-419c-aee7-90d7fe16a64f None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:03.457522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b3439-0d55-4fd1-b007-c73b20515b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:03.458932 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b3439-0d55-4fd1-b007-c73b20515b7f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:03.459097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266417aa-7d5b-4cd2-9914-9b96efb69b36 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:03.459505 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266417aa-7d5b-4cd2-9914-9b96efb69b36 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:03.459760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b3439-0d55-4fd1-b007-c73b20515b7f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:03.459954 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266417aa-7d5b-4cd2-9914-9b96efb69b36 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:03.460366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b3439-0d55-4fd1-b007-c73b20515b7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:03.461856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 369/722] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:03.467343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62461-1c46-4b5a-96d2-e6627d89cc58 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:03.467626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62461-1c46-4b5a-96d2-e6627d89cc58 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:03.468150 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62461-1c46-4b5a-96d2-e6627d89cc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:03.482335 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b3439-0d55-4fd1-b007-c73b20515b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-826b3439-0d55-4fd1-b007-c73b20515b7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:03.493239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:03.494634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:22:03.495828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:03.500121 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 354/723] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:22:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:03.508378 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-1bc62461-1c46-4b5a-96d2-e6627d89cc58 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:03.509868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-1bc62461-1c46-4b5a-96d2-e6627d89cc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:03.514435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b3439-0d55-4fd1-b007-c73b20515b7f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:03.586537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:03.678071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 370/724] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 09:22:03.698981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b3439-0d55-4fd1-b007-c73b20515b7f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:03.706252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc62461-1c46-4b5a-96d2-e6627d89cc58 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:03.772045 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:03.772682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:03.772964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:03.773244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:03.773984 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:03.774318 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:03.775715 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:03.791914 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 355/725] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:22:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:03.798363 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:03.940456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc62461-1c46-4b5a-96d2-e6627d89cc58 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:04.002498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:04.022959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343668-dd11-445d-bec5-163eb5d2a7a1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:04.087240 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:04.092373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:04.093382 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.093895 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.094418 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.104352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56f38f-c76f-4fff-a7ff-6ae5d59eb81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:04.105451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f38f-c76f-4fff-a7ff-6ae5d59eb81f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.106040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f38f-c76f-4fff-a7ff-6ae5d59eb81f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.106578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f38f-c76f-4fff-a7ff-6ae5d59eb81f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.118384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:04.120279 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:04.131304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56f38f-c76f-4fff-a7ff-6ae5d59eb81f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-4b56f38f-c76f-4fff-a7ff-6ae5d59eb81f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:04.137130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:04.138305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:04.139922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:04.145865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:04.147212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:04.148480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:04.148480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:04.148480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:04.148480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:04.148978 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 371/726] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:04.152078 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 356/727] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:04.152279 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56f38f-c76f-4fff-a7ff-6ae5d59eb81f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:04.159803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:04.322900 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4114a-079b-455a-85f5-1cd3bc435536 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:04.404349 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ed0ac-d7d5-4ec0-b614-520a6f74e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:04.406425 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed0ac-d7d5-4ec0-b614-520a6f74e20c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.407256 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed0ac-d7d5-4ec0-b614-520a6f74e20c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.408464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed0ac-d7d5-4ec0-b614-520a6f74e20c None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.410662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ed0ac-d7d5-4ec0-b614-520a6f74e20c None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:04.411217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ed0ac-d7d5-4ec0-b614-520a6f74e20c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:04.413828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ed0ac-d7d5-4ec0-b614-520a6f74e20c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:04.443181 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 357/728] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:04.447643 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56f38f-c76f-4fff-a7ff-6ae5d59eb81f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:04.449946 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016d4e-3420-4799-b91b-1112861de78a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.451076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016d4e-3420-4799-b91b-1112861de78a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.451773 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016d4e-3420-4799-b91b-1112861de78a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.455996 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 358/729] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:04 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:04.458052 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016d4e-3420-4799-b91b-1112861de78a None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:04.492158 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:04.529285 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e1e70-bcf7-4f4f-913f-0cc6040ecb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:04.530367 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1e70-bcf7-4f4f-913f-0cc6040ecb39 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.530893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1e70-bcf7-4f4f-913f-0cc6040ecb39 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.531356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1e70-bcf7-4f4f-913f-0cc6040ecb39 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.533578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e1e70-bcf7-4f4f-913f-0cc6040ecb39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:04.534096 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e1e70-bcf7-4f4f-913f-0cc6040ecb39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:04.538011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e1e70-bcf7-4f4f-913f-0cc6040ecb39 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:04.560080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 372/730] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:04.561915 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac92d-ca50-40cf-9235-49156a9a66e8 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.562394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac92d-ca50-40cf-9235-49156a9a66e8 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.563061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac92d-ca50-40cf-9235-49156a9a66e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.597332 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-d77ac92d-ca50-40cf-9235-49156a9a66e8 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:04.599208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-d77ac92d-ca50-40cf-9235-49156a9a66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:04.622640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016d4e-3420-4799-b91b-1112861de78a None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:04.653337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:04.691894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:04.693093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.693584 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.694205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.696198 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 373/731] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:22:04.697469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac92d-ca50-40cf-9235-49156a9a66e8 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:04.706868 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:04.708623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:04.724859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:04.726894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:04.729793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:04.738521 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 359/732] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:22:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:04.738521 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:04.842568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:04.888495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f286e3-9cf3-4075-b8a5-6b8542d62d42 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:04.902459 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac92d-ca50-40cf-9235-49156a9a66e8 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:04.956824 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:04.957969 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.958536 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.959063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.960178 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:04.960743 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:04.962584 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:04.984352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4cc55-c6b9-40e3-a84c-82a814a744d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:04.985044 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4cc55-c6b9-40e3-a84c-82a814a744d7 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:04.985334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4cc55-c6b9-40e3-a84c-82a814a744d7 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:04.985642 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4cc55-c6b9-40e3-a84c-82a814a744d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:04.991613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 360/733] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:22:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:04.993187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:05.001377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4cc55-c6b9-40e3-a84c-82a814a744d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:05.004344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4cc55-c6b9-40e3-a84c-82a814a744d7 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-86e4cc55-c6b9-40e3-a84c-82a814a744d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:05.026011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:05.033104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 374/734] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf83a88-396d-44ba-9b88-f4f1c5923d0d None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf83a88-396d-44ba-9b88-f4f1c5923d0d None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf83a88-396d-44ba-9b88-f4f1c5923d0d None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:05.035572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 375/735] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:05.036506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf83a88-396d-44ba-9b88-f4f1c5923d0d None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:05.071526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:05.161014 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6ed98-8ef8-467c-a09c-37bb4c0c784a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:05.204313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:05.204809 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf83a88-396d-44ba-9b88-f4f1c5923d0d None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:05.212631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b6c06-d2e4-4953-b56b-1b5aba638839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:05.213301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b6c06-d2e4-4953-b56b-1b5aba638839 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.213588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b6c06-d2e4-4953-b56b-1b5aba638839 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.213863 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b6c06-d2e4-4953-b56b-1b5aba638839 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:05.233246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b6c06-d2e4-4953-b56b-1b5aba638839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ac3b6c06-d2e4-4953-b56b-1b5aba638839 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:05.239731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:05.247588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 361/736] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:05.249406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc8f5-fe01-47ba-8f16-1cc19ae0f3a9 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.258944 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc8f5-fe01-47ba-8f16-1cc19ae0f3a9 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.258944 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc8f5-fe01-47ba-8f16-1cc19ae0f3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:05.264248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61421db-d91d-4673-b113-6d4034a63ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:05.265146 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61421db-d91d-4673-b113-6d4034a63ddc None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.265547 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61421db-d91d-4673-b113-6d4034a63ddc None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.266031 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61421db-d91d-4673-b113-6d4034a63ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:05.269892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-747fc8f5-fe01-47ba-8f16-1cc19ae0f3a9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:05.272169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-747fc8f5-fe01-47ba-8f16-1cc19ae0f3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:05.361367 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61421db-d91d-4673-b113-6d4034a63ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:05.363182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61421db-d91d-4673-b113-6d4034a63ddc None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:05.386168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 362/737] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:22:05.386168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fc8f5-fe01-47ba-8f16-1cc19ae0f3a9 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:05.407042 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:05.507847 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fc8f5-fe01-47ba-8f16-1cc19ae0f3a9 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:05.530766 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 376/738] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 09:22:05.530766 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61421db-d91d-4673-b113-6d4034a63ddc None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:05.538703 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:05.563450 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:05.564161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.564445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.564744 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:05.565495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:05.565818 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:05.567649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:05.586004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 363/739] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:05.587116 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:05.700901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61421db-d91d-4673-b113-6d4034a63ddc None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:05.730145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31424e-643c-49cd-8cd7-f6a4b2f01dd9 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:05.734747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:05.751401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:05.752100 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.752531 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.752824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:05.753754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:05.754072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:05.755696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:05.766422 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 377/740] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:05.767173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:05.773513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b9dc-76bf-4019-8d54-ccdf7987b9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:05.774912 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b9dc-76bf-4019-8d54-ccdf7987b9f8 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.775469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b9dc-76bf-4019-8d54-ccdf7987b9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.775826 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b9dc-76bf-4019-8d54-ccdf7987b9f8 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:05.776917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268b9dc-76bf-4019-8d54-ccdf7987b9f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:05.780023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268b9dc-76bf-4019-8d54-ccdf7987b9f8 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:05.800986 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 364/741] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:22:05 2022] POST /identity/v3/services => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:05.802134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b9dc-76bf-4019-8d54-ccdf7987b9f8 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:05.857337 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:05.940588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff7866-fde4-47da-ad7c-a05d5a4f293c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:05.945132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b9dc-76bf-4019-8d54-ccdf7987b9f8 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:05.985632 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9a967-86c3-4e53-bcac-b8c789143f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:05.986737 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9a967-86c3-4e53-bcac-b8c789143f21 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.987026 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9a967-86c3-4e53-bcac-b8c789143f21 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.987592 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9a967-86c3-4e53-bcac-b8c789143f21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:05.991934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:05.992616 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:05.992979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:05.993253 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.000091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9a967-86c3-4e53-bcac-b8c789143f21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:06.002875 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9a967-86c3-4e53-bcac-b8c789143f21 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:06.005415 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:06.006929 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-7cc9a967-86c3-4e53-bcac-b8c789143f21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:06.022208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:06.023328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:06.024419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:06.029223 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 365/742] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:22:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:06.029623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:06.030913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:06.032182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:06.032182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:06.032182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:06.032182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:06.032475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dbadd-1494-49df-9e80-73cde482dabd None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:06.032577 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 378/743] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:06.032782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dbadd-1494-49df-9e80-73cde482dabd None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:06.032782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dbadd-1494-49df-9e80-73cde482dabd None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.033922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:06.036473 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 366/744] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:06.037510 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dbadd-1494-49df-9e80-73cde482dabd None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:06.100391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:06.197587 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:06.205629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3e5bf-64c7-4e8d-8170-5b440c6ed7e8 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:06.209520 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dbadd-1494-49df-9e80-73cde482dabd None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:06.256529 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:06.257135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:06.257489 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:06.257768 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.258638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:06.258953 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:06.260435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:06.261009 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa790c4-89cd-49ff-b8a2-64f8acfa6e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:06.261790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa790c4-89cd-49ff-b8a2-64f8acfa6e2b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:06.262280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa790c4-89cd-49ff-b8a2-64f8acfa6e2b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:06.262682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa790c4-89cd-49ff-b8a2-64f8acfa6e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.277910 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa790c4-89cd-49ff-b8a2-64f8acfa6e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:06.279446 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa790c4-89cd-49ff-b8a2-64f8acfa6e2b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:06.289877 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 367/745] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:22:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:06.289877 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-caa790c4-89cd-49ff-b8a2-64f8acfa6e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:06.300633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:06.302081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:06.310421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:06.310421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:06.310421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:06.310421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:06.310421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 379/746] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:06.310421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cbe82-0ad5-4d0c-9a60-9cf44bde2372 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:06.310421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cbe82-0ad5-4d0c-9a60-9cf44bde2372 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:06.310421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cbe82-0ad5-4d0c-9a60-9cf44bde2372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.328246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-d36cbe82-0ad5-4d0c-9a60-9cf44bde2372 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:06.329618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-d36cbe82-0ad5-4d0c-9a60-9cf44bde2372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:06.430544 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 380/747] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 09:22:06.432198 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cbe82-0ad5-4d0c-9a60-9cf44bde2372 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:06.445480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9ebd8-d2e4-4c5b-bf00-bd3767cf2516 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:06.466853 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:06.490649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:06.492468 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:06.493401 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:06.493985 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.495279 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:06.495777 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:06.497429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:06.513073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 368/748] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:06.517265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:06.537884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:06.585099 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cbe82-0ad5-4d0c-9a60-9cf44bde2372 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:06.632262 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:06.632953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:06.633260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:06.633665 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.646476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:06.648070 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:06.652637 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3ff4c-5486-4819-94cf-7016d2f78b57 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:06.670524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:06.678613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:06.680166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:06.680166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:06.680166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:06.680166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:06.680166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 381/749] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:06.680166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:06.714464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:06.715244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:06.715532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:06.715849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.716667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:06.716988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:06.718693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:06.742675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 369/750] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:06.742675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:06.775163 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:06.824977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17795b6-402a-4343-8ce4-bf01fafa9305 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:06.872278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86631e-7f64-49fb-b7c3-7a1925f519ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:06.873057 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86631e-7f64-49fb-b7c3-7a1925f519ed None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:06.873388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86631e-7f64-49fb-b7c3-7a1925f519ed None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:06.873710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86631e-7f64-49fb-b7c3-7a1925f519ed None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:06.874575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86631e-7f64-49fb-b7c3-7a1925f519ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:06.876240 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86631e-7f64-49fb-b7c3-7a1925f519ed None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:06.898986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 382/751] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:22:06 2022] POST /identity/v3/services => generated 240 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:06.900787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86631e-7f64-49fb-b7c3-7a1925f519ed None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:06.977739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:07.010186 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb7c99-1661-4d61-bffc-3dd0f7901410 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:07.072895 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:07.074391 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:07.074974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:07.075657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:07.086425 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:07.089268 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:07.103597 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86631e-7f64-49fb-b7c3-7a1925f519ed None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:07.104831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:07.105964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:07.107023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:07.107023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:07.107023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:07.107023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:07.107023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:07.107023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:07.107023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:07.107023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:07.109679 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 370/752] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:07.111006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:07.149755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:07.163917 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:07.165356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:07.166030 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:07.166978 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:07.168465 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:07.169188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:07.171337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:07.201262 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 383/753] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:07.202953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:07.275524 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052289c-19bb-46b9-a86c-c8d91f244c3b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:07.333999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:07.344200 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:07.345596 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:07.346353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:07.346928 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] PATH_INFO: `/v3/users/swift` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:07.359906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:07.362482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:07.384282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:07.386755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:07.389297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 371/754] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:07.391065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:07.406760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31219767-d03e-4db7-8fff-a21c0dece871 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:07.476206 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:07.477451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:07.477963 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:07.478489 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:07.480068 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:07.480660 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:07.483751 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:07.508407 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 384/755] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:07.510033 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:07.532915 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74d33-c0af-42ad-b65b-3c6b1741ca5c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:07.579978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:07.579978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:07.579978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:07.579978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:07.586675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:07.586675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:07.602065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:07.604215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 372/756] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:07.612843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:07.614078 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:07.671478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:07.715024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b39317-a587-486e-af10-e762aa5a3229 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:07.739756 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2ff9a-14c7-421e-86d4-3dde69ea0079 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:07.783000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:07.785023 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:07.785212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:07.785933 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:07.786280 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:07.786683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:07.787026 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:07.787456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:07.788284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=a2c75126c70247d58c85cec5294cfc55) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:07.788966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:07.789702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:07.790500 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:07.791317 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:07.793810 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:07.806611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 373/757] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:07 2022] GET /identity/v3/role_assignments?user.id=a2c75126c70247d58c85cec5294cfc55&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:07.807708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:07.843333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 385/758] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:22:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:07.855873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:08.051166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51237fdf-d309-4116-a281-66e4878b2344 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:08.086552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:08.110288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41f0a3-c922-409e-b2d3-dc2404926d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:08.110288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41f0a3-c922-409e-b2d3-dc2404926d68 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.110288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41f0a3-c922-409e-b2d3-dc2404926d68 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.110288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41f0a3-c922-409e-b2d3-dc2404926d68 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.110781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41f0a3-c922-409e-b2d3-dc2404926d68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:08.110781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41f0a3-c922-409e-b2d3-dc2404926d68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:08.110781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41f0a3-c922-409e-b2d3-dc2404926d68 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:08.136529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5635d08-efa0-4461-bb79-640570f990da None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:08.136896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 374/759] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:08.138697 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c156ca6-d3b6-4bf1-b6e8-fc819296d84f None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.139327 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c156ca6-d3b6-4bf1-b6e8-fc819296d84f None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.139807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c156ca6-d3b6-4bf1-b6e8-fc819296d84f None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.141771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 375/760] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:08.142849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c156ca6-d3b6-4bf1-b6e8-fc819296d84f None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:08.151482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:08.209606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4488b-1c02-4259-9d08-396752a13745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:08.210779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4488b-1c02-4259-9d08-396752a13745 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.211343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4488b-1c02-4259-9d08-396752a13745 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.211817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4488b-1c02-4259-9d08-396752a13745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.300073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4488b-1c02-4259-9d08-396752a13745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:08.302046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4488b-1c02-4259-9d08-396752a13745 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:08.333860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c156ca6-d3b6-4bf1-b6e8-fc819296d84f None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:08.379949 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b468b4-1735-4702-9d2c-3893b3bffbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:08.381155 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b468b4-1735-4702-9d2c-3893b3bffbcf None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.381711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b468b4-1735-4702-9d2c-3893b3bffbcf None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.382279 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b468b4-1735-4702-9d2c-3893b3bffbcf None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/0970fc7ae258457ca70042acc0fc6c4c/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.385015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b468b4-1735-4702-9d2c-3893b3bffbcf None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=0970fc7ae258457ca70042acc0fc6c4c, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:08.387210 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b468b4-1735-4702-9d2c-3893b3bffbcf None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:08.410285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 376/761] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:22:08 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/0970fc7ae258457ca70042acc0fc6c4c/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:08.412326 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9c4e6-6a7c-4fd9-ae6b-bf7bea61b5e1 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.412905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9c4e6-6a7c-4fd9-ae6b-bf7bea61b5e1 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.413431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9c4e6-6a7c-4fd9-ae6b-bf7bea61b5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.450217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-9ea9c4e6-6a7c-4fd9-ae6b-bf7bea61b5e1 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:08.452005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-9ea9c4e6-6a7c-4fd9-ae6b-bf7bea61b5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:08.526448 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 386/762] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:08.532193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4488b-1c02-4259-9d08-396752a13745 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:08.553775 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:08.561804 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:08.681347 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 377/763] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:08.705780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9c4e6-6a7c-4fd9-ae6b-bf7bea61b5e1 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:08.774320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4488b-1c02-4259-9d08-396752a13745 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:08.836653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9c4e6-6a7c-4fd9-ae6b-bf7bea61b5e1 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:08.880872 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:08.882795 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.883472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.884101 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.886486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:08.887195 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.887482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.887776 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.888709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:08.890684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:08.903356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:08.905954 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:08.912644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:08.914739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:08.916733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:08.928076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 378/764] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 09:22:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:08.929615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:08.945403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:08.947255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:08.949299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:08.949299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:08.949299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:08.949299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:08.949299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 387/765] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:08.955473 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.032938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:09.033859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:09.136279 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe77e51-4814-478c-b739-6b7eddf24bc9 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.199870 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c997b-e903-4cb4-9e87-d5687b62e1cc None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.220994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.221655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.221939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.222250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.222981 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:09.223296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.224764 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.256485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 388/766] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:09.260382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.288807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.289480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.289802 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.290090 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.290885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:09.291202 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.292623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.303876 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 379/767] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:22:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:09.310904 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.427017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0809-65ba-41df-9d68-18bd80c2470f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.457429 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:09.496586 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:09.516749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.518299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.519073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.519843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.528653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025664b8-bbeb-4360-99c0-14f82f710748 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.550536 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.553403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:09.592957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:09.594193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:09.595229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:09.595229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:09.595229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:09.595229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:09.595229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:09.595229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:09.595229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:09.595229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:09.605902 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 389/768] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:09.607937 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.638580 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.639626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.640127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.640629 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.641948 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:09.642516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.645450 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.660489 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 380/769] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:09.670799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.831194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d2e89-5f42-4d93-b6da-4bf88d3bc835 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.858919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da5949-6eeb-4cbe-a5d2-777ce4b621e9 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.904805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080b6c1-e231-48eb-a982-703a4e4b3a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.905482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080b6c1-e231-48eb-a982-703a4e4b3a78 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.905767 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080b6c1-e231-48eb-a982-703a4e4b3a78 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.906039 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080b6c1-e231-48eb-a982-703a4e4b3a78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.906972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080b6c1-e231-48eb-a982-703a4e4b3a78 None admin] RBAC: Adding query filter params (interface=public, service_id=71ec86e0817b47bc81e6cdd15f3f01ca, region_id=RegionOne) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:09.907291 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080b6c1-e231-48eb-a982-703a4e4b3a78 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.908687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080b6c1-e231-48eb-a982-703a4e4b3a78 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.909181 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.910637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.911281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.911892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.913462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:09.914260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.917034 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.917593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 381/770] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/endpoints?service_id=71ec86e0817b47bc81e6cdd15f3f01ca&interface=public®ion_id=RegionOne => generated 194 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:09.919016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648d900-7b67-4c73-bcf6-eedabb0e7da7 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.919305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648d900-7b67-4c73-bcf6-eedabb0e7da7 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.919583 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648d900-7b67-4c73-bcf6-eedabb0e7da7 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.921208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 382/771] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:09.932221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf006b8-204d-46bf-bd9c-4868ed0aa12e None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.932884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf006b8-204d-46bf-bd9c-4868ed0aa12e None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.933387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf006b8-204d-46bf-bd9c-4868ed0aa12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.951807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 390/772] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:09.951807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.964003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-caf006b8-204d-46bf-bd9c-4868ed0aa12e None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:09.966336 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-caf006b8-204d-46bf-bd9c-4868ed0aa12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:09.993342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:10.009258 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:10.084970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20277b-2621-473b-b615-5053aced8753 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:10.094142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 383/773] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:10.094142 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf006b8-204d-46bf-bd9c-4868ed0aa12e None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:10.149668 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055041d7-6f4a-4f77-a8a4-1aae6581d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:10.150704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055041d7-6f4a-4f77-a8a4-1aae6581d266 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.151153 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055041d7-6f4a-4f77-a8a4-1aae6581d266 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.151609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055041d7-6f4a-4f77-a8a4-1aae6581d266 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.152480 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055041d7-6f4a-4f77-a8a4-1aae6581d266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:10.154331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055041d7-6f4a-4f77-a8a4-1aae6581d266 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:10.166278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 391/774] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/services => generated 772 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:10.167574 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055041d7-6f4a-4f77-a8a4-1aae6581d266 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:10.254152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf006b8-204d-46bf-bd9c-4868ed0aa12e None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:10.295191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecf191-2ce9-4285-b28f-abd4260863a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:10.296880 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecf191-2ce9-4285-b28f-abd4260863a6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.297187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecf191-2ce9-4285-b28f-abd4260863a6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.297473 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecf191-2ce9-4285-b28f-abd4260863a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.302149 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecf191-2ce9-4285-b28f-abd4260863a6 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=6726b8e1313a4181bcc32ed3697f4fd0) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:10.302149 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecf191-2ce9-4285-b28f-abd4260863a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:10.302149 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecf191-2ce9-4285-b28f-abd4260863a6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:10.309917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 384/775] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:10 2022] GET /identity/v3/role_assignments?user.id=6726b8e1313a4181bcc32ed3697f4fd0&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=af958d22229a4a0297bbe2e643f55181 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:10.312425 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76945926-f550-4d28-a12e-e0767f7d6c0a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.312724 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76945926-f550-4d28-a12e-e0767f7d6c0a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.313080 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76945926-f550-4d28-a12e-e0767f7d6c0a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.314870 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 385/776] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:10.325078 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdb700-08bf-4216-849f-450af2f9db16 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.325578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdb700-08bf-4216-849f-450af2f9db16 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.326031 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdb700-08bf-4216-849f-450af2f9db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.372642 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-35cdb700-08bf-4216-849f-450af2f9db16 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:10.377458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-35cdb700-08bf-4216-849f-450af2f9db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:10.408573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:10.426454 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055041d7-6f4a-4f77-a8a4-1aae6581d266 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:10.434561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:10.501824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:10.502559 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.502849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.503188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] PATH_INFO: `/v3/services/network` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.504056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:10.505660 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:10.518252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:10.519514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:10.520790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 392/777] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:10 2022] GET /identity/v3/services/network => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:10.529215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:10.551630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 386/778] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:10.591147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdb700-08bf-4216-849f-450af2f9db16 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:10.704917 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db0171-d982-4b27-a993-1f2f301ce6d6 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:10.757184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:10.757844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.758190 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.760311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.762957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] RBAC: Adding query filter params (name=network) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:10.763282 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:10.764759 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:10.780521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 393/779] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:10.787701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:10.839466 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:10.849273 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdb700-08bf-4216-849f-450af2f9db16 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:10.929825 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaaa88-70d9-42d3-a2c2-1144fd536a7b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:10.955061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50887da4-ea71-418e-9044-a25a5cc077c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:10.955660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50887da4-ea71-418e-9044-a25a5cc077c6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.955951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50887da4-ea71-418e-9044-a25a5cc077c6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.956239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50887da4-ea71-418e-9044-a25a5cc077c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.977566 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:10.978811 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.979440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.980009 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.981118 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] RBAC: Adding query filter params (type=network) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:10.981660 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:10.983342 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:10.987227 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50887da4-ea71-418e-9044-a25a5cc077c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:10.990888 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 394/780] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-50887da4-ea71-418e-9044-a25a5cc077c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:10.999108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:11.001234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:11.003312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 387/781] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:22:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:11.006611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50887da4-ea71-418e-9044-a25a5cc077c6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:11.007438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:11.020148 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:11.152556 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:11.168113 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d30532-934b-4c54-b673-dad3cda29333 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:11.175496 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50887da4-ea71-418e-9044-a25a5cc077c6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:11.236817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:11.237966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:11.239584 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:11.241147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:11.242392 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:11.243152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:11.246938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:11.269964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:11.270935 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:11.273280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:11.273814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:11.274906 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] RBAC: Adding query filter params (interface=public, service_id=06a52885bee14b61acb1ec25012001f6, region_id=RegionOne) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:11.275397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 388/782] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:22:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:11.275903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:11.277852 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:11.279862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:11.299027 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 395/783] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:22:11 2022] GET /identity/v3/endpoints?service_id=06a52885bee14b61acb1ec25012001f6&interface=public®ion_id=RegionOne => generated 194 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:11.305184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:11.359167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:11.403058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b40a9-d5d3-4be9-9f70-26c8b5b29027 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:11.460728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0a1c-5b68-4e20-aca7-b0cc8bd93fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:11.464960 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0a1c-5b68-4e20-aca7-b0cc8bd93fd9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:11.465335 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0a1c-5b68-4e20-aca7-b0cc8bd93fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:11.465789 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0a1c-5b68-4e20-aca7-b0cc8bd93fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:11.508091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0a1c-5b68-4e20-aca7-b0cc8bd93fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:11.510375 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0a1c-5b68-4e20-aca7-b0cc8bd93fd9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:11.555815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea678357-deb4-4415-96e2-7ddd8c513e24 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-d5ef0a1c-5b68-4e20-aca7-b0cc8bd93fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:11.556613 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:11.558247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:11.560235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:11.560235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:11.560235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:11.560235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:11.561340 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 389/784] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:11.565687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab672-81c2-4ff6-b095-20e9ab1728e7 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:11.565687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab672-81c2-4ff6-b095-20e9ab1728e7 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:11.565687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab672-81c2-4ff6-b095-20e9ab1728e7 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:11.568458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 390/785] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:11.569952 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ab672-81c2-4ff6-b095-20e9ab1728e7 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:11.612185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25cf4-86a1-424e-9ded-c8fa265bb893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:11.612944 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25cf4-86a1-424e-9ded-c8fa265bb893 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:11.613229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25cf4-86a1-424e-9ded-c8fa265bb893 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:11.613508 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25cf4-86a1-424e-9ded-c8fa265bb893 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:11.614309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd25cf4-86a1-424e-9ded-c8fa265bb893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:11.615711 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd25cf4-86a1-424e-9ded-c8fa265bb893 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:11.630320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 396/786] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:22:11 2022] GET /identity/v3/services => generated 772 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:11.632097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27a67c-8a80-4263-a04f-88f64f62e73a None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:11.632395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27a67c-8a80-4263-a04f-88f64f62e73a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:11.632832 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27a67c-8a80-4263-a04f-88f64f62e73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:11.671328 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-2b27a67c-8a80-4263-a04f-88f64f62e73a None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:11.672656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-2b27a67c-8a80-4263-a04f-88f64f62e73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:11.732319 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:11.817717 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ab672-81c2-4ff6-b095-20e9ab1728e7 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:11.856045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 397/787] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:11.873556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:11.904097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27a67c-8a80-4263-a04f-88f64f62e73a None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:11.905161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:11.905845 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:11.906162 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:11.906449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:11.907179 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:11.907492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:11.909801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:11.928791 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 391/788] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:11.930460 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:12.128537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27a67c-8a80-4263-a04f-88f64f62e73a None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:12.150515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fe8bb-2aa3-4a96-8e5d-87a0db1a4c61 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:12.223297 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3973cf5-fdd2-47ba-b549-c440a848cce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:12.224208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973cf5-fdd2-47ba-b549-c440a848cce7 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.224530 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973cf5-fdd2-47ba-b549-c440a848cce7 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.224872 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973cf5-fdd2-47ba-b549-c440a848cce7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.241783 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3973cf5-fdd2-47ba-b549-c440a848cce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a3973cf5-fdd2-47ba-b549-c440a848cce7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:12.247088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:12.248162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:12.249213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:12.250013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:12.250013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.250013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.250269 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 398/789] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:12.250346 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.257272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3973cf5-fdd2-47ba-b549-c440a848cce7 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:12.267332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:12.273906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:12.297690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:12.302909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:12.304056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:12.305167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 392/790] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:22:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:12.310636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:12.364378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:12.499243 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3973cf5-fdd2-47ba-b549-c440a848cce7 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:12.508598 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2765c11-a4c0-42ce-9bdc-907d5ab5cb3a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:12.552348 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:12.553373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.553839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.554388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.555397 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:12.555879 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:12.558477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:12.573443 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 399/791] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:12.582895 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:12.591949 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:12.592604 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.592889 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.593175 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.594016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:12.594359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:12.595848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:12.641562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 393/792] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:22:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:12.644081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:12.754130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:12.778539 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:12.791428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4405339-71a6-4cb6-af47-5c2c4a445c52 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:12.816961 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009ff46-d56e-42c4-8088-be1bcb2cdb7f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:12.857094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:12.857813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.858251 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.858533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.878826 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511cfba-9f43-42ee-a8d4-f86421692bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:12.879515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cfba-9f43-42ee-a8d4-f86421692bc2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.880145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cfba-9f43-42ee-a8d4-f86421692bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.880441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cfba-9f43-42ee-a8d4-f86421692bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.890223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:12.893622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511cfba-9f43-42ee-a8d4-f86421692bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:12.895192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511cfba-9f43-42ee-a8d4-f86421692bc2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:12.898817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-b511cfba-9f43-42ee-a8d4-f86421692bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:12.922301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:12.923657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:12.924687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:12.924687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:12.924687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:12.924687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:12.925674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:12.926669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:12.927675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:12.927675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:12.927675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:12.927675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:12.928679 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 394/793] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:12.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ad2ed-68d5-4984-9f39-087f6ad15e74 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ad2ed-68d5-4984-9f39-087f6ad15e74 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ad2ed-68d5-4984-9f39-087f6ad15e74 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 395/795] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:12.934289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ad2ed-68d5-4984-9f39-087f6ad15e74 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:12.934769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 400/795] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:12.936939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.085088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b207cd-635e-4953-afb8-f1357215404f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.103322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ad2ed-68d5-4984-9f39-087f6ad15e74 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.127977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:13.139320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8dc48-da9b-477f-a879-aae9a55fe446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.140224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8dc48-da9b-477f-a879-aae9a55fe446 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.140504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8dc48-da9b-477f-a879-aae9a55fe446 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.141121 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8dc48-da9b-477f-a879-aae9a55fe446 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.142328 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8dc48-da9b-477f-a879-aae9a55fe446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:13.142627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8dc48-da9b-477f-a879-aae9a55fe446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.144517 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8dc48-da9b-477f-a879-aae9a55fe446 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.146822 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ee253-d57c-4a8c-9a9c-554588aee72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.147853 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ee253-d57c-4a8c-9a9c-554588aee72f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.148293 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ee253-d57c-4a8c-9a9c-554588aee72f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.148641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ee253-d57c-4a8c-9a9c-554588aee72f None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.149725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ee253-d57c-4a8c-9a9c-554588aee72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:13.150303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ee253-d57c-4a8c-9a9c-554588aee72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.152711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ee253-d57c-4a8c-9a9c-554588aee72f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.164690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 401/796] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:13.164690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a01d1d-c25a-4b9b-b2fd-5c6d15e8d698 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.164690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a01d1d-c25a-4b9b-b2fd-5c6d15e8d698 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.164690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a01d1d-c25a-4b9b-b2fd-5c6d15e8d698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.171936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 396/797] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:13.171936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da1ce9-8212-4aec-8a0a-bef8c5fbd364 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.171936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da1ce9-8212-4aec-8a0a-bef8c5fbd364 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.171936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da1ce9-8212-4aec-8a0a-bef8c5fbd364 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.171936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 397/798] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:13.172463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da1ce9-8212-4aec-8a0a-bef8c5fbd364 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.192806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-c7a01d1d-c25a-4b9b-b2fd-5c6d15e8d698 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:13.193149 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:13.195140 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-c7a01d1d-c25a-4b9b-b2fd-5c6d15e8d698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:13.293846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 402/799] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:13.294548 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a01d1d-c25a-4b9b-b2fd-5c6d15e8d698 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.336118 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da1ce9-8212-4aec-8a0a-bef8c5fbd364 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.391512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e98af-d892-4f7a-b2e0-57a01ade4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.392425 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e98af-d892-4f7a-b2e0-57a01ade4af1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.392776 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e98af-d892-4f7a-b2e0-57a01ade4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.393069 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e98af-d892-4f7a-b2e0-57a01ade4af1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.401160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e98af-d892-4f7a-b2e0-57a01ade4af1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.402751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e98af-d892-4f7a-b2e0-57a01ade4af1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-d74e98af-d892-4f7a-b2e0-57a01ade4af1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:13.418035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:13.419456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 398/800] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:13.420993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40022776-d21c-4515-8322-d6792cc2cc25 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.421887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40022776-d21c-4515-8322-d6792cc2cc25 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.421887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40022776-d21c-4515-8322-d6792cc2cc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.434580 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a01d1d-c25a-4b9b-b2fd-5c6d15e8d698 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.442509 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-40022776-d21c-4515-8322-d6792cc2cc25 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:13.443876 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-40022776-d21c-4515-8322-d6792cc2cc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:13.458010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:13.487087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.487829 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.488160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.488496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.499599 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.501240 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:13.514771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:13.516084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:13.517812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:13.517812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:13.517812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:13.517812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:13.517812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:13.517812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:13.517812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:13.517812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:13.522474 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 403/801] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:13.523488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.532801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:13.546320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 399/802] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:13.546320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40022776-d21c-4515-8322-d6792cc2cc25 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.664671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b53aa-3b5b-4757-a409-0d34076c6ce1 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.687862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40022776-d21c-4515-8322-d6792cc2cc25 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.711697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.712395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.712678 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.713051 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.713729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.715423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:13.730952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:13.733450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 09:22:13.735911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:13.738650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 404/803] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:13.739846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.756412 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.756813 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.756998 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.757156 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.758215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:13.758532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.760637 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.787131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 400/804] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:13.787131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.802716 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:13.852192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:13.882486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35360cf-ec3b-4eea-aebd-0d52e01ab6c2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.916377 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4080fc2-38ac-4874-a3bc-55ce8cd07e47 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.930233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.931105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.931525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.931960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.932915 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:13.933380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.935184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.960951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b715a38-a22b-4f27-86ee-11812b7acab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.961610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715a38-a22b-4f27-86ee-11812b7acab6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.961884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715a38-a22b-4f27-86ee-11812b7acab6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.962202 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715a38-a22b-4f27-86ee-11812b7acab6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.962610 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 405/805] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:13.963865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.976957 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b715a38-a22b-4f27-86ee-11812b7acab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-2b715a38-a22b-4f27-86ee-11812b7acab6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:13.982640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:13.984793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 401/806] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:13.986683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b715a38-a22b-4f27-86ee-11812b7acab6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.113769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b715a38-a22b-4f27-86ee-11812b7acab6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.116279 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e9a17-8f7e-4e70-bc98-c439d06e5349 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.118849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:14.143506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:14.164444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.165343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.165758 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.166131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.167133 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:14.167479 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.168031 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed49ff-57c4-4e84-b9a0-4c9718f5b4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.168820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed49ff-57c4-4e84-b9a0-4c9718f5b4fe None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.169148 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed49ff-57c4-4e84-b9a0-4c9718f5b4fe None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.169432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.169551 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed49ff-57c4-4e84-b9a0-4c9718f5b4fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.182525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 406/807] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:14.184272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.262886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed49ff-57c4-4e84-b9a0-4c9718f5b4fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.265058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed49ff-57c4-4e84-b9a0-4c9718f5b4fe None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.326056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9643a-7686-4a98-804f-9d14a6ca83b3 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.385488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ded29-c36b-460e-a021-b0c0ea8c0d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.386406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ded29-c36b-460e-a021-b0c0ea8c0d6a None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.386834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ded29-c36b-460e-a021-b0c0ea8c0d6a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.387144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ded29-c36b-460e-a021-b0c0ea8c0d6a None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/a2c75126c70247d58c85cec5294cfc55/roles/aec37946c06745cfb2ea73e9b3c9a547` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.392873 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 402/808] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:14.393829 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed49ff-57c4-4e84-b9a0-4c9718f5b4fe None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.412386 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ded29-c36b-460e-a021-b0c0ea8c0d6a None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=a2c75126c70247d58c85cec5294cfc55, role_id=aec37946c06745cfb2ea73e9b3c9a547)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.413812 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ded29-c36b-460e-a021-b0c0ea8c0d6a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.416464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:14.432650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 407/809] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:22:14 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/a2c75126c70247d58c85cec5294cfc55/roles/aec37946c06745cfb2ea73e9b3c9a547 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:14.432650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ded29-c36b-460e-a021-b0c0ea8c0d6a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.471275 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:14.568006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed49ff-57c4-4e84-b9a0-4c9718f5b4fe None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.630769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ded29-c36b-460e-a021-b0c0ea8c0d6a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.650766 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.651309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.651591 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.651862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.652651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:14.652963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.654922 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.669345 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 403/810] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:14.670942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.697589 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.702888 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.703519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.703721 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.704387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:14.704853 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.707065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.716636 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 408/811] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:14.717567 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.829434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:14.844565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:14.866891 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dba77-a962-4f96-9126-b7a4d9cfa2e1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.869712 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df910d8-5b4e-49d0-b72b-a3cf4c306e03 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.914020 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.914790 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.915174 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.915561 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.918047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a13d0c-7272-4bbd-9616-171500ac7b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.921403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a13d0c-7272-4bbd-9616-171500ac7b61 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.921403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a13d0c-7272-4bbd-9616-171500ac7b61 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.921403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a13d0c-7272-4bbd-9616-171500ac7b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.932350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.934080 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.944212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a13d0c-7272-4bbd-9616-171500ac7b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.947361 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a13d0c-7272-4bbd-9616-171500ac7b61 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:14.951057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:14.952209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:14.953495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:14.953495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:14.953495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:14.953495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:14.956971 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 404/812] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:14.957959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-93a13d0c-7272-4bbd-9616-171500ac7b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:14.977296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:14.979173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:14.980731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:14.980731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:14.980731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:14.980731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:14.980731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 409/813] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:14.980731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523c67-2047-46fa-9374-dee8d0180424 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.980731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523c67-2047-46fa-9374-dee8d0180424 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.981228 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523c67-2047-46fa-9374-dee8d0180424 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.982839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 410/814] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:14.983715 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523c67-2047-46fa-9374-dee8d0180424 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:15.113725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b3af-6f6b-485b-970a-b5e39154e1df None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:15.140271 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523c67-2047-46fa-9374-dee8d0180424 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:15.158397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:15.165749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207042db-7573-4ffe-82de-e5e9d1563618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:15.166290 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207042db-7573-4ffe-82de-e5e9d1563618 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:15.166602 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207042db-7573-4ffe-82de-e5e9d1563618 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:15.166910 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207042db-7573-4ffe-82de-e5e9d1563618 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:15.167716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207042db-7573-4ffe-82de-e5e9d1563618 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:15.169246 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207042db-7573-4ffe-82de-e5e9d1563618 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:15.192151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:15.210138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 405/815] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:15.210138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207042db-7573-4ffe-82de-e5e9d1563618 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:15.235423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb81a5-064f-414e-8658-5b1477be2c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:15.236587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb81a5-064f-414e-8658-5b1477be2c26 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:15.237208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb81a5-064f-414e-8658-5b1477be2c26 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:15.237703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb81a5-064f-414e-8658-5b1477be2c26 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:15.238645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb81a5-064f-414e-8658-5b1477be2c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:15.238965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb81a5-064f-414e-8658-5b1477be2c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:15.241199 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb81a5-064f-414e-8658-5b1477be2c26 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:15.262341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 411/816] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:15.263963 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f728d-65fd-4d7d-9188-8d36a89304bf None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:15.264384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f728d-65fd-4d7d-9188-8d36a89304bf None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:15.264741 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f728d-65fd-4d7d-9188-8d36a89304bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:15.298138 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-111f728d-65fd-4d7d-9188-8d36a89304bf None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:15.301349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-111f728d-65fd-4d7d-9188-8d36a89304bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:15.363068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207042db-7573-4ffe-82de-e5e9d1563618 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:15.412519 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:15.412519 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:15.413144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:15.413415 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:15.414579 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:15.415155 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:15.418222 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:15.439411 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 406/817] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:15.441541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:15.463962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 412/818] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:15.465661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f728d-65fd-4d7d-9188-8d36a89304bf None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:15.531320 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:15.625547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c87bf-f11e-4b41-8beb-a726d803852c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:15.651896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f728d-65fd-4d7d-9188-8d36a89304bf None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:15.680558 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:15.681543 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:15.682514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:15.682991 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:15.683365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:15.691892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:15.693515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:15.715100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:15.716682 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:15.718147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:15.718147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:15.718147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:15.718147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:15.718147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:15.718147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:15.718147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:15.718147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:15.722511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 407/819] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:15.723410 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:15.724025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:15.727077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:15.727773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:15.728830 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:15.743684 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:15.745807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:15.768151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:15.769630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:15.770930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 413/820] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:15.771800 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:15.890160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab7d82-b981-404d-9d65-49ba87f705ec None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:15.900959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:15.913808 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d647e4-7d18-46ba-a439-13c51823f5ab None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:15.948855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:15.950214 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:15.950904 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:15.951404 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] PATH_INFO: `/v3/services/network` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:15.952831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:15.955985 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:15.968492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:15.971493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 09:22:15.974120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:15.983343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:15.983343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:15.983343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:15.983343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:15.983807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 408/821] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:15 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:15.983923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:15.984015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:15.984106 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:15.984106 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:16.006452 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 414/822] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:16.008016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:16.070806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:16.126858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918663f-0fb7-4198-9392-8f33113c234b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:16.140149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a2678-fa04-4328-a952-b71b13026a4b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:16.169354 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:16.170407 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.170906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.171473 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.172494 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:16.173000 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:16.174689 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:16.179785 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:16.180435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.180722 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.181001 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.181704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] RBAC: Adding query filter params (name=network) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:16.182020 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:16.183781 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:16.195019 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 415/823] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:16.195693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:16.204869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 409/824] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:22:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:16.206211 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:16.239570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:16.323345 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e79653-5215-47f8-b5dd-83c4d1b94ecd None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:16.325066 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:16.335532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb857-bd1f-49fe-9ade-7d0f374ae825 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:16.363505 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:16.364412 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.364844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.365298 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.366900 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=0970fc7ae258457ca70042acc0fc6c4c) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:16.367292 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:16.369753 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:16.375621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:16.376619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.377089 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.377561 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.378509 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] RBAC: Adding query filter params (type=network) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:16.379040 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:16.380728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:16.387502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 410/826] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:16.387769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 416/826] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:16 2022] GET /identity/v3/role_assignments?user.id=0970fc7ae258457ca70042acc0fc6c4c&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=af958d22229a4a0297bbe2e643f55181 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:16.388714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:16.395285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:16.588626 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:16.603633 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790450a-579e-4f44-88fa-6e53edefcfc1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:16.619380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9f80c-1c4d-41d6-a5a0-35e1b1efd26c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:16.659449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b5e5a-7e1e-4d9c-83ad-68188f033337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:16.660582 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b5e5a-7e1e-4d9c-83ad-68188f033337 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.661107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b5e5a-7e1e-4d9c-83ad-68188f033337 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.661630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b5e5a-7e1e-4d9c-83ad-68188f033337 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.663063 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b5e5a-7e1e-4d9c-83ad-68188f033337 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:16.664750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b5e5a-7e1e-4d9c-83ad-68188f033337 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:16.679042 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:16.680003 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.680320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.680754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.699751 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:16.701623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:16.706681 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 411/827] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:16.720047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:16.721347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:16.722671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:16.722671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:16.722671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:16.722671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:16.724471 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 417/828] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:16.731667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b5e5a-7e1e-4d9c-83ad-68188f033337 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:16.884981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b5e5a-7e1e-4d9c-83ad-68188f033337 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:16.925013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c433f7-fa8a-4b8c-9e63-daf46ccfd3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:16.925695 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c433f7-fa8a-4b8c-9e63-daf46ccfd3cc None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.925978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c433f7-fa8a-4b8c-9e63-daf46ccfd3cc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.926321 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c433f7-fa8a-4b8c-9e63-daf46ccfd3cc None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.927087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c433f7-fa8a-4b8c-9e63-daf46ccfd3cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:16.927397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c433f7-fa8a-4b8c-9e63-daf46ccfd3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:16.928794 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c433f7-fa8a-4b8c-9e63-daf46ccfd3cc None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:16.943653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 412/829] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:16.949778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:16.961503 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446cef9-5a02-4787-bd07-1d7d2da1468b None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.962165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446cef9-5a02-4787-bd07-1d7d2da1468b None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.962165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446cef9-5a02-4787-bd07-1d7d2da1468b None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.963859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 413/830] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:16.981501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b746-1db1-415b-8724-6bc2e0a8ebba None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:16.981833 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b746-1db1-415b-8724-6bc2e0a8ebba None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:16.982313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b746-1db1-415b-8724-6bc2e0a8ebba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:16.996650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:17.018488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:17.021310 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-6485b746-1db1-415b-8724-6bc2e0a8ebba None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:17.022955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-6485b746-1db1-415b-8724-6bc2e0a8ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:17.103569 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb08a-0eee-4e69-8c4e-01f867d23298 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.112789 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 414/831] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:17.144291 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485b746-1db1-415b-8724-6bc2e0a8ebba None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.164119 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.165018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.165302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.165582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.178993 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:17.180802 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:17.200836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:17.202432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:17.203445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:17.203445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:17.203445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:17.203445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:17.203445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:17.203445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:17.203445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:17.203445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:17.206302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 418/832] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:17.209436 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.309247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485b746-1db1-415b-8724-6bc2e0a8ebba None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.340990 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:17.367988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c7965-d3d0-4168-88bd-92f28542bcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.368691 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c7965-d3d0-4168-88bd-92f28542bcff None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.368989 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c7965-d3d0-4168-88bd-92f28542bcff None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.369270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c7965-d3d0-4168-88bd-92f28542bcff None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.372015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:17.384635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91da8-418f-4b58-a5c2-712ddf5f2556 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.385970 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c7965-d3d0-4168-88bd-92f28542bcff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-767c7965-d3d0-4168-88bd-92f28542bcff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:17.393209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:17.394861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:17.396145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 415/833] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:22:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:17.401784 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c7965-d3d0-4168-88bd-92f28542bcff None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.445281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.446048 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.446452 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.446802 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.448327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:17.448760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:17.450795 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:17.474232 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 419/834] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:17.476733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.553429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c7965-d3d0-4168-88bd-92f28542bcff None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.600322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.601408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.601955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.602472 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.603598 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:17.604119 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:17.605918 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:17.617035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 416/835] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:22:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:17.625343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.630843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f7bb1-786e-4bdb-a4c6-54a40d57d45b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.695626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a92f31-72f4-471e-98cf-bafda487ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.696291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a92f31-72f4-471e-98cf-bafda487ecb2 None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.696623 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a92f31-72f4-471e-98cf-bafda487ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.696913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a92f31-72f4-471e-98cf-bafda487ecb2 None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/6726b8e1313a4181bcc32ed3697f4fd0/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.700471 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a92f31-72f4-471e-98cf-bafda487ecb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=6726b8e1313a4181bcc32ed3697f4fd0, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:17.702960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a92f31-72f4-471e-98cf-bafda487ecb2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:17.708204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:17.719092 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:17.730560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 420/836] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:22:17 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/6726b8e1313a4181bcc32ed3697f4fd0/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:17.797540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f712c-425a-4b9c-90df-dd458b949e70 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.801949 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941a21a-fd3e-4606-a8de-1d48251cc594 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.802400 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941a21a-fd3e-4606-a8de-1d48251cc594 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.802582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941a21a-fd3e-4606-a8de-1d48251cc594 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.804434 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 421/837] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:17.811715 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987edb24-b7f5-4593-bc09-a950e717b24e None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.812136 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987edb24-b7f5-4593-bc09-a950e717b24e None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.812621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987edb24-b7f5-4593-bc09-a950e717b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.841436 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-987edb24-b7f5-4593-bc09-a950e717b24e None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:17.843189 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-987edb24-b7f5-4593-bc09-a950e717b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:17.849186 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.850324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.850860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.851361 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.864576 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:17.866528 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:17.883095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:17.884192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:17.885225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:17.885225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:17.885225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:17.885225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:17.892271 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 417/838] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:17.894137 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.965833 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 422/839] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:18.003304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987edb24-b7f5-4593-bc09-a950e717b24e None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.038270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c2521-1350-4b9e-b0d6-b27dcd52b656 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.050054 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:18.096162 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.097580 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.098086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.098763 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.100497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:18.101945 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.104960 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.135335 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 418/840] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:18.141001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.145131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987edb24-b7f5-4593-bc09-a950e717b24e None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.180550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:18.190996 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.192089 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.192452 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.192793 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.193513 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.195291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:18.205169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:18.207431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:18.208634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:18.211696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 423/841] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:18.213165 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.288201 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9073769-cd6c-474e-ba42-be3781390696 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.340349 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.341704 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.342369 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.342943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.343494 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca745e-8f46-4283-95dd-a72a28fc549b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.357688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.360286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:18.381364 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:18.382558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:18.383641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:18.388066 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.388950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.389426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.389886 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.390250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 419/842] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:18.391087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:18.391584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.392247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.393497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.400630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 424/843] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:18.409078 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.432679 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:18.492797 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:18.530610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd17de-8fd9-47c3-b8f2-08aac323c62e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.558157 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad6770-9c74-4f6d-acd6-4dce7019e585 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.574278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.575304 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.575897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.576425 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.577568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:18.578093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.579930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.607718 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 420/844] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:18.610241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.611330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864acb5d-85f9-425c-8dcf-30917e76c1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.611985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864acb5d-85f9-425c-8dcf-30917e76c1a2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.612268 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864acb5d-85f9-425c-8dcf-30917e76c1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.612546 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864acb5d-85f9-425c-8dcf-30917e76c1a2 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.613253 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864acb5d-85f9-425c-8dcf-30917e76c1a2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:18.613562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864acb5d-85f9-425c-8dcf-30917e76c1a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.615230 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864acb5d-85f9-425c-8dcf-30917e76c1a2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.626162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 425/845] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:18.722615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:18.736284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bc3dd-cc24-4cc9-8e5c-7d725367e1a0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.776451 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33154c32-ecbb-4159-b797-981f1afb5f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.777146 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33154c32-ecbb-4159-b797-981f1afb5f42 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.777462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33154c32-ecbb-4159-b797-981f1afb5f42 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.777739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33154c32-ecbb-4159-b797-981f1afb5f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.856472 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33154c32-ecbb-4159-b797-981f1afb5f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.859620 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33154c32-ecbb-4159-b797-981f1afb5f42 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.020770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:19.030488 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2bc7-b788-421c-9aac-590f921a2f4b None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.030856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2bc7-b788-421c-9aac-590f921a2f4b None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.031055 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2bc7-b788-421c-9aac-590f921a2f4b None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.032761 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 426/846] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:19.038486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64b4d-c368-4c22-81f0-7b5c8d4861d9 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.038752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64b4d-c368-4c22-81f0-7b5c8d4861d9 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.039076 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64b4d-c368-4c22-81f0-7b5c8d4861d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.052393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 421/847] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:19.062152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33154c32-ecbb-4159-b797-981f1afb5f42 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.066640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-64a64b4d-c368-4c22-81f0-7b5c8d4861d9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:19.068016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-64a64b4d-c368-4c22-81f0-7b5c8d4861d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:19.158295 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 427/848] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:19.179496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a64b4d-c368-4c22-81f0-7b5c8d4861d9 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.189795 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33154c32-ecbb-4159-b797-981f1afb5f42 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.238767 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.239437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.239713 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.240274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.241524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:19.257719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.259341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:19.280658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:19.282938 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:19.285170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:19.285170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:19.285170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:19.285170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:19.289289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 422/849] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:19.289289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.306641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a64b4d-c368-4c22-81f0-7b5c8d4861d9 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.343956 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:19.351036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.352280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.352785 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.353330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.354310 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.356224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:19.367965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:19.369152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:19.370309 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:19.379390 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 428/850] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:19.380718 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.418744 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186c99-99cb-4a09-a7c7-ad28bcc8f9fa None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.462637 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.463274 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.463740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.464021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.465076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:19.465397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.467526 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.486206 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 423/851] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:19.490514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.511912 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456d1bd-6528-456d-b30b-2f03ee65291c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.528034 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:19.554631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.555550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.555782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.556066 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.556859 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:19.557596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.559135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.570909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 429/852] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:19.576867 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.631465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:19.636020 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a312287-2a2a-4d04-a169-62f18432004c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.677225 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.678546 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.679044 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.679529 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.687868 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.689803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:19.703437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:19.704605 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:19.705700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:19.705700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:19.705700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:19.705700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:19.705700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:19.705700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:19.705700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:19.705700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:19.706773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de15a-bcf8-4a24-b735-a7ee413e858a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.713608 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 424/853] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:19.715581 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.754036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.754857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.755172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.755534 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.756292 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:19.756603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.758528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.770264 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 430/854] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:19.841906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a609059-0d59-4bdc-8c33-b005de3068c3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.884128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e475a-6361-4092-89e3-1cdb94d3d3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.885175 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e475a-6361-4092-89e3-1cdb94d3d3ba None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.885673 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e475a-6361-4092-89e3-1cdb94d3d3ba None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.886270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e475a-6361-4092-89e3-1cdb94d3d3ba None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.887431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e475a-6361-4092-89e3-1cdb94d3d3ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:19.887981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e475a-6361-4092-89e3-1cdb94d3d3ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.889696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e475a-6361-4092-89e3-1cdb94d3d3ba None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.913734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 425/855] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:19.919339 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.990860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:20.048703 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0b44-7a59-48aa-953f-3a9ec438d0b1 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:20.049426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0b44-7a59-48aa-953f-3a9ec438d0b1 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:20.050122 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0b44-7a59-48aa-953f-3a9ec438d0b1 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:20.057438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 426/856] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:20.063500 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991c456-2ae0-4e13-b0c5-5149d225f8ec None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:20.064013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991c456-2ae0-4e13-b0c5-5149d225f8ec None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:20.064628 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991c456-2ae0-4e13-b0c5-5149d225f8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:20.083031 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460cfd0-60b9-45e3-8033-ce64b420dd94 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:20.099417 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-e991c456-2ae0-4e13-b0c5-5149d225f8ec None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:20.101588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-e991c456-2ae0-4e13-b0c5-5149d225f8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:20.120028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:20.120709 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:20.121166 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:20.121455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:20.122357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=a2c75126c70247d58c85cec5294cfc55) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:20.122683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:20.124298 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:20.131392 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 431/857] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/role_assignments?user.id=a2c75126c70247d58c85cec5294cfc55&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:20.134590 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:20.239447 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 427/858] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:20.263361 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:20.395765 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef97c3e-262b-4e69-ba47-91155df82cc2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:20.404137 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:20.446206 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92e78d-5b7d-4051-9e18-4502bf9b8997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:20.447433 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e78d-5b7d-4051-9e18-4502bf9b8997 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:20.447923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e78d-5b7d-4051-9e18-4502bf9b8997 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:20.448533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e78d-5b7d-4051-9e18-4502bf9b8997 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:20.465520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92e78d-5b7d-4051-9e18-4502bf9b8997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-7d92e78d-5b7d-4051-9e18-4502bf9b8997 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:20.474084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:20.475406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:20.476486 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:20.477303 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 432/859] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:20.487282 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991c456-2ae0-4e13-b0c5-5149d225f8ec None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:20.644258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991c456-2ae0-4e13-b0c5-5149d225f8ec None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:20.684363 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:20.685140 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:20.685427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:20.685705 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:20.686493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:20.686801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:20.688208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:20.699245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 428/860] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:20.720320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92e78d-5b7d-4051-9e18-4502bf9b8997 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:20.847225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92e78d-5b7d-4051-9e18-4502bf9b8997 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:20.896965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:20.898813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:20.898813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:20.898813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:20.930630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:20.932192 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:20.952054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:20.953216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:20.954304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:20.954304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:20.954304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:20.954304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:20.954304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 433/861] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:20.963400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:21.002024 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:21.159957 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c9dcd-5bbe-49e9-a919-dbd6432fd2bd None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:21.209461 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4430a-2ce5-4684-8f2f-72fd1e1a54f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:21.214136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4430a-2ce5-4684-8f2f-72fd1e1a54f2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:21.214136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4430a-2ce5-4684-8f2f-72fd1e1a54f2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:21.214136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4430a-2ce5-4684-8f2f-72fd1e1a54f2 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:21.214136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4430a-2ce5-4684-8f2f-72fd1e1a54f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:21.214136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4430a-2ce5-4684-8f2f-72fd1e1a54f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:21.216302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4430a-2ce5-4684-8f2f-72fd1e1a54f2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:21.238798 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 429/862] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:21.244678 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:21.263254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e215f-c16d-4c53-893a-d4846e5d1646 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:21.263638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e215f-c16d-4c53-893a-d4846e5d1646 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:21.263832 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e215f-c16d-4c53-893a-d4846e5d1646 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:21.265864 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 430/863] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:21.277017 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f2f62-9e10-40e6-8b94-4c681e4cc2ff None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:21.278256 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f2f62-9e10-40e6-8b94-4c681e4cc2ff None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:21.279217 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:21.280963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f2f62-9e10-40e6-8b94-4c681e4cc2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:21.321516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-639f2f62-9e10-40e6-8b94-4c681e4cc2ff None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:21.324494 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-639f2f62-9e10-40e6-8b94-4c681e4cc2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:21.408823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b242-63de-40e8-8ce3-1420fa834f13 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:21.442369 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:21.455859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 431/864] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:21.467519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:21.468204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:21.468504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:21.468760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] PATH_INFO: `/v3/users/swift` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:21.481854 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f2f62-9e10-40e6-8b94-4c681e4cc2ff None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:21.486207 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:21.488835 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:21.506300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:21.507702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 09:22:21.509229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:21.512960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 434/865] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:21.519237 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:21.597196 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f2f62-9e10-40e6-8b94-4c681e4cc2ff None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:21.635293 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91320cd4-db91-4f16-b20a-1153af756c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:21.635935 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320cd4-db91-4f16-b20a-1153af756c28 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:21.636220 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320cd4-db91-4f16-b20a-1153af756c28 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:21.636518 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320cd4-db91-4f16-b20a-1153af756c28 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:21.637242 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91320cd4-db91-4f16-b20a-1153af756c28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:21.638855 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91320cd4-db91-4f16-b20a-1153af756c28 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:21.670658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 432/866] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:22:21 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:21.671830 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:21.709154 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64ca1f-eb11-432e-a081-38ed33fb213b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:21.774460 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:21.776201 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:21.776917 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:21.777717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:21.779665 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:21.780619 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:21.784065 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:21.821046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 435/867] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 09:22:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:21.827215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91320cd4-db91-4f16-b20a-1153af756c28 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:21.903578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:21.954829 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91320cd4-db91-4f16-b20a-1153af756c28 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:21.999686 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5f8f2-d6d6-4779-88de-2a2b595ab3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:22.000283 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5f8f2-d6d6-4779-88de-2a2b595ab3ec None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.000606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5f8f2-d6d6-4779-88de-2a2b595ab3ec None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.000922 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5f8f2-d6d6-4779-88de-2a2b595ab3ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.086568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5f8f2-d6d6-4779-88de-2a2b595ab3ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:22.088799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5f8f2-d6d6-4779-88de-2a2b595ab3ec None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:22.163786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:22.217567 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 433/868] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:21 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:22.224386 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:22.275051 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8aa763-be3b-4345-97da-0eec20f3d304 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.275264 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8aa763-be3b-4345-97da-0eec20f3d304 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.275608 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8aa763-be3b-4345-97da-0eec20f3d304 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.277297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 434/869] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:22.283625 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bd3d3-76af-4ced-8c9c-833bd8bc3230 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.284012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bd3d3-76af-4ced-8c9c-833bd8bc3230 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.284212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bd3d3-76af-4ced-8c9c-833bd8bc3230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.314152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-422bd3d3-76af-4ced-8c9c-833bd8bc3230 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:22.314152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-422bd3d3-76af-4ced-8c9c-833bd8bc3230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:22.354533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24168b0-2569-43c2-9f3d-6e6f4a83482d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:22.388976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:22.395692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:22.396721 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.397212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.397690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.403607 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 435/870] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:22.414390 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:22.416234 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:22.426479 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422bd3d3-76af-4ced-8c9c-833bd8bc3230 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:22.434784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:22.435880 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:22.436925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:22.436925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:22.436925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:22.436925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:22.439278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 436/871] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:22.447589 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:22.598041 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:22.601635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972dacb-96f8-4bf4-8cdd-576721c24137 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:22.628099 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422bd3d3-76af-4ced-8c9c-833bd8bc3230 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:22.655223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa98b5-4d6d-408e-a6c0-9a6d78824394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:22.656644 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa98b5-4d6d-408e-a6c0-9a6d78824394 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.657085 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa98b5-4d6d-408e-a6c0-9a6d78824394 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.657733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa98b5-4d6d-408e-a6c0-9a6d78824394 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.659037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa98b5-4d6d-408e-a6c0-9a6d78824394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:22.659610 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa98b5-4d6d-408e-a6c0-9a6d78824394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:22.662136 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa98b5-4d6d-408e-a6c0-9a6d78824394 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:22.683606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 437/872] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:22.684959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cd49b-162c-447d-ab46-ed1f0ad07066 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.685257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cd49b-162c-447d-ab46-ed1f0ad07066 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.685532 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cd49b-162c-447d-ab46-ed1f0ad07066 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.687148 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 438/873] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:22.692844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef902e2b-d5ba-4fb5-9b99-e8deb27f3b9e None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.693149 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef902e2b-d5ba-4fb5-9b99-e8deb27f3b9e None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.693450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef902e2b-d5ba-4fb5-9b99-e8deb27f3b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.696325 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d12ee0-44a6-4bd1-a328-a288785776b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:22.697309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d12ee0-44a6-4bd1-a328-a288785776b1 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.697774 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d12ee0-44a6-4bd1-a328-a288785776b1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.698260 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d12ee0-44a6-4bd1-a328-a288785776b1 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.699825 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d12ee0-44a6-4bd1-a328-a288785776b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:22.701533 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d12ee0-44a6-4bd1-a328-a288785776b1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:22.716353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-ef902e2b-d5ba-4fb5-9b99-e8deb27f3b9e None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:22.717851 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-ef902e2b-d5ba-4fb5-9b99-e8deb27f3b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:22.730618 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 436/874] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:22:22 2022] POST /identity/v3/services => generated 269 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:22.730618 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d12ee0-44a6-4bd1-a328-a288785776b1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:22.780003 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:22.823878 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 439/875] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:22.848794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef902e2b-d5ba-4fb5-9b99-e8deb27f3b9e None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:22.887388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d12ee0-44a6-4bd1-a328-a288785776b1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:22.933313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:22.934618 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:22.935176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:22.935675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:22.945286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:22.947431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:22.961660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:22.963163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:22.964435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:22.964435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:22.964435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:22.964435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:22.964435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:22.964435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:22.964435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:22.964435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:22.967095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 437/876] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:22.972640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef902e2b-d5ba-4fb5-9b99-e8deb27f3b9e None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:22.975933 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:22.995731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:23.018337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc5855-068f-4f43-af64-3e1e8cfb9118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:23.019273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc5855-068f-4f43-af64-3e1e8cfb9118 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:23.019470 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc5855-068f-4f43-af64-3e1e8cfb9118 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:23.019749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc5855-068f-4f43-af64-3e1e8cfb9118 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:23.033413 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc5855-068f-4f43-af64-3e1e8cfb9118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-aadc5855-068f-4f43-af64-3e1e8cfb9118 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:23.040215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:23.042976 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:23.044570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 440/877] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:22:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:23.048561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc5855-068f-4f43-af64-3e1e8cfb9118 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:23.097730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:23.159135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3f3fc-d728-4598-acc1-0ab50ff19877 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:23.175129 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc5855-068f-4f43-af64-3e1e8cfb9118 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:23.214336 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:23.215014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:23.215014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:23.215363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:23.215596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:23.216410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:23.216644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:23.216757 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:23.217231 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:23.217710 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:23.218139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:23.218954 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:23.219476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:23.221137 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:23.228045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 441/878] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:22:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:23.238882 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:23.239963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 438/879] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:23.246276 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:23.363502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:23.401806 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1c961-3a4a-4f47-b2be-d214f5a78ae7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:23.408779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:23.425375 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937a3a9-d7c1-4d0c-981f-4d950eac19de None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:23.467458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:23.468486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:23.468958 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:23.469411 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:23.470837 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=6726b8e1313a4181bcc32ed3697f4fd0) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:23.471363 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:23.471854 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:23.472781 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:23.473258 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:23.473735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:23.473950 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:23.487697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:23.489532 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:23.494324 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 439/880] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:23 2022] GET /identity/v3/role_assignments?user.id=6726b8e1313a4181bcc32ed3697f4fd0&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=af958d22229a4a0297bbe2e643f55181 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:23.510627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:23.511816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:23.512855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:23.512855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:23.512855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:23.512855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:23.515215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 442/881] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:23.521632 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:23.686016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e80dfb-c4fe-4959-9d99-086ef2da8da3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:23.734844 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859afd9-da45-4aab-a174-a0583efe9996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:23.735937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859afd9-da45-4aab-a174-a0583efe9996 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:23.736434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859afd9-da45-4aab-a174-a0583efe9996 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:23.736918 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859afd9-da45-4aab-a174-a0583efe9996 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:23.737999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859afd9-da45-4aab-a174-a0583efe9996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:23.738461 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859afd9-da45-4aab-a174-a0583efe9996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:23.740651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859afd9-da45-4aab-a174-a0583efe9996 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:23.743735 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:23.763869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 440/882] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:23.770379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:23.925266 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ac45b-c6b9-47bf-a200-e4dc53b74dc8 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:23.974127 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:23.975178 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:23.975683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:23.976193 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:23.991509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:23.994328 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.015834 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e4ad-d0d1-481c-a2bc-10996c0197a1 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.016347 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e4ad-d0d1-481c-a2bc-10996c0197a1 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:24.016637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:24.017667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:24.018873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:24.019611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e4ad-d0d1-481c-a2bc-10996c0197a1 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.019702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 443/883] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:22:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:24.019869 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 441/884] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:24.027246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.031469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fdc1-4536-412f-b6ce-8b388118fc73 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.031983 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fdc1-4536-412f-b6ce-8b388118fc73 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.032492 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fdc1-4536-412f-b6ce-8b388118fc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.032974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:24.068178 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-dca5fdc1-4536-412f-b6ce-8b388118fc73 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:24.069765 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-dca5fdc1-4536-412f-b6ce-8b388118fc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:24.161054 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 442/885] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:24.184962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a2dc8-1ea1-416e-8aa5-d55f2fd595c1 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.185196 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5fdc1-4536-412f-b6ce-8b388118fc73 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.235524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.236605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.237170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.238207 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.239783 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:24.240396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.242795 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.277733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 444/886] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:24.278570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.327398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5fdc1-4536-412f-b6ce-8b388118fc73 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.361458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:24.377778 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.378857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.379362 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.379847 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.381252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.383479 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:24.393815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:24.395428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:24.396566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:24.400455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 443/887] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:24.401384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.402487 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:24.437994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe00ddb-e871-4ef1-bf4f-3e64c8ecd5cc None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.485797 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d149d-dc80-48ad-9154-98fdd7a6a42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.487419 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d149d-dc80-48ad-9154-98fdd7a6a42d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.487992 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d149d-dc80-48ad-9154-98fdd7a6a42d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.489117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d149d-dc80-48ad-9154-98fdd7a6a42d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.558383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4b14e-77fa-458f-85e6-991b99a82533 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.606903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d149d-dc80-48ad-9154-98fdd7a6a42d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.606903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d149d-dc80-48ad-9154-98fdd7a6a42d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.612217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.613269 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.613805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.614422 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.615632 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:24.616278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.618956 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.633930 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 444/888] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:24.635065 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.697530 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:24.720407 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:24.783533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 445/889] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 09:22:24.787360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d149d-dc80-48ad-9154-98fdd7a6a42d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.805309 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dce3f-a96b-4f2f-b6df-81845282bdaa None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.855047 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.856431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.856893 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.857365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.858848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:24.859401 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.862076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.869067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 445/890] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:24.875408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.929970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d149d-dc80-48ad-9154-98fdd7a6a42d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.985924 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.987067 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.987566 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.988208 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.001696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85c7a1-064b-4e7b-adc6-0a233208fe91 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.014754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.017578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.021004 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:25.042621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:25.044362 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:25.045690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:25.045690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:25.045690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:25.045690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:25.048692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 446/891] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:25.049953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.051419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.052376 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.053188 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.054759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.055980 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] RBAC: Adding query filter params (interface=public, service_id=2b70f57760d84aa593ba811f8c32eb40, region_id=RegionOne) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:25.056569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.058612 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.067134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 446/892] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/endpoints?service_id=2b70f57760d84aa593ba811f8c32eb40&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:25.073703 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.078244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:25.201036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3887c6-8f29-43fe-ae8c-e92f86a80787 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.242205 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c2092-d668-4f97-8456-f29a585e08ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.243173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c2092-d668-4f97-8456-f29a585e08ce None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.243467 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c2092-d668-4f97-8456-f29a585e08ce None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.243906 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c2092-d668-4f97-8456-f29a585e08ce None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.244853 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c2092-d668-4f97-8456-f29a585e08ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:25.245308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c2092-d668-4f97-8456-f29a585e08ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.247332 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c2092-d668-4f97-8456-f29a585e08ce None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.250854 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75aceb-3327-443b-a6ec-fef542fb0432 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.269532 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 447/893] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:25.269532 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a98d3-0aee-403b-9419-3e85612bda17 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.269532 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a98d3-0aee-403b-9419-3e85612bda17 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.269532 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a98d3-0aee-403b-9419-3e85612bda17 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.270050 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 448/894] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:25.271946 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a98d3-0aee-403b-9419-3e85612bda17 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.300367 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3dac2-d838-424b-9890-6905f1b0990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.301204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3dac2-d838-424b-9890-6905f1b0990e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.301204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3dac2-d838-424b-9890-6905f1b0990e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.301549 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3dac2-d838-424b-9890-6905f1b0990e None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.302132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3dac2-d838-424b-9890-6905f1b0990e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.303588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3dac2-d838-424b-9890-6905f1b0990e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.318134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 447/895] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:22:25 2022] GET /identity/v3/services => generated 1268 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:22:25.318134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97863b-bb0e-4663-98ae-459718f104b5 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.318134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97863b-bb0e-4663-98ae-459718f104b5 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.318134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97863b-bb0e-4663-98ae-459718f104b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.346684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-7f97863b-bb0e-4663-98ae-459718f104b5 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:25.348890 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-7f97863b-bb0e-4663-98ae-459718f104b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:25.397142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:25.435704 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:25.450315 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a98d3-0aee-403b-9419-3e85612bda17 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.471960 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 448/896] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:25.499839 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97863b-bb0e-4663-98ae-459718f104b5 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.515834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.516528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.516805 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.517084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.526512 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.528317 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:25.543618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:25.544985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:25.546308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 449/897] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:25.551110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.661719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97863b-bb0e-4663-98ae-459718f104b5 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.682975 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7c73d-184a-4703-b52c-ddf872418104 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.709610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.711134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.712073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.712717 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.713810 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.716161 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.723474 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.723973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.724255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.724494 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.725370 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:25.725658 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.727225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:25.734772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:25.736266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:25.737463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:25.741705 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 449/898] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 09:22:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:25.741910 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:25.750707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 450/899] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:25.750975 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.752080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.814628 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:25.879902 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96907722-d71e-42f5-95b0-c2dc7354ad00 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.904134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe26569-07b0-4e42-8bca-1197c019e198 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.926480 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d837f68-98e9-42a2-836f-3df3964d3944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.927696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d837f68-98e9-42a2-836f-3df3964d3944 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.928871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d837f68-98e9-42a2-836f-3df3964d3944 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.929585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d837f68-98e9-42a2-836f-3df3964d3944 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.931016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d837f68-98e9-42a2-836f-3df3964d3944 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:25.931838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d837f68-98e9-42a2-836f-3df3964d3944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.934957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d837f68-98e9-42a2-836f-3df3964d3944 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.948623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.949332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.949639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.949925 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.950779 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] RBAC: Adding query filter params (role.id=8d82d0ddc65e4d268c8039cfbd1335a5, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=a2c75126c70247d58c85cec5294cfc55) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:25.951110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.952603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.958420 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 451/900] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:25.959909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 450/901] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:25 2022] GET /identity/v3/role_assignments?user.id=a2c75126c70247d58c85cec5294cfc55&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=8d82d0ddc65e4d268c8039cfbd1335a5 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:25.960547 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3f7f-37da-49d3-b91a-b51a6fd0d207 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.961158 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3f7f-37da-49d3-b91a-b51a6fd0d207 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.961610 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3f7f-37da-49d3-b91a-b51a6fd0d207 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.963696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 452/902] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:25.971872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.975423 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d71a6-8eb6-4be2-93bc-242aa8e442de None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.975818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d71a6-8eb6-4be2-93bc-242aa8e442de None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.976052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d71a6-8eb6-4be2-93bc-242aa8e442de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:26.003083 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-638d71a6-8eb6-4be2-93bc-242aa8e442de None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:26.004914 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-638d71a6-8eb6-4be2-93bc-242aa8e442de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:26.083570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:26.110174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 453/903] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:26.140618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d71a6-8eb6-4be2-93bc-242aa8e442de None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:26.141955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abdb9c-7c1a-4a70-8aa2-0488aa244e90 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:26.147443 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:26.184998 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:26.185658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:26.186007 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:26.186367 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:26.187169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:26.187589 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:26.189457 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:26.196412 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 451/904] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:26.203023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:26.258777 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d71a6-8eb6-4be2-93bc-242aa8e442de None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:26.298351 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:26.299023 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:26.299301 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:26.299578 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:26.300294 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:26.301689 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:26.311855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:26.313641 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:26.315426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:26.319723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 454/905] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:26.322511 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:26.350936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9c717-071a-4e07-9f64-2029e73d535e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:26.384595 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:26.406653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:26.408125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:26.408677 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:26.409339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:26.411248 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] RBAC: Adding query filter params (interface=public, service_id=d9c8796422f64adb89f0961e56dfa6a1, region_id=RegionOne) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:26.412295 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:26.415733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:26.437468 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 452/906] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:22:26 2022] GET /identity/v3/endpoints?service_id=d9c8796422f64adb89f0961e56dfa6a1&interface=public®ion_id=RegionOne => generated 194 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:26.440310 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:26.444621 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad113572-9d54-4c34-be89-249687e71e16 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:26.484649 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:26.485840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:26.485840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:26.485840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:26.486723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:26.487117 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:26.488695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:26.499324 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 455/907] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:26.502631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:26.517171 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:26.630513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cb947-cfeb-498a-88f4-cd0641e7773a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:26.652857 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15d9ce-8498-4548-848a-4c22402f454f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:26.674793 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:26.696607 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6f7b4-c7f2-4cef-9e7c-3f15f1d1b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:26.697339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f7b4-c7f2-4cef-9e7c-3f15f1d1b078 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:26.698023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f7b4-c7f2-4cef-9e7c-3f15f1d1b078 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:26.698500 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f7b4-c7f2-4cef-9e7c-3f15f1d1b078 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:26.699408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f7b4-c7f2-4cef-9e7c-3f15f1d1b078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:26.701329 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f7b4-c7f2-4cef-9e7c-3f15f1d1b078 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:26.703551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18feea67-5ce1-40e1-89b2-d43e8d414940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:26.704609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feea67-5ce1-40e1-89b2-d43e8d414940 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:26.705240 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feea67-5ce1-40e1-89b2-d43e8d414940 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:26.705769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feea67-5ce1-40e1-89b2-d43e8d414940 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:26.707024 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18feea67-5ce1-40e1-89b2-d43e8d414940 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:26.707743 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18feea67-5ce1-40e1-89b2-d43e8d414940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:26.709660 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18feea67-5ce1-40e1-89b2-d43e8d414940 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:26.718141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 453/908] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:22:26 2022] GET /identity/v3/services => generated 1268 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:22:26.719263 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 456/909] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:27.670885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561cdeb-7f9b-41e6-8413-77a62d674a9a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:27.671472 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561cdeb-7f9b-41e6-8413-77a62d674a9a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:27.671472 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561cdeb-7f9b-41e6-8413-77a62d674a9a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:27.673136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 454/910] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:27.679528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca87d41-90ed-4dd8-8aaf-db94d691b6cf None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:27.680039 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca87d41-90ed-4dd8-8aaf-db94d691b6cf None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:27.680137 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca87d41-90ed-4dd8-8aaf-db94d691b6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:27.701465 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-4ca87d41-90ed-4dd8-8aaf-db94d691b6cf None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:27.702719 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-4ca87d41-90ed-4dd8-8aaf-db94d691b6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:27.813714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 457/911] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:27.839918 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561cdeb-7f9b-41e6-8413-77a62d674a9a None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:27.950411 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:27.961085 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561cdeb-7f9b-41e6-8413-77a62d674a9a None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:28.002621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:28.003296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:28.003579 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:28.003857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:28.004503 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:28.005900 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:28.016887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:28.018516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:28.020110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 455/912] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:28.025096 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca87d41-90ed-4dd8-8aaf-db94d691b6cf None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:28.142044 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca87d41-90ed-4dd8-8aaf-db94d691b6cf None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:28.154671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:28.185882 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cb983-b14c-4012-8d50-b9f4fcb16014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:28.186561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cb983-b14c-4012-8d50-b9f4fcb16014 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:28.186868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cb983-b14c-4012-8d50-b9f4fcb16014 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:28.187162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cb983-b14c-4012-8d50-b9f4fcb16014 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:28.187908 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cb983-b14c-4012-8d50-b9f4fcb16014 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:28.188236 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cb983-b14c-4012-8d50-b9f4fcb16014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:28.189643 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cb983-b14c-4012-8d50-b9f4fcb16014 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:28.195767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 458/913] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:28.203351 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:28.205230 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97466-0399-420d-b8e7-63495eec4dec None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:28.205516 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97466-0399-420d-b8e7-63495eec4dec None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:28.205791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97466-0399-420d-b8e7-63495eec4dec None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:28.207181 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 459/914] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:28.212386 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598055a-2194-46a6-8576-98a908b6f15f None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:28.212653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598055a-2194-46a6-8576-98a908b6f15f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:28.212934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598055a-2194-46a6-8576-98a908b6f15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:28.232840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-7598055a-2194-46a6-8576-98a908b6f15f None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:28.234242 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-7598055a-2194-46a6-8576-98a908b6f15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:28.342271 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 460/915] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:28.351366 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bfaf2-233b-4a27-a2f9-68bede1c1469 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:28.363997 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598055a-2194-46a6-8576-98a908b6f15f None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:28.415434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:28.416529 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:28.417090 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:28.417665 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:28.418638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:28.419194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:28.421696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:28.429423 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 456/916] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 09:22:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:28.436121 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:28.465288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:28.517147 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598055a-2194-46a6-8576-98a908b6f15f None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:28.520080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:28.552395 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e094442-c089-4d6a-987e-6ad61a4bc7b5 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:28.559194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0667b-25f0-4378-820a-393c6055208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:28.559868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0667b-25f0-4378-820a-393c6055208b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:28.560155 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0667b-25f0-4378-820a-393c6055208b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:28.560437 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0667b-25f0-4378-820a-393c6055208b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:28.573184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0667b-25f0-4378-820a-393c6055208b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-60f0667b-25f0-4378-820a-393c6055208b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:28.578932 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:28.580071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:28.581182 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:28.586092 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 461/917] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:22:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:28.586977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0667b-25f0-4378-820a-393c6055208b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:28.603976 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d077f-37b2-4b96-9893-da57755c73a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:28.605222 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d077f-37b2-4b96-9893-da57755c73a8 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:28.605872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d077f-37b2-4b96-9893-da57755c73a8 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:28.606442 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d077f-37b2-4b96-9893-da57755c73a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:28.607393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d077f-37b2-4b96-9893-da57755c73a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:28.609228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d077f-37b2-4b96-9893-da57755c73a8 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:28.642934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 457/918] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:28.734799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0667b-25f0-4378-820a-393c6055208b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:28.790388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:28.791492 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:28.791989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:28.792521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:28.793603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:28.794191 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:28.795870 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:28.806711 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 462/919] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:22:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:28.813687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d077f-37b2-4b96-9893-da57755c73a8 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:28.929179 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:28.959062 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d077f-37b2-4b96-9893-da57755c73a8 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:29.001350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b4c22-7511-4828-a65b-9eb207b9eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:29.001987 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b4c22-7511-4828-a65b-9eb207b9eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.002313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b4c22-7511-4828-a65b-9eb207b9eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.002640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b4c22-7511-4828-a65b-9eb207b9eaa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.020738 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b4c22-7511-4828-a65b-9eb207b9eaa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:29.023719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b4c22-7511-4828-a65b-9eb207b9eaa3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-258b4c22-7511-4828-a65b-9eb207b9eaa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:29.050820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:29.051902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:29.052913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:29.052913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:29.052913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:29.052913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:29.053611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 458/920] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:29.059215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:29.102842 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:29.140733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947913-baa0-4243-a680-7400752eba6d None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.141025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947913-baa0-4243-a680-7400752eba6d None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.141311 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947913-baa0-4243-a680-7400752eba6d None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.143264 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 459/921] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:29.152021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595232f9-3c0d-4860-b467-db132fa41597 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.152303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595232f9-3c0d-4860-b467-db132fa41597 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.152587 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595232f9-3c0d-4860-b467-db132fa41597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.178556 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-595232f9-3c0d-4860-b467-db132fa41597 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:29.180150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-595232f9-3c0d-4860-b467-db132fa41597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:29.204133 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00129ff5-5f8e-483c-9559-8b5eb2fde0c0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:29.255077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea7baa-7939-423b-a67c-2a24e9fd265b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:29.255614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea7baa-7939-423b-a67c-2a24e9fd265b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.255902 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea7baa-7939-423b-a67c-2a24e9fd265b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.256188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea7baa-7939-423b-a67c-2a24e9fd265b None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.256921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea7baa-7939-423b-a67c-2a24e9fd265b None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:29.257293 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea7baa-7939-423b-a67c-2a24e9fd265b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:29.258722 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea7baa-7939-423b-a67c-2a24e9fd265b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:29.279421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 463/922] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:29.279421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98ace-16e7-435e-a69e-e15a9433d8af None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.279421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98ace-16e7-435e-a69e-e15a9433d8af None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.279421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98ace-16e7-435e-a69e-e15a9433d8af None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.279421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 464/923] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:29.282849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 460/924] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:29.285210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e98ace-16e7-435e-a69e-e15a9433d8af None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:29.287890 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bca56-5820-4bbf-9765-30ce12190bf0 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.288523 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bca56-5820-4bbf-9765-30ce12190bf0 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.289285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bca56-5820-4bbf-9765-30ce12190bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.312463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-1b2bca56-5820-4bbf-9765-30ce12190bf0 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:29.314497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-1b2bca56-5820-4bbf-9765-30ce12190bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:29.398361 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:29.399529 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:29.415150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 461/925] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:29.416021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bca56-5820-4bbf-9765-30ce12190bf0 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:29.439773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e98ace-16e7-435e-a69e-e15a9433d8af None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:29.487130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:29.487990 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.488323 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.488630 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.499985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:29.501701 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:29.518173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:29.519748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 465/926] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:22:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:29.527791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:29.546517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bca56-5820-4bbf-9765-30ce12190bf0 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:29.610627 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:29.611845 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.612209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.612582 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.613455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:29.614915 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:29.624803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:29.626213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:29.627711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 462/927] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 09:22:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:29.629095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:29.667451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade3c0-2a9b-4843-a8fc-bc1f63c64486 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:29.708940 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:29.720376 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0394bfb-ebd7-4e8a-8971-ba201cc16ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:29.721000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394bfb-ebd7-4e8a-8971-ba201cc16ee1 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.721286 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394bfb-ebd7-4e8a-8971-ba201cc16ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.721573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394bfb-ebd7-4e8a-8971-ba201cc16ee1 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.722384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0394bfb-ebd7-4e8a-8971-ba201cc16ee1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:29.723833 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0394bfb-ebd7-4e8a-8971-ba201cc16ee1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:29.742485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 466/928] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:22:29 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:29.743333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0394bfb-ebd7-4e8a-8971-ba201cc16ee1 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:29.749160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:29.791147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7b998-e537-443a-98e8-9db31839cd0d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:29.853562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:29.854625 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.855160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.855625 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.856849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:29.857428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:29.859999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:29.887746 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 463/929] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:22:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:29.895731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:29.916324 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0394bfb-ebd7-4e8a-8971-ba201cc16ee1 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:29.962874 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:29.963592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:29.963882 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:29.964195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:29.964986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:29.965304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:29.966792 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:29.978046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 467/930] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:29.980785 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:30.059503 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007a00-acaf-4ff2-82c6-0af0d25adea3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:30.075596 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:30.079627 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:30.113095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:30.114086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.114409 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.114695 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:30.132189 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d88a62-6ead-4daf-ba1c-0223327079ee None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:30.134578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:30.137611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:30.160688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:30.161828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:30.162897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:30.162897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:30.162897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:30.162897 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:30.163906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 464/931] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:30.170020 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:30.183154 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:30.184797 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.185305 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.185799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:30.187280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:30.187845 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:30.189994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:30.199085 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 468/932] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:30.205697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:30.311236 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4dfba-d65e-4832-8e6a-c8ef23aa2abf None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:30.367024 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:30.367998 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.368411 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.368969 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:30.370029 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec1b4d-07d1-4919-8bcb-7ea7dc76b4d7 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:30.370233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:30.371032 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:30.373105 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:30.392793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 465/933] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:30.400496 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:30.404795 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:30.412446 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720e3fe-2c17-4083-aa7d-812cf770d1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:30.413444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720e3fe-2c17-4083-aa7d-812cf770d1a2 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.413965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720e3fe-2c17-4083-aa7d-812cf770d1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.414464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720e3fe-2c17-4083-aa7d-812cf770d1a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:30.415398 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720e3fe-2c17-4083-aa7d-812cf770d1a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:30.417001 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720e3fe-2c17-4083-aa7d-812cf770d1a2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:30.440641 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:30.445084 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 469/934] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:30.574806 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d3426-864b-4e53-ae11-7548ae22fe16 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:30.625057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:30.625743 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.626033 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.626359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:30.634808 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:30.636312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:30.649585 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:30.650731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:30.651780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 466/935] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:30.659268 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720e3fe-2c17-4083-aa7d-812cf770d1a2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:30.779748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720e3fe-2c17-4083-aa7d-812cf770d1a2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:30.816978 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bbec9-2be5-49bd-935e-3abe2310a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:30.818071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bbec9-2be5-49bd-935e-3abe2310a09c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.818653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bbec9-2be5-49bd-935e-3abe2310a09c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.819139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bbec9-2be5-49bd-935e-3abe2310a09c None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:30.820144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bbec9-2be5-49bd-935e-3abe2310a09c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:30.820679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bbec9-2be5-49bd-935e-3abe2310a09c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:30.822376 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bbec9-2be5-49bd-935e-3abe2310a09c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:30.836383 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 470/936] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:30.842849 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:30.874377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d54eb-8532-4851-b0f6-c2bd7e97cb3c None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.875021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d54eb-8532-4851-b0f6-c2bd7e97cb3c None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.875508 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d54eb-8532-4851-b0f6-c2bd7e97cb3c None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:30.877425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 471/937] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:30.882767 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1092a7-40b3-4719-8520-01d9871d8b21 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.883364 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1092a7-40b3-4719-8520-01d9871d8b21 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.883889 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1092a7-40b3-4719-8520-01d9871d8b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:30.905215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-6c1092a7-40b3-4719-8520-01d9871d8b21 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:30.906637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-6c1092a7-40b3-4719-8520-01d9871d8b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:30.942035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:30.955938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:30.969397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63400a-b20b-4e20-bbad-6a912b1c7ef5 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:30.984222 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 472/938] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:31.007285 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1092a7-40b3-4719-8520-01d9871d8b21 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:31.010289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94627ac0-c932-4797-b35a-f949d4ae1856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:31.012273 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94627ac0-c932-4797-b35a-f949d4ae1856 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:31.012563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94627ac0-c932-4797-b35a-f949d4ae1856 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:31.012850 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94627ac0-c932-4797-b35a-f949d4ae1856 None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/a2c75126c70247d58c85cec5294cfc55/roles/8d82d0ddc65e4d268c8039cfbd1335a5` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.015515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94627ac0-c932-4797-b35a-f949d4ae1856 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=a2c75126c70247d58c85cec5294cfc55, role_id=8d82d0ddc65e4d268c8039cfbd1335a5)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:31.017907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94627ac0-c932-4797-b35a-f949d4ae1856 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:31.037261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 467/939] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:22:30 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/a2c75126c70247d58c85cec5294cfc55/roles/8d82d0ddc65e4d268c8039cfbd1335a5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:31.134622 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1092a7-40b3-4719-8520-01d9871d8b21 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:31.179406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:31.180106 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:31.180417 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:31.180725 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.181483 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:31.183165 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:31.186015 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: WARNING keystone.server.flask.common [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 09:22:31.207073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.sql.core [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 09:22:31.207073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 09:22:31.207073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [parameters: {'id': '918f114732c940a1a19cd7a7784c43a7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 09:22:31.207073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=474607) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 918f114732c940a1a19cd7a7784c43a7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 918f114732c940a1a19cd7a7784c43a7. Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:22:31.225737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result.read() Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 09:22:31.226896 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application yield resource Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise self.value Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application session.commit() Jul 28 09:22:31.228021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.session.flush() Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application compat.raise_( Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 09:22:31.229091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 09:22:31.230995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result.read() Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:22:31.232693 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [parameters: {'id': '918f114732c940a1a19cd7a7784c43a7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:31.233930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:31.235509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 918f114732c940a1a19cd7a7784c43a7. Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:31.237782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 473/940] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 09:22:31 2022] POST /identity/v3/projects => generated 262 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:31.239291 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94627ac0-c932-4797-b35a-f949d4ae1856 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:31.370870 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94627ac0-c932-4797-b35a-f949d4ae1856 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:31.410344 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8a01d-de12-43e0-9758-35f2d7cdc156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:31.410866 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a01d-de12-43e0-9758-35f2d7cdc156 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:31.411212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a01d-de12-43e0-9758-35f2d7cdc156 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:31.411497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a01d-de12-43e0-9758-35f2d7cdc156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.414819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:31.420996 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8a01d-de12-43e0-9758-35f2d7cdc156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:31.422599 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8a01d-de12-43e0-9758-35f2d7cdc156 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-91c8a01d-de12-43e0-9758-35f2d7cdc156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:31.436005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:31.439029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:31.440883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 468/941] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:31.448617 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:31.499502 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:31.590068 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e21127-7178-4c44-b865-0f7ec3d24fbc None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:31.648528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f090e-f62b-40c8-9de6-8eb18bc38ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:31.649574 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f090e-f62b-40c8-9de6-8eb18bc38ddd None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:31.650087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f090e-f62b-40c8-9de6-8eb18bc38ddd None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:31.650638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f090e-f62b-40c8-9de6-8eb18bc38ddd None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.652087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f090e-f62b-40c8-9de6-8eb18bc38ddd None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:31.652658 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f090e-f62b-40c8-9de6-8eb18bc38ddd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:31.655884 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f090e-f62b-40c8-9de6-8eb18bc38ddd None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:31.696264 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 474/942] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:31.946819 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41959bb-015f-4fe5-9719-f0d2017f444e None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:31.947514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41959bb-015f-4fe5-9719-f0d2017f444e None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:31.947514 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41959bb-015f-4fe5-9719-f0d2017f444e None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.948985 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 469/943] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:31.954708 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d75c0-9e3e-4bb9-b6ff-bef91f787179 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:31.955394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d75c0-9e3e-4bb9-b6ff-bef91f787179 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:31.955394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d75c0-9e3e-4bb9-b6ff-bef91f787179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.975655 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-2c4d75c0-9e3e-4bb9-b6ff-bef91f787179 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:31.977387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-2c4d75c0-9e3e-4bb9-b6ff-bef91f787179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:32.050562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 475/944] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:32.073417 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41959bb-015f-4fe5-9719-f0d2017f444e None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:32.183871 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41959bb-015f-4fe5-9719-f0d2017f444e None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:32.220873 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:32.221461 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:32.221749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:32.222026 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:32.222771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:32.224296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:32.237801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:32.241400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:32.243593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 470/945] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:32.245979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d75c0-9e3e-4bb9-b6ff-bef91f787179 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:32.278839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:32.371557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d75c0-9e3e-4bb9-b6ff-bef91f787179 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:32.410498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:32.410498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:32.410498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:32.410498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:32.415576 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:32.415576 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:32.415576 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:32.418954 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 476/946] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:32.424575 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:32.489073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:32.533102 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80850a27-e137-412d-9606-417f48131cb7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:32.570281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:32.570921 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:32.571204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:32.571479 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:32.572180 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:32.572486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:32.573937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:32.580023 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 471/947] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:32.585752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:32.677247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:32.693707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b07d7-3f5e-4cba-82db-b7efb1cbcd39 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:32.730629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a5256-a514-4774-bf8e-e5bc1b5adf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:32.731279 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a5256-a514-4774-bf8e-e5bc1b5adf72 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:32.731592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a5256-a514-4774-bf8e-e5bc1b5adf72 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:32.731839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a5256-a514-4774-bf8e-e5bc1b5adf72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:32.732610 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a5256-a514-4774-bf8e-e5bc1b5adf72 None admin] RBAC: Adding query filter params (interface=public, service_id=d62f6086d6b54119acd9eda3b0132fac, region_id=RegionOne) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:32.732928 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a5256-a514-4774-bf8e-e5bc1b5adf72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:32.734375 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a5256-a514-4774-bf8e-e5bc1b5adf72 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:32.740784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 477/948] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:22:32 2022] GET /identity/v3/endpoints?service_id=d62f6086d6b54119acd9eda3b0132fac&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:32.746740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:32.804618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bdd6b-970d-440f-b560-629cff042aaa None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:32.804974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bdd6b-970d-440f-b560-629cff042aaa None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:32.805194 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bdd6b-970d-440f-b560-629cff042aaa None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:32.806910 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 478/949] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:32.812730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd88357-ada4-46b7-b4a5-1ca02e7a3a24 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:32.813073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd88357-ada4-46b7-b4a5-1ca02e7a3a24 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:32.813406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd88357-ada4-46b7-b4a5-1ca02e7a3a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:32.833081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-2bd88357-ada4-46b7-b4a5-1ca02e7a3a24 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:32.834341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-2bd88357-ada4-46b7-b4a5-1ca02e7a3a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:32.862177 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef5247-1ad2-4ef4-8de2-84d07976fa12 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:32.890482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:32.904643 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179c1ba-3a1a-4562-a01f-3f36bfcf5ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:32.905658 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179c1ba-3a1a-4562-a01f-3f36bfcf5ce0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:32.906195 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179c1ba-3a1a-4562-a01f-3f36bfcf5ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:32.906783 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179c1ba-3a1a-4562-a01f-3f36bfcf5ce0 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:32.907727 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179c1ba-3a1a-4562-a01f-3f36bfcf5ce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:32.909312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179c1ba-3a1a-4562-a01f-3f36bfcf5ce0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:32.909500 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 479/950] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:32.916962 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 472/951] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:22:32 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:32.931942 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd88357-ada4-46b7-b4a5-1ca02e7a3a24 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:33.031111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:33.052252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd88357-ada4-46b7-b4a5-1ca02e7a3a24 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:33.092455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:33.093137 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:33.093479 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:33.093760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] PATH_INFO: `/v3/services/compute` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:33.094590 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:33.096010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:33.113380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:33.114755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:33.116098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 480/952] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:33.120785 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179c1ba-3a1a-4562-a01f-3f36bfcf5ce0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:33.232800 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179c1ba-3a1a-4562-a01f-3f36bfcf5ce0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:33.270751 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989567a4-6b9d-4d3f-8c31-c4abf0146a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:33.271822 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989567a4-6b9d-4d3f-8c31-c4abf0146a2e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:33.272331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989567a4-6b9d-4d3f-8c31-c4abf0146a2e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:33.272857 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989567a4-6b9d-4d3f-8c31-c4abf0146a2e None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:33.273777 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989567a4-6b9d-4d3f-8c31-c4abf0146a2e None admin] RBAC: Adding query filter params (name=compute) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:33.274349 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989567a4-6b9d-4d3f-8c31-c4abf0146a2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:33.275959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989567a4-6b9d-4d3f-8c31-c4abf0146a2e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:33.282266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 473/953] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:33.290751 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:33.332327 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee476-3f8f-47ef-8060-7148ee625bfc None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:33.332692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee476-3f8f-47ef-8060-7148ee625bfc None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:33.332962 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee476-3f8f-47ef-8060-7148ee625bfc None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:33.334817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 474/954] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:33.340404 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc588fb-f5dd-4338-ad6e-5b204076a3a0 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:33.340766 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc588fb-f5dd-4338-ad6e-5b204076a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:33.341116 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc588fb-f5dd-4338-ad6e-5b204076a3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:33.360995 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-7dc588fb-f5dd-4338-ad6e-5b204076a3a0 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:33.362578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-7dc588fb-f5dd-4338-ad6e-5b204076a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:33.418493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b70a4-0929-480e-aa4b-844c9af73124 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:33.423237 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:33.462138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 475/955] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:33.470615 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:33.471401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:33.471837 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:33.472213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:33.473030 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:33.473443 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:33.475010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:33.480808 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 481/956] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:33.497562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc588fb-f5dd-4338-ad6e-5b204076a3a0 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:33.633944 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc588fb-f5dd-4338-ad6e-5b204076a3a0 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:33.680480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0ca60-2272-4bf4-a410-38cab9580dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:33.681171 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0ca60-2272-4bf4-a410-38cab9580dd1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:33.681459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0ca60-2272-4bf4-a410-38cab9580dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:33.681740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0ca60-2272-4bf4-a410-38cab9580dd1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:33.694889 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0ca60-2272-4bf4-a410-38cab9580dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-cfa0ca60-2272-4bf4-a410-38cab9580dd1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:33.700266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:33.701476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:33.702506 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 476/957] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:22:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:33.710094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:33.725457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:33.830022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fc4bb-b3d9-45ea-bdf9-5f52876086f1 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:33.865183 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85435090-95d9-4945-a5be-0987261dd24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:33.865729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435090-95d9-4945-a5be-0987261dd24c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:33.866008 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435090-95d9-4945-a5be-0987261dd24c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:33.866310 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435090-95d9-4945-a5be-0987261dd24c None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:33.867040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85435090-95d9-4945-a5be-0987261dd24c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:33.867343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85435090-95d9-4945-a5be-0987261dd24c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:33.868709 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85435090-95d9-4945-a5be-0987261dd24c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:33.878018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 482/958] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:22:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:33.883362 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0ca60-2272-4bf4-a410-38cab9580dd1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:33.932803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:33.998842 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91542f-b39c-4f3c-8b06-5fe72f8d565a None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:33.999130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91542f-b39c-4f3c-8b06-5fe72f8d565a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:33.999434 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91542f-b39c-4f3c-8b06-5fe72f8d565a None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.000948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 483/959] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:34.004445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0ca60-2272-4bf4-a410-38cab9580dd1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:34.006601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cabd59-4be3-4ffc-af8d-690a9a9e0527 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.006745 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cabd59-4be3-4ffc-af8d-690a9a9e0527 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.007028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cabd59-4be3-4ffc-af8d-690a9a9e0527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.025584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-00cabd59-4be3-4ffc-af8d-690a9a9e0527 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:34.026974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-00cabd59-4be3-4ffc-af8d-690a9a9e0527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:34.044334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:34.045393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.045910 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.046423 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.058125 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:34.060001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:34.077226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:34.078372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:34.079408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:34.079408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:34.079408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:34.079408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:34.081712 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 477/960] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:34.088892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:34.107877 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 484/961] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:34.109023 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:34.132395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cabd59-4be3-4ffc-af8d-690a9a9e0527 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:34.198872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70f65d-533f-4a68-a91b-a08ac14bdaa2 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:34.206457 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:34.236610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:34.237263 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.237560 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.237847 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.238631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:34.238953 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:34.240430 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:34.240738 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cabd59-4be3-4ffc-af8d-690a9a9e0527 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:34.258449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 478/962] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:34.259416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:34.277980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:34.278661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.279010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.279296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.280056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:34.281588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:34.297848 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.sql.core [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 09:22:34.297848 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 09:22:34.297848 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [parameters: {'user_id': '0cdb008bb14845d085cf3c0b9580aa0c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 09:22:34.297848 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=474607) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result.read() Jul 28 09:22:34.306241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:22:34.307312 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application yield resource Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application next(self.gen) Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise self.value Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application session.commit() Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 09:22:34.308717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.session.flush() Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application compat.raise_( Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 09:22:34.309909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 09:22:34.311359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application conn.query(q) Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application result.read() Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 09:22:34.312669 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [parameters: {'user_id': '0cdb008bb14845d085cf3c0b9580aa0c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:34.313671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 09:22:34.314860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 09:22:34.316457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 09:22:34.316457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 09:22:34.316457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 09:22:34.316457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 09:22:34.316457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 09:22:34.316457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:34.316457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 485/963] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 09:22:34 2022] POST /identity/v3/users => generated 160 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:34.318679 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:34.377884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407db5b-f9d8-4a50-9a62-d05a55ded250 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:34.414080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:34.414313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:34.414797 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.415084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.415741 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.424943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:34.426450 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:34.435041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fd95a-24ef-4904-9d0d-f00e41a66b00 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:34.438881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:34.440165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 09:22:34.441862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:34.445512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 479/964] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:22:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:34.446425 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:34.462572 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:34.479472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:34.480547 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.481087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.481514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.489782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:34.491744 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:34.505382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:34.506710 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 09:22:34.507948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:34.510346 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 486/965] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:34.516009 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:34.560571 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef5d17-072a-436f-874e-2b910ba6ef93 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:34.604002 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:34.604765 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.605213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.605593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.606611 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:34.607028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:34.609033 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:34.628618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53dbf0-e9fd-4933-85fa-4fbb47ba7775 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:34.629421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 480/966] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 09:22:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:34.634844 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:34.654690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:34.674682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:34.676393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.677079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.677614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.679476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] RBAC: Adding query filter params (name=cinder) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:34.680340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:34.682131 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:34.724967 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:34.735832 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 487/967] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 09:22:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:34.768114 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b6c0b-6720-4d63-a684-50ac1916ec52 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:34.810300 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8e6f5-7be2-4379-9f73-edf85ab3ab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:34.810962 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8e6f5-7be2-4379-9f73-edf85ab3ab2e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:34.811249 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8e6f5-7be2-4379-9f73-edf85ab3ab2e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:34.811531 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8e6f5-7be2-4379-9f73-edf85ab3ab2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:34.891843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8e6f5-7be2-4379-9f73-edf85ab3ab2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:34.893356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8e6f5-7be2-4379-9f73-edf85ab3ab2e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:34.967790 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:35.000963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 481/968] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:34 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:35.006705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.122250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af951e89-6200-4183-b334-f7a2b72ac9aa None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:35.142991 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8ac0-dcc5-4735-b761-ec9c48da5d02 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.143270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8ac0-dcc5-4735-b761-ec9c48da5d02 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.143544 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8ac0-dcc5-4735-b761-ec9c48da5d02 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.145081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 482/969] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:35.150017 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506abdec-e052-4d73-b9cf-20f9c0952fce None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.150348 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506abdec-e052-4d73-b9cf-20f9c0952fce None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.150638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506abdec-e052-4d73-b9cf-20f9c0952fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.159840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:35.160509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.160794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.161071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.169951 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-506abdec-e052-4d73-b9cf-20f9c0952fce None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:35.171242 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-506abdec-e052-4d73-b9cf-20f9c0952fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:35.172524 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:35.174212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:35.190980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:35.192585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:35.193849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:35.193849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:35.193849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:35.193849 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:35.197506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 488/970] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:35.198809 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.235602 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:35.254481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 483/971] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:35.274564 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506abdec-e052-4d73-b9cf-20f9c0952fce None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.318363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4fcb8-7aa0-4283-a4ac-cdc7737b4c35 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:35.357052 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:35.358090 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.358410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.358688 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.359403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:35.359714 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:35.361346 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:35.379372 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 489/972] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:35.381280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.391413 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506abdec-e052-4d73-b9cf-20f9c0952fce None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:35.393277 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:35.431836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:35.432501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.432789 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.433102 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.433785 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:35.435284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:35.445726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:35.446988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:35.448001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 484/973] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:35.453297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.481626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:35.494041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf8e2-607e-4930-b6eb-d861599c67eb None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:35.545087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:35.545776 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.546060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.546378 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.554521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:35.556063 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:35.563111 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d4539-d427-4370-96c5-9f2e5ae136a3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:35.569211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:35.570444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:35.571493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:35.571493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:35.571493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:35.571493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:35.571493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:35.571493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:35.571493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:35.571493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:35.575480 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 490/974] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:35.576344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.608944 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:35.609630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.609922 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.610272 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.610934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:35.611247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:35.612639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:35.618520 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 485/975] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:35.624600 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.667078 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:35.707821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0143662-4f70-4991-9dcc-d18f5345f4bf None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:35.748129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77118125-43c1-43ec-a318-8622c58b1b80 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:35.754991 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b624a2-5be6-4851-bffb-9c6265a68c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:35.755954 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b624a2-5be6-4851-bffb-9c6265a68c26 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.756352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b624a2-5be6-4851-bffb-9c6265a68c26 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.756790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b624a2-5be6-4851-bffb-9c6265a68c26 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.757877 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b624a2-5be6-4851-bffb-9c6265a68c26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:35.758366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b624a2-5be6-4851-bffb-9c6265a68c26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:35.760310 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b624a2-5be6-4851-bffb-9c6265a68c26 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:35.783369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 491/976] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:35.783369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f248d-e6e3-42bf-a673-062203a1e783 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.783369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f248d-e6e3-42bf-a673-062203a1e783 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.783369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f248d-e6e3-42bf-a673-062203a1e783 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.783369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 492/977] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:35.785451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f248d-e6e3-42bf-a673-062203a1e783 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.801455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460413c-15af-46d1-8f5a-899114756ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:35.802176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460413c-15af-46d1-8f5a-899114756ce0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.802462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460413c-15af-46d1-8f5a-899114756ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.802737 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460413c-15af-46d1-8f5a-899114756ce0 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.803429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460413c-15af-46d1-8f5a-899114756ce0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:35.803737 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460413c-15af-46d1-8f5a-899114756ce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:35.805139 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:35.805325 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460413c-15af-46d1-8f5a-899114756ce0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:35.811629 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 486/978] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:35.812867 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99264225-b227-44fe-8059-711d9a9ea8c4 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.813155 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99264225-b227-44fe-8059-711d9a9ea8c4 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.813421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99264225-b227-44fe-8059-711d9a9ea8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.832510 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-99264225-b227-44fe-8059-711d9a9ea8c4 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:35.833711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-99264225-b227-44fe-8059-711d9a9ea8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:35.899893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f248d-e6e3-42bf-a673-062203a1e783 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:35.908090 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 487/979] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:35.911162 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99264225-b227-44fe-8059-711d9a9ea8c4 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:35.940630 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:35.942545 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:35.943180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:35.943470 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:35.943760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:35.944628 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] RBAC: Adding query filter params (role.id=8d82d0ddc65e4d268c8039cfbd1335a5, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=a2c75126c70247d58c85cec5294cfc55) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:35.944946 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:35.946429 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:35.952582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 493/980] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:35 2022] GET /identity/v3/role_assignments?user.id=a2c75126c70247d58c85cec5294cfc55&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=8d82d0ddc65e4d268c8039cfbd1335a5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:35.953374 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:36.049363 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99264225-b227-44fe-8059-711d9a9ea8c4 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:36.060967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:36.094047 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23d703-6d70-4dc7-96e2-a20a76818410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:36.094720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23d703-6d70-4dc7-96e2-a20a76818410 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:36.094874 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252658f6-5096-4a30-9a6d-6b9ef381c188 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:36.094997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23d703-6d70-4dc7-96e2-a20a76818410 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:36.095271 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23d703-6d70-4dc7-96e2-a20a76818410 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:36.096009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23d703-6d70-4dc7-96e2-a20a76818410 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:36.098669 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23d703-6d70-4dc7-96e2-a20a76818410 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:36.131058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 488/981] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:36.151636 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a6e56-db0f-412a-8736-8c309c4736fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:36.152973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a6e56-db0f-412a-8736-8c309c4736fb None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:36.153507 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a6e56-db0f-412a-8736-8c309c4736fb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:36.153789 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a6e56-db0f-412a-8736-8c309c4736fb None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:36.154550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a6e56-db0f-412a-8736-8c309c4736fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:36.156019 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a6e56-db0f-412a-8736-8c309c4736fb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:36.175314 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 494/982] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:22:35 2022] POST /identity/v3/services => generated 242 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:38.044544 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e2d66-c78a-4d4d-940a-47fda5aabcca None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.045247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e2d66-c78a-4d4d-940a-47fda5aabcca None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.045247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e2d66-c78a-4d4d-940a-47fda5aabcca None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.050151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 489/983] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:38.052562 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91251f-a277-4a36-85d6-289693c99812 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.053026 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91251f-a277-4a36-85d6-289693c99812 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.053124 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91251f-a277-4a36-85d6-289693c99812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.075660 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-2a91251f-a277-4a36-85d6-289693c99812 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:38.077293 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-2a91251f-a277-4a36-85d6-289693c99812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:38.166146 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 495/984] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:38.187068 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e2d66-c78a-4d4d-940a-47fda5aabcca None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.216872 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcfae9-12f6-4016-8adc-d8d1e83c9060 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.217160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcfae9-12f6-4016-8adc-d8d1e83c9060 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.217440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcfae9-12f6-4016-8adc-d8d1e83c9060 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.220005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 496/985] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:38.224396 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c720a7-7638-4ee8-a0d2-adf5b800c74c None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.224670 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c720a7-7638-4ee8-a0d2-adf5b800c74c None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.224948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c720a7-7638-4ee8-a0d2-adf5b800c74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.244078 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-92c720a7-7638-4ee8-a0d2-adf5b800c74c None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:38.245318 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-92c720a7-7638-4ee8-a0d2-adf5b800c74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:38.290358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:38.304827 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:38.312268 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e2d66-c78a-4d4d-940a-47fda5aabcca None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:38.321356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 497/986] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:38.322792 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6d073-93db-416c-b54b-3fed573a0853 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.323099 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6d073-93db-416c-b54b-3fed573a0853 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.323731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6d073-93db-416c-b54b-3fed573a0853 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.326157 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 498/987] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:38.331356 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fc3b6-8a92-4b75-af6d-2223bdcec375 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.331724 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fc3b6-8a92-4b75-af6d-2223bdcec375 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.331998 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fc3b6-8a92-4b75-af6d-2223bdcec375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.351257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-888fc3b6-8a92-4b75-af6d-2223bdcec375 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:38.352504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-888fc3b6-8a92-4b75-af6d-2223bdcec375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:38.353411 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:38.354064 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.354373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.354650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] PATH_INFO: `/v3/services/image` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.355359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:38.356730 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:38.365942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:38.367170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.368141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 490/988] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:38.369070 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.424749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 499/989] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:38.425628 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fc3b6-8a92-4b75-af6d-2223bdcec375 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.478477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8f39d-30f1-4e2d-a4be-03e5ebd159fa None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:38.513043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:38.513688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.513977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.514283 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.514925 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:38.516319 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:38.525380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:38.526733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.527708 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 491/990] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:38.528633 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.533601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fc3b6-8a92-4b75-af6d-2223bdcec375 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:38.538814 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:38.553280 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:38.571371 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:38.571925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.572210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.572485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.573172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] RBAC: Adding query filter params (name=image) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:38.573634 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:38.574958 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:38.582829 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 500/991] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:38.583656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.631496 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e089b52-0d07-4d6a-9ea1-8b2b754d1c16 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:38.674244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:38.674907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.675194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.675479 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] PATH_INFO: `/v3/services/compute` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.676128 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:38.677515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:38.686650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:38.687747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:38.688707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 492/992] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:38.689628 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.702860 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a5405-214d-494e-bf4c-9af642acdbbf None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:38.740100 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:38.740780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.741160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.741436 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.742151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:38.742462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:38.743834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:38.748792 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 501/993] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:38.749572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.764217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:38.796526 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d9e23-5dcc-4536-8b14-0698dd6cf75f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:38.800181 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:38.832583 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:38.833155 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.833445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.833707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.834429 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] RBAC: Adding query filter params (type=image) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:38.834740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:38.836201 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:38.840922 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 493/994] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:38.841727 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.864070 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cc920-3327-4090-bc39-42304843d38d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:38.907349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:38.908384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:38.908676 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:38.909249 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:38.910040 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] RBAC: Adding query filter params (name=compute) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:38.910585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:38.912757 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:38.918651 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 502/995] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:38.924614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:38.964187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51af47-d4cd-44c5-9e73-92e95b946ed1 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:39.008711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:39.009351 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:39.009633 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:39.009909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:39.010633 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:39.010947 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:39.012934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:39.015550 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:39.019758 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 494/996] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:39.027816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:39.040911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264397fe-02f2-4df5-a9cd-32daa17f60ce None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:39.053262 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:39.079656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:39.080424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:39.080708 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:39.081077 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:39.081781 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] RBAC: Adding query filter params (type=compute) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:39.082071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:39.083592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:39.093838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 503/997] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:39.095327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:39.144719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b2d1a-f93c-4eb3-b472-82ccaa8245d6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:39.183009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:39.183769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:39.184060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:39.184343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:39.185392 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] RBAC: Adding query filter params (interface=admin, service_id=d62f6086d6b54119acd9eda3b0132fac, region_id=RegionOne) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:39.185536 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:39.186912 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:39.198191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 495/998] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jul 28 09:22:39 2022] GET /identity/v3/endpoints?service_id=d62f6086d6b54119acd9eda3b0132fac&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:39.199720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:39.211631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6ae6b-5e30-4d83-bccf-2462f28a79cb None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:39.249999 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:39.251074 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:39.251561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:39.252036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:39.252959 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] RBAC: Adding query filter params (interface=public, service_id=823058bf28364e25a56a21edc173e3f0, region_id=RegionOne) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:39.253426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:39.255220 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:39.261606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 504/999] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:22:39 2022] GET /identity/v3/endpoints?service_id=823058bf28364e25a56a21edc173e3f0&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:39.267016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:39.274247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:39.308433 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:39.321667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d929f-f2e0-47d0-94ea-1bd35f878d76 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:39.361050 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf4e71-2b62-4eaa-b29b-24f1d5165e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:39.361657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4e71-2b62-4eaa-b29b-24f1d5165e55 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:39.361940 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4e71-2b62-4eaa-b29b-24f1d5165e55 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:39.362251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4e71-2b62-4eaa-b29b-24f1d5165e55 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:39.362942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf4e71-2b62-4eaa-b29b-24f1d5165e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:39.364329 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf4e71-2b62-4eaa-b29b-24f1d5165e55 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:39.371780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 496/1000] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:22:39 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:39.384676 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bc6b9-81cb-4952-a93e-6b40ce7b35f4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:39.423760 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd11df4-010a-4174-a51d-96974574ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:39.424410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11df4-010a-4174-a51d-96974574ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:39.424690 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11df4-010a-4174-a51d-96974574ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:39.424965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11df4-010a-4174-a51d-96974574ccc7 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:39.425647 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd11df4-010a-4174-a51d-96974574ccc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:39.427132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd11df4-010a-4174-a51d-96974574ccc7 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:39.434670 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 505/1001] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:22:39 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:22:40.876963 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c63e-a6e9-463e-8699-a4e7285866d7 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:40.877511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c63e-a6e9-463e-8699-a4e7285866d7 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:40.877511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c63e-a6e9-463e-8699-a4e7285866d7 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:40.879080 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 497/1002] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:40.884399 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dc12b-90bc-4667-bf7a-546ceece95e7 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:40.884840 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dc12b-90bc-4667-bf7a-546ceece95e7 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:40.884925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dc12b-90bc-4667-bf7a-546ceece95e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:40.904088 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-280dc12b-90bc-4667-bf7a-546ceece95e7 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:40.905290 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-280dc12b-90bc-4667-bf7a-546ceece95e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:40.978881 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 506/1003] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:41.001468 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33c63e-a6e9-463e-8699-a4e7285866d7 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:41.113315 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33c63e-a6e9-463e-8699-a4e7285866d7 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:41.142061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:41.151380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d30af-38df-41d9-a704-c3e360140841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:41.152091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d30af-38df-41d9-a704-c3e360140841 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.152372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d30af-38df-41d9-a704-c3e360140841 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.152650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d30af-38df-41d9-a704-c3e360140841 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.153363 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d30af-38df-41d9-a704-c3e360140841 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:41.154847 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d30af-38df-41d9-a704-c3e360140841 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:41.178135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 498/1004] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:22:41 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:41.422651 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb6487-9cb5-46d0-a360-1981a4f08b14 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.422874 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb6487-9cb5-46d0-a360-1981a4f08b14 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.423158 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb6487-9cb5-46d0-a360-1981a4f08b14 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.424671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 507/1005] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:41.430172 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890709b-7012-459e-a5f5-01d6b240bd4d None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.430508 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890709b-7012-459e-a5f5-01d6b240bd4d None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.430744 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890709b-7012-459e-a5f5-01d6b240bd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.450840 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-9890709b-7012-459e-a5f5-01d6b240bd4d None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:41.452509 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-9890709b-7012-459e-a5f5-01d6b240bd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:41.531274 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf94b8-b159-4add-a5b8-1ea48f150cdf None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.531550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf94b8-b159-4add-a5b8-1ea48f150cdf None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.531834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf94b8-b159-4add-a5b8-1ea48f150cdf None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.533224 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 508/1006] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:41.534019 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 499/1007] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:41 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:41.538316 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fe781-35bd-4f01-beaa-e9c4572e56a7 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.538650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fe781-35bd-4f01-beaa-e9c4572e56a7 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.538901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fe781-35bd-4f01-beaa-e9c4572e56a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.557196 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-ec6fe781-35bd-4f01-beaa-e9c4572e56a7 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:41.558185 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-ec6fe781-35bd-4f01-beaa-e9c4572e56a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:41.558387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890709b-7012-459e-a5f5-01d6b240bd4d None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:41.590196 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:41.637432 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 509/1008] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:41 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:41.660056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fe781-35bd-4f01-beaa-e9c4572e56a7 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:41.682950 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890709b-7012-459e-a5f5-01d6b240bd4d None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:41.699148 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:41.724924 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:41.725591 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.725881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.726186 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.726839 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:41.728238 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:41.737414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:41.742254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:41.743289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 500/1009] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 09:22:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:41.745617 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:41.775388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fe781-35bd-4f01-beaa-e9c4572e56a7 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:41.813232 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:41.813898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.814213 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.814492 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] PATH_INFO: `/v3/services/compute` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.815143 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:41.816888 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:41.826028 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:41.827349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:41.828343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 510/1010] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:41.833322 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:41.857934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d47c-52bb-4c74-a929-809be8e6ce60 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:41.864335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:41.894572 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:41.895177 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.895457 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.895733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.896440 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:41.896734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:41.898133 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:41.908234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 501/1011] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:41.909463 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:41.950354 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77ad8d-cc5a-4535-acbe-16abf5ca14fd None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:41.955575 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:41.986834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:41.987573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:41.987771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:41.988044 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:41.988736 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] RBAC: Adding query filter params (name=compute) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:41.989049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:41.990483 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:42.000563 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 511/1012] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:42.001782 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:42.029220 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81636-d0e3-4aac-96bc-12abd5f2bdac None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:42.064654 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:42.065307 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:42.065584 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:42.065861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:42.066884 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:42.067188 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:42.068563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:42.078154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 502/1013] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 09:22:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:42.079217 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:42.101245 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:42.111537 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3790a23-5903-48de-bb31-8278d3795794 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:42.153997 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:42.154823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:42.155107 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:42.155385 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:42.156547 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:42.157007 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:42.158434 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:42.164154 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 512/1014] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 09:22:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:42.169907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:42.193439 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27704cd6-382d-4935-abb1-b598a1d10d7e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:42.201233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:42.229700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420079a-1d2f-4988-aa31-f9321bc8eeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:42.230414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420079a-1d2f-4988-aa31-f9321bc8eeed None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:42.230698 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420079a-1d2f-4988-aa31-f9321bc8eeed None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:42.230969 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420079a-1d2f-4988-aa31-f9321bc8eeed None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:42.231707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420079a-1d2f-4988-aa31-f9321bc8eeed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:42.233089 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420079a-1d2f-4988-aa31-f9321bc8eeed None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:42.251190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 503/1015] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:42.290639 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678fa6e-15bb-4631-b29c-709e345392e0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:42.339718 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93682c15-703e-4fa8-b805-1218f8feba45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:42.340375 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93682c15-703e-4fa8-b805-1218f8feba45 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:42.340661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93682c15-703e-4fa8-b805-1218f8feba45 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:42.340939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93682c15-703e-4fa8-b805-1218f8feba45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:42.341692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93682c15-703e-4fa8-b805-1218f8feba45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:42.343147 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93682c15-703e-4fa8-b805-1218f8feba45 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:42.367775 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 513/1016] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:43.240162 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef7bb-e42d-4979-9444-97b170761c76 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:43.240739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef7bb-e42d-4979-9444-97b170761c76 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:43.240739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef7bb-e42d-4979-9444-97b170761c76 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:43.242286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 504/1017] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:43.247551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7eb29-2871-426d-a279-b6702f1fe1cb None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:43.248049 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7eb29-2871-426d-a279-b6702f1fe1cb None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:43.248135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7eb29-2871-426d-a279-b6702f1fe1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:43.261518 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:43.267047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-fde7eb29-2871-426d-a279-b6702f1fe1cb None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:43.268783 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-fde7eb29-2871-426d-a279-b6702f1fe1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:43.348359 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 514/1018] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:43.371087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ef7bb-e42d-4979-9444-97b170761c76 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:43.484683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ef7bb-e42d-4979-9444-97b170761c76 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:43.520233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:43.520802 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:43.521095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:43.521372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] PATH_INFO: `/v3/services/image` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:43.522015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:43.523615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:43.533197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:43.539644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:43.540725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 505/1019] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 09:22:43 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:43.542854 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7eb29-2871-426d-a279-b6702f1fe1cb None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:43.657075 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7eb29-2871-426d-a279-b6702f1fe1cb None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:43.688221 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:43.694557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:43.695197 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:43.695482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:43.695759 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:43.696451 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] RBAC: Adding query filter params (name=image) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:43.696764 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:43.698156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:43.703707 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 515/1020] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:43.709597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:43.743140 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:43.819265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6577fc-bc82-4bca-b657-1f76d9ef6942 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:43.857645 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:43.858378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:43.858656 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:43.858940 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:43.860937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] RBAC: Adding query filter params (type=image) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:43.860937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:43.861663 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:43.872310 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 506/1021] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:43.873964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:43.982363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2e3f-2a82-43b9-a858-5c424039e530 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:44.017205 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f8d7f-2f95-4f8a-80bd-aa1aae9c88cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:44.017836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f8d7f-2f95-4f8a-80bd-aa1aae9c88cd None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.018132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f8d7f-2f95-4f8a-80bd-aa1aae9c88cd None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.018430 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f8d7f-2f95-4f8a-80bd-aa1aae9c88cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.019204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f8d7f-2f95-4f8a-80bd-aa1aae9c88cd None admin] RBAC: Adding query filter params (interface=public, service_id=b5d655bd3c55421b95222b48be165c8f, region_id=RegionOne) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:44.019519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f8d7f-2f95-4f8a-80bd-aa1aae9c88cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:44.020897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f8d7f-2f95-4f8a-80bd-aa1aae9c88cd None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:44.026456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 516/1022] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:22:43 2022] GET /identity/v3/endpoints?service_id=b5d655bd3c55421b95222b48be165c8f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:44.032228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:44.130692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:44.139010 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeac549-ebea-4955-b479-4adeea9dda85 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:44.176942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb426e27-63b9-419b-8352-5db477a59f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:44.177579 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb426e27-63b9-419b-8352-5db477a59f6e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.177860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb426e27-63b9-419b-8352-5db477a59f6e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.178167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb426e27-63b9-419b-8352-5db477a59f6e None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.178846 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb426e27-63b9-419b-8352-5db477a59f6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:44.180234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb426e27-63b9-419b-8352-5db477a59f6e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:44.187592 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 507/1023] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:22:44 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:44.286241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3f1c6-9cab-4eb9-9813-c2d72e880954 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.286813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3f1c6-9cab-4eb9-9813-c2d72e880954 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.286813 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3f1c6-9cab-4eb9-9813-c2d72e880954 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.288376 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 517/1024] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:44.294344 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df56d-adfb-4667-bdb5-1d477e5ba097 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.294909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df56d-adfb-4667-bdb5-1d477e5ba097 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.294909 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df56d-adfb-4667-bdb5-1d477e5ba097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.315101 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-588df56d-adfb-4667-bdb5-1d477e5ba097 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:44.316664 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-588df56d-adfb-4667-bdb5-1d477e5ba097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:44.392375 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 508/1025] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:44.414755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3f1c6-9cab-4eb9-9813-c2d72e880954 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:44.481702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:44.491451 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae98e71-139d-4cc3-b84b-e189f5428f88 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.491879 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae98e71-139d-4cc3-b84b-e189f5428f88 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.492155 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae98e71-139d-4cc3-b84b-e189f5428f88 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.493726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 509/1026] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:44.498932 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579bd8d-731b-4741-b40b-691715c85ec5 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.499220 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579bd8d-731b-4741-b40b-691715c85ec5 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.499497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579bd8d-731b-4741-b40b-691715c85ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.518478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-f579bd8d-731b-4741-b40b-691715c85ec5 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:44.519684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-f579bd8d-731b-4741-b40b-691715c85ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:44.529425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3f1c6-9cab-4eb9-9813-c2d72e880954 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:44.568115 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a09ff-5087-4c4f-8614-07fda2e22b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:44.568766 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a09ff-5087-4c4f-8614-07fda2e22b5c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.569046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a09ff-5087-4c4f-8614-07fda2e22b5c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.569317 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a09ff-5087-4c4f-8614-07fda2e22b5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.570973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a09ff-5087-4c4f-8614-07fda2e22b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:44.572420 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a09ff-5087-4c4f-8614-07fda2e22b5c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:44.575029 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 518/1027] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:44.580749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a09ff-5087-4c4f-8614-07fda2e22b5c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:44.607821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 510/1028] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:44.631181 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579bd8d-731b-4741-b40b-691715c85ec5 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:44.651827 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:44.695572 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a09ff-5087-4c4f-8614-07fda2e22b5c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:44.727610 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:44.737411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d1bbf-ad7f-4564-bee0-ca61805aced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:44.738072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d1bbf-ad7f-4564-bee0-ca61805aced3 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.738350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d1bbf-ad7f-4564-bee0-ca61805aced3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.738624 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d1bbf-ad7f-4564-bee0-ca61805aced3 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.739316 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d1bbf-ad7f-4564-bee0-ca61805aced3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:44.740831 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d1bbf-ad7f-4564-bee0-ca61805aced3 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:44.743025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579bd8d-731b-4741-b40b-691715c85ec5 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:44.771485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 519/1029] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:22:44 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:44.792877 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1068e3-d8f7-4c57-a558-9cde44d38049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:44.793569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1068e3-d8f7-4c57-a558-9cde44d38049 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.793852 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1068e3-d8f7-4c57-a558-9cde44d38049 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.794308 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1068e3-d8f7-4c57-a558-9cde44d38049 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.807882 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1068e3-d8f7-4c57-a558-9cde44d38049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-7a1068e3-d8f7-4c57-a558-9cde44d38049 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:44.813154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:44.814959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:44.817066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:44.820179 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 511/1030] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:22:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:44.821606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d1bbf-ad7f-4564-bee0-ca61805aced3 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:44.940705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d1bbf-ad7f-4564-bee0-ca61805aced3 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:44.976882 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:44.977504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:44.977786 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:44.978061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:44.978854 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:44.979163 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:44.980565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:44.989726 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 520/1031] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:22:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:44.995762 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1068e3-d8f7-4c57-a558-9cde44d38049 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:45.081945 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:45.108445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1068e3-d8f7-4c57-a558-9cde44d38049 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:45.145199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:45.145859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:45.146146 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:45.146431 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:45.157878 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:45.159418 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:45.176966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:45.178083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:45.179192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:45.179192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:45.179192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:45.179192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:45.179192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 512/1032] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:45.183977 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:45.234448 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:45.300508 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379093d-df96-46e6-ae88-ac5d33e432c6 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:45.340656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:45.341278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:45.341608 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:45.341883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:45.342580 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:45.342887 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:45.344275 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:45.357671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 521/1033] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:45.364215 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:45.483501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e626-d0ce-4e81-a634-a0157e23fb2e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:45.514925 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:45.522051 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:45.522738 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:45.523020 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:45.523296 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:45.531659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:45.533126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:45.545606 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:45.547036 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:45.548131 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:45.550219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 513/1034] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:22:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:45.553108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:45.670311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abe735-ac9a-449f-9e20-120ca512a238 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:45.671949 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:45.708963 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:45.709606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:45.709922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:45.710371 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:45.711144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:45.711457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:45.713010 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:45.733869 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 522/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:22:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:45.739808 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:45.851228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bbe48-66e1-4f7a-a9f5-cdd63db31ea9 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:45.896134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246047f-ac22-4e2a-960f-c6e17c23d97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:45.896776 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246047f-ac22-4e2a-960f-c6e17c23d97e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:45.897060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246047f-ac22-4e2a-960f-c6e17c23d97e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:45.897334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246047f-ac22-4e2a-960f-c6e17c23d97e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:45.964902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:45.977744 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246047f-ac22-4e2a-960f-c6e17c23d97e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:45.979304 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246047f-ac22-4e2a-960f-c6e17c23d97e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.098767 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 514/1036] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:46.105293 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.211045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596cf5-4ba1-449b-af85-02ddd74910f0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.225570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714f5be-55c2-4c66-a57b-da83f99f2c6a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.225858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714f5be-55c2-4c66-a57b-da83f99f2c6a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.226170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714f5be-55c2-4c66-a57b-da83f99f2c6a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.227678 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 515/1037] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:46.233060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033545c-3e5e-4777-8ebb-15097cf275fa None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.233333 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033545c-3e5e-4777-8ebb-15097cf275fa None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.233768 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033545c-3e5e-4777-8ebb-15097cf275fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.250719 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:46.251383 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.251668 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.254987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.255171 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-f033545c-3e5e-4777-8ebb-15097cf275fa None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:46.255806 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-f033545c-3e5e-4777-8ebb-15097cf275fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:46.271233 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:46.272737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:46.288909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:46.290025 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:46.291110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:46.291110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:46.291110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:46.291110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:46.294499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 523/1038] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:46.295895 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.311872 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:46.346187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 516/1039] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:46.361550 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033545c-3e5e-4777-8ebb-15097cf275fa None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.385879 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:46.406536 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18a5af-bf3c-4687-b508-216232463c12 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.444105 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:46.445272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.445551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.445826 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.446565 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:46.447236 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:46.449046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.461908 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 524/1040] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:46.467509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.475454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033545c-3e5e-4777-8ebb-15097cf275fa None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.512707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:46.513375 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.513693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.513941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] PATH_INFO: `/v3/services/image` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.514722 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:46.516081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:46.525647 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:46.526978 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:46.527977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 517/1041] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 09:22:46 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:46.532934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.547156 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:46.583107 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ba57e-1d9f-4f45-ad07-2e8963a3edf4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.623826 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:46.624494 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.624781 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.625061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.632968 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:46.633122 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:46.634960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:46.646855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:46.647925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:46.649334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 525/1042] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:46.653893 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.657138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6aab-3936-4a75-9bc6-aa406ff08f86 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.694593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:46.695230 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.695512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.695791 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.696485 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] RBAC: Adding query filter params (name=image) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:46.696792 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:46.698241 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.703248 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 518/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:46.709283 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.766389 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dc78e-1a9d-46ab-85b3-9b4f40fef213 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.799326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:46.805689 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:46.806350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.806629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.806898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.807657 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:46.807989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:46.809414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.812927 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54869-676f-417a-8a30-97edc4221b70 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.831247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 526/1044] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:46.832785 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.851563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:46.852194 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.852476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.852754 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.853436 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] RBAC: Adding query filter params (type=image) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:46.853796 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:46.855279 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.860906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 519/1045] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:46.866646 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:46.882322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:46.945584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541cc45-407c-4cc6-8324-5451f136818d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.974639 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652989a7-8c9a-43d5-b9a8-919ae31d092a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:46.981453 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b805bc5-dad8-4fc8-8675-8561fa7580bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:46.982094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b805bc5-dad8-4fc8-8675-8561fa7580bb None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.982407 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b805bc5-dad8-4fc8-8675-8561fa7580bb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.982685 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b805bc5-dad8-4fc8-8675-8561fa7580bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.983491 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b805bc5-dad8-4fc8-8675-8561fa7580bb None admin] RBAC: Adding query filter params (role.id=8d82d0ddc65e4d268c8039cfbd1335a5, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=0970fc7ae258457ca70042acc0fc6c4c) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:46.983817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b805bc5-dad8-4fc8-8675-8561fa7580bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:46.985261 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b805bc5-dad8-4fc8-8675-8561fa7580bb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:46.991338 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 527/1046] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:46 2022] GET /identity/v3/role_assignments?user.id=0970fc7ae258457ca70042acc0fc6c4c&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=8d82d0ddc65e4d268c8039cfbd1335a5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:46.992437 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ec7ba-e64c-468a-b2c4-6d48ec0c99ec None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:46.992734 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ec7ba-e64c-468a-b2c4-6d48ec0c99ec None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:46.993016 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ec7ba-e64c-468a-b2c4-6d48ec0c99ec None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:46.994311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 528/1047] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:47.000463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2e448-c540-4f0e-8cb9-4cbd8c48344e None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:47.000754 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2e448-c540-4f0e-8cb9-4cbd8c48344e None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:47.001037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2e448-c540-4f0e-8cb9-4cbd8c48344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:47.019051 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc553-5ab4-4b20-a07d-f02662fa1a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:47.020469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc553-5ab4-4b20-a07d-f02662fa1a7e None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:47.020908 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc553-5ab4-4b20-a07d-f02662fa1a7e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:47.021337 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc553-5ab4-4b20-a07d-f02662fa1a7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:47.021463 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-c2c2e448-c540-4f0e-8cb9-4cbd8c48344e None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:47.022444 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc553-5ab4-4b20-a07d-f02662fa1a7e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:47.022783 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-c2c2e448-c540-4f0e-8cb9-4cbd8c48344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:47.024192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc553-5ab4-4b20-a07d-f02662fa1a7e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:47.057265 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 520/1048] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:47.093496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:47.121341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 529/1049] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:47.146971 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc553-5ab4-4b20-a07d-f02662fa1a7e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:47.225359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:47.264793 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc553-5ab4-4b20-a07d-f02662fa1a7e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:47.302270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d9048-27ff-435a-8085-008c136c071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:47.302934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d9048-27ff-435a-8085-008c136c071a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:47.303221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d9048-27ff-435a-8085-008c136c071a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:47.303499 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d9048-27ff-435a-8085-008c136c071a None admin] PATH_INFO: `/v3/domains/default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:47.304687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d9048-27ff-435a-8085-008c136c071a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:47.306150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d9048-27ff-435a-8085-008c136c071a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:47.308100 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 521/1050] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:47.313979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2e448-c540-4f0e-8cb9-4cbd8c48344e None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:47.422472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2e448-c540-4f0e-8cb9-4cbd8c48344e None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:47.459454 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8bea1-f5ca-4659-a0cf-e2096c46271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:47.459454 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8bea1-f5ca-4659-a0cf-e2096c46271e None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:47.459454 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8bea1-f5ca-4659-a0cf-e2096c46271e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:47.459454 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8bea1-f5ca-4659-a0cf-e2096c46271e None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:47.459929 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8bea1-f5ca-4659-a0cf-e2096c46271e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:47.461346 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8bea1-f5ca-4659-a0cf-e2096c46271e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:47.494140 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 530/1051] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:22:47 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:49.021977 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151afb-adab-4949-9bdb-3fecd5654942 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.022576 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151afb-adab-4949-9bdb-3fecd5654942 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.022576 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151afb-adab-4949-9bdb-3fecd5654942 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.024103 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 522/1052] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:49.029584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65d0c-7a9e-4b31-95b5-2ed80479c647 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.030255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65d0c-7a9e-4b31-95b5-2ed80479c647 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.030255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65d0c-7a9e-4b31-95b5-2ed80479c647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.057392 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-a6f65d0c-7a9e-4b31-95b5-2ed80479c647 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:49.058676 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-a6f65d0c-7a9e-4b31-95b5-2ed80479c647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:49.090443 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0225c8e-467f-4248-a0f3-09f284f312cd None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.090805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0225c8e-467f-4248-a0f3-09f284f312cd None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.090999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0225c8e-467f-4248-a0f3-09f284f312cd None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.092535 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 523/1053] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:49.097841 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590de1f-6473-4fef-b1f0-5e19998322de None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.098152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590de1f-6473-4fef-b1f0-5e19998322de None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.098407 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590de1f-6473-4fef-b1f0-5e19998322de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.099814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:49.118841 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-1590de1f-6473-4fef-b1f0-5e19998322de None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:49.120044 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-1590de1f-6473-4fef-b1f0-5e19998322de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:49.148591 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 531/1054] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:49.171001 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65d0c-7a9e-4b31-95b5-2ed80479c647 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:49.205474 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 524/1055] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:49.228066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590de1f-6473-4fef-b1f0-5e19998322de None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:49.290415 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65d0c-7a9e-4b31-95b5-2ed80479c647 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:49.311732 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:49.344657 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7319e-b461-40cf-81bf-b0153a882626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:49.345311 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7319e-b461-40cf-81bf-b0153a882626 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.345592 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7319e-b461-40cf-81bf-b0153a882626 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.345868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7319e-b461-40cf-81bf-b0153a882626 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.357325 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7319e-b461-40cf-81bf-b0153a882626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:49.360061 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590de1f-6473-4fef-b1f0-5e19998322de None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-4ee7319e-b461-40cf-81bf-b0153a882626 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:49.362331 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:49.363506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 532/1056] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:22:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:49.369839 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7319e-b461-40cf-81bf-b0153a882626 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:49.409719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe89451-b06e-43d0-91ce-7066afbb6c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:49.410403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe89451-b06e-43d0-91ce-7066afbb6c2f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.410690 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe89451-b06e-43d0-91ce-7066afbb6c2f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.410967 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe89451-b06e-43d0-91ce-7066afbb6c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.412822 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:49.489990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe89451-b06e-43d0-91ce-7066afbb6c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:49.491509 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe89451-b06e-43d0-91ce-7066afbb6c2f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:49.502989 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7319e-b461-40cf-81bf-b0153a882626 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:49.541737 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:49.542352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.542644 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.542919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.543662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:49.543978 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:49.545391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:49.554574 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 533/1057] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:22:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:49.560431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:49.575381 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:49.642681 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 525/1058] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:49.644863 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeb90e-d6b6-41fe-9363-a811ea8ad95a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.645335 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeb90e-d6b6-41fe-9363-a811ea8ad95a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.645860 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeb90e-d6b6-41fe-9363-a811ea8ad95a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.650134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 526/1059] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:49.650134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeb90e-d6b6-41fe-9363-a811ea8ad95a None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:49.671489 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:49.685070 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e432be4-8253-4e9f-a727-b6e3ddc67e53 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:49.729776 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e617732-e509-48a4-99d7-a87e5970f67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:49.730506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617732-e509-48a4-99d7-a87e5970f67f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.730791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617732-e509-48a4-99d7-a87e5970f67f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.731061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617732-e509-48a4-99d7-a87e5970f67f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.742778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e617732-e509-48a4-99d7-a87e5970f67f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:49.744265 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e617732-e509-48a4-99d7-a87e5970f67f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-6e617732-e509-48a4-99d7-a87e5970f67f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:49.765748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:49.767072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:49.768094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:49.768094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:49.768094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:49.768094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:49.768094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 534/1060] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:49.769426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefc0aa-a765-41d7-a146-6252cabc85e0 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.769784 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefc0aa-a765-41d7-a146-6252cabc85e0 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.770189 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefc0aa-a765-41d7-a146-6252cabc85e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.786382 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeb90e-d6b6-41fe-9363-a811ea8ad95a None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:49.796868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-ccefc0aa-a765-41d7-a146-6252cabc85e0 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:49.798283 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-ccefc0aa-a765-41d7-a146-6252cabc85e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:49.829786 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:49.830945 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.831445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.831953 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.849153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:49.851958 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:49.873286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:49.880199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:49.881713 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:49.881713 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:49.881713 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:49.881713 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:49.881713 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 527/1061] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:49.881713 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:49.884875 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 535/1062] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:49.885991 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefc0aa-a765-41d7-a146-6252cabc85e0 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:49.902640 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:49.968683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:50.002195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefc0aa-a765-41d7-a146-6252cabc85e0 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.014894 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482899f5-ed39-4091-8fc4-546eebc8a538 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.055295 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.055862 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.056127 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.056401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.057132 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:50.057599 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.059130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.067238 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ec445-544a-40f7-904c-002db0be68db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.067910 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec445-544a-40f7-904c-002db0be68db None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.068239 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec445-544a-40f7-904c-002db0be68db None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.068593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec445-544a-40f7-904c-002db0be68db None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.069454 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ec445-544a-40f7-904c-002db0be68db None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:50.069815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ec445-544a-40f7-904c-002db0be68db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.071387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ec445-544a-40f7-904c-002db0be68db None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.088377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 536/1063] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:50.088377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.088686 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 528/1064] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:50.094311 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ec445-544a-40f7-904c-002db0be68db None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.194901 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:50.224828 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9c39e-5213-4dc3-9407-0117d3cd79d9 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.233403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ec445-544a-40f7-904c-002db0be68db None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.278173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:50.283514 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e15c8-433d-49bb-9bc3-ae38d57dd963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.285899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e15c8-433d-49bb-9bc3-ae38d57dd963 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.286215 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e15c8-433d-49bb-9bc3-ae38d57dd963 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.286505 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e15c8-433d-49bb-9bc3-ae38d57dd963 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.290392 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.291208 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.291540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.291826 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] PATH_INFO: `/v3/users/nova` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.302314 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e15c8-433d-49bb-9bc3-ae38d57dd963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:50.306418 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-6b9e15c8-433d-49bb-9bc3-ae38d57dd963 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:50.308424 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:50.309519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:50.310551 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:50.311319 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.311406 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 537/1065] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:50.318515 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e15c8-433d-49bb-9bc3-ae38d57dd963 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:50.328199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:50.329399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:50.330547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 529/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:50.337421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.464640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e15c8-433d-49bb-9bc3-ae38d57dd963 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.474228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e2c1f-f8e9-4971-96f2-e3e61fa5357e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.510146 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.510931 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.511221 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.511499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.511843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.512239 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:50.512507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.512652 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.512794 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.513066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.513922 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:50.514304 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.514412 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.515785 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.525127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:50.525835 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 538/1067] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:50.531846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.537079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 530/1068] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:50.543091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.595444 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:50.647535 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151215-b3dc-41a5-8a0b-513ecf056d67 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.659345 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0440e40-efe0-4de6-af16-7141c5e10655 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.696717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b57e2-b30a-422d-b93d-3ad9d4e0e2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.697535 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b57e2-b30a-422d-b93d-3ad9d4e0e2be None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.697696 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.697919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b57e2-b30a-422d-b93d-3ad9d4e0e2be None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.698602 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b57e2-b30a-422d-b93d-3ad9d4e0e2be None admin] PATH_INFO: `/v3/users/57f3ca3711de4fff8559e3e167de565a` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.698760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.698760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.699188 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.711997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.713896 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.718227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b57e2-b30a-422d-b93d-3ad9d4e0e2be None admin] RBAC: Authorizing `identity:get_user(user_id=57f3ca3711de4fff8559e3e167de565a)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.719824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b57e2-b30a-422d-b93d-3ad9d4e0e2be None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.721729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 539/1069] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/users/57f3ca3711de4fff8559e3e167de565a => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:50.728817 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b57e2-b30a-422d-b93d-3ad9d4e0e2be None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:50.740469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:50.742441 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:50.744742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:50.744742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:50.744742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:50.744742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:50.744742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 531/1070] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:50.750081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.830651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:50.882178 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b57e2-b30a-422d-b93d-3ad9d4e0e2be None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.898478 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:50.917866 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e4821-df1a-4a65-9e44-8a97df7cb9fd None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.922615 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b00607-a5ff-4ec6-92af-a1a5b9fd5190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.923381 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b00607-a5ff-4ec6-92af-a1a5b9fd5190 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.923698 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b00607-a5ff-4ec6-92af-a1a5b9fd5190 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.924099 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b00607-a5ff-4ec6-92af-a1a5b9fd5190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.964137 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.964798 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.965076 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.965358 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.966084 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:50.966439 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.967931 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.981644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 532/1071] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:50.988519 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:51.013418 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b00607-a5ff-4ec6-92af-a1a5b9fd5190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:51.015333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b00607-a5ff-4ec6-92af-a1a5b9fd5190 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:51.101414 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc4e73-a5b7-4f44-b187-6aeb5ad7267a None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:51.111803 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:51.140853 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:51.141425 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:51.141718 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:51.141993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:51.142834 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 540/1072] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:51.149433 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:51.151141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:51.151796 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b00607-a5ff-4ec6-92af-a1a5b9fd5190 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:51.163603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:51.165043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:51.166355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 533/1073] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:51.169698 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:51.171372 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:51.275538 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b00607-a5ff-4ec6-92af-a1a5b9fd5190 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:51.297495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963396c-7984-41e0-b7b3-5287a60a27d6 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:51.320387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390414e-a52f-45fd-99f2-dce15af28d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:51.321257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390414e-a52f-45fd-99f2-dce15af28d3c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:51.321504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390414e-a52f-45fd-99f2-dce15af28d3c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:51.321787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390414e-a52f-45fd-99f2-dce15af28d3c None admin] PATH_INFO: `/v3/projects/b1caa9b5266c42a69295ceb3d70d4943` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:51.323101 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390414e-a52f-45fd-99f2-dce15af28d3c None admin] RBAC: Authorizing `identity:get_project(project_id=b1caa9b5266c42a69295ceb3d70d4943)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:51.325540 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390414e-a52f-45fd-99f2-dce15af28d3c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:51.327908 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 541/1074] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:22:51 2022] GET /identity/v3/projects/b1caa9b5266c42a69295ceb3d70d4943 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:51.335005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390414e-a52f-45fd-99f2-dce15af28d3c None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:51.342043 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:51.342723 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:51.343012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:51.343293 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:51.344069 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:51.344384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:51.345848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:51.359783 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 534/1075] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:51.368922 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:51.401944 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:51.452102 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:51.484302 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390414e-a52f-45fd-99f2-dce15af28d3c None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:51.517162 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f32b3-0947-4aae-8ee8-c0cc5c0b107e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:51.523284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab7670-8d17-4230-8f28-5d78591897f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:51.524232 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7670-8d17-4230-8f28-5d78591897f1 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:51.524691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7670-8d17-4230-8f28-5d78591897f1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:51.525128 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7670-8d17-4230-8f28-5d78591897f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:51.526227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab7670-8d17-4230-8f28-5d78591897f1 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=b1caa9b5266c42a69295ceb3d70d4943, user.id=57f3ca3711de4fff8559e3e167de565a) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:51.526807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab7670-8d17-4230-8f28-5d78591897f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:51.528598 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab7670-8d17-4230-8f28-5d78591897f1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:51.535527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 542/1076] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:51 2022] GET /identity/v3/role_assignments?user.id=57f3ca3711de4fff8559e3e167de565a&scope.project.id=b1caa9b5266c42a69295ceb3d70d4943&role.id=af958d22229a4a0297bbe2e643f55181 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:51.577247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158eb91-9aa3-431c-9c9a-7b7a0929263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:51.578764 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158eb91-9aa3-431c-9c9a-7b7a0929263b None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:51.579450 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158eb91-9aa3-431c-9c9a-7b7a0929263b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:51.580110 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158eb91-9aa3-431c-9c9a-7b7a0929263b None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/0970fc7ae258457ca70042acc0fc6c4c/roles/8d82d0ddc65e4d268c8039cfbd1335a5` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:51.584185 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158eb91-9aa3-431c-9c9a-7b7a0929263b None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=0970fc7ae258457ca70042acc0fc6c4c, role_id=8d82d0ddc65e4d268c8039cfbd1335a5)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:51.585820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158eb91-9aa3-431c-9c9a-7b7a0929263b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:51.605352 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 535/1077] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:22:51 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/0970fc7ae258457ca70042acc0fc6c4c/roles/8d82d0ddc65e4d268c8039cfbd1335a5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:22:53.611254 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7616c67-a8ce-46c4-9a51-c0d9651f8d03 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.611830 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7616c67-a8ce-46c4-9a51-c0d9651f8d03 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.612020 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7616c67-a8ce-46c4-9a51-c0d9651f8d03 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.613540 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 543/1078] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:53.619615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfd3ad-be3b-435b-99e7-19e5a0983d95 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.620164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfd3ad-be3b-435b-99e7-19e5a0983d95 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.620276 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfd3ad-be3b-435b-99e7-19e5a0983d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.643190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-93dfd3ad-be3b-435b-99e7-19e5a0983d95 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:53.644439 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-93dfd3ad-be3b-435b-99e7-19e5a0983d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:53.649257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b6a5-df63-4f07-bac8-957f0ef9605d None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.649683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b6a5-df63-4f07-bac8-957f0ef9605d None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.650170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b6a5-df63-4f07-bac8-957f0ef9605d None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.651684 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 544/1079] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:22:53.657530 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a3e71-4c69-4e52-b489-dd8bbd7555ba None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.657822 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a3e71-4c69-4e52-b489-dd8bbd7555ba None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.658152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a3e71-4c69-4e52-b489-dd8bbd7555ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.676733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-c63a3e71-4c69-4e52-b489-dd8bbd7555ba None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:53.678031 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-c63a3e71-4c69-4e52-b489-dd8bbd7555ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:53.724592 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 536/1080] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:22:53.757189 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 545/1081] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:53.761285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfd3ad-be3b-435b-99e7-19e5a0983d95 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:53.784604 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:53.787184 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a3e71-4c69-4e52-b489-dd8bbd7555ba None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:53.868629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:53.907104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a3e71-4c69-4e52-b489-dd8bbd7555ba None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:53.910776 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfd3ad-be3b-435b-99e7-19e5a0983d95 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:53.952663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea1201-28e2-467a-adfe-b550897341c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:53.953569 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea1201-28e2-467a-adfe-b550897341c3 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.953935 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea1201-28e2-467a-adfe-b550897341c3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.954363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea1201-28e2-467a-adfe-b550897341c3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.957080 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da95c60-444b-4e27-b427-cad9a2e77541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:53.957574 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95c60-444b-4e27-b427-cad9a2e77541 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.957861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95c60-444b-4e27-b427-cad9a2e77541 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.958121 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95c60-444b-4e27-b427-cad9a2e77541 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.971464 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea1201-28e2-467a-adfe-b550897341c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:53.976236 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da95c60-444b-4e27-b427-cad9a2e77541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-49ea1201-28e2-467a-adfe-b550897341c3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:53.980861 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:53.982170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 09:22:53.983170 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-6da95c60-444b-4e27-b427-cad9a2e77541 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:53.985683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:53.987936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:53.989929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:53.991336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 546/1082] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 09:22:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:53.991562 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 537/1083] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:53.992411 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea1201-28e2-467a-adfe-b550897341c3 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:53.994541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da95c60-444b-4e27-b427-cad9a2e77541 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.078738 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:54.105709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da95c60-444b-4e27-b427-cad9a2e77541 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.106003 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea1201-28e2-467a-adfe-b550897341c3 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.149704 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.150380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.150664 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.150939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.151455 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.151674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:54.151990 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.152136 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.152375 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.152656 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.153409 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:54.153553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.153737 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.155225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:54.155437 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.164770 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 547/1084] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:22:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:54.166513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 538/1085] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:54.171823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.172355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.289889 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816e148-53af-4e8a-8cdd-8f9750198d2b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.291705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad5c3a-1662-417d-a985-522501f9fbd2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.330483 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14ad28-663e-4b4e-bdb0-989075e6932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.331327 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14ad28-663e-4b4e-bdb0-989075e6932d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.331706 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14ad28-663e-4b4e-bdb0-989075e6932d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.331907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.332163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14ad28-663e-4b4e-bdb0-989075e6932d None admin] PATH_INFO: `/v3/users/57f3ca3711de4fff8559e3e167de565a` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.332379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.332671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.332952 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.333858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14ad28-663e-4b4e-bdb0-989075e6932d None admin] RBAC: Authorizing `identity:get_user(user_id=57f3ca3711de4fff8559e3e167de565a)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.335602 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14ad28-663e-4b4e-bdb0-989075e6932d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.337603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 539/1086] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/users/57f3ca3711de4fff8559e3e167de565a => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:54.344003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14ad28-663e-4b4e-bdb0-989075e6932d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.345642 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.348161 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.362319 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:54.374836 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:54.376188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:54.377472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:54.377472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:54.377472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:54.377472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:54.384168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 548/1087] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:54.385653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.448231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:54.486008 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14ad28-663e-4b4e-bdb0-989075e6932d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.504465 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510f22-4bb6-4690-9bd3-347b2d750e11 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.523773 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273149af-e967-44c1-a5e0-28be2b3b11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.524445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273149af-e967-44c1-a5e0-28be2b3b11ae None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.524728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273149af-e967-44c1-a5e0-28be2b3b11ae None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.525013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273149af-e967-44c1-a5e0-28be2b3b11ae None admin] PATH_INFO: `/v3/projects/b1caa9b5266c42a69295ceb3d70d4943` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.526480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273149af-e967-44c1-a5e0-28be2b3b11ae None admin] RBAC: Authorizing `identity:get_project(project_id=b1caa9b5266c42a69295ceb3d70d4943)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.527948 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273149af-e967-44c1-a5e0-28be2b3b11ae None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.529702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 540/1088] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/projects/b1caa9b5266c42a69295ceb3d70d4943 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:54.536127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273149af-e967-44c1-a5e0-28be2b3b11ae None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.539169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.539806 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.540095 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.540370 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.541078 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:54.541377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.542835 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.555620 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 549/1089] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:54.562176 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.647649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:54.677493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273149af-e967-44c1-a5e0-28be2b3b11ae None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.709380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da755d-c605-4d13-87ee-25b0ae16bcc1 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.717756 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21677362-ca02-4a68-ad04-a5d0af97c2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.718725 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677362-ca02-4a68-ad04-a5d0af97c2e3 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.719073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677362-ca02-4a68-ad04-a5d0af97c2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.719526 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677362-ca02-4a68-ad04-a5d0af97c2e3 None admin] PATH_INFO: `/v3/projects/b1caa9b5266c42a69295ceb3d70d4943/users/57f3ca3711de4fff8559e3e167de565a/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.720814 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:54.723164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21677362-ca02-4a68-ad04-a5d0af97c2e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1caa9b5266c42a69295ceb3d70d4943, user_id=57f3ca3711de4fff8559e3e167de565a, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.725149 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21677362-ca02-4a68-ad04-a5d0af97c2e3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.742305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 541/1090] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:22:54 2022] PUT /identity/v3/projects/b1caa9b5266c42a69295ceb3d70d4943/users/57f3ca3711de4fff8559e3e167de565a/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:22:54.770897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.771853 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.772320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.772677 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] PATH_INFO: `/v3/users/nova` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.783475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.785417 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:22:54.803614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:54.804921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:54.806456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 550/1091] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:22:54.813005 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21677362-ca02-4a68-ad04-a5d0af97c2e3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.953982 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21677362-ca02-4a68-ad04-a5d0af97c2e3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.995761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.996303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.996588 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.996870 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.997734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:54.998078 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.999990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.003328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:55.032981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 542/1092] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:55.034619 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.164014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825e77-2fa6-45fa-a6f1-a0eb65780b3f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.202079 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bb36-be15-4b18-ba04-05fc926e198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.202747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bb36-be15-4b18-ba04-05fc926e198d None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.203041 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bb36-be15-4b18-ba04-05fc926e198d None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.203319 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bb36-be15-4b18-ba04-05fc926e198d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.243696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:55.284336 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358bb36-be15-4b18-ba04-05fc926e198d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.287355 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358bb36-be15-4b18-ba04-05fc926e198d None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.428250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 551/1093] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:55.435189 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.567486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1051d-e54c-46c4-b400-ad1fb8f66887 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.612532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.613636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.614164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.614657 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.629907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.632258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:22:55.659067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:55.661028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:55.662570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:22:55.662570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:22:55.662570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:22:55.662570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:55.662570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 543/1094] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:55.670983 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bb36-be15-4b18-ba04-05fc926e198d None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.765313 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:55.818092 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bb36-be15-4b18-ba04-05fc926e198d None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.864615 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.865333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.865644 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.865987 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.866756 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:55.867129 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.868629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.882123 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 552/1095] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:55.888949 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.946048 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:56.028464 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f64cee-8722-43a9-b34a-9f7eb561aaba None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:56.070303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:56.070948 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:56.071235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:56.071511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:56.080025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:56.081677 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:22:56.095488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:56.096636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:22:56.097671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 544/1096] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:56.105091 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:56.231770 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a2e76-4655-422d-ab50-181e5a2846de None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:56.263730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:56.276210 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49588e7-0f79-4a5d-8d42-7fc73b3d73ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:56.277128 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49588e7-0f79-4a5d-8d42-7fc73b3d73ec None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:56.277586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49588e7-0f79-4a5d-8d42-7fc73b3d73ec None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:56.278011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49588e7-0f79-4a5d-8d42-7fc73b3d73ec None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:56.279220 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49588e7-0f79-4a5d-8d42-7fc73b3d73ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:56.279657 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49588e7-0f79-4a5d-8d42-7fc73b3d73ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:56.281845 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49588e7-0f79-4a5d-8d42-7fc73b3d73ec None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:56.310448 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 553/1097] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:22:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:56.312453 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:56.445329 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccdf50-1cc2-485e-9639-723aff6544fe None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:56.453770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:56.485152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55917487-6c4a-49b9-a247-11fab65e2a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:56.485807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55917487-6c4a-49b9-a247-11fab65e2a75 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:56.486088 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55917487-6c4a-49b9-a247-11fab65e2a75 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:56.486398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55917487-6c4a-49b9-a247-11fab65e2a75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:56.487213 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55917487-6c4a-49b9-a247-11fab65e2a75 None admin] RBAC: Adding query filter params (role.id=8d82d0ddc65e4d268c8039cfbd1335a5, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=0970fc7ae258457ca70042acc0fc6c4c) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:56.487528 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55917487-6c4a-49b9-a247-11fab65e2a75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:56.489126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55917487-6c4a-49b9-a247-11fab65e2a75 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:56.497481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 545/1098] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:56 2022] GET /identity/v3/role_assignments?user.id=0970fc7ae258457ca70042acc0fc6c4c&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=8d82d0ddc65e4d268c8039cfbd1335a5 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:56.831307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652267-39ac-4170-bae0-4f4bdb58ff92 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:56.832090 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652267-39ac-4170-bae0-4f4bdb58ff92 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:56.832090 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652267-39ac-4170-bae0-4f4bdb58ff92 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:56.833410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 554/1099] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:22:56.841494 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9189ebe-b3ca-4332-aac0-9be064591e7d None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:56.841494 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9189ebe-b3ca-4332-aac0-9be064591e7d None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:56.841494 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9189ebe-b3ca-4332-aac0-9be064591e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:56.871499 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-d9189ebe-b3ca-4332-aac0-9be064591e7d None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:56.872877 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-d9189ebe-b3ca-4332-aac0-9be064591e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:56.961800 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 546/1100] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:22:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:22:56.994674 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652267-39ac-4170-bae0-4f4bdb58ff92 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.130140 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652267-39ac-4170-bae0-4f4bdb58ff92 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.175665 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dfe1f-9a13-4b58-aca0-7d6484ea56ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.176347 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dfe1f-9a13-4b58-aca0-7d6484ea56ff None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.176628 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dfe1f-9a13-4b58-aca0-7d6484ea56ff None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.176917 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dfe1f-9a13-4b58-aca0-7d6484ea56ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.190600 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dfe1f-9a13-4b58-aca0-7d6484ea56ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-ee7dfe1f-9a13-4b58-aca0-7d6484ea56ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:22:57.197173 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:22:57.199272 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:22:57.200402 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 555/1101] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:22:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:22:57.206162 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9189ebe-b3ca-4332-aac0-9be064591e7d None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.332288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:57.335885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9189ebe-b3ca-4332-aac0-9be064591e7d None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.379069 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.379734 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.380015 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.380359 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.381072 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:57.381559 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:57.383062 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:57.396069 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 547/1102] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:22:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:57.403155 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dfe1f-9a13-4b58-aca0-7d6484ea56ff None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.473951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:57.541772 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dfe1f-9a13-4b58-aca0-7d6484ea56ff None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.600414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcd261-a6eb-4dc7-a83e-c543d34cdfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.601445 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd261-a6eb-4dc7-a83e-c543d34cdfb4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.602069 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd261-a6eb-4dc7-a83e-c543d34cdfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.602485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd261-a6eb-4dc7-a83e-c543d34cdfb4 None admin] PATH_INFO: `/v3/users/57f3ca3711de4fff8559e3e167de565a` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.606441 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcd261-a6eb-4dc7-a83e-c543d34cdfb4 None admin] RBAC: Authorizing `identity:get_user(user_id=57f3ca3711de4fff8559e3e167de565a)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:57.608852 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcd261-a6eb-4dc7-a83e-c543d34cdfb4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:57.612180 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 556/1103] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:22:57 2022] GET /identity/v3/users/57f3ca3711de4fff8559e3e167de565a => generated 285 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:57.619345 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.751070 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517ebf-1e36-4b39-904d-e380177bd93e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.793865 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53437cef-263c-4685-a586-9e25e785cfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.794609 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53437cef-263c-4685-a586-9e25e785cfc2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.794923 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53437cef-263c-4685-a586-9e25e785cfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.795272 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53437cef-263c-4685-a586-9e25e785cfc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.833266 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:57.887280 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53437cef-263c-4685-a586-9e25e785cfc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:57.888794 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53437cef-263c-4685-a586-9e25e785cfc2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:58.019886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 548/1104] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:22:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:22:58.026807 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcd261-a6eb-4dc7-a83e-c543d34cdfb4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:58.143614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcd261-a6eb-4dc7-a83e-c543d34cdfb4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:58.163873 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:58.184021 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a8458-80eb-40a2-943e-900d45e8cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:58.184686 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a8458-80eb-40a2-943e-900d45e8cd78 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:58.184970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a8458-80eb-40a2-943e-900d45e8cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:58.185252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a8458-80eb-40a2-943e-900d45e8cd78 None admin] PATH_INFO: `/v3/projects/b1caa9b5266c42a69295ceb3d70d4943` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:58.186503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a8458-80eb-40a2-943e-900d45e8cd78 None admin] RBAC: Authorizing `identity:get_project(project_id=b1caa9b5266c42a69295ceb3d70d4943)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:58.187965 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a8458-80eb-40a2-943e-900d45e8cd78 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:58.189711 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 557/1105] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:22:58 2022] GET /identity/v3/projects/b1caa9b5266c42a69295ceb3d70d4943 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:58.196817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53437cef-263c-4685-a586-9e25e785cfc2 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:58.267069 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:58.311198 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53437cef-263c-4685-a586-9e25e785cfc2 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:58.350253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655bb9a-48b9-40a3-a21a-09b6d10aff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:58.350964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655bb9a-48b9-40a3-a21a-09b6d10aff00 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:58.351248 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655bb9a-48b9-40a3-a21a-09b6d10aff00 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:58.351527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655bb9a-48b9-40a3-a21a-09b6d10aff00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:58.352343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655bb9a-48b9-40a3-a21a-09b6d10aff00 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=b1caa9b5266c42a69295ceb3d70d4943, user.id=57f3ca3711de4fff8559e3e167de565a) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:58.352659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655bb9a-48b9-40a3-a21a-09b6d10aff00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:58.354221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655bb9a-48b9-40a3-a21a-09b6d10aff00 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:58.361163 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 549/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:22:58 2022] GET /identity/v3/role_assignments?user.id=57f3ca3711de4fff8559e3e167de565a&scope.project.id=b1caa9b5266c42a69295ceb3d70d4943&role.id=af958d22229a4a0297bbe2e643f55181 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:00.504549 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb683a9-08b3-4531-8bfd-eb7a7b84c214 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:00.505153 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb683a9-08b3-4531-8bfd-eb7a7b84c214 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:00.505153 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb683a9-08b3-4531-8bfd-eb7a7b84c214 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:00.506502 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 558/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:00.512298 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41cf33-0e3f-44c9-a712-57fc61768a90 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:00.512997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41cf33-0e3f-44c9-a712-57fc61768a90 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:00.512997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41cf33-0e3f-44c9-a712-57fc61768a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:00.532124 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-2b41cf33-0e3f-44c9-a712-57fc61768a90 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:00.533368 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-2b41cf33-0e3f-44c9-a712-57fc61768a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:00.607470 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 550/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:23:00.632604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb683a9-08b3-4531-8bfd-eb7a7b84c214 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:00.748644 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb683a9-08b3-4531-8bfd-eb7a7b84c214 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:00.788177 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666350a-1813-42ef-ab99-028906518354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:00.788732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666350a-1813-42ef-ab99-028906518354 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:00.789014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666350a-1813-42ef-ab99-028906518354 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:00.789282 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666350a-1813-42ef-ab99-028906518354 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:00.790501 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666350a-1813-42ef-ab99-028906518354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:00.791975 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666350a-1813-42ef-ab99-028906518354 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:00.793594 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 559/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:23:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:00.799607 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41cf33-0e3f-44c9-a712-57fc61768a90 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:00.861521 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:00.915632 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41cf33-0e3f-44c9-a712-57fc61768a90 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:00.956745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d654c-df47-45ae-8e62-110d2f938cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:00.957421 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d654c-df47-45ae-8e62-110d2f938cd4 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:00.957688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d654c-df47-45ae-8e62-110d2f938cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:00.957966 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d654c-df47-45ae-8e62-110d2f938cd4 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:00.958718 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d654c-df47-45ae-8e62-110d2f938cd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:00.960192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d654c-df47-45ae-8e62-110d2f938cd4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:00.985357 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 551/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:23:00 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:03.214733 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861de6e-33f0-4826-80a1-18033cddf610 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:03.215318 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861de6e-33f0-4826-80a1-18033cddf610 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:03.215318 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861de6e-33f0-4826-80a1-18033cddf610 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:03.216638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 560/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:03.222124 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6aa3b-52a9-4a3f-a0fb-992c0900f050 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:03.222631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6aa3b-52a9-4a3f-a0fb-992c0900f050 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:03.222826 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6aa3b-52a9-4a3f-a0fb-992c0900f050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:03.241818 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-aff6aa3b-52a9-4a3f-a0fb-992c0900f050 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:03.243093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-aff6aa3b-52a9-4a3f-a0fb-992c0900f050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:03.320482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 552/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:03.347607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861de6e-33f0-4826-80a1-18033cddf610 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:03.411159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:03.463282 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861de6e-33f0-4826-80a1-18033cddf610 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:03.499087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82410f0-9220-4383-bec0-8239611df365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:03.499650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82410f0-9220-4383-bec0-8239611df365 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:03.499939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82410f0-9220-4383-bec0-8239611df365 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:03.500218 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82410f0-9220-4383-bec0-8239611df365 None admin] PATH_INFO: `/v3/roles/6b79a8b00a964a8fbcbaaa8371e71edb` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:03.501780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82410f0-9220-4383-bec0-8239611df365 None admin] RBAC: Authorizing `identity:get_role(role_id=6b79a8b00a964a8fbcbaaa8371e71edb)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:03.503207 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82410f0-9220-4383-bec0-8239611df365 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:03.504603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 561/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:03 2022] GET /identity/v3/roles/6b79a8b00a964a8fbcbaaa8371e71edb => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:03.511074 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6aa3b-52a9-4a3f-a0fb-992c0900f050 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:03.558204 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:03.618740 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6aa3b-52a9-4a3f-a0fb-992c0900f050 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:03.653695 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c31a0-257c-439b-a8b6-2c7a5f65674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:03.654378 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c31a0-257c-439b-a8b6-2c7a5f65674f None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:03.654662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c31a0-257c-439b-a8b6-2c7a5f65674f None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:03.654937 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c31a0-257c-439b-a8b6-2c7a5f65674f None admin] PATH_INFO: `/v3/users/bb5f9331c4d44025b9c996f57f67cd70` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:03.668902 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c31a0-257c-439b-a8b6-2c7a5f65674f None admin] RBAC: Authorizing `identity:get_user(user_id=bb5f9331c4d44025b9c996f57f67cd70)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:03.670398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c31a0-257c-439b-a8b6-2c7a5f65674f None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:03.672089 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 553/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:03 2022] GET /identity/v3/users/bb5f9331c4d44025b9c996f57f67cd70 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:03.678237 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82410f0-9220-4383-bec0-8239611df365 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:03.794560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82410f0-9220-4383-bec0-8239611df365 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:03.816253 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:03.831858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80411fb-8bcb-424a-8b4f-69922a066d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:03.832519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80411fb-8bcb-424a-8b4f-69922a066d3f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:03.832780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80411fb-8bcb-424a-8b4f-69922a066d3f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:03.833062 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80411fb-8bcb-424a-8b4f-69922a066d3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:03.903020 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80411fb-8bcb-424a-8b4f-69922a066d3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:03.904501 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80411fb-8bcb-424a-8b4f-69922a066d3f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.010979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 562/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:23:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:23:04.017245 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c31a0-257c-439b-a8b6-2c7a5f65674f None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.120342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:04.123499 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c31a0-257c-439b-a8b6-2c7a5f65674f None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.159439 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed051229-97a4-4a88-a2ad-cd01f5f4cb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.160086 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051229-97a4-4a88-a2ad-cd01f5f4cb73 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.160380 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051229-97a4-4a88-a2ad-cd01f5f4cb73 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.160660 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051229-97a4-4a88-a2ad-cd01f5f4cb73 None admin] PATH_INFO: `/v3/projects/b1caa9b5266c42a69295ceb3d70d4943` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.161898 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed051229-97a4-4a88-a2ad-cd01f5f4cb73 None admin] RBAC: Authorizing `identity:get_project(project_id=b1caa9b5266c42a69295ceb3d70d4943)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.163404 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed051229-97a4-4a88-a2ad-cd01f5f4cb73 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.165117 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 554/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:04 2022] GET /identity/v3/projects/b1caa9b5266c42a69295ceb3d70d4943 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:04.171728 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80411fb-8bcb-424a-8b4f-69922a066d3f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.194220 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:04.278520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80411fb-8bcb-424a-8b4f-69922a066d3f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.313971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189aca5-16ec-483b-aed8-93d51800760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.314640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189aca5-16ec-483b-aed8-93d51800760c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.314918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189aca5-16ec-483b-aed8-93d51800760c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.315159 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189aca5-16ec-483b-aed8-93d51800760c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.316278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189aca5-16ec-483b-aed8-93d51800760c None admin] RBAC: Adding query filter params (role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=b1caa9b5266c42a69295ceb3d70d4943, user.id=bb5f9331c4d44025b9c996f57f67cd70) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:04.316614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189aca5-16ec-483b-aed8-93d51800760c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.318060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189aca5-16ec-483b-aed8-93d51800760c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.323928 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 563/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:23:04 2022] GET /identity/v3/role_assignments?user.id=bb5f9331c4d44025b9c996f57f67cd70&scope.project.id=b1caa9b5266c42a69295ceb3d70d4943&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:06.345291 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94459b-f906-4e1d-acbf-e7fd7d2bf0a6 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.345906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94459b-f906-4e1d-acbf-e7fd7d2bf0a6 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.345906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94459b-f906-4e1d-acbf-e7fd7d2bf0a6 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.347233 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 555/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:06.352831 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af4fb-3f80-4e4b-9c96-72162d9cf844 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.353296 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af4fb-3f80-4e4b-9c96-72162d9cf844 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.353408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af4fb-3f80-4e4b-9c96-72162d9cf844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.373253 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-fa5af4fb-3f80-4e4b-9c96-72162d9cf844 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:06.374547 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-fa5af4fb-3f80-4e4b-9c96-72162d9cf844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:06.449635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 564/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:06.473160 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94459b-f906-4e1d-acbf-e7fd7d2bf0a6 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:06.585197 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94459b-f906-4e1d-acbf-e7fd7d2bf0a6 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:06.623315 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5d1f2-adab-4029-b84a-b179adc655d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:06.623969 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5d1f2-adab-4029-b84a-b179adc655d0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.624258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5d1f2-adab-4029-b84a-b179adc655d0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.624542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5d1f2-adab-4029-b84a-b179adc655d0 None admin] PATH_INFO: `/v3/roles/6b79a8b00a964a8fbcbaaa8371e71edb` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.626075 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5d1f2-adab-4029-b84a-b179adc655d0 None admin] RBAC: Authorizing `identity:get_role(role_id=6b79a8b00a964a8fbcbaaa8371e71edb)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:06.627497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5d1f2-adab-4029-b84a-b179adc655d0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:06.628876 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 556/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:06 2022] GET /identity/v3/roles/6b79a8b00a964a8fbcbaaa8371e71edb => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:06.635190 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5af4fb-3f80-4e4b-9c96-72162d9cf844 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:06.651527 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:06.746225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5af4fb-3f80-4e4b-9c96-72162d9cf844 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:06.790476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b395c8-7146-4347-9160-fe855d7c503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:06.791133 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b395c8-7146-4347-9160-fe855d7c503a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.791420 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b395c8-7146-4347-9160-fe855d7c503a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.791706 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b395c8-7146-4347-9160-fe855d7c503a None admin] PATH_INFO: `/v3/users/bb5f9331c4d44025b9c996f57f67cd70` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.793000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b395c8-7146-4347-9160-fe855d7c503a None admin] RBAC: Authorizing `identity:get_user(user_id=bb5f9331c4d44025b9c996f57f67cd70)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:06.794535 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b395c8-7146-4347-9160-fe855d7c503a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:06.796229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 565/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:06 2022] GET /identity/v3/users/bb5f9331c4d44025b9c996f57f67cd70 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:06.805040 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5d1f2-adab-4029-b84a-b179adc655d0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:06.844298 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:06.912436 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5d1f2-adab-4029-b84a-b179adc655d0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:06.947168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84566ce4-6716-4c5e-97fe-68aa8741123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:06.947850 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84566ce4-6716-4c5e-97fe-68aa8741123b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.948158 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84566ce4-6716-4c5e-97fe-68aa8741123b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.948438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84566ce4-6716-4c5e-97fe-68aa8741123b None admin] PATH_INFO: `/v3/projects/b1caa9b5266c42a69295ceb3d70d4943` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.949642 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84566ce4-6716-4c5e-97fe-68aa8741123b None admin] RBAC: Authorizing `identity:get_project(project_id=b1caa9b5266c42a69295ceb3d70d4943)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:06.951123 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84566ce4-6716-4c5e-97fe-68aa8741123b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:06.952850 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 557/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:06 2022] GET /identity/v3/projects/b1caa9b5266c42a69295ceb3d70d4943 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:06.959603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b395c8-7146-4347-9160-fe855d7c503a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:07.046404 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:07.078946 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b395c8-7146-4347-9160-fe855d7c503a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:07.117660 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08906f80-9962-456a-b323-4833026c000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:07.118366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08906f80-9962-456a-b323-4833026c000c None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.118644 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08906f80-9962-456a-b323-4833026c000c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.118915 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08906f80-9962-456a-b323-4833026c000c None admin] PATH_INFO: `/v3/projects/b1caa9b5266c42a69295ceb3d70d4943/users/bb5f9331c4d44025b9c996f57f67cd70/roles/6b79a8b00a964a8fbcbaaa8371e71edb` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.121326 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08906f80-9962-456a-b323-4833026c000c None admin] RBAC: Authorizing `identity:create_grant(project_id=b1caa9b5266c42a69295ceb3d70d4943, user_id=bb5f9331c4d44025b9c996f57f67cd70, role_id=6b79a8b00a964a8fbcbaaa8371e71edb)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:07.122838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08906f80-9962-456a-b323-4833026c000c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:07.138030 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 566/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:23:06 2022] PUT /identity/v3/projects/b1caa9b5266c42a69295ceb3d70d4943/users/bb5f9331c4d44025b9c996f57f67cd70/roles/6b79a8b00a964a8fbcbaaa8371e71edb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:09.162822 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8723263-df1a-4733-8a71-1ca169aa3e8a None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:09.163430 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8723263-df1a-4733-8a71-1ca169aa3e8a None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:09.163430 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8723263-df1a-4733-8a71-1ca169aa3e8a None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:09.164750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 558/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:09.170731 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59b53-5ced-4b2b-834c-58567e114646 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:09.171181 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59b53-5ced-4b2b-834c-58567e114646 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:09.171282 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59b53-5ced-4b2b-834c-58567e114646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:09.190128 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-22e59b53-5ced-4b2b-834c-58567e114646 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:09.191637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-22e59b53-5ced-4b2b-834c-58567e114646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:09.263994 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 567/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:09.288623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8723263-df1a-4733-8a71-1ca169aa3e8a None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:09.402283 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8723263-df1a-4733-8a71-1ca169aa3e8a None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:09.404742 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:09.441341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529cb93-3980-4506-b705-dfe648783b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:09.441906 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529cb93-3980-4506-b705-dfe648783b75 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:09.442400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529cb93-3980-4506-b705-dfe648783b75 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:09.442678 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529cb93-3980-4506-b705-dfe648783b75 None admin] PATH_INFO: `/v3/roles/6b79a8b00a964a8fbcbaaa8371e71edb` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:09.444226 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529cb93-3980-4506-b705-dfe648783b75 None admin] RBAC: Authorizing `identity:get_role(role_id=6b79a8b00a964a8fbcbaaa8371e71edb)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:09.445614 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529cb93-3980-4506-b705-dfe648783b75 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:09.447000 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 559/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:09 2022] GET /identity/v3/roles/6b79a8b00a964a8fbcbaaa8371e71edb => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:09.453241 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e59b53-5ced-4b2b-834c-58567e114646 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:09.528692 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:09.563032 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e59b53-5ced-4b2b-834c-58567e114646 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:09.604360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25843ff-f494-4d06-8f3d-7742ea85fdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:09.605054 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25843ff-f494-4d06-8f3d-7742ea85fdd2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:09.605337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25843ff-f494-4d06-8f3d-7742ea85fdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:09.605614 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25843ff-f494-4d06-8f3d-7742ea85fdd2 None admin] PATH_INFO: `/v3/users/bb5f9331c4d44025b9c996f57f67cd70` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:09.606899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25843ff-f494-4d06-8f3d-7742ea85fdd2 None admin] RBAC: Authorizing `identity:get_user(user_id=bb5f9331c4d44025b9c996f57f67cd70)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:09.608349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25843ff-f494-4d06-8f3d-7742ea85fdd2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:09.609996 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 568/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:09 2022] GET /identity/v3/users/bb5f9331c4d44025b9c996f57f67cd70 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:09.616097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529cb93-3980-4506-b705-dfe648783b75 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:09.723465 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529cb93-3980-4506-b705-dfe648783b75 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:09.765401 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852073a-8340-4be0-9244-026674ed1379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:09.766166 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852073a-8340-4be0-9244-026674ed1379 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:09.766649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852073a-8340-4be0-9244-026674ed1379 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:09.767151 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852073a-8340-4be0-9244-026674ed1379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:09.814404 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:09.844905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852073a-8340-4be0-9244-026674ed1379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:09.846415 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852073a-8340-4be0-9244-026674ed1379 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:09.949316 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 560/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:23:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:23:09.955482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25843ff-f494-4d06-8f3d-7742ea85fdd2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:10.059503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25843ff-f494-4d06-8f3d-7742ea85fdd2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:10.098960 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595092c-d933-4410-8988-bf262413d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:10.099637 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595092c-d933-4410-8988-bf262413d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:10.099927 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595092c-d933-4410-8988-bf262413d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:10.100207 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595092c-d933-4410-8988-bf262413d3b9 None admin] PATH_INFO: `/v3/projects/b1caa9b5266c42a69295ceb3d70d4943` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:10.101493 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595092c-d933-4410-8988-bf262413d3b9 None admin] RBAC: Authorizing `identity:get_project(project_id=b1caa9b5266c42a69295ceb3d70d4943)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:10.103011 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595092c-d933-4410-8988-bf262413d3b9 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:10.104947 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 569/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:09 2022] GET /identity/v3/projects/b1caa9b5266c42a69295ceb3d70d4943 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:10.111601 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852073a-8340-4be0-9244-026674ed1379 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:10.185183 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:10.218627 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852073a-8340-4be0-9244-026674ed1379 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:10.257134 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f197599-8e80-43a9-abe5-3f0990d639d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:10.257766 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f197599-8e80-43a9-abe5-3f0990d639d1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:10.258057 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f197599-8e80-43a9-abe5-3f0990d639d1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:10.258373 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f197599-8e80-43a9-abe5-3f0990d639d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:10.259176 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f197599-8e80-43a9-abe5-3f0990d639d1 None admin] RBAC: Adding query filter params (role.id=6b79a8b00a964a8fbcbaaa8371e71edb, scope.project.id=b1caa9b5266c42a69295ceb3d70d4943, user.id=bb5f9331c4d44025b9c996f57f67cd70) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:10.259493 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f197599-8e80-43a9-abe5-3f0990d639d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:10.260947 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f197599-8e80-43a9-abe5-3f0990d639d1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:10.267636 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 561/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:23:10 2022] GET /identity/v3/role_assignments?user.id=bb5f9331c4d44025b9c996f57f67cd70&scope.project.id=b1caa9b5266c42a69295ceb3d70d4943&role.id=6b79a8b00a964a8fbcbaaa8371e71edb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:12.307973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9333-7c38-4147-b40b-d38f3f4349a2 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.308609 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9333-7c38-4147-b40b-d38f3f4349a2 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.308735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9333-7c38-4147-b40b-d38f3f4349a2 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.310533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 570/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:12.316617 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b1455-d08b-4738-866b-899589893a06 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.317129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b1455-d08b-4738-866b-899589893a06 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.317129 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b1455-d08b-4738-866b-899589893a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.336494 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-4f2b1455-d08b-4738-866b-899589893a06 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:12.337720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-4f2b1455-d08b-4738-866b-899589893a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:12.414286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 562/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:12.438337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9333-7c38-4147-b40b-d38f3f4349a2 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:12.450961 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:12.563981 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9333-7c38-4147-b40b-d38f3f4349a2 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:12.600691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99867d21-bc78-48ab-9a17-94d9533a798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:12.601363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99867d21-bc78-48ab-9a17-94d9533a798e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.601648 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99867d21-bc78-48ab-9a17-94d9533a798e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.601925 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99867d21-bc78-48ab-9a17-94d9533a798e None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.603171 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99867d21-bc78-48ab-9a17-94d9533a798e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:12.604652 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99867d21-bc78-48ab-9a17-94d9533a798e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:12.606338 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 571/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:23:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:12.612343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b1455-d08b-4738-866b-899589893a06 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:12.691547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:12.729820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b1455-d08b-4738-866b-899589893a06 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:12.766126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4affe9-bca5-481e-a4c7-fee87a77fecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:12.766787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4affe9-bca5-481e-a4c7-fee87a77fecc None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.767071 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4affe9-bca5-481e-a4c7-fee87a77fecc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.767353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4affe9-bca5-481e-a4c7-fee87a77fecc None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.768067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4affe9-bca5-481e-a4c7-fee87a77fecc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:12.769566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4affe9-bca5-481e-a4c7-fee87a77fecc None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:12.799972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 563/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:23:12 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:14.992575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc48322-2ac4-4b17-bbbe-51e3b4ff3565 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.993203 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc48322-2ac4-4b17-bbbe-51e3b4ff3565 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.993203 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc48322-2ac4-4b17-bbbe-51e3b4ff3565 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.994824 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 572/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:15.000242 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4690c9a-1689-4d32-9553-99761cfd2963 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.000694 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4690c9a-1689-4d32-9553-99761cfd2963 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.000818 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4690c9a-1689-4d32-9553-99761cfd2963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.019890 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-b4690c9a-1689-4d32-9553-99761cfd2963 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:15.021258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-b4690c9a-1689-4d32-9553-99761cfd2963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:15.101123 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 564/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:15.131720 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc48322-2ac4-4b17-bbbe-51e3b4ff3565 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.222163 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:15.248740 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc48322-2ac4-4b17-bbbe-51e3b4ff3565 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.288235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10ac27-1d94-4f2c-8d6f-dafa9388ff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.288882 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10ac27-1d94-4f2c-8d6f-dafa9388ff54 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.289168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10ac27-1d94-4f2c-8d6f-dafa9388ff54 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.289442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10ac27-1d94-4f2c-8d6f-dafa9388ff54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.290718 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10ac27-1d94-4f2c-8d6f-dafa9388ff54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.292204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10ac27-1d94-4f2c-8d6f-dafa9388ff54 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.293895 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 573/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:23:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:15.300021 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4690c9a-1689-4d32-9553-99761cfd2963 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.350085 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:15.404297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4690c9a-1689-4d32-9553-99761cfd2963 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.446850 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d62e1-1004-444d-9911-be5fc25e9fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.447485 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d62e1-1004-444d-9911-be5fc25e9fcd None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.447817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d62e1-1004-444d-9911-be5fc25e9fcd None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.448104 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d62e1-1004-444d-9911-be5fc25e9fcd None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.448805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d62e1-1004-444d-9911-be5fc25e9fcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.450365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d62e1-1004-444d-9911-be5fc25e9fcd None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.473955 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 565/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:23:15 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:17.668200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831db68-e33d-4fd0-923c-e4bfd71ada82 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:17.668775 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831db68-e33d-4fd0-923c-e4bfd71ada82 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:17.668775 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831db68-e33d-4fd0-923c-e4bfd71ada82 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:17.670133 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 574/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:17.677087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661c717-d61d-4cdb-ab0d-d4c1e09b51e6 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:17.677686 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661c717-d61d-4cdb-ab0d-d4c1e09b51e6 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:17.677686 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661c717-d61d-4cdb-ab0d-d4c1e09b51e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:17.698169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-e661c717-d61d-4cdb-ab0d-d4c1e09b51e6 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:17.699458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-e661c717-d61d-4cdb-ab0d-d4c1e09b51e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:17.771357 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 566/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:17.794848 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831db68-e33d-4fd0-923c-e4bfd71ada82 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:17.909958 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831db68-e33d-4fd0-923c-e4bfd71ada82 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:17.949971 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98360b3-609f-4b41-805e-44a9660efee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:17.950663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98360b3-609f-4b41-805e-44a9660efee9 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:17.950944 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98360b3-609f-4b41-805e-44a9660efee9 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:17.951245 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98360b3-609f-4b41-805e-44a9660efee9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:17.963171 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98360b3-609f-4b41-805e-44a9660efee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-e98360b3-609f-4b41-805e-44a9660efee9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:23:17.968200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:23:17.969380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:23:17.971357 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 575/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:23:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:23:17.975947 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661c717-d61d-4cdb-ab0d-d4c1e09b51e6 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:18.009342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:18.095264 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661c717-d61d-4cdb-ab0d-d4c1e09b51e6 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:18.133313 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:18.133932 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:18.134240 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:18.134518 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:18.135343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:18.135650 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:18.137037 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:18.153844 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 567/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:23:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:18.159838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98360b3-609f-4b41-805e-44a9660efee9 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:18.164545 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:18.268315 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98360b3-609f-4b41-805e-44a9660efee9 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:18.304539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba17fb5-8910-40ca-89bf-7064c076b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:18.305205 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba17fb5-8910-40ca-89bf-7064c076b08a None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:18.305490 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba17fb5-8910-40ca-89bf-7064c076b08a None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:18.305762 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba17fb5-8910-40ca-89bf-7064c076b08a None admin] PATH_INFO: `/v3/users/fde59ffe260e46d08fca0159a2dfc4f3` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:18.321889 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba17fb5-8910-40ca-89bf-7064c076b08a None admin] RBAC: Authorizing `identity:get_user(user_id=fde59ffe260e46d08fca0159a2dfc4f3)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:18.323772 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba17fb5-8910-40ca-89bf-7064c076b08a None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:18.325796 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 576/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:18 2022] GET /identity/v3/users/fde59ffe260e46d08fca0159a2dfc4f3 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:18.332386 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:18.435563 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:18.442859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f0f7a-8404-4bfc-829a-6cec2859c356 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:18.478422 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110fce6-2c97-4c13-b359-28f1b9a44588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:18.479094 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110fce6-2c97-4c13-b359-28f1b9a44588 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:18.479382 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110fce6-2c97-4c13-b359-28f1b9a44588 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:18.479649 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110fce6-2c97-4c13-b359-28f1b9a44588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:18.555139 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110fce6-2c97-4c13-b359-28f1b9a44588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:18.556620 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110fce6-2c97-4c13-b359-28f1b9a44588 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:18.664403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:18.671872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 568/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:23:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:23:18.680341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba17fb5-8910-40ca-89bf-7064c076b08a None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:18.767492 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:18.811689 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba17fb5-8910-40ca-89bf-7064c076b08a None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:18.849675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61031b-1f32-4285-a739-1f61436ef462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:18.850350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61031b-1f32-4285-a739-1f61436ef462 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:18.850624 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61031b-1f32-4285-a739-1f61436ef462 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:18.850897 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61031b-1f32-4285-a739-1f61436ef462 None admin] PATH_INFO: `/v3/projects/5d48337e9038486489985abe2db85a28` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:18.852154 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61031b-1f32-4285-a739-1f61436ef462 None admin] RBAC: Authorizing `identity:get_project(project_id=5d48337e9038486489985abe2db85a28)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:18.853606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61031b-1f32-4285-a739-1f61436ef462 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:18.855443 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 577/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:18 2022] GET /identity/v3/projects/5d48337e9038486489985abe2db85a28 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:23:18.862133 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110fce6-2c97-4c13-b359-28f1b9a44588 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:18.969210 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110fce6-2c97-4c13-b359-28f1b9a44588 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:19.010016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf0f86-2766-4f67-b5e1-c656417c84e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:19.010780 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0f86-2766-4f67-b5e1-c656417c84e2 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:19.011064 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0f86-2766-4f67-b5e1-c656417c84e2 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:19.011341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0f86-2766-4f67-b5e1-c656417c84e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:19.012170 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf0f86-2766-4f67-b5e1-c656417c84e2 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=5d48337e9038486489985abe2db85a28, user.id=fde59ffe260e46d08fca0159a2dfc4f3) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:19.012459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf0f86-2766-4f67-b5e1-c656417c84e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:19.013944 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf0f86-2766-4f67-b5e1-c656417c84e2 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:19.020406 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 569/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:23:18 2022] GET /identity/v3/role_assignments?user.id=fde59ffe260e46d08fca0159a2dfc4f3&scope.project.id=5d48337e9038486489985abe2db85a28&role.id=af958d22229a4a0297bbe2e643f55181 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:21.016400 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bfa47-3f96-46ac-80cd-5b78981cc379 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:21.017026 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bfa47-3f96-46ac-80cd-5b78981cc379 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:21.017026 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bfa47-3f96-46ac-80cd-5b78981cc379 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:21.018380 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 578/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:21.023985 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f651a-7935-4fd0-9d14-14f8fe089e94 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:21.024513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f651a-7935-4fd0-9d14-14f8fe089e94 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:21.024513 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f651a-7935-4fd0-9d14-14f8fe089e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:21.046140 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-724f651a-7935-4fd0-9d14-14f8fe089e94 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:21.047399 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-724f651a-7935-4fd0-9d14-14f8fe089e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:21.118042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:21.131049 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 570/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:21.154349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bfa47-3f96-46ac-80cd-5b78981cc379 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:21.272062 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bfa47-3f96-46ac-80cd-5b78981cc379 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:21.304438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:21.311334 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9acb4e-915b-4063-9fc2-5c90c7177564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:21.312023 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9acb4e-915b-4063-9fc2-5c90c7177564 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:21.312307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9acb4e-915b-4063-9fc2-5c90c7177564 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:21.312583 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9acb4e-915b-4063-9fc2-5c90c7177564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:21.325019 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9acb4e-915b-4063-9fc2-5c90c7177564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-3c9acb4e-915b-4063-9fc2-5c90c7177564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:23:21.330142 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:23:21.331557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:23:21.333247 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 579/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:23:21.337801 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f651a-7935-4fd0-9d14-14f8fe089e94 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:21.460285 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f651a-7935-4fd0-9d14-14f8fe089e94 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:21.497942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:21.498631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:21.498945 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:21.499238 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:21.499940 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:21.500254 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:21.501684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:21.511661 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 571/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:23:21.517773 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9acb4e-915b-4063-9fc2-5c90c7177564 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:21.652691 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9acb4e-915b-4063-9fc2-5c90c7177564 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:21.690095 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681838-ff8d-42f2-811a-e8e5768a072f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:21.690792 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681838-ff8d-42f2-811a-e8e5768a072f None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:21.691438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681838-ff8d-42f2-811a-e8e5768a072f None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:21.691722 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681838-ff8d-42f2-811a-e8e5768a072f None admin] PATH_INFO: `/v3/users/fde59ffe260e46d08fca0159a2dfc4f3` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:21.692957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09681838-ff8d-42f2-811a-e8e5768a072f None admin] RBAC: Authorizing `identity:get_user(user_id=fde59ffe260e46d08fca0159a2dfc4f3)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:21.694462 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09681838-ff8d-42f2-811a-e8e5768a072f None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:21.696209 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 580/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:21 2022] GET /identity/v3/users/fde59ffe260e46d08fca0159a2dfc4f3 => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:23:21.702521 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:21.758139 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:21.814528 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9b320-bf1c-4732-805b-78b04c4ad248 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:21.853459 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ffce1-df37-41d4-b8bd-de780ba7e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:21.854087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ffce1-df37-41d4-b8bd-de780ba7e96d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:21.854371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ffce1-df37-41d4-b8bd-de780ba7e96d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:21.854667 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ffce1-df37-41d4-b8bd-de780ba7e96d None admin] PATH_INFO: `/v3/projects/5d48337e9038486489985abe2db85a28` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:21.855881 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ffce1-df37-41d4-b8bd-de780ba7e96d None admin] RBAC: Authorizing `identity:get_project(project_id=5d48337e9038486489985abe2db85a28)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:21.857330 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ffce1-df37-41d4-b8bd-de780ba7e96d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:21.859081 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 572/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:21 2022] GET /identity/v3/projects/5d48337e9038486489985abe2db85a28 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:21.865477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681838-ff8d-42f2-811a-e8e5768a072f None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:21.932031 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:21.978455 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681838-ff8d-42f2-811a-e8e5768a072f None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:22.014752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f683e-2890-4cf0-b14c-4dc98a8896d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:22.015485 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f683e-2890-4cf0-b14c-4dc98a8896d1 None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:22.015755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f683e-2890-4cf0-b14c-4dc98a8896d1 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:22.016071 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f683e-2890-4cf0-b14c-4dc98a8896d1 None admin] PATH_INFO: `/v3/projects/5d48337e9038486489985abe2db85a28/users/fde59ffe260e46d08fca0159a2dfc4f3/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:22.018211 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f683e-2890-4cf0-b14c-4dc98a8896d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d48337e9038486489985abe2db85a28, user_id=fde59ffe260e46d08fca0159a2dfc4f3, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:22.019682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f683e-2890-4cf0-b14c-4dc98a8896d1 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:22.034202 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 581/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:23:21 2022] PUT /identity/v3/projects/5d48337e9038486489985abe2db85a28/users/fde59ffe260e46d08fca0159a2dfc4f3/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:24.056943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3facd08-f5e7-4bbb-aaf4-e8917fd8e60e None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:24.057587 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3facd08-f5e7-4bbb-aaf4-e8917fd8e60e None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:24.057587 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3facd08-f5e7-4bbb-aaf4-e8917fd8e60e None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:24.059025 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 573/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:24.064980 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edadf0e-5f5a-4203-8c7e-dc3e4f18b0c5 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:24.065475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edadf0e-5f5a-4203-8c7e-dc3e4f18b0c5 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:24.065475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edadf0e-5f5a-4203-8c7e-dc3e4f18b0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:24.084908 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-0edadf0e-5f5a-4203-8c7e-dc3e4f18b0c5 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:24.086127 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-0edadf0e-5f5a-4203-8c7e-dc3e4f18b0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:24.155337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 582/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:24.179017 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3facd08-f5e7-4bbb-aaf4-e8917fd8e60e None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:24.286568 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3facd08-f5e7-4bbb-aaf4-e8917fd8e60e None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:24.307619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:24.325210 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8909ec-87d8-48a6-9c71-02ee9eb2ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:24.325859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8909ec-87d8-48a6-9c71-02ee9eb2ac0d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:24.326144 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8909ec-87d8-48a6-9c71-02ee9eb2ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:24.326438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8909ec-87d8-48a6-9c71-02ee9eb2ac0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:24.337448 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8909ec-87d8-48a6-9c71-02ee9eb2ac0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-6f8909ec-87d8-48a6-9c71-02ee9eb2ac0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:23:24.342371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:23:24.343782 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:23:24.344972 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 574/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:23:24.349978 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edadf0e-5f5a-4203-8c7e-dc3e4f18b0c5 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:24.401606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:24.456097 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edadf0e-5f5a-4203-8c7e-dc3e4f18b0c5 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:24.492287 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:24.492923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:24.493206 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:24.493482 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:24.494285 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:24.494844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:24.496257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:24.505513 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 583/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:23:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:24.511811 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8909ec-87d8-48a6-9c71-02ee9eb2ac0d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:24.628670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8909ec-87d8-48a6-9c71-02ee9eb2ac0d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:24.667427 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5cfdc-37d5-4f52-845b-b92b287c575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:24.668093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5cfdc-37d5-4f52-845b-b92b287c575c None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:24.668368 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5cfdc-37d5-4f52-845b-b92b287c575c None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:24.668648 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5cfdc-37d5-4f52-845b-b92b287c575c None admin] PATH_INFO: `/v3/users/fde59ffe260e46d08fca0159a2dfc4f3` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:24.669915 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5cfdc-37d5-4f52-845b-b92b287c575c None admin] RBAC: Authorizing `identity:get_user(user_id=fde59ffe260e46d08fca0159a2dfc4f3)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:24.671428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5cfdc-37d5-4f52-845b-b92b287c575c None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:24.673088 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 575/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:24 2022] GET /identity/v3/users/fde59ffe260e46d08fca0159a2dfc4f3 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:24.679281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:24.786838 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4fc17-2768-414d-bbd2-7e7a0a29ea60 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:24.797533 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:24.822842 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461957c-3a6e-4434-aeec-b01aa8b41e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:24.823536 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461957c-3a6e-4434-aeec-b01aa8b41e21 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:24.823811 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461957c-3a6e-4434-aeec-b01aa8b41e21 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:24.824087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461957c-3a6e-4434-aeec-b01aa8b41e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:24.894520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461957c-3a6e-4434-aeec-b01aa8b41e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:24.895985 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461957c-3a6e-4434-aeec-b01aa8b41e21 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:25.007636 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 584/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:23:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:23:25.014318 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5cfdc-37d5-4f52-845b-b92b287c575c None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:25.068190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:25.121480 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5cfdc-37d5-4f52-845b-b92b287c575c None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:25.156228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b11ae-8308-4ba6-9c65-b6e95dd71ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:25.156798 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b11ae-8308-4ba6-9c65-b6e95dd71ce1 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:25.157079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b11ae-8308-4ba6-9c65-b6e95dd71ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:25.157319 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b11ae-8308-4ba6-9c65-b6e95dd71ce1 None admin] PATH_INFO: `/v3/projects/5d48337e9038486489985abe2db85a28` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:25.158599 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b11ae-8308-4ba6-9c65-b6e95dd71ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=5d48337e9038486489985abe2db85a28)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:25.160045 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b11ae-8308-4ba6-9c65-b6e95dd71ce1 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:25.161726 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 576/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:25 2022] GET /identity/v3/projects/5d48337e9038486489985abe2db85a28 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:25.168560 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461957c-3a6e-4434-aeec-b01aa8b41e21 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:25.180829 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:25.275902 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461957c-3a6e-4434-aeec-b01aa8b41e21 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:25.311969 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3ad0f-b9af-4ae8-a19e-da59ca0daf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:25.312626 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3ad0f-b9af-4ae8-a19e-da59ca0daf77 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:25.312934 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3ad0f-b9af-4ae8-a19e-da59ca0daf77 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:25.313216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3ad0f-b9af-4ae8-a19e-da59ca0daf77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:25.314018 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3ad0f-b9af-4ae8-a19e-da59ca0daf77 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=5d48337e9038486489985abe2db85a28, user.id=fde59ffe260e46d08fca0159a2dfc4f3) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:25.314370 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3ad0f-b9af-4ae8-a19e-da59ca0daf77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:25.315815 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3ad0f-b9af-4ae8-a19e-da59ca0daf77 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:25.322703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 585/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:23:25 2022] GET /identity/v3/role_assignments?user.id=fde59ffe260e46d08fca0159a2dfc4f3&scope.project.id=5d48337e9038486489985abe2db85a28&role.id=af958d22229a4a0297bbe2e643f55181 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:27.391352 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c5aa-6802-4293-9822-bd47bb5143c1 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:27.392009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c5aa-6802-4293-9822-bd47bb5143c1 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:27.392009 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c5aa-6802-4293-9822-bd47bb5143c1 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:27.393301 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 577/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:27.398662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea517-17be-43fa-9ec1-9d25241a5162 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:27.399116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea517-17be-43fa-9ec1-9d25241a5162 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:27.399227 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea517-17be-43fa-9ec1-9d25241a5162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:27.418245 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-4ebea517-17be-43fa-9ec1-9d25241a5162 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:27.419460 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-4ebea517-17be-43fa-9ec1-9d25241a5162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:27.491652 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 586/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:27.513507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8c5aa-6802-4293-9822-bd47bb5143c1 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:27.638749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8c5aa-6802-4293-9822-bd47bb5143c1 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:27.654944 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:27.675933 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc1ed-7399-4df7-9cf1-24252ceb2e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:27.676558 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc1ed-7399-4df7-9cf1-24252ceb2e2b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:27.676839 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc1ed-7399-4df7-9cf1-24252ceb2e2b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:27.677112 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc1ed-7399-4df7-9cf1-24252ceb2e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:27.762883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cc1ed-7399-4df7-9cf1-24252ceb2e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:27.764354 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cc1ed-7399-4df7-9cf1-24252ceb2e2b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:27.870154 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 578/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:23:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:23:27.876446 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebea517-17be-43fa-9ec1-9d25241a5162 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:27.879673 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:27.986660 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebea517-17be-43fa-9ec1-9d25241a5162 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.023730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cabc8-8e15-4506-b82a-bdad64a645cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.024410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cabc8-8e15-4506-b82a-bdad64a645cc None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.024696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cabc8-8e15-4506-b82a-bdad64a645cc None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.024974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cabc8-8e15-4506-b82a-bdad64a645cc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.033508 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cabc8-8e15-4506-b82a-bdad64a645cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.035037 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cabc8-8e15-4506-b82a-bdad64a645cc None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-4d2cabc8-8e15-4506-b82a-bdad64a645cc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:23:28.047656 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:23:28.049204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:23:28.050509 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 587/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:23:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:23:28.056219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc1ed-7399-4df7-9cf1-24252ceb2e2b None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:28.076998 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:28.169848 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc1ed-7399-4df7-9cf1-24252ceb2e2b None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.204941 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939b0f0-8c19-4d66-a58f-74ac6e39c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.205567 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939b0f0-8c19-4d66-a58f-74ac6e39c278 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.205851 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939b0f0-8c19-4d66-a58f-74ac6e39c278 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.206147 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939b0f0-8c19-4d66-a58f-74ac6e39c278 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.206864 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939b0f0-8c19-4d66-a58f-74ac6e39c278 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:28.207174 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939b0f0-8c19-4d66-a58f-74ac6e39c278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.208537 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939b0f0-8c19-4d66-a58f-74ac6e39c278 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.217634 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 579/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 09:23:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:30.186585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4c03e-644e-40cc-b58d-3b6b419eab6a None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:30.187209 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4c03e-644e-40cc-b58d-3b6b419eab6a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:30.187209 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4c03e-644e-40cc-b58d-3b6b419eab6a None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:30.188529 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 588/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:30.194597 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41946811-cc07-4248-b829-24825e1045d9 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:30.195107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41946811-cc07-4248-b829-24825e1045d9 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:30.195227 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41946811-cc07-4248-b829-24825e1045d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:30.214631 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-41946811-cc07-4248-b829-24825e1045d9 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:30.215875 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-41946811-cc07-4248-b829-24825e1045d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:30.288419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 580/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:30.311368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4c03e-644e-40cc-b58d-3b6b419eab6a None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:30.382126 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:30.424573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4c03e-644e-40cc-b58d-3b6b419eab6a None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:30.460702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb96b3-937f-46bf-bfa4-364286c02c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:30.461235 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb96b3-937f-46bf-bfa4-364286c02c72 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:30.461519 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb96b3-937f-46bf-bfa4-364286c02c72 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:30.461795 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb96b3-937f-46bf-bfa4-364286c02c72 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:30.462546 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb96b3-937f-46bf-bfa4-364286c02c72 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:30.463995 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb96b3-937f-46bf-bfa4-364286c02c72 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:30.481248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 589/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jul 28 09:23:30 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:32.497250 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c4942-e6d2-44c0-b2cc-4a2c412e7fef None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.497871 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c4942-e6d2-44c0-b2cc-4a2c412e7fef None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.497871 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c4942-e6d2-44c0-b2cc-4a2c412e7fef None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.500162 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 581/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:32.507869 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171a96-25c7-4c83-88c8-921dc40c8a03 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.508490 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171a96-25c7-4c83-88c8-921dc40c8a03 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.508490 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171a96-25c7-4c83-88c8-921dc40c8a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.527014 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-7f171a96-25c7-4c83-88c8-921dc40c8a03 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:32.528226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-7f171a96-25c7-4c83-88c8-921dc40c8a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:32.598734 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 590/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:32.623055 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c4942-e6d2-44c0-b2cc-4a2c412e7fef None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.639029 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:32.746807 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c4942-e6d2-44c0-b2cc-4a2c412e7fef None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.782946 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fa8d5-ccb4-435d-a169-0fbfa27176d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.783509 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa8d5-ccb4-435d-a169-0fbfa27176d4 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.783792 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa8d5-ccb4-435d-a169-0fbfa27176d4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.784066 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa8d5-ccb4-435d-a169-0fbfa27176d4 None admin] PATH_INFO: `/v3/domains/7d609c50905e468db80707e7bb1285a7` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.795087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fa8d5-ccb4-435d-a169-0fbfa27176d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7d609c50905e468db80707e7bb1285a7)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.796560 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fa8d5-ccb4-435d-a169-0fbfa27176d4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.798311 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 582/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 09:23:32 2022] GET /identity/v3/domains/7d609c50905e468db80707e7bb1285a7 => generated 267 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:32.804697 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f171a96-25c7-4c83-88c8-921dc40c8a03 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.866273 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:32.914252 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f171a96-25c7-4c83-88c8-921dc40c8a03 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.949984 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c00541-102b-484d-b3ab-ffde6631b8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.950650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00541-102b-484d-b3ab-ffde6631b8da None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.950904 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00541-102b-484d-b3ab-ffde6631b8da None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.951209 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00541-102b-484d-b3ab-ffde6631b8da None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.951899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c00541-102b-484d-b3ab-ffde6631b8da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.953984 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c00541-102b-484d-b3ab-ffde6631b8da None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.988394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 591/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:23:32 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:35.110482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b45e6-7b07-47b8-baa5-9d7fb3f9b44d None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:35.111107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b45e6-7b07-47b8-baa5-9d7fb3f9b44d None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:35.111107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b45e6-7b07-47b8-baa5-9d7fb3f9b44d None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:35.112424 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 583/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:35.117930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed779-ca6a-4812-a41e-e81606fba043 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:35.118414 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed779-ca6a-4812-a41e-e81606fba043 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:35.118531 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed779-ca6a-4812-a41e-e81606fba043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:35.141835 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-0e1ed779-ca6a-4812-a41e-e81606fba043 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:35.142982 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-0e1ed779-ca6a-4812-a41e-e81606fba043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:35.216903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 592/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:35.241716 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b45e6-7b07-47b8-baa5-9d7fb3f9b44d None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:35.324988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:35.360481 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b45e6-7b07-47b8-baa5-9d7fb3f9b44d None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:35.398702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8f9be-a94d-4d3e-a53a-c49180eb7d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:35.399367 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8f9be-a94d-4d3e-a53a-c49180eb7d5e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:35.399654 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8f9be-a94d-4d3e-a53a-c49180eb7d5e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:35.399932 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8f9be-a94d-4d3e-a53a-c49180eb7d5e None admin] PATH_INFO: `/v3/domains/7d609c50905e468db80707e7bb1285a7` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:35.401187 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8f9be-a94d-4d3e-a53a-c49180eb7d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=7d609c50905e468db80707e7bb1285a7)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:35.402702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8f9be-a94d-4d3e-a53a-c49180eb7d5e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:35.404361 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 584/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 09:23:35 2022] GET /identity/v3/domains/7d609c50905e468db80707e7bb1285a7 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:35.410606 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ed779-ca6a-4812-a41e-e81606fba043 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:35.447104 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:35.517056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ed779-ca6a-4812-a41e-e81606fba043 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:35.551846 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829ca8e-af73-4ba6-9b0c-8fcdafa5f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:35.552470 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829ca8e-af73-4ba6-9b0c-8fcdafa5f012 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:35.552749 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829ca8e-af73-4ba6-9b0c-8fcdafa5f012 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:35.553020 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829ca8e-af73-4ba6-9b0c-8fcdafa5f012 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:35.553709 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829ca8e-af73-4ba6-9b0c-8fcdafa5f012 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:35.555195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829ca8e-af73-4ba6-9b0c-8fcdafa5f012 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:35.578595 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 593/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:23:35 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:37.627676 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd701e-56f6-40b7-9aa8-53a1888f15d8 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:37.628191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd701e-56f6-40b7-9aa8-53a1888f15d8 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:37.628191 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd701e-56f6-40b7-9aa8-53a1888f15d8 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:37.629603 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 585/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:37.635090 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7c5ae-8c7e-4f63-a645-1fb8c5323b4a None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:37.635558 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7c5ae-8c7e-4f63-a645-1fb8c5323b4a None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:37.635696 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7c5ae-8c7e-4f63-a645-1fb8c5323b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:37.654764 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-d3f7c5ae-8c7e-4f63-a645-1fb8c5323b4a None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:37.656031 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-d3f7c5ae-8c7e-4f63-a645-1fb8c5323b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:37.729379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 594/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:37.753108 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd701e-56f6-40b7-9aa8-53a1888f15d8 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:37.860331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd701e-56f6-40b7-9aa8-53a1888f15d8 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:37.898124 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4263b6d-2540-4aaf-b1f5-1e793998c16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:37.898798 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4263b6d-2540-4aaf-b1f5-1e793998c16d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:37.899085 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4263b6d-2540-4aaf-b1f5-1e793998c16d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:37.899356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4263b6d-2540-4aaf-b1f5-1e793998c16d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:37.907115 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:37.911843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4263b6d-2540-4aaf-b1f5-1e793998c16d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-a4263b6d-2540-4aaf-b1f5-1e793998c16d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:23:37.917168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:23:37.918532 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:23:37.919815 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 586/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:23:37.924957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7c5ae-8c7e-4f63-a645-1fb8c5323b4a None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:37.951055 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:38.032275 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7c5ae-8c7e-4f63-a645-1fb8c5323b4a None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.069000 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.069635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.069918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.070226 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.070957 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:38.071271 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.072699 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.080988 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 595/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:38.087295 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4263b6d-2540-4aaf-b1f5-1e793998c16d None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.195572 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4263b6d-2540-4aaf-b1f5-1e793998c16d None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.232434 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe8d55-e205-46ff-b8c7-0ed0e895b2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.233091 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe8d55-e205-46ff-b8c7-0ed0e895b2f0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.233386 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe8d55-e205-46ff-b8c7-0ed0e895b2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.233656 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe8d55-e205-46ff-b8c7-0ed0e895b2f0 None admin] PATH_INFO: `/v3/users/d0d84e2439724844b9ac418f8e51127e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.248158 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe8d55-e205-46ff-b8c7-0ed0e895b2f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d0d84e2439724844b9ac418f8e51127e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.249638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe8d55-e205-46ff-b8c7-0ed0e895b2f0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.251503 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 587/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/users/d0d84e2439724844b9ac418f8e51127e => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:38.257739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.360738 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:38.379217 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc42d1e-e4e6-4581-ba1e-d273d9607185 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.417420 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb984129-81ab-44f5-a0ce-318f534a3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.418058 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984129-81ab-44f5-a0ce-318f534a3d00 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.418379 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984129-81ab-44f5-a0ce-318f534a3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.418752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984129-81ab-44f5-a0ce-318f534a3d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.490540 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984129-81ab-44f5-a0ce-318f534a3d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.491938 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984129-81ab-44f5-a0ce-318f534a3d00 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.578031 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:38.596890 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 596/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:23:38.603252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe8d55-e205-46ff-b8c7-0ed0e895b2f0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.664056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:38.712384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe8d55-e205-46ff-b8c7-0ed0e895b2f0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.750547 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428628-7aa1-448b-ad26-1405261bc43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.751138 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428628-7aa1-448b-ad26-1405261bc43d None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.751428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428628-7aa1-448b-ad26-1405261bc43d None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.751704 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428628-7aa1-448b-ad26-1405261bc43d None admin] PATH_INFO: `/v3/projects/adb222b5176941e096a53862813f7357` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.753035 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428628-7aa1-448b-ad26-1405261bc43d None admin] RBAC: Authorizing `identity:get_project(project_id=adb222b5176941e096a53862813f7357)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.754559 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428628-7aa1-448b-ad26-1405261bc43d None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.756349 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 588/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/projects/adb222b5176941e096a53862813f7357 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:38.765350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb984129-81ab-44f5-a0ce-318f534a3d00 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.882651 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb984129-81ab-44f5-a0ce-318f534a3d00 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.919422 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28bf60-5958-4973-b9cf-bc0c8ee4de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.920050 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28bf60-5958-4973-b9cf-bc0c8ee4de42 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.920337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28bf60-5958-4973-b9cf-bc0c8ee4de42 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.920607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28bf60-5958-4973-b9cf-bc0c8ee4de42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.921407 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28bf60-5958-4973-b9cf-bc0c8ee4de42 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=adb222b5176941e096a53862813f7357, user.id=d0d84e2439724844b9ac418f8e51127e) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:38.921718 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28bf60-5958-4973-b9cf-bc0c8ee4de42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.923206 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28bf60-5958-4973-b9cf-bc0c8ee4de42 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.928956 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 597/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/role_assignments?user.id=d0d84e2439724844b9ac418f8e51127e&scope.project.id=adb222b5176941e096a53862813f7357&role.id=af958d22229a4a0297bbe2e643f55181 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:40.923858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f80be-91ef-47eb-89e4-aadd0e2f6b50 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:40.924442 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f80be-91ef-47eb-89e4-aadd0e2f6b50 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:40.924442 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f80be-91ef-47eb-89e4-aadd0e2f6b50 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:40.925805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 589/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:40.931561 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc48cd-bac9-457f-9ee1-47bdfa6b79f6 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:40.932001 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc48cd-bac9-457f-9ee1-47bdfa6b79f6 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:40.932130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc48cd-bac9-457f-9ee1-47bdfa6b79f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:40.952996 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-c7dc48cd-bac9-457f-9ee1-47bdfa6b79f6 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:40.954313 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-c7dc48cd-bac9-457f-9ee1-47bdfa6b79f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:41.008378 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:41.036377 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 598/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:41.060114 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f80be-91ef-47eb-89e4-aadd0e2f6b50 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.164687 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f80be-91ef-47eb-89e4-aadd0e2f6b50 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.190593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:41.211920 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012c3c0-c92d-45d3-9a4a-66bedd12bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.212936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012c3c0-c92d-45d3-9a4a-66bedd12bbca None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.213214 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012c3c0-c92d-45d3-9a4a-66bedd12bbca None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.213487 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012c3c0-c92d-45d3-9a4a-66bedd12bbca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.224303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012c3c0-c92d-45d3-9a4a-66bedd12bbca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-9012c3c0-c92d-45d3-9a4a-66bedd12bbca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:23:41.229234 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:23:41.230671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:23:41.232042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 590/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:23:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:23:41.236777 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc48cd-bac9-457f-9ee1-47bdfa6b79f6 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.340585 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc48cd-bac9-457f-9ee1-47bdfa6b79f6 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.377029 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.377660 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.377947 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.378250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.378990 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:41.379294 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.380682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.392155 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 599/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:23:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:41.399931 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012c3c0-c92d-45d3-9a4a-66bedd12bbca None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.511302 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012c3c0-c92d-45d3-9a4a-66bedd12bbca None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.547952 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f04c2-82ff-40f7-bbd0-2c1e9c5102e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.548617 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f04c2-82ff-40f7-bbd0-2c1e9c5102e8 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.548998 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f04c2-82ff-40f7-bbd0-2c1e9c5102e8 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.549280 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f04c2-82ff-40f7-bbd0-2c1e9c5102e8 None admin] PATH_INFO: `/v3/users/d0d84e2439724844b9ac418f8e51127e` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.550510 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f04c2-82ff-40f7-bbd0-2c1e9c5102e8 None admin] RBAC: Authorizing `identity:get_user(user_id=d0d84e2439724844b9ac418f8e51127e)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.552010 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f04c2-82ff-40f7-bbd0-2c1e9c5102e8 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.553659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 591/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:41 2022] GET /identity/v3/users/d0d84e2439724844b9ac418f8e51127e => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:41.559920 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.611238 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:41.680596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a5257-1d6b-45e7-984f-fbcd0638ad01 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.716337 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bef1a-a1d9-4045-9873-f22fde27a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.716997 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bef1a-a1d9-4045-9873-f22fde27a619 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.717284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bef1a-a1d9-4045-9873-f22fde27a619 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.717557 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bef1a-a1d9-4045-9873-f22fde27a619 None admin] PATH_INFO: `/v3/projects/adb222b5176941e096a53862813f7357` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.718844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bef1a-a1d9-4045-9873-f22fde27a619 None admin] RBAC: Authorizing `identity:get_project(project_id=adb222b5176941e096a53862813f7357)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.720438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bef1a-a1d9-4045-9873-f22fde27a619 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.722197 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 600/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:41 2022] GET /identity/v3/projects/adb222b5176941e096a53862813f7357 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:41.728594 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f04c2-82ff-40f7-bbd0-2c1e9c5102e8 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.785724 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:41.841258 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f04c2-82ff-40f7-bbd0-2c1e9c5102e8 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.878593 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f2783-516f-46d3-8c26-3a58d35633dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.879326 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f2783-516f-46d3-8c26-3a58d35633dc None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.879610 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f2783-516f-46d3-8c26-3a58d35633dc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.879893 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f2783-516f-46d3-8c26-3a58d35633dc None admin] PATH_INFO: `/v3/projects/adb222b5176941e096a53862813f7357/users/d0d84e2439724844b9ac418f8e51127e/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.881974 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f2783-516f-46d3-8c26-3a58d35633dc None admin] RBAC: Authorizing `identity:create_grant(project_id=adb222b5176941e096a53862813f7357, user_id=d0d84e2439724844b9ac418f8e51127e, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.883478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f2783-516f-46d3-8c26-3a58d35633dc None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.898449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 592/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:23:41 2022] PUT /identity/v3/projects/adb222b5176941e096a53862813f7357/users/d0d84e2439724844b9ac418f8e51127e/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:43.945506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7486-373f-4d91-9c08-2c2691998475 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:43.946152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7486-373f-4d91-9c08-2c2691998475 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:43.946152 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7486-373f-4d91-9c08-2c2691998475 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:43.947616 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 601/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:23:43.953001 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d069e2a-8a4f-42db-a6b6-9da7bfcc8ece None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:43.953461 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d069e2a-8a4f-42db-a6b6-9da7bfcc8ece None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:43.953565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d069e2a-8a4f-42db-a6b6-9da7bfcc8ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:43.972469 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-0d069e2a-8a4f-42db-a6b6-9da7bfcc8ece None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:43.973691 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-0d069e2a-8a4f-42db-a6b6-9da7bfcc8ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:44.046132 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 593/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:23:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:23:44.070169 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b7486-373f-4d91-9c08-2c2691998475 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:44.180116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b7486-373f-4d91-9c08-2c2691998475 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:44.190060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:44.217732 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951aefa-093b-4517-97b5-f8b2cd012915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:44.218343 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951aefa-093b-4517-97b5-f8b2cd012915 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:44.218631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951aefa-093b-4517-97b5-f8b2cd012915 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:44.218911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951aefa-093b-4517-97b5-f8b2cd012915 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:44.230389 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951aefa-093b-4517-97b5-f8b2cd012915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-2951aefa-093b-4517-97b5-f8b2cd012915 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:23:44.237223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:23:44.238735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:23:44.239843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 602/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:23:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:23:44.245729 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d069e2a-8a4f-42db-a6b6-9da7bfcc8ece None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:44.294420 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:44.356516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d069e2a-8a4f-42db-a6b6-9da7bfcc8ece None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:44.394179 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:44.394820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:44.395137 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:44.395416 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:44.396886 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:44.397289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:44.398728 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:44.407312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 594/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:44.413495 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951aefa-093b-4517-97b5-f8b2cd012915 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:44.522898 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951aefa-093b-4517-97b5-f8b2cd012915 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:44.561615 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17773068-b833-4a35-a598-cba38d87a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:44.562315 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17773068-b833-4a35-a598-cba38d87a838 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:44.562595 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17773068-b833-4a35-a598-cba38d87a838 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:44.562871 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17773068-b833-4a35-a598-cba38d87a838 None admin] PATH_INFO: `/v3/users/d0d84e2439724844b9ac418f8e51127e` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:44.564118 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17773068-b833-4a35-a598-cba38d87a838 None admin] RBAC: Authorizing `identity:get_user(user_id=d0d84e2439724844b9ac418f8e51127e)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:44.565581 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17773068-b833-4a35-a598-cba38d87a838 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:44.567620 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 603/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 09:23:44 2022] GET /identity/v3/users/d0d84e2439724844b9ac418f8e51127e => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:44.574262 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:44.681126 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed19a8-b726-40b5-bfa0-8bec133869f3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:44.683707 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:44.717501 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f43c89-40b3-49fb-a29c-0750e6226980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:44.718169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43c89-40b3-49fb-a29c-0750e6226980 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:44.718447 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43c89-40b3-49fb-a29c-0750e6226980 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:44.718721 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43c89-40b3-49fb-a29c-0750e6226980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:44.788851 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f43c89-40b3-49fb-a29c-0750e6226980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:44.790288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f43c89-40b3-49fb-a29c-0750e6226980 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:44.891485 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 595/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:23:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:23:44.897723 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17773068-b833-4a35-a598-cba38d87a838 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:44.935393 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:45.004816 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17773068-b833-4a35-a598-cba38d87a838 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:45.040342 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da33e20-8d82-48e4-8c70-23f850b75dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:45.040946 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33e20-8d82-48e4-8c70-23f850b75dfa None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:45.041217 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33e20-8d82-48e4-8c70-23f850b75dfa None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:45.041494 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33e20-8d82-48e4-8c70-23f850b75dfa None admin] PATH_INFO: `/v3/projects/adb222b5176941e096a53862813f7357` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:45.042751 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da33e20-8d82-48e4-8c70-23f850b75dfa None admin] RBAC: Authorizing `identity:get_project(project_id=adb222b5176941e096a53862813f7357)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:45.044207 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da33e20-8d82-48e4-8c70-23f850b75dfa None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:45.045921 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 604/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 09:23:44 2022] GET /identity/v3/projects/adb222b5176941e096a53862813f7357 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:45.052495 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f43c89-40b3-49fb-a29c-0750e6226980 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:45.055830 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:45.157251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f43c89-40b3-49fb-a29c-0750e6226980 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:45.193739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1ffb0-a114-48e5-ae89-ebc7f45b20ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:45.194393 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1ffb0-a114-48e5-ae89-ebc7f45b20ab None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:45.194757 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1ffb0-a114-48e5-ae89-ebc7f45b20ab None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:45.194931 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1ffb0-a114-48e5-ae89-ebc7f45b20ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:45.195772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1ffb0-a114-48e5-ae89-ebc7f45b20ab None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=adb222b5176941e096a53862813f7357, user.id=d0d84e2439724844b9ac418f8e51127e) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:45.196087 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1ffb0-a114-48e5-ae89-ebc7f45b20ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:45.197609 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1ffb0-a114-48e5-ae89-ebc7f45b20ab None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:45.203737 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 596/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:23:45 2022] GET /identity/v3/role_assignments?user.id=d0d84e2439724844b9ac418f8e51127e&scope.project.id=adb222b5176941e096a53862813f7357&role.id=af958d22229a4a0297bbe2e643f55181 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:55.035629 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=48 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:55.201006 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=66 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:41.410433 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae64e42-cece-44ef-a450-c3609f85ad56 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:41.411778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae64e42-cece-44ef-a450-c3609f85ad56 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:41.412645 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae64e42-cece-44ef-a450-c3609f85ad56 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:41.415443 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 605/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:24:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:24:41.423824 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d982d-f95c-41ee-b24d-29a40b1aa3e3 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:41.424699 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d982d-f95c-41ee-b24d-29a40b1aa3e3 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:41.424699 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d982d-f95c-41ee-b24d-29a40b1aa3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:41.447064 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-ec1d982d-f95c-41ee-b24d-29a40b1aa3e3 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:41.448453 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-ec1d982d-f95c-41ee-b24d-29a40b1aa3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:41.523982 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 597/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:24:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:24:41.558662 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae64e42-cece-44ef-a450-c3609f85ad56 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:41.718555 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae64e42-cece-44ef-a450-c3609f85ad56 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:41.764550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:41.765686 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:41.766216 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:41.766790 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:41.779967 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:41.781891 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:24:41.799231 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:41.800671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:41.802060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:24:41.802060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:24:41.802060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:41.802060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:24:41.802060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 606/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:41.809511 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d982d-f95c-41ee-b24d-29a40b1aa3e3 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:41.921517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d982d-f95c-41ee-b24d-29a40b1aa3e3 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:41.963097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:41.970520 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11813a-f8b6-4333-a35c-a9bf739c4144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:41.971095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11813a-f8b6-4333-a35c-a9bf739c4144 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:41.971381 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11813a-f8b6-4333-a35c-a9bf739c4144 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:41.971661 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11813a-f8b6-4333-a35c-a9bf739c4144 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:41.972379 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11813a-f8b6-4333-a35c-a9bf739c4144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:41.972688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11813a-f8b6-4333-a35c-a9bf739c4144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:41.974133 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11813a-f8b6-4333-a35c-a9bf739c4144 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:41.994335 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 598/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:24:41.999929 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:42.075162 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:42.179366 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546a349-8b8f-4000-ae15-070107d7fada None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:42.239703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9853b-46b5-4268-a3d3-2fc50aa0dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:42.240476 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9853b-46b5-4268-a3d3-2fc50aa0dd16 None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:42.240864 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9853b-46b5-4268-a3d3-2fc50aa0dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:42.241243 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9853b-46b5-4268-a3d3-2fc50aa0dd16 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:42.242308 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9853b-46b5-4268-a3d3-2fc50aa0dd16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:42.244448 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9853b-46b5-4268-a3d3-2fc50aa0dd16 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:42.279313 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 607/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 09:24:41 2022] POST /identity/v3/users => generated 251 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:24:44.515280 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b01019-7fcc-4d8c-9b14-12da346c1049 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:44.516228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b01019-7fcc-4d8c-9b14-12da346c1049 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:44.516228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b01019-7fcc-4d8c-9b14-12da346c1049 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:44.517589 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 599/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:24:44.523841 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98999683-577f-42dd-8eb0-e219e22b78a2 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:44.524682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98999683-577f-42dd-8eb0-e219e22b78a2 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:44.524682 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98999683-577f-42dd-8eb0-e219e22b78a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:44.545102 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-98999683-577f-42dd-8eb0-e219e22b78a2 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:44.546520 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-98999683-577f-42dd-8eb0-e219e22b78a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:44.668419 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 608/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:24:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:24:44.699739 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b01019-7fcc-4d8c-9b14-12da346c1049 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:44.864275 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b01019-7fcc-4d8c-9b14-12da346c1049 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:44.905169 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76992c-1fd4-4858-bf6e-1feea72cd446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:44.905872 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76992c-1fd4-4858-bf6e-1feea72cd446 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:44.906199 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76992c-1fd4-4858-bf6e-1feea72cd446 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:44.906520 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76992c-1fd4-4858-bf6e-1feea72cd446 None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:44.919847 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76992c-1fd4-4858-bf6e-1feea72cd446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-ec76992c-1fd4-4858-bf6e-1feea72cd446 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:24:44.927060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:44.928475 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:24:44.929552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:44.934042 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 600/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:44.935668 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98999683-577f-42dd-8eb0-e219e22b78a2 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:44.987297 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:45.074399 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98999683-577f-42dd-8eb0-e219e22b78a2 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:45.123618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:45.124323 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:45.124612 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:45.124945 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:45.125718 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:45.126080 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:45.127588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:45.140661 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 609/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:45.148229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76992c-1fd4-4858-bf6e-1feea72cd446 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:45.238069 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:45.333135 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76992c-1fd4-4858-bf6e-1feea72cd446 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:45.393298 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:45.394097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:45.394486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:45.394927 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:45.409401 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:45.411062 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:24:45.429235 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:45.431759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:45.433993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:24:45.433993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:24:45.433993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:45.433993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:45.433993 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 601/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:45.441163 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:45.631395 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:45.636911 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8a5fe-4d19-4455-aff5-510844a78434 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:45.698172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:45.699200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:45.699586 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:45.699986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:45.701318 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:45.701739 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:45.703698 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:45.722654 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 610/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:45.732468 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:45.915003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf222f-e7d2-48ef-9246-d62d318236e8 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:45.917861 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:45.978248 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:45.978973 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:45.979333 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:45.979692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:45.992153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:45.993843 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:24:46.012557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:46.022659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:46.024278 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 602/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:46.025729 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:46.180945 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211a2d9-c67e-44ee-b836-15936cb63f02 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:46.230951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:46.231502 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:46.231783 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:46.232031 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:46.232942 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:46.233255 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:46.234811 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:46.260558 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 611/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:24:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:24:46.267745 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:46.452763 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bea92e-4968-49bc-ab1b-0c6a09cd3ac0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:46.508167 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68949fd-7736-4072-ac75-ce65338d89e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:46.508969 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68949fd-7736-4072-ac75-ce65338d89e0 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:46.509386 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68949fd-7736-4072-ac75-ce65338d89e0 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:46.509887 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68949fd-7736-4072-ac75-ce65338d89e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:46.562403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:46.645075 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68949fd-7736-4072-ac75-ce65338d89e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:46.648269 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68949fd-7736-4072-ac75-ce65338d89e0 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:46.790280 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 603/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:24:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:24:46.793827 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:46.834151 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:46.977350 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0256d-7299-479d-9043-12fe2e7cebc3 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:47.050061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:47.051328 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:47.051872 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:47.052436 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:47.071472 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:47.074009 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:24:47.103973 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:47.106705 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:47.109048 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:24:47.109048 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:24:47.109048 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:47.109048 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:24:47.109048 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 612/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:24:47.113175 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68949fd-7736-4072-ac75-ce65338d89e0 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:47.237153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:47.312747 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68949fd-7736-4072-ac75-ce65338d89e0 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:47.371965 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:47.372683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:47.372981 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:47.373221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:47.374003 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:47.374355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:47.375905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:47.401528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:47.402247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 604/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:47.545638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:47.609991 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631cdee-ad72-438e-80ca-ac38e87935f8 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:47.674401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:47.674401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:47.674401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:47.674401 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:47.685951 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:47.687619 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:24:47.702640 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:47.704919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:24:47.707892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 613/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:24:47.713485 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:47.868467 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed496f1-b166-4fc0-9725-1f2fa4a901db None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:47.909189 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:47.934679 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94a34a-9b9d-40de-9d87-9079814ef1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:47.935343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94a34a-9b9d-40de-9d87-9079814ef1cb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:47.935622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94a34a-9b9d-40de-9d87-9079814ef1cb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:47.935952 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94a34a-9b9d-40de-9d87-9079814ef1cb None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:47.936746 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94a34a-9b9d-40de-9d87-9079814ef1cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:47.937060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94a34a-9b9d-40de-9d87-9079814ef1cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:47.939182 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94a34a-9b9d-40de-9d87-9079814ef1cb None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:47.960334 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 605/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:24:47.967177 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:48.115426 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ec900-02ab-484b-acb9-7bf7eaac0ae4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:48.176158 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:48.186172 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb889a-092e-479e-bfc4-4c9e86b4d653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:48.186841 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb889a-092e-479e-bfc4-4c9e86b4d653 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:48.187130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb889a-092e-479e-bfc4-4c9e86b4d653 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:48.187397 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb889a-092e-479e-bfc4-4c9e86b4d653 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:48.188605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb889a-092e-479e-bfc4-4c9e86b4d653 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=a117c23f88cd4b7ab7b4c0f0d24341d5) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:48.189060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb889a-092e-479e-bfc4-4c9e86b4d653 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:48.191695 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb889a-092e-479e-bfc4-4c9e86b4d653 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:48.200086 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 614/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:24:47 2022] GET /identity/v3/role_assignments?user.id=a117c23f88cd4b7ab7b4c0f0d24341d5&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:50.833253 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717e489-3913-4af3-a132-f61069f67655 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:50.833889 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717e489-3913-4af3-a132-f61069f67655 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:50.833997 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717e489-3913-4af3-a132-f61069f67655 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:50.835845 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 606/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:24:50.842803 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9699a-b015-4d6a-9aa4-cc6f598ac7cd None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:50.843678 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9699a-b015-4d6a-9aa4-cc6f598ac7cd None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:50.843678 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9699a-b015-4d6a-9aa4-cc6f598ac7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:50.864891 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-93f9699a-b015-4d6a-9aa4-cc6f598ac7cd None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:50.866095 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-93f9699a-b015-4d6a-9aa4-cc6f598ac7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:50.964341 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 615/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:24:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:24:51.007964 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717e489-3913-4af3-a132-f61069f67655 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:51.147032 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717e489-3913-4af3-a132-f61069f67655 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:51.195889 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84996bd-7e83-45c2-a26f-fc088552bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:51.196922 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84996bd-7e83-45c2-a26f-fc088552bbfb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:51.197260 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84996bd-7e83-45c2-a26f-fc088552bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:51.197675 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84996bd-7e83-45c2-a26f-fc088552bbfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:51.216331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84996bd-7e83-45c2-a26f-fc088552bbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c84996bd-7e83-45c2-a26f-fc088552bbfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:24:51.224476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:51.227130 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:51.229770 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 607/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:51.236799 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9699a-b015-4d6a-9aa4-cc6f598ac7cd None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:51.385297 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9699a-b015-4d6a-9aa4-cc6f598ac7cd None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:51.416936 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:51.432888 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:51.433582 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:51.433891 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:51.434160 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:51.434920 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:51.435229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:51.436702 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:51.446734 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 616/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:51.453229 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84996bd-7e83-45c2-a26f-fc088552bbfb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:51.503583 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:51.574942 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84996bd-7e83-45c2-a26f-fc088552bbfb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:51.624530 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:51.625461 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:51.625781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:51.626207 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:51.640858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:51.642420 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:24:51.659512 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:51.661400 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:51.662892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:24:51.662892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:24:51.662892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:51.662892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:51.662892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 608/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:24:51.667981 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:51.804314 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb61e2-0a64-4e4e-90dc-0653ad8bfcd8 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:51.850526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:51.851675 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:51.852110 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:51.852535 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:51.853525 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:51.853970 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:51.855933 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:51.875558 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 617/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:51.881777 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:52.005305 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:52.014362 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ec785-6c57-4a54-b4fb-f4d419f044ce None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:52.054156 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:52.055079 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.055471 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.055875 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.065684 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:52.067520 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:24:52.080781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:52.082303 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:52.083507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 609/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:52.091569 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:52.146892 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:52.224397 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b4880-2b46-4e32-9d89-2be17d125ca7 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:52.264588 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:52.265118 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.265387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.265667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.266589 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:52.266903 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:52.268503 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:52.292910 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 618/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:24:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:52.300219 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:52.438671 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d55766-3189-4076-8d2b-f4c787cb9b36 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:52.485018 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:52.486350 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.486646 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.486929 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.500284 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:52.501892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:24:52.520107 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:52.521456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:52.522771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:24:52.522771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:24:52.522771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:52.522771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:52.522771 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 610/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:52.527929 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:52.696559 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df6bb-345c-433b-a896-7e8c589ff6b4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:52.700535 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:52.747471 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:52.748190 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.748496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.748830 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.749541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:52.749881 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:52.751407 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:52.765327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 619/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:24:52.773530 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:52.783901 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:52.903749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35ee97-0d88-4a30-88e2-f3a1280a7718 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:52.946342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:52.947046 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.947377 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.947686 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.957183 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:52.959536 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:24:52.973919 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:52.976816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:52.978905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 611/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:52.985360 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:53.138367 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77169e-f0a4-4879-8cd8-1e247e913a03 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:53.185373 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640a4b6-d3e6-42cf-80bf-d1d6952df275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:53.186435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a4b6-d3e6-42cf-80bf-d1d6952df275 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:53.186918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a4b6-d3e6-42cf-80bf-d1d6952df275 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:53.187399 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a4b6-d3e6-42cf-80bf-d1d6952df275 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:53.188798 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640a4b6-d3e6-42cf-80bf-d1d6952df275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:53.189346 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640a4b6-d3e6-42cf-80bf-d1d6952df275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:53.191835 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640a4b6-d3e6-42cf-80bf-d1d6952df275 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:53.210752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 620/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:53.218582 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:53.312071 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:53.363821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56f16f-a191-451b-9276-e36ab5e723da None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:53.413788 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccfbf1-b614-4bed-b21b-a6c39d5b8691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:53.414653 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccfbf1-b614-4bed-b21b-a6c39d5b8691 None admin] REQUEST_METHOD: `PUT` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:53.415106 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccfbf1-b614-4bed-b21b-a6c39d5b8691 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:53.415560 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccfbf1-b614-4bed-b21b-a6c39d5b8691 None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/a117c23f88cd4b7ab7b4c0f0d24341d5/roles/aec37946c06745cfb2ea73e9b3c9a547` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:53.440488 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccfbf1-b614-4bed-b21b-a6c39d5b8691 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=a117c23f88cd4b7ab7b4c0f0d24341d5, role_id=aec37946c06745cfb2ea73e9b3c9a547)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:53.442552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccfbf1-b614-4bed-b21b-a6c39d5b8691 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:53.458333 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 612/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:24:53 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/a117c23f88cd4b7ab7b4c0f0d24341d5/roles/aec37946c06745cfb2ea73e9b3c9a547 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:24:55.674954 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c87f7-1b42-48af-8258-cb9731785850 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.675781 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c87f7-1b42-48af-8258-cb9731785850 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.675781 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c87f7-1b42-48af-8258-cb9731785850 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.677045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 621/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:24:55.683619 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89c81e-40f6-4e5e-826e-07fe96eb5eca None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.684339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89c81e-40f6-4e5e-826e-07fe96eb5eca None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.684339 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89c81e-40f6-4e5e-826e-07fe96eb5eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.706686 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-4e89c81e-40f6-4e5e-826e-07fe96eb5eca None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:55.708183 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-4e89c81e-40f6-4e5e-826e-07fe96eb5eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:55.792718 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 613/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:24:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:24:55.819856 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c87f7-1b42-48af-8258-cb9731785850 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.856438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:55.967752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c87f7-1b42-48af-8258-cb9731785850 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:56.021291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3a29-659b-48e9-a340-d1c070da70ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:56.022382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3a29-659b-48e9-a340-d1c070da70ff None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:56.022858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3a29-659b-48e9-a340-d1c070da70ff None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:56.023330 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3a29-659b-48e9-a340-d1c070da70ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:56.042405 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf3a29-659b-48e9-a340-d1c070da70ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-4daf3a29-659b-48e9-a340-d1c070da70ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:24:56.052108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:56.054045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:24:56.056382 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 622/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 09:24:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:56.060925 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89c81e-40f6-4e5e-826e-07fe96eb5eca None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:56.122622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:56.195120 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89c81e-40f6-4e5e-826e-07fe96eb5eca None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:56.250056 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:56.250829 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:56.251190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:56.251542 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:56.252361 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] RBAC: Adding query filter params (name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:56.252744 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:56.254268 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:56.265198 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 614/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 09:24:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:56.271865 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3a29-659b-48e9-a340-d1c070da70ff None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:56.402469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:56.425212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3a29-659b-48e9-a340-d1c070da70ff None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:56.465023 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:56.465728 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:56.465984 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:56.466300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:56.477821 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:56.479368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:24:56.495818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:56.497408 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:56.498573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:24:56.498573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:24:56.498573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:56.498573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:24:56.498573 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 623/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:56.503717 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:56.622759 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a3af9-8884-4fb0-8100-110605d3d2fa None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:56.632892 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:56.665882 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:56.666428 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:56.666719 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:56.667016 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:56.667762 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:56.668071 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:56.669474 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:56.683356 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 615/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:56.689475 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:56.819607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c4217-fcae-4081-8dfa-8d82dd4cd41e None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:56.863060 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:56.863713 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:56.863986 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:56.864257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:56.867809 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:56.873440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:56.875375 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:24:56.896442 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:56.897779 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:24:56.899504 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 624/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:24:56.909192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:57.041244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27aa61-f9e1-430a-8013-0f4ee6af6e14 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:57.089458 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:57.090243 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:57.090604 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:57.090917 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:57.092010 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:57.092332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:57.093809 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:57.119711 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 616/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:24:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:57.125939 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:57.256886 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa9d73-f3e9-45b6-aa35-d01fd5bc7ec2 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:57.306458 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24c613-83d5-46de-a536-ccb3362bf1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:57.306992 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24c613-83d5-46de-a536-ccb3362bf1d4 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:57.307278 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24c613-83d5-46de-a536-ccb3362bf1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:57.307554 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24c613-83d5-46de-a536-ccb3362bf1d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:57.401363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:57.411438 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24c613-83d5-46de-a536-ccb3362bf1d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:57.413284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24c613-83d5-46de-a536-ccb3362bf1d4 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:57.533368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 625/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:24:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:24:57.540209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:57.572571 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:57.659252 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007d47-07ea-4f38-9129-a72181c7e422 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:57.704438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:57.705541 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:57.706032 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:57.706557 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:57.725577 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:57.728315 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:24:57.747387 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:57.748769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:57.749911 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:24:57.749911 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:24:57.749911 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:24:57.749911 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:57.749911 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 617/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:24:57.754948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24c613-83d5-46de-a536-ccb3362bf1d4 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:57.860317 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:57.870174 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24c613-83d5-46de-a536-ccb3362bf1d4 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:57.922653 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:57.923526 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:57.923937 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:57.924333 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:57.925340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:57.925789 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:57.927730 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:57.947042 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 626/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:57.954209 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:58.049152 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:58.084623 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef0c47-93a9-4ca4-95d3-5a0ffedaf258 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:58.142491 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:58.143578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:58.144095 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:58.144586 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:58.157195 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:58.159943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:24:58.183127 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:24:58.184554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:24:58.185659 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 618/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:24:58.192998 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:58.333300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd382a-9d80-48eb-86c5-7bd65852820b None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:58.379397 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c74022-2295-4523-a3d4-7bac05098bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:58.380053 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c74022-2295-4523-a3d4-7bac05098bae None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:58.380335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c74022-2295-4523-a3d4-7bac05098bae None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:58.380610 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c74022-2295-4523-a3d4-7bac05098bae None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:58.381394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c74022-2295-4523-a3d4-7bac05098bae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:58.381716 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c74022-2295-4523-a3d4-7bac05098bae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:58.383321 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c74022-2295-4523-a3d4-7bac05098bae None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:58.397962 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 627/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:24:58.404733 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:58.551403 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a4c12-7243-4df3-922f-846827884310 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:58.596174 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e00e3-2bdb-45b0-819e-631be9d80621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:58.596802 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e00e3-2bdb-45b0-819e-631be9d80621 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:58.597093 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e00e3-2bdb-45b0-819e-631be9d80621 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:58.597371 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e00e3-2bdb-45b0-819e-631be9d80621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:58.598212 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e00e3-2bdb-45b0-819e-631be9d80621 None admin] RBAC: Adding query filter params (role.id=aec37946c06745cfb2ea73e9b3c9a547, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=a117c23f88cd4b7ab7b4c0f0d24341d5) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:58.598662 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e00e3-2bdb-45b0-819e-631be9d80621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:58.600622 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e00e3-2bdb-45b0-819e-631be9d80621 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:58.609576 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 619/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:24:58 2022] GET /identity/v3/role_assignments?user.id=a117c23f88cd4b7ab7b4c0f0d24341d5&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=aec37946c06745cfb2ea73e9b3c9a547 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:01.142823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a4662-c1d8-4e34-aec5-6d41eda0825f None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.143794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a4662-c1d8-4e34-aec5-6d41eda0825f None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.143794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a4662-c1d8-4e34-aec5-6d41eda0825f None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.145635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 628/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:25:01.152383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14711750-91b8-4942-8e97-1027cc5cc65d None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.153330 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14711750-91b8-4942-8e97-1027cc5cc65d None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.153330 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14711750-91b8-4942-8e97-1027cc5cc65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.162790 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:01.177943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-14711750-91b8-4942-8e97-1027cc5cc65d None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:01.179295 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-14711750-91b8-4942-8e97-1027cc5cc65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:01.270165 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 620/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:25:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:25:01.291434 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a4662-c1d8-4e34-aec5-6d41eda0825f None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:01.318958 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:01.462499 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a4662-c1d8-4e34-aec5-6d41eda0825f None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:01.510868 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae3f49-ac0a-41b8-8110-00f34053e1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:01.511613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae3f49-ac0a-41b8-8110-00f34053e1f3 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.512081 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae3f49-ac0a-41b8-8110-00f34053e1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.512328 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae3f49-ac0a-41b8-8110-00f34053e1f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.527477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae3f49-ac0a-41b8-8110-00f34053e1f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-87ae3f49-ac0a-41b8-8110-00f34053e1f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:25:01.533384 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:01.535780 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:01.537277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 629/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:25:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:01.542218 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14711750-91b8-4942-8e97-1027cc5cc65d None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:01.675763 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14711750-91b8-4942-8e97-1027cc5cc65d None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:01.729555 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:01.737270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:01.738202 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.738638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.739050 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.740060 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:01.740527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:01.742578 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:01.756504 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 621/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:01.765496 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae3f49-ac0a-41b8-8110-00f34053e1f3 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:01.860587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:01.892277 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae3f49-ac0a-41b8-8110-00f34053e1f3 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:01.935958 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:01.936635 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.936919 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.937189 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.950979 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:01.952642 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:25:01.971250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:01.973431 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:01.975416 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:25:01.975416 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:25:01.975416 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:01.975416 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:01.975416 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 630/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:01.982438 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:02.150291 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55c830-8964-44e8-94b4-7dee7342918e None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:02.196291 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:02.197270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:02.197474 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:02.197858 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:02.198908 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:02.199341 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:02.201408 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:02.225644 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 622/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:02.227420 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:02.387076 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8a332-ec0e-40cb-b0f6-62c6f3d474bf None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:02.430918 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:02.444953 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:02.446212 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:02.446713 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:02.447316 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] PATH_INFO: `/v3/users/placement` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:02.458858 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:02.460323 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:25:02.473755 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:02.476603 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:02.478270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 631/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:02.481838 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:02.561936 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:02.622643 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785e75c-0b33-488f-82d7-4e6237bea119 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:02.669810 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:02.670379 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:02.670661 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:02.670943 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:02.671965 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:02.672450 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:02.673988 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:02.698216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 623/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:25:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:02.705483 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:02.854301 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3959ed-50e1-4c5f-aa43-d726a2377deb None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:02.905136 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabd5f8-2848-4719-a587-db3d72f85097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:02.905855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabd5f8-2848-4719-a587-db3d72f85097 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:02.906168 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabd5f8-2848-4719-a587-db3d72f85097 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:02.906505 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabd5f8-2848-4719-a587-db3d72f85097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:02.985144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:03.002427 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabd5f8-2848-4719-a587-db3d72f85097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:03.005223 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabd5f8-2848-4719-a587-db3d72f85097 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:03.143188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 632/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:25:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:25:03.149599 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:03.275269 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5764dfe-d51a-420e-bbf0-9357f3c46671 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:03.291426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:03.323247 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:03.323979 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:03.324275 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:03.324599 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:03.337202 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:03.338905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:25:03.357435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:03.364150 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:03.365507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:25:03.365507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:25:03.365507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:03.365507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:03.365507 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 624/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:03.365964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabd5f8-2848-4719-a587-db3d72f85097 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:03.500506 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:03.515250 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabd5f8-2848-4719-a587-db3d72f85097 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:03.561085 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:03.561636 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:03.561909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:03.562229 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:03.562949 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:03.563254 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:03.564671 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:03.580246 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 633/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:03.585940 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:03.712103 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde5795-9d6a-4297-8678-9e8dbd9ed5d7 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:03.752145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:03.752982 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:03.753259 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:03.753570 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:03.762478 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:03.764097 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:25:03.778058 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:03.779328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:03.781384 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 625/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:25:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:03.785899 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:03.913542 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3079dc-4d39-480f-9123-dd996e2077f5 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:03.956056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8349b2-67e7-4c27-9e9c-3fd95470db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:03.957116 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8349b2-67e7-4c27-9e9c-3fd95470db67 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:03.957450 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8349b2-67e7-4c27-9e9c-3fd95470db67 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:03.957738 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8349b2-67e7-4c27-9e9c-3fd95470db67 None admin] PATH_INFO: `/v3/projects` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:03.958576 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8349b2-67e7-4c27-9e9c-3fd95470db67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:03.958882 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8349b2-67e7-4c27-9e9c-3fd95470db67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:03.960387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8349b2-67e7-4c27-9e9c-3fd95470db67 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:03.975221 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 634/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:25:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:03.981836 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:04.012781 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:04.124954 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a57f1-86de-4c08-9d04-abbc9ea231ef None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:04.180891 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145e638-1d88-4f1c-856e-78ee453ebbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:04.181456 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145e638-1d88-4f1c-856e-78ee453ebbb9 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:04.181761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145e638-1d88-4f1c-856e-78ee453ebbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:04.182013 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145e638-1d88-4f1c-856e-78ee453ebbb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:04.182859 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145e638-1d88-4f1c-856e-78ee453ebbb9 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=a117c23f88cd4b7ab7b4c0f0d24341d5) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:04.183158 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145e638-1d88-4f1c-856e-78ee453ebbb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:04.184615 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145e638-1d88-4f1c-856e-78ee453ebbb9 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:04.191816 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 626/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:25:03 2022] GET /identity/v3/role_assignments?user.id=a117c23f88cd4b7ab7b4c0f0d24341d5&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=af958d22229a4a0297bbe2e643f55181 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:25:06.411735 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513873bb-39b4-4aed-b53e-a4c55c40a702 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:06.412335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513873bb-39b4-4aed-b53e-a4c55c40a702 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:06.412717 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513873bb-39b4-4aed-b53e-a4c55c40a702 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:06.415855 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 635/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:25:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:25:06.422075 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632588c-5d59-4a8f-b398-b3310b14cbb0 None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:06.422957 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632588c-5d59-4a8f-b398-b3310b14cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:06.422957 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632588c-5d59-4a8f-b398-b3310b14cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:06.445903 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-2632588c-5d59-4a8f-b398-b3310b14cbb0 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:06.447177 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-2632588c-5d59-4a8f-b398-b3310b14cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:06.529688 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 627/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:25:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 09:25:06.565667 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513873bb-39b4-4aed-b53e-a4c55c40a702 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:06.584017 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:06.703191 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513873bb-39b4-4aed-b53e-a4c55c40a702 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:06.755605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f9fdb-dfdf-4bcf-b231-5614a1cfd577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:06.756647 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f9fdb-dfdf-4bcf-b231-5614a1cfd577 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:06.757058 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f9fdb-dfdf-4bcf-b231-5614a1cfd577 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:06.757570 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f9fdb-dfdf-4bcf-b231-5614a1cfd577 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:06.774440 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f9fdb-dfdf-4bcf-b231-5614a1cfd577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-dc0f9fdb-dfdf-4bcf-b231-5614a1cfd577 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:25:06.780843 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:06.783195 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:06.785260 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 636/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:06.789156 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632588c-5d59-4a8f-b398-b3310b14cbb0 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:06.812203 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:06.947327 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632588c-5d59-4a8f-b398-b3310b14cbb0 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:06.997632 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:06.998216 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:06.998545 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:06.998820 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] PATH_INFO: `/v3/roles` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:06.999581 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:06.999914 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:07.001270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:07.016821 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 628/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:25:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:07.023550 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f9fdb-dfdf-4bcf-b231-5614a1cfd577 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:07.113498 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:07.165045 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f9fdb-dfdf-4bcf-b231-5614a1cfd577 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:07.208556 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:07.209279 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:07.209566 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:07.209847 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:07.225061 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:07.226775 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:25:07.245747 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:07.247335 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:07.248601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:25:07.248601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:25:07.248601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:07.248601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:07.248601 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 637/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:07.254263 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:07.399934 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:07.438052 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc308c9c-766e-402b-929b-c9a5cdbde288 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:07.501760 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:07.502767 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:07.503251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:07.503749 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:07.505055 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:07.505566 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:07.508328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:07.532342 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 629/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:07.539683 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:07.669056 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8db6b8-5371-497c-89c6-2be56c6eeb4e None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:07.693243 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:07.711072 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:07.711744 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:07.712030 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:07.712307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] PATH_INFO: `/v3/users/placement` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:07.721274 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:07.722841 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:25:07.736787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:07.739425 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:07.741801 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 638/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:07.746449 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:07.872471 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37dbabb-c4df-4d48-87ea-da7f47c436cc None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:07.911388 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:07.912047 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:07.912331 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:07.912601 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] PATH_INFO: `/v3/users` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:07.913445 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:07.913754 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:07.915353 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:07.925828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:07.936899 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 630/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:25:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:07.943564 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:08.052827 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d0b72-a9c4-4973-a832-6b317fd72514 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:08.091368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:08.092023 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:08.092304 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:08.092584 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:08.104293 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:08.105769 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:25:08.121552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:08.123646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:08.125638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:25:08.125638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:25:08.125638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:08.125638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:08.125638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 639/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:08.129190 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:08.255693 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9adfd-b86b-45d5-a0e2-cf5038056a32 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:08.299306 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:08.299999 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:08.300289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:08.300573 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:08.301328 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:08.301638 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:08.303153 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:08.318432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 631/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:08.326353 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:08.380469 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:08.473452 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a7b7f-006e-43af-82e2-7338e9bd9a45 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:08.524319 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:08.525067 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:08.525352 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:08.525631 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:08.534190 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:08.535719 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:25:08.549200 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:08.551596 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:08.553552 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 640/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:08.556967 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:08.627700 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:08.673237 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b37de-c946-40df-846f-7f6a8cdc6e07 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:08.713540 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df1b6f-82df-4d2c-af1d-d26c28e83ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:08.714237 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df1b6f-82df-4d2c-af1d-d26c28e83ab3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:08.714516 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df1b6f-82df-4d2c-af1d-d26c28e83ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:08.714789 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df1b6f-82df-4d2c-af1d-d26c28e83ab3 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:08.715565 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df1b6f-82df-4d2c-af1d-d26c28e83ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:08.715876 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df1b6f-82df-4d2c-af1d-d26c28e83ab3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:08.717327 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df1b6f-82df-4d2c-af1d-d26c28e83ab3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:08.730878 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 632/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:08.738093 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:08.836794 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:08.847492 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b1a5f-a8ff-41fa-8aca-1bc97c6ded42 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:08.887111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3528d1-1979-4229-a229-4798473b6d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:08.887111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3528d1-1979-4229-a229-4798473b6d5e None admin] REQUEST_METHOD: `PUT` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:08.887111 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3528d1-1979-4229-a229-4798473b6d5e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:08.887930 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3528d1-1979-4229-a229-4798473b6d5e None admin] PATH_INFO: `/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/a117c23f88cd4b7ab7b4c0f0d24341d5/roles/af958d22229a4a0297bbe2e643f55181` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:08.892394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3528d1-1979-4229-a229-4798473b6d5e None admin] RBAC: Authorizing `identity:create_grant(project_id=9a54bfaeb3c04288a7942306702edba9, user_id=a117c23f88cd4b7ab7b4c0f0d24341d5, role_id=af958d22229a4a0297bbe2e643f55181)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:08.892394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3528d1-1979-4229-a229-4798473b6d5e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:08.906721 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 641/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 09:25:08 2022] PUT /identity/v3/projects/9a54bfaeb3c04288a7942306702edba9/users/a117c23f88cd4b7ab7b4c0f0d24341d5/roles/af958d22229a4a0297bbe2e643f55181 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:25:10.901034 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b63bf8-52e2-400d-804a-469d8a4fdf44 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:10.901618 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b63bf8-52e2-400d-804a-469d8a4fdf44 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:10.901618 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b63bf8-52e2-400d-804a-469d8a4fdf44 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:10.903772 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 633/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:25:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:25:10.912497 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fb20d-6993-4aa8-894c-da693e8c8478 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:10.912909 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fb20d-6993-4aa8-894c-da693e8c8478 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:10.913062 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fb20d-6993-4aa8-894c-da693e8c8478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:10.934394 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-984fb20d-6993-4aa8-894c-da693e8c8478 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:10.935600 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-984fb20d-6993-4aa8-894c-da693e8c8478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:11.005862 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 642/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:25:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:25:11.031769 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b63bf8-52e2-400d-804a-469d8a4fdf44 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:11.153989 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b63bf8-52e2-400d-804a-469d8a4fdf44 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:11.171731 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:11.191192 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b43e4-3ab9-488c-9039-0d9aff7650fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:11.191850 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b43e4-3ab9-488c-9039-0d9aff7650fb None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:11.192137 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b43e4-3ab9-488c-9039-0d9aff7650fb None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:11.192419 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b43e4-3ab9-488c-9039-0d9aff7650fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:11.203383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b43e4-3ab9-488c-9039-0d9aff7650fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-8a3b43e4-3ab9-488c-9039-0d9aff7650fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:25:11.208398 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:11.209720 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:11.211270 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 634/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 09:25:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:11.216410 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fb20d-6993-4aa8-894c-da693e8c8478 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:11.323047 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:11.343171 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fb20d-6993-4aa8-894c-da693e8c8478 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:11.388020 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:11.388703 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:11.388983 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:11.389270 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] PATH_INFO: `/v3/roles` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:11.390005 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:11.390349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:11.391791 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:11.401064 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 643/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 09:25:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:11.408381 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b43e4-3ab9-488c-9039-0d9aff7650fb None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:11.525332 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b43e4-3ab9-488c-9039-0d9aff7650fb None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:11.574461 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:11.575111 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:11.575397 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:11.575683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:11.588383 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:11.589864 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:25:11.608365 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:11.609651 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:11.611196 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:25:11.611196 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:25:11.611196 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:11.611196 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:11.611196 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 635/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:11.616094 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:11.728403 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40edf33-4688-4570-abb0-807e6045b117 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:11.764587 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:11.765221 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:11.765500 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:11.765771 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] PATH_INFO: `/v3/domains` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:11.766510 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:11.766818 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:11.768188 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:11.786069 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 644/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:11.795417 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:11.835288 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:11.917028 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e6dc7-d20b-4b5d-ae0f-43f48173aa22 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:11.953577 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:11.954261 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:11.954535 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:11.954805 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:11.962616 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:11.964073 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:25:11.975297 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:11.976502 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:11.978343 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 636/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:11.985320 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:11.989554 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:12.101070 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d092601-daae-47c3-a49e-c6ca6e583b19 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:12.138344 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:12.138975 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:12.139261 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:12.139539 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] PATH_INFO: `/v3/users` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:12.140340 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:12.140650 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:12.142085 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:12.159236 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 645/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 09:25:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:25:12.167415 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:12.280913 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:12.293083 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612fe16-bc99-4ed4-939e-a83404af6aa4 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:12.332319 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff50a-be4c-4156-b1ef-ac09593d0ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:12.333515 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff50a-be4c-4156-b1ef-ac09593d0ded None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:12.334041 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff50a-be4c-4156-b1ef-ac09593d0ded None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:12.334522 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff50a-be4c-4156-b1ef-ac09593d0ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:12.420011 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff50a-be4c-4156-b1ef-ac09593d0ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:12.421503 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff50a-be4c-4156-b1ef-ac09593d0ded None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:12.519632 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:12.539435 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 637/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 09:25:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:25:12.546495 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:12.614397 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:12.666369 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd47fb-49c2-4769-b4c5-c1c965fee8b0 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:12.705291 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:12.705936 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:12.706281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:12.706513 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:12.718923 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:12.720528 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:25:12.736244 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:12.737823 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:12.739575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:25:12.739575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:25:12.739575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:25:12.739575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:12.739575 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 646/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 09:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:12.744326 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff50a-be4c-4156-b1ef-ac09593d0ded None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:12.859864 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff50a-be4c-4156-b1ef-ac09593d0ded None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:12.895967 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:12.896598 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:12.896882 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:12.897168 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] PATH_INFO: `/v3/domains` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:12.897874 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:12.898214 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:12.899599 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:12.911281 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 638/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 09:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:12.917532 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:13.044204 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4694a-8b14-4242-ad28-2727d29e150e None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:13.062368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:13.087605 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:13.088354 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:13.088633 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:13.088910 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:13.095808 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:13.097248 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:25:13.109225 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:13.110477 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: ERROR keystone.server.flask.application Jul 28 09:25:13.111950 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 647/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 09:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:13.116683 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:13.173629 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:13.230761 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47245c-25e8-46fc-b01e-dbb53312e949 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:13.266635 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbf706-0744-492a-9c90-d99f0a699755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:13.267262 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf706-0744-492a-9c90-d99f0a699755 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:13.267554 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf706-0744-492a-9c90-d99f0a699755 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:13.267833 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf706-0744-492a-9c90-d99f0a699755 None admin] PATH_INFO: `/v3/projects` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:13.268607 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbf706-0744-492a-9c90-d99f0a699755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:13.268921 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbf706-0744-492a-9c90-d99f0a699755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:13.270413 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbf706-0744-492a-9c90-d99f0a699755 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:13.283322 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 639/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 09:25:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:13.292974 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:13.410349 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f6893-00d3-46c5-864c-5bf5decd0f81 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:13.451122 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693e4a-4bd4-422b-97b3-1e106a7d61e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:13.451752 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693e4a-4bd4-422b-97b3-1e106a7d61e6 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:13.452036 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693e4a-4bd4-422b-97b3-1e106a7d61e6 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:13.452300 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693e4a-4bd4-422b-97b3-1e106a7d61e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:13.453102 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2693e4a-4bd4-422b-97b3-1e106a7d61e6 None admin] RBAC: Adding query filter params (role.id=af958d22229a4a0297bbe2e643f55181, scope.project.id=9a54bfaeb3c04288a7942306702edba9, user.id=a117c23f88cd4b7ab7b4c0f0d24341d5) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:13.453418 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2693e4a-4bd4-422b-97b3-1e106a7d61e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:13.454907 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2693e4a-4bd4-422b-97b3-1e106a7d61e6 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:13.461022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 648/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 09:25:13 2022] GET /identity/v3/role_assignments?user.id=a117c23f88cd4b7ab7b4c0f0d24341d5&scope.project.id=9a54bfaeb3c04288a7942306702edba9&role.id=af958d22229a4a0297bbe2e643f55181 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:15.538287 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991dae5-71e8-41d9-915f-42b008a9660c None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:15.538924 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991dae5-71e8-41d9-915f-42b008a9660c None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:15.538924 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991dae5-71e8-41d9-915f-42b008a9660c None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:15.540264 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 640/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:25:15.545883 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fec7-24cb-42eb-bc3b-27fb53ef2732 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:15.546435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fec7-24cb-42eb-bc3b-27fb53ef2732 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:15.546435 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fec7-24cb-42eb-bc3b-27fb53ef2732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:15.566512 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-9754fec7-24cb-42eb-bc3b-27fb53ef2732 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:15.567742 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-9754fec7-24cb-42eb-bc3b-27fb53ef2732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:15.602466 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:15.644031 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 649/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:25:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:25:15.666530 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991dae5-71e8-41d9-915f-42b008a9660c None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:15.777527 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991dae5-71e8-41d9-915f-42b008a9660c None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:15.788916 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:15.815256 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:15.816286 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:15.816559 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:15.816831 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:15.817473 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:15.818901 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:15.832012 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:15.833432 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:15.834569 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 641/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:25:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:15.839864 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754fec7-24cb-42eb-bc3b-27fb53ef2732 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:15.948748 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754fec7-24cb-42eb-bc3b-27fb53ef2732 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:15.991409 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd7c3-ae39-42f5-b35f-65623a7beb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:15.992046 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd7c3-ae39-42f5-b35f-65623a7beb50 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:15.992332 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd7c3-ae39-42f5-b35f-65623a7beb50 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:15.992608 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd7c3-ae39-42f5-b35f-65623a7beb50 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:15.993299 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd7c3-ae39-42f5-b35f-65623a7beb50 None admin] RBAC: Adding query filter params (name=placement) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:15.993604 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd7c3-ae39-42f5-b35f-65623a7beb50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:15.995098 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd7c3-ae39-42f5-b35f-65623a7beb50 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:16.000457 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 650/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 09:25:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:16.006862 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:16.112318 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c14f1c-1afe-4996-9d07-3a7b18cfdeb3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:16.151255 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e384a-dc4d-47b1-92d3-581fe80e090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:16.151900 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e384a-dc4d-47b1-92d3-581fe80e090b None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:16.152186 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e384a-dc4d-47b1-92d3-581fe80e090b None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:16.152462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e384a-dc4d-47b1-92d3-581fe80e090b None admin] PATH_INFO: `/v3/services` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:16.153164 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e384a-dc4d-47b1-92d3-581fe80e090b None admin] RBAC: Adding query filter params (type=placement) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:16.153476 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e384a-dc4d-47b1-92d3-581fe80e090b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:16.154905 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e384a-dc4d-47b1-92d3-581fe80e090b None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:16.159827 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 642/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 09:25:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:18.222441 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1786fab-d9ae-4947-87c5-e7afe6dea528 None None] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:18.223038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1786fab-d9ae-4947-87c5-e7afe6dea528 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:18.223038 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1786fab-d9ae-4947-87c5-e7afe6dea528 None None] PATH_INFO: `/` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:18.224363 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 651/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:25:18.229883 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1b06-b35f-496f-853c-3582428879ba None None] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:18.230497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1b06-b35f-496f-853c-3582428879ba None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:18.230497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1b06-b35f-496f-853c-3582428879ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:18.252882 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.auth.core [None req-c7ae1b06-b35f-496f-853c-3582428879ba None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:18.254244 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.fernet_utils [None req-c7ae1b06-b35f-496f-853c-3582428879ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:18.292497 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:18.337986 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 643/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:25:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:25:18.364391 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1786fab-d9ae-4947-87c5-e7afe6dea528 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:18.405388 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:18.492144 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1786fab-d9ae-4947-87c5-e7afe6dea528 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:18.530822 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31d228-92ba-46c1-a283-f9268a67ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:18.531456 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31d228-92ba-46c1-a283-f9268a67ef4e None admin] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:18.531742 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31d228-92ba-46c1-a283-f9268a67ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:18.532022 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31d228-92ba-46c1-a283-f9268a67ef4e None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:18.532736 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31d228-92ba-46c1-a283-f9268a67ef4e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:18.534148 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31d228-92ba-46c1-a283-f9268a67ef4e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:18.551483 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 652/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 09:25:18 2022] POST /identity/v3/services => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:25:20.621709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4ad2-704b-4864-996c-a1a4c199fbd5 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:20.622312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4ad2-704b-4864-996c-a1a4c199fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:20.622312 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4ad2-704b-4864-996c-a1a4c199fbd5 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:20.623692 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 644/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:25:20.630327 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5f999-a2b4-4f53-9f58-66985831f818 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:20.631095 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5f999-a2b4-4f53-9f58-66985831f818 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:20.631287 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5f999-a2b4-4f53-9f58-66985831f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:20.658421 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-27c5f999-a2b4-4f53-9f58-66985831f818 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:20.659663 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-27c5f999-a2b4-4f53-9f58-66985831f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:20.742607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 653/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:25:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:25:20.766188 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4ad2-704b-4864-996c-a1a4c199fbd5 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:20.886702 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4ad2-704b-4864-996c-a1a4c199fbd5 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:20.922090 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:20.922787 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:20.923070 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:20.923355 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:20.923990 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:20.925390 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:20.935141 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:20.936670 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:20.937959 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:20.942088 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 645/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:25:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:20.943222 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5f999-a2b4-4f53-9f58-66985831f818 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:20.962618 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:21.049161 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5f999-a2b4-4f53-9f58-66985831f818 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:21.084901 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:21.085521 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:21.085800 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:21.086073 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:21.086787 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] RBAC: Adding query filter params (name=placement) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:21.087087 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:21.088607 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:21.093913 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 654/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 09:25:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:21.100579 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:21.135907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:21.208412 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c7a60-8bfe-4217-afe4-10f52198dac3 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:21.245499 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a6ca6-6574-4f32-bb37-5bb963036ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:21.246145 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a6ca6-6574-4f32-bb37-5bb963036ecc None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:21.246426 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a6ca6-6574-4f32-bb37-5bb963036ecc None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:21.246709 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a6ca6-6574-4f32-bb37-5bb963036ecc None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:21.247482 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a6ca6-6574-4f32-bb37-5bb963036ecc None admin] RBAC: Adding query filter params (interface=public, service_id=d2bdf7bc2aa243dbb4f990fa42827e91, region_id=RegionOne) {{(pid=474608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:21.247799 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a6ca6-6574-4f32-bb37-5bb963036ecc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:21.249227 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a6ca6-6574-4f32-bb37-5bb963036ecc None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:21.255477 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 646/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 09:25:21 2022] GET /identity/v3/endpoints?service_id=d2bdf7bc2aa243dbb4f990fa42827e91&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:25:21.262508 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:21.369368 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:21.397964 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89104f95-9586-4496-98e5-9c0072116798 None admin] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:21.439638 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ffb9d-fa12-4c10-9499-50608ab97b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:21.440307 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ffb9d-fa12-4c10-9499-50608ab97b0e None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:21.440595 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ffb9d-fa12-4c10-9499-50608ab97b0e None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:21.441187 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ffb9d-fa12-4c10-9499-50608ab97b0e None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:21.441922 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ffb9d-fa12-4c10-9499-50608ab97b0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:21.443461 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ffb9d-fa12-4c10-9499-50608ab97b0e None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:21.452284 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 655/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 09:25:21 2022] GET /identity/v3/services => generated 2191 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:25:23.525251 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905a1d-e622-4f41-aaaf-174583640975 None None] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:23.525885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905a1d-e622-4f41-aaaf-174583640975 None None] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:23.525885 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905a1d-e622-4f41-aaaf-174583640975 None None] PATH_INFO: `/` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:23.527289 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 647/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 09:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:25:23.532613 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4621a-6201-43ee-9d02-314442edc3f1 None None] REQUEST_METHOD: `POST` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:23.533108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4621a-6201-43ee-9d02-314442edc3f1 None None] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:23.533108 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4621a-6201-43ee-9d02-314442edc3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:23.552553 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.auth.core [None req-07a4621a-6201-43ee-9d02-314442edc3f1 None None] MFA Rules not processed for user `530dca3bb7524c5cbcfad367381348a5`. Rule list: `[]` (Enabled: `True`). {{(pid=474607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:23.553778 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.fernet_utils [None req-07a4621a-6201-43ee-9d02-314442edc3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=474607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:23.626687 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 656/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 09:25:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:25:23.649764 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45905a1d-e622-4f41-aaaf-174583640975 None None] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:23.757361 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=100 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:23.765612 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45905a1d-e622-4f41-aaaf-174583640975 None None] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:23.813907 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:23.814817 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] REQUEST_METHOD: `GET` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:23.815205 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:23.815552 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] PATH_INFO: `/v3/services/placement` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:23.816866 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:23.819067 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:25:23.829517 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:25:23.831486 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application with Lock( Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: ERROR keystone.server.flask.application Jul 28 09:25:23.833750 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 648/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 09:25:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:25:23.837646 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4621a-6201-43ee-9d02-314442edc3f1 None None] Authenticating user token {{(pid=474607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:23.939387 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=100 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:23.965541 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4621a-6201-43ee-9d02-314442edc3f1 None None] Validating token access rules against request {{(pid=474607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:24.003257 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41222ae-fd10-44a2-809a-8382bbc6238c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:24.003885 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41222ae-fd10-44a2-809a-8382bbc6238c None admin] REQUEST_METHOD: `GET` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:24.004130 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41222ae-fd10-44a2-809a-8382bbc6238c None admin] SCRIPT_NAME: `/identity` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:24.004429 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41222ae-fd10-44a2-809a-8382bbc6238c None admin] PATH_INFO: `/v3/services` {{(pid=474607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:24.005135 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41222ae-fd10-44a2-809a-8382bbc6238c None admin] RBAC: Adding query filter params (name=placement) {{(pid=474607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:25:24.005439 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41222ae-fd10-44a2-809a-8382bbc6238c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:24.006844 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41222ae-fd10-44a2-809a-8382bbc6238c None admin] RBAC: Authorization granted {{(pid=474607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:24.012281 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: [pid: 474607|app: 0|req: 657/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 09:25:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:25:24.018829 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] Authenticating user token {{(pid=474608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:24.128598 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca849d-5fa5-4b13-ba77-913deaa1d578 None admin] Validating token access rules against request {{(pid=474608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:24.164828 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f78612-6e8a-4106-a59b-e4e5c8443a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '530dca3bb7524c5cbcfad367381348a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=474608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:24.165462 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f78612-6e8a-4106-a59b-e4e5c8443a92 None admin] REQUEST_METHOD: `POST` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:24.165755 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f78612-6e8a-4106-a59b-e4e5c8443a92 None admin] SCRIPT_NAME: `/identity` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:24.166059 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f78612-6e8a-4106-a59b-e4e5c8443a92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=474608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:24.166826 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f78612-6e8a-4106-a59b-e4e5c8443a92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:24.168228 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f78612-6e8a-4106-a59b-e4e5c8443a92 None admin] RBAC: Authorization granted {{(pid=474608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:24.189221 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: [pid: 474608|app: 0|req: 649/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 09:25:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:25:34.008948 dv-cinder-851351-1-iscsi devstack@keystone.service[474607]: DEBUG dbcounter [-] [474607] Writing DB stats keystone:SELECT=30 {{(pid=474607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:34.184621 dv-cinder-851351-1-iscsi devstack@keystone.service[474608]: DEBUG dbcounter [-] [474608] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=474608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}